From webhook-mailer at python.org Sat Sep 1 04:57:02 2018 From: webhook-mailer at python.org (Ivan Levkivskyi) Date: Sat, 01 Sep 2018 08:57:02 -0000 Subject: [Python-checkins] Fix typo in typing.py module docstring (#9014) Message-ID: https://github.com/python/cpython/commit/5265b3a98b376684e361b62d0728483b26f493f2 commit: 5265b3a98b376684e361b62d0728483b26f493f2 branch: master author: Tim McNamara committer: Ivan Levkivskyi date: 2018-09-01T09:56:58+01:00 summary: Fix typo in typing.py module docstring (#9014) "explicitelly" ? "explicitly" files: M Lib/typing.py diff --git a/Lib/typing.py b/Lib/typing.py index 48992633ac34..445a42492b6b 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -2,7 +2,7 @@ The typing module: Support for gradual typing as defined by PEP 484. At large scale, the structure of the module is following: -* Imports and exports, all public names should be explicitelly added to __all__. +* Imports and exports, all public names should be explicitly added to __all__. * Internal helper functions: these should never be used in code outside this module. * _SpecialForm and its instances (special forms): Any, NoReturn, ClassVar, Union, Optional * Two classes whose instances can be type arguments in addition to types: ForwardRef and TypeVar From solipsis at pitrou.net Sat Sep 1 05:10:43 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 01 Sep 2018 09:10:43 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=-4 Message-ID: <20180901091043.1.CE09C2F9448AA2BC@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [-7, 1, 0] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [0, 0, -2] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogxJiVO_', '--timeout', '7200'] From webhook-mailer at python.org Sat Sep 1 05:15:45 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 01 Sep 2018 09:15:45 -0000 Subject: [Python-checkins] Fix typo in typing.py module docstring (GH-9014) Message-ID: https://github.com/python/cpython/commit/f6d70b8b1708382f09b008bd1cd8792e00b82afb commit: f6d70b8b1708382f09b008bd1cd8792e00b82afb branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-01T05:15:40-04:00 summary: Fix typo in typing.py module docstring (GH-9014) "explicitelly" ? "explicitly" (cherry picked from commit 5265b3a98b376684e361b62d0728483b26f493f2) Co-authored-by: Tim McNamara files: M Lib/typing.py diff --git a/Lib/typing.py b/Lib/typing.py index 48992633ac34..445a42492b6b 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -2,7 +2,7 @@ The typing module: Support for gradual typing as defined by PEP 484. At large scale, the structure of the module is following: -* Imports and exports, all public names should be explicitelly added to __all__. +* Imports and exports, all public names should be explicitly added to __all__. * Internal helper functions: these should never be used in code outside this module. * _SpecialForm and its instances (special forms): Any, NoReturn, ClassVar, Union, Optional * Two classes whose instances can be type arguments in addition to types: ForwardRef and TypeVar From webhook-mailer at python.org Sat Sep 1 18:14:01 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Sat, 01 Sep 2018 22:14:01 -0000 Subject: [Python-checkins] closes bpo-34555: Fix incorrectly nested test for HAVE_LINUX_VM_SOCKETS_H (GH-9016) Message-ID: https://github.com/python/cpython/commit/2d7102e726e973ab2d307aa9748c7ec433677877 commit: 2d7102e726e973ab2d307aa9748c7ec433677877 branch: master author: Thomas Herzog committer: Benjamin Peterson date: 2018-09-01T15:13:57-07:00 summary: closes bpo-34555: Fix incorrectly nested test for HAVE_LINUX_VM_SOCKETS_H (GH-9016) files: A Misc/NEWS.d/next/Build/2018-08-31-19-41-09.bpo-34555.dfQcnm.rst M Modules/socketmodule.h diff --git a/Misc/NEWS.d/next/Build/2018-08-31-19-41-09.bpo-34555.dfQcnm.rst b/Misc/NEWS.d/next/Build/2018-08-31-19-41-09.bpo-34555.dfQcnm.rst new file mode 100644 index 000000000000..7e61c4fb20ab --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-08-31-19-41-09.bpo-34555.dfQcnm.rst @@ -0,0 +1,2 @@ +Fix for case where it was not possible to have both +``HAVE_LINUX_VM_SOCKETS_H`` and ``HAVE_SOCKADDR_ALG`` be undefined. diff --git a/Modules/socketmodule.h b/Modules/socketmodule.h index bce74c3da8fd..0b2edc158782 100644 --- a/Modules/socketmodule.h +++ b/Modules/socketmodule.h @@ -101,39 +101,40 @@ typedef int socklen_t; #include #endif -#ifdef HAVE_SOCKADDR_ALG -#include -#ifndef AF_ALG -#define AF_ALG 38 -#endif -#ifndef SOL_ALG -#define SOL_ALG 279 -#endif - #ifdef HAVE_LINUX_VM_SOCKETS_H # include #else # undef AF_VSOCK #endif +#ifdef HAVE_SOCKADDR_ALG + +# include +# ifndef AF_ALG +# define AF_ALG 38 +# endif +# ifndef SOL_ALG +# define SOL_ALG 279 +# endif + /* Linux 3.19 */ -#ifndef ALG_SET_AEAD_ASSOCLEN -#define ALG_SET_AEAD_ASSOCLEN 4 -#endif -#ifndef ALG_SET_AEAD_AUTHSIZE -#define ALG_SET_AEAD_AUTHSIZE 5 -#endif +# ifndef ALG_SET_AEAD_ASSOCLEN +# define ALG_SET_AEAD_ASSOCLEN 4 +# endif +# ifndef ALG_SET_AEAD_AUTHSIZE +# define ALG_SET_AEAD_AUTHSIZE 5 +# endif /* Linux 4.8 */ -#ifndef ALG_SET_PUBKEY -#define ALG_SET_PUBKEY 6 -#endif +# ifndef ALG_SET_PUBKEY +# define ALG_SET_PUBKEY 6 +# endif -#ifndef ALG_OP_SIGN -#define ALG_OP_SIGN 2 -#endif -#ifndef ALG_OP_VERIFY -#define ALG_OP_VERIFY 3 -#endif +# ifndef ALG_OP_SIGN +# define ALG_OP_SIGN 2 +# endif +# ifndef ALG_OP_VERIFY +# define ALG_OP_VERIFY 3 +# endif #endif /* HAVE_SOCKADDR_ALG */ From webhook-mailer at python.org Sat Sep 1 18:30:49 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 01 Sep 2018 22:30:49 -0000 Subject: [Python-checkins] closes bpo-34555: Fix incorrectly nested test for HAVE_LINUX_VM_SOCKETS_H (GH-9016) Message-ID: https://github.com/python/cpython/commit/4c532da1209bd20ba07f18448134f32ace8c54f7 commit: 4c532da1209bd20ba07f18448134f32ace8c54f7 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-01T18:30:44-04:00 summary: closes bpo-34555: Fix incorrectly nested test for HAVE_LINUX_VM_SOCKETS_H (GH-9016) (cherry picked from commit 2d7102e726e973ab2d307aa9748c7ec433677877) Co-authored-by: Thomas Herzog files: A Misc/NEWS.d/next/Build/2018-08-31-19-41-09.bpo-34555.dfQcnm.rst M Modules/socketmodule.h diff --git a/Misc/NEWS.d/next/Build/2018-08-31-19-41-09.bpo-34555.dfQcnm.rst b/Misc/NEWS.d/next/Build/2018-08-31-19-41-09.bpo-34555.dfQcnm.rst new file mode 100644 index 000000000000..7e61c4fb20ab --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-08-31-19-41-09.bpo-34555.dfQcnm.rst @@ -0,0 +1,2 @@ +Fix for case where it was not possible to have both +``HAVE_LINUX_VM_SOCKETS_H`` and ``HAVE_SOCKADDR_ALG`` be undefined. diff --git a/Modules/socketmodule.h b/Modules/socketmodule.h index bce74c3da8fd..0b2edc158782 100644 --- a/Modules/socketmodule.h +++ b/Modules/socketmodule.h @@ -101,39 +101,40 @@ typedef int socklen_t; #include #endif -#ifdef HAVE_SOCKADDR_ALG -#include -#ifndef AF_ALG -#define AF_ALG 38 -#endif -#ifndef SOL_ALG -#define SOL_ALG 279 -#endif - #ifdef HAVE_LINUX_VM_SOCKETS_H # include #else # undef AF_VSOCK #endif +#ifdef HAVE_SOCKADDR_ALG + +# include +# ifndef AF_ALG +# define AF_ALG 38 +# endif +# ifndef SOL_ALG +# define SOL_ALG 279 +# endif + /* Linux 3.19 */ -#ifndef ALG_SET_AEAD_ASSOCLEN -#define ALG_SET_AEAD_ASSOCLEN 4 -#endif -#ifndef ALG_SET_AEAD_AUTHSIZE -#define ALG_SET_AEAD_AUTHSIZE 5 -#endif +# ifndef ALG_SET_AEAD_ASSOCLEN +# define ALG_SET_AEAD_ASSOCLEN 4 +# endif +# ifndef ALG_SET_AEAD_AUTHSIZE +# define ALG_SET_AEAD_AUTHSIZE 5 +# endif /* Linux 4.8 */ -#ifndef ALG_SET_PUBKEY -#define ALG_SET_PUBKEY 6 -#endif +# ifndef ALG_SET_PUBKEY +# define ALG_SET_PUBKEY 6 +# endif -#ifndef ALG_OP_SIGN -#define ALG_OP_SIGN 2 -#endif -#ifndef ALG_OP_VERIFY -#define ALG_OP_VERIFY 3 -#endif +# ifndef ALG_OP_SIGN +# define ALG_OP_SIGN 2 +# endif +# ifndef ALG_OP_VERIFY +# define ALG_OP_VERIFY 3 +# endif #endif /* HAVE_SOCKADDR_ALG */ From webhook-mailer at python.org Sat Sep 1 21:59:30 2018 From: webhook-mailer at python.org (Zachary Ware) Date: Sun, 02 Sep 2018 01:59:30 -0000 Subject: [Python-checkins] Fix struct sequence glossary entry grammar (GH-9030) Message-ID: https://github.com/python/cpython/commit/98b976a2f82ba5f50cf6846338f644ca6c64f47d commit: 98b976a2f82ba5f50cf6846338f644ca6c64f47d branch: master author: Zachary Ware committer: GitHub date: 2018-09-01T20:59:27-05:00 summary: Fix struct sequence glossary entry grammar (GH-9030) ... by removing a superfluous "either". Reported by ?????? ???????? on docs at . files: M Doc/glossary.rst diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 69960579f2c0..16fc7f0783c8 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -1012,7 +1012,7 @@ Glossary struct sequence A tuple with named elements. Struct sequences expose an interface similar - to :term:`named tuple` in that elements can either be accessed either by + to :term:`named tuple` in that elements can be accessed either by index or as an attribute. However, they do not have any of the named tuple methods like :meth:`~collections.somenamedtuple._make` or :meth:`~collections.somenamedtuple._asdict`. Examples of struct sequences From webhook-mailer at python.org Sat Sep 1 22:13:38 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 02 Sep 2018 02:13:38 -0000 Subject: [Python-checkins] Fix struct sequence glossary entry grammar (GH-9030) Message-ID: https://github.com/python/cpython/commit/f4c865eda8f57dedaeaa16530ab6f1024bfb94de commit: f4c865eda8f57dedaeaa16530ab6f1024bfb94de branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-01T22:13:34-04:00 summary: Fix struct sequence glossary entry grammar (GH-9030) ... by removing a superfluous "either". Reported by ?????? ???????? on docs at . (cherry picked from commit 98b976a2f82ba5f50cf6846338f644ca6c64f47d) Co-authored-by: Zachary Ware files: M Doc/glossary.rst diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 69960579f2c0..16fc7f0783c8 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -1012,7 +1012,7 @@ Glossary struct sequence A tuple with named elements. Struct sequences expose an interface similar - to :term:`named tuple` in that elements can either be accessed either by + to :term:`named tuple` in that elements can be accessed either by index or as an attribute. However, they do not have any of the named tuple methods like :meth:`~collections.somenamedtuple._make` or :meth:`~collections.somenamedtuple._asdict`. Examples of struct sequences From webhook-mailer at python.org Sat Sep 1 22:13:46 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 02 Sep 2018 02:13:46 -0000 Subject: [Python-checkins] Fix struct sequence glossary entry grammar (GH-9030) Message-ID: https://github.com/python/cpython/commit/f4c4d994ee1d49d9455f72445e9cb689a215afbf commit: f4c4d994ee1d49d9455f72445e9cb689a215afbf branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-01T22:13:43-04:00 summary: Fix struct sequence glossary entry grammar (GH-9030) ... by removing a superfluous "either". Reported by ?????? ???????? on docs at . (cherry picked from commit 98b976a2f82ba5f50cf6846338f644ca6c64f47d) Co-authored-by: Zachary Ware files: M Doc/glossary.rst diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 12f88a9d736d..cf2ca671f26d 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -1006,7 +1006,7 @@ Glossary struct sequence A tuple with named elements. Struct sequences expose an interface similar - to :term:`named tuple` in that elements can either be accessed either by + to :term:`named tuple` in that elements can be accessed either by index or as an attribute. However, they do not have any of the named tuple methods like :meth:`~collections.somenamedtuple._make` or :meth:`~collections.somenamedtuple._asdict`. Examples of struct sequences From webhook-mailer at python.org Sat Sep 1 22:18:26 2018 From: webhook-mailer at python.org (Zachary Ware) Date: Sun, 02 Sep 2018 02:18:26 -0000 Subject: [Python-checkins] [2.7] Fix struct sequence glossary entry grammar (GH-9030) Message-ID: https://github.com/python/cpython/commit/71f2dadf66c8f9f513bb67f3b06d320c406ac2ff commit: 71f2dadf66c8f9f513bb67f3b06d320c406ac2ff branch: 2.7 author: Zachary Ware committer: GitHub date: 2018-09-01T21:18:22-05:00 summary: [2.7] Fix struct sequence glossary entry grammar (GH-9030) ... by removing a superfluous "either". Reported by ?????? ???????? on docs@ (cherry picked from commit 98b976a2f82ba5f50cf6846338f644ca6c64f47d) files: M Doc/glossary.rst diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 34c71c6f3eee..9e6bf233bbcf 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -710,7 +710,7 @@ Glossary struct sequence A tuple with named elements. Struct sequences expose an interface similiar - to :term:`named tuple` in that elements can either be accessed either by + to :term:`named tuple` in that elements can be accessed either by index or as an attribute. However, they do not have any of the named tuple methods like :meth:`~collections.somenamedtuple._make` or :meth:`~collections.somenamedtuple._asdict`. Examples of struct sequences From solipsis at pitrou.net Sun Sep 2 05:09:23 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 02 Sep 2018 09:09:23 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=9 Message-ID: <20180902090923.1.43F73A409E5C8E39@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 3] memory blocks, sum=3 test_collections leaked [0, 7, -7] memory blocks, sum=0 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [-1, 2, 0] memory blocks, sum=1 test_multiprocessing_spawn leaked [-1, 0, 2] memory blocks, sum=1 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog0xUvUl', '--timeout', '7200'] From webhook-mailer at python.org Sun Sep 2 16:34:26 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 02 Sep 2018 20:34:26 -0000 Subject: [Python-checkins] Minor improvement to code clarity (GH-9036) Message-ID: https://github.com/python/cpython/commit/f3267144269b873bcb87a9fcafe94b37be1bcfdc commit: f3267144269b873bcb87a9fcafe94b37be1bcfdc branch: master author: Raymond Hettinger committer: GitHub date: 2018-09-02T13:34:21-07:00 summary: Minor improvement to code clarity (GH-9036) Make it clear that the n==0 case is included. Otherwise, you have to know that max==0.0 whenever n==0. files: M Modules/mathmodule.c diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 8015a95d2aa7..06a969cebacb 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -2074,7 +2074,7 @@ vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) if (found_nan) { return Py_NAN; } - if (max == 0.0 || n == 1) { + if (max == 0.0 || n <= 1) { return max; } for (i=0 ; i < n ; i++) { From webhook-mailer at python.org Sun Sep 2 21:48:17 2018 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Mon, 03 Sep 2018 01:48:17 -0000 Subject: [Python-checkins] bpo-34500: Fix ResourceWarning in difflib.py (GH-8926) Message-ID: https://github.com/python/cpython/commit/30af2e737aad427d4da97f8dadeeecff6c2b28f5 commit: 30af2e737aad427d4da97f8dadeeecff6c2b28f5 branch: 2.7 author: Micka?l Schoentgen committer: Terry Jan Reedy date: 2018-09-02T21:48:08-04:00 summary: bpo-34500: Fix ResourceWarning in difflib.py (GH-8926) The change to Tools/scripts/diff.py effectively backports part of a2637729f23dc993e820fd92f0d1759ad714c9b2. The test code changed in Doc/library/difflib.rst is not present in current 3.x. files: A Misc/NEWS.d/next/Tools-Demos/2018-08-25-17-13-24.bpo-34500.Edz41x.rst M Doc/library/difflib.rst M Tools/scripts/diff.py diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index c6bf3ef6775d..01a3bfc2cd13 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -757,8 +757,10 @@ It is also contained in the Python source distribution, as # we're passing these as arguments to the diff function fromdate = time.ctime(os.stat(fromfile).st_mtime) todate = time.ctime(os.stat(tofile).st_mtime) - fromlines = open(fromfile, 'U').readlines() - tolines = open(tofile, 'U').readlines() + with open(fromfile, 'U') as f: + fromlines = f.readlines() + with open(tofile, 'U') as f: + tolines = f.readlines() if options.u: diff = difflib.unified_diff(fromlines, tolines, fromfile, tofile, diff --git a/Misc/NEWS.d/next/Tools-Demos/2018-08-25-17-13-24.bpo-34500.Edz41x.rst b/Misc/NEWS.d/next/Tools-Demos/2018-08-25-17-13-24.bpo-34500.Edz41x.rst new file mode 100644 index 000000000000..27ca06f1cbcd --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2018-08-25-17-13-24.bpo-34500.Edz41x.rst @@ -0,0 +1 @@ +Fix 2 ResourceWarning in difflib.py. Patch by Micka?l Schoentgen. diff --git a/Tools/scripts/diff.py b/Tools/scripts/diff.py index 513e2a7112de..c4c2e101c60e 100755 --- a/Tools/scripts/diff.py +++ b/Tools/scripts/diff.py @@ -32,8 +32,10 @@ def main(): fromdate = time.ctime(os.stat(fromfile).st_mtime) todate = time.ctime(os.stat(tofile).st_mtime) - fromlines = open(fromfile, 'U').readlines() - tolines = open(tofile, 'U').readlines() + with open(fromfile, 'U') as f: + fromlines = f.readlines() + with open(tofile, 'U') as f: + tolines = f.readlines() if options.u: diff = difflib.unified_diff(fromlines, tolines, fromfile, tofile, fromdate, todate, n=n) From solipsis at pitrou.net Mon Sep 3 05:09:17 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 03 Sep 2018 09:09:17 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=4 Message-ID: <20180903090917.1.FA8EC9172D155D61@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [7, 0, -7] memory blocks, sum=0 test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogG5g4N6', '--timeout', '7200'] From webhook-mailer at python.org Mon Sep 3 08:38:24 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 03 Sep 2018 12:38:24 -0000 Subject: [Python-checkins] bpo-34544: Fix setlocale() in pymain_read_conf() (GH-9041) Message-ID: https://github.com/python/cpython/commit/f01b2a1b84ee08df73a78cf1017eecf15e3cb995 commit: f01b2a1b84ee08df73a78cf1017eecf15e3cb995 branch: master author: Victor Stinner committer: GitHub date: 2018-09-03T14:38:21+02:00 summary: bpo-34544: Fix setlocale() in pymain_read_conf() (GH-9041) bpo-34485, bpo-34544: On some FreeBSD, nl_langinfo(CODESET) fails if LC_ALL or LC_CTYPE is set to an invalid locale name. Replace _Py_SetLocaleFromEnv(LC_CTYPE) with _Py_SetLocaleFromEnv(LC_ALL) to initialize properly locales. Partially revert commit 177d921c8c03d30daa32994362023f777624b10d. files: M Modules/main.c diff --git a/Modules/main.c b/Modules/main.c index 1ab555b42de4..974a0a6b78b9 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -1291,10 +1291,17 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, int init_legacy_encoding = Py_LegacyWindowsFSEncodingFlag; #endif _PyCoreConfig save_config = _PyCoreConfig_INIT; + char *oldloc = NULL; int res = -1; - /* Set LC_CTYPE to the user preferred locale */ - _Py_SetLocaleFromEnv(LC_CTYPE); + oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL)); + if (oldloc == NULL) { + pymain->err = _Py_INIT_NO_MEMORY(); + goto done; + } + + /* Reconfigure the locale to the default for this process */ + _Py_SetLocaleFromEnv(LC_ALL); int locale_coerced = 0; int loops = 0; @@ -1385,6 +1392,10 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, done: _PyCoreConfig_Clear(&save_config); + if (oldloc != NULL) { + setlocale(LC_ALL, oldloc); + PyMem_RawFree(oldloc); + } Py_UTF8Mode = init_utf8_mode ; #ifdef MS_WINDOWS Py_LegacyWindowsFSEncodingFlag = init_legacy_encoding; From webhook-mailer at python.org Mon Sep 3 11:05:23 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 03 Sep 2018 15:05:23 -0000 Subject: [Python-checkins] _Py_CoerceLegacyLocale() restores LC_CTYPE on fail (GH-9044) Message-ID: https://github.com/python/cpython/commit/8ea09110d413829f71d979d8c7073008cb87fb03 commit: 8ea09110d413829f71d979d8c7073008cb87fb03 branch: master author: Victor Stinner committer: GitHub date: 2018-09-03T17:05:18+02:00 summary: _Py_CoerceLegacyLocale() restores LC_CTYPE on fail (GH-9044) bpo-34544: If _Py_CoerceLegacyLocale() fails to coerce the C locale, restore the LC_CTYPE locale to the its previous value. files: M Python/pylifecycle.c diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 7d17f2e65e42..33ca802fd56d 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -363,6 +363,13 @@ void _Py_CoerceLegacyLocale(int warn) { #ifdef PY_COERCE_C_LOCALE + char *oldloc = NULL; + + oldloc = _PyMem_RawStrdup(setlocale(LC_CTYPE, NULL)); + if (oldloc == NULL) { + return; + } + const char *locale_override = getenv("LC_ALL"); if (locale_override == NULL || *locale_override == '\0') { /* LC_ALL is also not set (or is set to an empty string) */ @@ -384,11 +391,16 @@ defined(HAVE_LANGINFO_H) && defined(CODESET) #endif /* Successfully configured locale, so make it the default */ _coerce_default_locale_settings(warn, target); - return; + goto done; } } } /* No C locale warning here, as Py_Initialize will emit one later */ + + setlocale(LC_CTYPE, oldloc); + +done: + PyMem_RawFree(oldloc); #endif } From webhook-mailer at python.org Mon Sep 3 11:06:42 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 03 Sep 2018 15:06:42 -0000 Subject: [Python-checkins] bpo-34567: pythoninfo gets coreconfig (GH-9043) Message-ID: https://github.com/python/cpython/commit/2094c2bea4f79c31819994d8f0afa97ccc52cca8 commit: 2094c2bea4f79c31819994d8f0afa97ccc52cca8 branch: master author: Victor Stinner committer: GitHub date: 2018-09-03T17:06:39+02:00 summary: bpo-34567: pythoninfo gets coreconfig (GH-9043) * Add _testcapi.get_coreconfig() to get the _PyCoreConfig of the interpreter * test.pythoninfo now gets the core configuration using _testcapi.get_coreconfig() files: M Include/coreconfig.h M Lib/test/pythoninfo.py M Modules/_testcapimodule.c M Modules/main.c diff --git a/Include/coreconfig.h b/Include/coreconfig.h index 431c292b8164..ef043ab02df6 100644 --- a/Include/coreconfig.h +++ b/Include/coreconfig.h @@ -366,6 +366,10 @@ PyAPI_FUNC(int) _Py_SetFileSystemEncoding( PyAPI_FUNC(void) _Py_ClearFileSystemEncoding(void); #endif +#ifndef Py_LIMITED_API +PyAPI_FUNC(PyObject*) _Py_wstrlist_as_pylist(int len, wchar_t **list); +#endif + #ifdef __cplusplus } diff --git a/Lib/test/pythoninfo.py b/Lib/test/pythoninfo.py index f058185c2f80..21763d58c039 100644 --- a/Lib/test/pythoninfo.py +++ b/Lib/test/pythoninfo.py @@ -534,6 +534,17 @@ def collect_gdbm(info_add): info_add('gdbm.GDBM_VERSION', '.'.join(map(str, _GDBM_VERSION))) +def collect_get_coreconfig(info_add): + try: + from _testcapi import get_coreconfig + except ImportError: + return + + config = get_coreconfig() + for key in sorted(config): + info_add('coreconfig[%s]' % key, repr(config[key])) + + def collect_info(info): error = False info_add = info.add @@ -562,6 +573,7 @@ def collect_info(info): collect_resource, collect_cc, collect_gdbm, + collect_get_coreconfig, # Collecting from tests should be last as they have side effects. collect_test_socket, diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 7c2c57b98001..add642f223af 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -4642,6 +4642,160 @@ decode_locale_ex(PyObject *self, PyObject *args) } +static PyObject * +get_coreconfig(PyObject *self, PyObject *Py_UNUSED(args)) +{ + PyInterpreterState *interp = _PyInterpreterState_Get(); + const _PyCoreConfig *config = &interp->core_config; + PyObject *dict, *obj; + + dict = PyDict_New(); + if (dict == NULL) { + return NULL; + } + +#define FROM_STRING(STR) \ + ((STR != NULL) ? \ + PyUnicode_FromString(STR) \ + : (Py_INCREF(Py_None), Py_None)) +#define FROM_WSTRING(STR) \ + ((STR != NULL) ? \ + PyUnicode_FromWideChar(STR, -1) \ + : (Py_INCREF(Py_None), Py_None)) +#define SET_ITEM(KEY, EXPR) \ + do { \ + obj = (EXPR); \ + if (obj == NULL) { \ + return NULL; \ + } \ + int res = PyDict_SetItemString(dict, (KEY), obj); \ + Py_DECREF(obj); \ + if (res < 0) { \ + goto fail; \ + } \ + } while (0) + + SET_ITEM("install_signal_handlers", + PyLong_FromLong(config->install_signal_handlers)); + SET_ITEM("use_environment", + PyLong_FromLong(config->use_environment)); + SET_ITEM("use_hash_seed", + PyLong_FromLong(config->use_hash_seed)); + SET_ITEM("hash_seed", + PyLong_FromUnsignedLong(config->hash_seed)); + SET_ITEM("allocator", + FROM_STRING(config->allocator)); + SET_ITEM("dev_mode", + PyLong_FromLong(config->dev_mode)); + SET_ITEM("faulthandler", + PyLong_FromLong(config->faulthandler)); + SET_ITEM("tracemalloc", + PyLong_FromLong(config->tracemalloc)); + SET_ITEM("import_time", + PyLong_FromLong(config->import_time)); + SET_ITEM("show_ref_count", + PyLong_FromLong(config->show_ref_count)); + SET_ITEM("show_alloc_count", + PyLong_FromLong(config->show_alloc_count)); + SET_ITEM("dump_refs", + PyLong_FromLong(config->dump_refs)); + SET_ITEM("malloc_stats", + PyLong_FromLong(config->malloc_stats)); + SET_ITEM("coerce_c_locale", + PyLong_FromLong(config->coerce_c_locale)); + SET_ITEM("coerce_c_locale_warn", + PyLong_FromLong(config->coerce_c_locale_warn)); + SET_ITEM("filesystem_encoding", + FROM_STRING(config->filesystem_encoding)); + SET_ITEM("filesystem_errors", + FROM_STRING(config->filesystem_errors)); + SET_ITEM("stdio_encoding", + FROM_STRING(config->stdio_encoding)); + SET_ITEM("utf8_mode", + PyLong_FromLong(config->utf8_mode)); + SET_ITEM("pycache_prefix", + FROM_WSTRING(config->pycache_prefix)); + SET_ITEM("program_name", + FROM_WSTRING(config->program_name)); + SET_ITEM("argv", + _Py_wstrlist_as_pylist(config->argc, config->argv)); + SET_ITEM("program", + FROM_WSTRING(config->program)); + SET_ITEM("warnoptions", + _Py_wstrlist_as_pylist(config->nwarnoption, config->warnoptions)); + SET_ITEM("module_search_path_env", + FROM_WSTRING(config->module_search_path_env)); + SET_ITEM("home", + FROM_WSTRING(config->home)); + SET_ITEM("module_search_paths", + _Py_wstrlist_as_pylist(config->nmodule_search_path, config->module_search_paths)); + SET_ITEM("executable", + FROM_WSTRING(config->executable)); + SET_ITEM("prefix", + FROM_WSTRING(config->prefix)); + SET_ITEM("base_prefix", + FROM_WSTRING(config->base_prefix)); + SET_ITEM("exec_prefix", + FROM_WSTRING(config->exec_prefix)); + SET_ITEM("base_exec_prefix", + FROM_WSTRING(config->base_exec_prefix)); +#ifdef MS_WINDOWS + SET_ITEM("dll_path", + FROM_WSTRING(config->dll_path)); +#endif + SET_ITEM("isolated", + PyLong_FromLong(config->isolated)); + SET_ITEM("site_import", + PyLong_FromLong(config->site_import)); + SET_ITEM("bytes_warning", + PyLong_FromLong(config->bytes_warning)); + SET_ITEM("inspect", + PyLong_FromLong(config->inspect)); + SET_ITEM("interactive", + PyLong_FromLong(config->interactive)); + SET_ITEM("optimization_level", + PyLong_FromLong(config->optimization_level)); + SET_ITEM("parser_debug", + PyLong_FromLong(config->parser_debug)); + SET_ITEM("write_bytecode", + PyLong_FromLong(config->write_bytecode)); + SET_ITEM("verbose", + PyLong_FromLong(config->verbose)); + SET_ITEM("quiet", + PyLong_FromLong(config->quiet)); + SET_ITEM("user_site_directory", + PyLong_FromLong(config->user_site_directory)); + SET_ITEM("buffered_stdio", + PyLong_FromLong(config->buffered_stdio)); + SET_ITEM("stdio_encoding", + FROM_STRING(config->stdio_encoding)); + SET_ITEM("stdio_errors", + FROM_STRING(config->stdio_errors)); +#ifdef MS_WINDOWS + SET_ITEM("legacy_windows_fs_encoding", + PyLong_FromLong(config->legacy_windows_fs_encoding)); + SET_ITEM("legacy_windows_stdio", + PyLong_FromLong(config->legacy_windows_stdio)); +#endif + SET_ITEM("_install_importlib", + PyLong_FromLong(config->_install_importlib)); + SET_ITEM("_check_hash_pycs_mode", + FROM_STRING(config->_check_hash_pycs_mode)); + SET_ITEM("_frozen", + PyLong_FromLong(config->_frozen)); + + return dict; + +fail: + Py_DECREF(dict); + return NULL; + +#undef FROM_STRING +#undef FROM_WSTRING +#undef SET_ITEM +} + + static PyMethodDef TestMethods[] = { {"raise_exception", raise_exception, METH_VARARGS}, {"raise_memoryerror", raise_memoryerror, METH_NOARGS}, @@ -4865,6 +5019,7 @@ static PyMethodDef TestMethods[] = { {"hamt", new_hamt, METH_NOARGS}, {"EncodeLocaleEx", encode_locale_ex, METH_VARARGS}, {"DecodeLocaleEx", decode_locale_ex, METH_VARARGS}, + {"get_coreconfig", get_coreconfig, METH_NOARGS}, {NULL, NULL} /* sentinel */ }; diff --git a/Modules/main.c b/Modules/main.c index 974a0a6b78b9..6f817b6c84fd 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -1018,8 +1018,8 @@ pymain_init_core_argv(_PyMain *pymain, _PyCoreConfig *config, _PyCmdline *cmdlin } -static PyObject* -wstrlist_as_pylist(int len, wchar_t **list) +PyObject* +_Py_wstrlist_as_pylist(int len, wchar_t **list) { assert(list != NULL || len < 1); @@ -1502,7 +1502,7 @@ _PyMainInterpreterConfig_Read(_PyMainInterpreterConfig *main_config, #define COPY_WSTRLIST(ATTR, LEN, LIST) \ do { \ if (ATTR == NULL) { \ - ATTR = wstrlist_as_pylist(LEN, LIST); \ + ATTR = _Py_wstrlist_as_pylist(LEN, LIST); \ if (ATTR == NULL) { \ return _Py_INIT_NO_MEMORY(); \ } \ From webhook-mailer at python.org Mon Sep 3 11:32:35 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 03 Sep 2018 15:32:35 -0000 Subject: [Python-checkins] bpo-34544: pymain_read_conf() don't change LC_ALL (GH-9045) Message-ID: https://github.com/python/cpython/commit/73b00becbdd40f6a80cfa00abf1ae650a2b5e454 commit: 73b00becbdd40f6a80cfa00abf1ae650a2b5e454 branch: master author: Victor Stinner committer: GitHub date: 2018-09-03T17:32:31+02:00 summary: bpo-34544: pymain_read_conf() don't change LC_ALL (GH-9045) bpo-34485, bpo-34544: Again, pymain_read_conf() leaves LC_ALL locale unchanged: only modify LC_CTYPE. files: M Modules/main.c diff --git a/Modules/main.c b/Modules/main.c index 6f817b6c84fd..455178af8bab 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -1291,18 +1291,7 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, int init_legacy_encoding = Py_LegacyWindowsFSEncodingFlag; #endif _PyCoreConfig save_config = _PyCoreConfig_INIT; - char *oldloc = NULL; int res = -1; - - oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL)); - if (oldloc == NULL) { - pymain->err = _Py_INIT_NO_MEMORY(); - goto done; - } - - /* Reconfigure the locale to the default for this process */ - _Py_SetLocaleFromEnv(LC_ALL); - int locale_coerced = 0; int loops = 0; @@ -1311,6 +1300,9 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, goto done; } + /* Set LC_CTYPE to the user preferred locale */ + _Py_SetLocaleFromEnv(LC_CTYPE); + while (1) { int utf8_mode = config->utf8_mode; int encoding_changed = 0; @@ -1392,10 +1384,6 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, done: _PyCoreConfig_Clear(&save_config); - if (oldloc != NULL) { - setlocale(LC_ALL, oldloc); - PyMem_RawFree(oldloc); - } Py_UTF8Mode = init_utf8_mode ; #ifdef MS_WINDOWS Py_LegacyWindowsFSEncodingFlag = init_legacy_encoding; From webhook-mailer at python.org Mon Sep 3 16:17:11 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 03 Sep 2018 20:17:11 -0000 Subject: [Python-checkins] _Py_CoerceLegacyLocale() restores LC_CTYPE on fail (GH-9044) (GH-9046) Message-ID: https://github.com/python/cpython/commit/84b0129b5e0a0e22aad22ae8db2e3833a228aa57 commit: 84b0129b5e0a0e22aad22ae8db2e3833a228aa57 branch: 3.7 author: Victor Stinner committer: GitHub date: 2018-09-03T22:17:07+02:00 summary: _Py_CoerceLegacyLocale() restores LC_CTYPE on fail (GH-9044) (GH-9046) bpo-34544: If _Py_CoerceLegacyLocale() fails to coerce the C locale, restore the LC_CTYPE locale to the its previous value. (cherry picked from commit 8ea09110d413829f71d979d8c7073008cb87fb03) files: M Python/pylifecycle.c diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 539d62a2f0f4..ba4b54864fd8 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -475,6 +475,13 @@ void _Py_CoerceLegacyLocale(const _PyCoreConfig *config) { #ifdef PY_COERCE_C_LOCALE + char *oldloc = NULL; + + oldloc = _PyMem_RawStrdup(setlocale(LC_CTYPE, NULL)); + if (oldloc == NULL) { + return; + } + const char *locale_override = getenv("LC_ALL"); if (locale_override == NULL || *locale_override == '\0') { /* LC_ALL is also not set (or is set to an empty string) */ @@ -496,11 +503,16 @@ defined(HAVE_LANGINFO_H) && defined(CODESET) #endif /* Successfully configured locale, so make it the default */ _coerce_default_locale_settings(config, target); - return; + goto done; } } } /* No C locale warning here, as Py_Initialize will emit one later */ + + setlocale(LC_CTYPE, oldloc); + +done: + PyMem_RawFree(oldloc); #endif } From webhook-mailer at python.org Mon Sep 3 17:17:24 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 03 Sep 2018 21:17:24 -0000 Subject: [Python-checkins] bpo-26901: Fix the Argument Clinic test suite (GH-8879) Message-ID: https://github.com/python/cpython/commit/65fc98e7b1f62c2e621f04780a3a77c3498cc195 commit: 65fc98e7b1f62c2e621f04780a3a77c3498cc195 branch: master author: Victor Stinner committer: GitHub date: 2018-09-03T23:17:20+02:00 summary: bpo-26901: Fix the Argument Clinic test suite (GH-8879) * Fix Tools/clinic/clinic_test.py: add missing FakeClinic.destination_buffers attribute and pass a file argument to Clinic(). * Rename Tools/clinic/clinic_test.py to Lib/test/test_clinic.py: add temporary Tools/clinic/ to sys.path to import the clinic module. Co-Authored-By: Pablo Galindo files: A Lib/test/test_clinic.py D Tools/clinic/clinic_test.py diff --git a/Tools/clinic/clinic_test.py b/Lib/test/test_clinic.py similarity index 97% rename from Tools/clinic/clinic_test.py rename to Lib/test/test_clinic.py index a9479a6e44e5..ba4ae3401056 100644 --- a/Tools/clinic/clinic_test.py +++ b/Lib/test/test_clinic.py @@ -1,16 +1,26 @@ # Argument Clinic # Copyright 2012-2013 by Larry Hastings. # Licensed to the PSF under a contributor agreement. -# -import clinic -from clinic import DSLParser +from test import support +from unittest import TestCase import collections import inspect -from test import support +import os.path import sys import unittest -from unittest import TestCase + + +clinic_path = os.path.join(os.path.dirname(__file__), '..', '..', 'Tools', 'clinic') +clinic_path = os.path.normpath(clinic_path) +if not os.path.exists(clinic_path): + raise unittest.SkipTest(f'{clinic_path!r} path does not exist') +sys.path.append(clinic_path) +try: + import clinic + from clinic import DSLParser +finally: + del sys.path[-1] class FakeConverter: @@ -35,7 +45,7 @@ def get(self, name, default): return self.used_converters.setdefault(name, FakeConverterFactory(name)) clinic.Clinic.presets_text = '' -c = clinic.Clinic(language='C') +c = clinic.Clinic(language='C', filename = "file") class FakeClinic: def __init__(self): @@ -43,6 +53,7 @@ def __init__(self): self.legacy_converters = FakeConvertersDict() self.language = clinic.CLanguage(None) self.filename = None + self.destination_buffers = {} self.block_parser = clinic.BlockParser('', self.language) self.modules = collections.OrderedDict() self.classes = collections.OrderedDict() @@ -93,7 +104,7 @@ def test_eol(self): # so it would spit out an end line for you. # and since you really already had one, # the last line of the block got corrupted. - c = clinic.Clinic(clinic.CLanguage(None)) + c = clinic.Clinic(clinic.CLanguage(None), filename="file") raw = "/*[clinic]\nfoo\n[clinic]*/" cooked = c.parse(raw).splitlines() end_line = cooked[2].rstrip() @@ -252,7 +263,7 @@ def test_round_trip_2(self): def _test_clinic(self, input, output): language = clinic.CLanguage(None) - c = clinic.Clinic(language) + c = clinic.Clinic(language, filename="file") c.parsers['inert'] = InertParser(c) c.parsers['copy'] = CopyParser(c) computed = c.parse(input) From webhook-mailer at python.org Mon Sep 3 17:20:09 2018 From: webhook-mailer at python.org (Pablo Galindo) Date: Mon, 03 Sep 2018 21:20:09 -0000 Subject: [Python-checkins] bpo-33083 - Make math.factorial reject arguments that are not int-like (GH-6149) Message-ID: https://github.com/python/cpython/commit/e9ba3705de656215d52b8f8f4a2e7ad60190e944 commit: e9ba3705de656215d52b8f8f4a2e7ad60190e944 branch: master author: Pablo Galindo committer: GitHub date: 2018-09-03T22:20:06+01:00 summary: bpo-33083 - Make math.factorial reject arguments that are not int-like (GH-6149) math.factorial() was accepting non-integral Decimal instances. This is inconsistent with the actual behaviour for floats, which are not accepted. files: A Misc/NEWS.d/next/Core and Builtins/2018-03-19-00-59-20.bpo-33083.Htztjl.rst M Lib/test/test_math.py M Modules/mathmodule.c diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py index fff82fe7fadb..608789f7202f 100644 --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -5,6 +5,7 @@ from test import support import unittest import itertools +import decimal import math import os import platform @@ -510,6 +511,10 @@ def testFactorial(self): self.assertRaises(ValueError, math.factorial, -1e100) self.assertRaises(ValueError, math.factorial, math.pi) + def testFactorialNonIntegers(self): + self.assertRaises(TypeError, math.factorial, decimal.Decimal(5.2)) + self.assertRaises(TypeError, math.factorial, "5") + # Other implementations may place different upper bounds. @support.cpython_only def testFactorialHugeInputs(self): diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-03-19-00-59-20.bpo-33083.Htztjl.rst b/Misc/NEWS.d/next/Core and Builtins/2018-03-19-00-59-20.bpo-33083.Htztjl.rst new file mode 100644 index 000000000000..81df83989e48 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-03-19-00-59-20.bpo-33083.Htztjl.rst @@ -0,0 +1,2 @@ +``math.factorial`` no longer accepts arguments that are not int-like. +Patch by Pablo Galindo. diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 06a969cebacb..e872e473f5fe 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -1656,7 +1656,7 @@ math_factorial(PyObject *module, PyObject *arg) { long x; int overflow; - PyObject *result, *odd_part, *two_valuation; + PyObject *result, *odd_part, *two_valuation, *pyint_form; if (PyFloat_Check(arg)) { PyObject *lx; @@ -1672,8 +1672,14 @@ math_factorial(PyObject *module, PyObject *arg) x = PyLong_AsLongAndOverflow(lx, &overflow); Py_DECREF(lx); } - else - x = PyLong_AsLongAndOverflow(arg, &overflow); + else { + pyint_form = PyNumber_Index(arg); + if (pyint_form == NULL) { + return NULL; + } + x = PyLong_AsLongAndOverflow(pyint_form, &overflow); + Py_DECREF(pyint_form); + } if (x == -1 && PyErr_Occurred()) { return NULL; From webhook-mailer at python.org Mon Sep 3 18:24:20 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 03 Sep 2018 22:24:20 -0000 Subject: [Python-checkins] bpo-26901: Fix the Argument Clinic test suite (GH-8879) (GH-9048) Message-ID: https://github.com/python/cpython/commit/1e921236d7ecc77299d34380d6a2159e9db05a1a commit: 1e921236d7ecc77299d34380d6a2159e9db05a1a branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Victor Stinner date: 2018-09-04T00:24:17+02:00 summary: bpo-26901: Fix the Argument Clinic test suite (GH-8879) (GH-9048) * Fix Tools/clinic/clinic_test.py: add missing FakeClinic.destination_buffers attribute and pass a file argument to Clinic(). * Rename Tools/clinic/clinic_test.py to Lib/test/test_clinic.py: add temporary Tools/clinic/ to sys.path to import the clinic module. Co-Authored-By: Pablo Galindo (cherry picked from commit 65fc98e7b1f62c2e621f04780a3a77c3498cc195) Co-authored-by: Victor Stinner files: A Lib/test/test_clinic.py D Tools/clinic/clinic_test.py diff --git a/Tools/clinic/clinic_test.py b/Lib/test/test_clinic.py similarity index 97% rename from Tools/clinic/clinic_test.py rename to Lib/test/test_clinic.py index a9479a6e44e5..ba4ae3401056 100644 --- a/Tools/clinic/clinic_test.py +++ b/Lib/test/test_clinic.py @@ -1,16 +1,26 @@ # Argument Clinic # Copyright 2012-2013 by Larry Hastings. # Licensed to the PSF under a contributor agreement. -# -import clinic -from clinic import DSLParser +from test import support +from unittest import TestCase import collections import inspect -from test import support +import os.path import sys import unittest -from unittest import TestCase + + +clinic_path = os.path.join(os.path.dirname(__file__), '..', '..', 'Tools', 'clinic') +clinic_path = os.path.normpath(clinic_path) +if not os.path.exists(clinic_path): + raise unittest.SkipTest(f'{clinic_path!r} path does not exist') +sys.path.append(clinic_path) +try: + import clinic + from clinic import DSLParser +finally: + del sys.path[-1] class FakeConverter: @@ -35,7 +45,7 @@ def get(self, name, default): return self.used_converters.setdefault(name, FakeConverterFactory(name)) clinic.Clinic.presets_text = '' -c = clinic.Clinic(language='C') +c = clinic.Clinic(language='C', filename = "file") class FakeClinic: def __init__(self): @@ -43,6 +53,7 @@ def __init__(self): self.legacy_converters = FakeConvertersDict() self.language = clinic.CLanguage(None) self.filename = None + self.destination_buffers = {} self.block_parser = clinic.BlockParser('', self.language) self.modules = collections.OrderedDict() self.classes = collections.OrderedDict() @@ -93,7 +104,7 @@ def test_eol(self): # so it would spit out an end line for you. # and since you really already had one, # the last line of the block got corrupted. - c = clinic.Clinic(clinic.CLanguage(None)) + c = clinic.Clinic(clinic.CLanguage(None), filename="file") raw = "/*[clinic]\nfoo\n[clinic]*/" cooked = c.parse(raw).splitlines() end_line = cooked[2].rstrip() @@ -252,7 +263,7 @@ def test_round_trip_2(self): def _test_clinic(self, input, output): language = clinic.CLanguage(None) - c = clinic.Clinic(language) + c = clinic.Clinic(language, filename="file") c.parsers['inert'] = InertParser(c) c.parsers['copy'] = CopyParser(c) computed = c.parse(input) From webhook-mailer at python.org Tue Sep 4 04:53:59 2018 From: webhook-mailer at python.org (Antoine Pitrou) Date: Tue, 04 Sep 2018 08:53:59 -0000 Subject: [Python-checkins] bpo-33613, test_semaphore_tracker_sigint: fix race condition (#7850) Message-ID: https://github.com/python/cpython/commit/ec74d187f50a8a48f94eb37023300583fbd644cc commit: ec74d187f50a8a48f94eb37023300583fbd644cc branch: master author: Pablo Galindo committer: Antoine Pitrou date: 2018-09-04T10:53:54+02:00 summary: bpo-33613, test_semaphore_tracker_sigint: fix race condition (#7850) Fail `test_semaphore_tracker_sigint` if no warnings are expected and one is received. Fix race condition when the child receives SIGINT before it can register signal handlers for it. The race condition occurs when the parent calls `_semaphore_tracker.ensure_running()` (which in turn spawns the semaphore_tracker using `_posixsubprocess.fork_exec`), the child registers the signal handlers and the parent tries to kill the child. What seem to happen is that in some slow systems, the parent sends the signal to kill the child before the child protects against the signal. files: A Misc/NEWS.d/next/Library/2018-07-31-23-33-06.bpo-33613.Cdnt0i.rst M Lib/multiprocessing/semaphore_tracker.py M Lib/test/_test_multiprocessing.py diff --git a/Lib/multiprocessing/semaphore_tracker.py b/Lib/multiprocessing/semaphore_tracker.py index 3e31bf8402ee..82833bcf861a 100644 --- a/Lib/multiprocessing/semaphore_tracker.py +++ b/Lib/multiprocessing/semaphore_tracker.py @@ -23,6 +23,9 @@ __all__ = ['ensure_running', 'register', 'unregister'] +_HAVE_SIGMASK = hasattr(signal, 'pthread_sigmask') +_IGNORED_SIGNALS = (signal.SIGINT, signal.SIGTERM) + class SemaphoreTracker(object): @@ -43,10 +46,16 @@ def ensure_running(self): with self._lock: if self._pid is not None: # semaphore tracker was launched before, is it still running? - pid, status = os.waitpid(self._pid, os.WNOHANG) - if not pid: - # => still alive - return + try: + pid, _ = os.waitpid(self._pid, os.WNOHANG) + except ChildProcessError: + # The process terminated + pass + else: + if not pid: + # => still alive + return + # => dead, launch it again os.close(self._fd) self._fd = None @@ -68,7 +77,19 @@ def ensure_running(self): exe = spawn.get_executable() args = [exe] + util._args_from_interpreter_flags() args += ['-c', cmd % r] - pid = util.spawnv_passfds(exe, args, fds_to_pass) + # bpo-33613: Register a signal mask that will block the signals. + # This signal mask will be inherited by the child that is going + # to be spawned and will protect the child from a race condition + # that can make the child die before it registers signal handlers + # for SIGINT and SIGTERM. The mask is unregistered after spawning + # the child. + try: + if _HAVE_SIGMASK: + signal.pthread_sigmask(signal.SIG_BLOCK, _IGNORED_SIGNALS) + pid = util.spawnv_passfds(exe, args, fds_to_pass) + finally: + if _HAVE_SIGMASK: + signal.pthread_sigmask(signal.SIG_UNBLOCK, _IGNORED_SIGNALS) except: os.close(w) raise @@ -104,12 +125,13 @@ def _send(self, cmd, name): unregister = _semaphore_tracker.unregister getfd = _semaphore_tracker.getfd - def main(fd): '''Run semaphore tracker.''' # protect the process from ^C and "killall python" etc signal.signal(signal.SIGINT, signal.SIG_IGN) signal.signal(signal.SIGTERM, signal.SIG_IGN) + if _HAVE_SIGMASK: + signal.pthread_sigmask(signal.SIG_UNBLOCK, _IGNORED_SIGNALS) for f in (sys.stdin, sys.stdout): try: diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index 749cf8c4fd2c..a5509ce9135a 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -20,6 +20,7 @@ import struct import operator import weakref +import warnings import test.support import test.support.script_helper from test import support @@ -4517,17 +4518,19 @@ def check_semaphore_tracker_death(self, signum, should_die): # bpo-31310: if the semaphore tracker process has died, it should # be restarted implicitly. from multiprocessing.semaphore_tracker import _semaphore_tracker - _semaphore_tracker.ensure_running() pid = _semaphore_tracker._pid + if pid is not None: + os.kill(pid, signal.SIGKILL) + os.waitpid(pid, 0) + with warnings.catch_warnings(record=True) as all_warn: + _semaphore_tracker.ensure_running() + pid = _semaphore_tracker._pid + os.kill(pid, signum) time.sleep(1.0) # give it time to die ctx = multiprocessing.get_context("spawn") - with contextlib.ExitStack() as stack: - if should_die: - stack.enter_context(self.assertWarnsRegex( - UserWarning, - "semaphore_tracker: process died")) + with warnings.catch_warnings(record=True) as all_warn: sem = ctx.Semaphore() sem.acquire() sem.release() @@ -4537,11 +4540,23 @@ def check_semaphore_tracker_death(self, signum, should_die): del sem gc.collect() self.assertIsNone(wr()) + if should_die: + self.assertEqual(len(all_warn), 1) + the_warn = all_warn[0] + issubclass(the_warn.category, UserWarning) + self.assertTrue("semaphore_tracker: process died" + in str(the_warn.message)) + else: + self.assertEqual(len(all_warn), 0) def test_semaphore_tracker_sigint(self): # Catchable signal (ignored by semaphore tracker) self.check_semaphore_tracker_death(signal.SIGINT, False) + def test_semaphore_tracker_sigterm(self): + # Catchable signal (ignored by semaphore tracker) + self.check_semaphore_tracker_death(signal.SIGTERM, False) + def test_semaphore_tracker_sigkill(self): # Uncatchable signal. self.check_semaphore_tracker_death(signal.SIGKILL, True) diff --git a/Misc/NEWS.d/next/Library/2018-07-31-23-33-06.bpo-33613.Cdnt0i.rst b/Misc/NEWS.d/next/Library/2018-07-31-23-33-06.bpo-33613.Cdnt0i.rst new file mode 100644 index 000000000000..9574e43fcc60 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-31-23-33-06.bpo-33613.Cdnt0i.rst @@ -0,0 +1,3 @@ +Fix a race condition in ``multiprocessing.semaphore_tracker`` when the +tracker receives SIGINT before it can register signal handlers for ignoring +it. From webhook-mailer at python.org Tue Sep 4 05:01:13 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 04 Sep 2018 09:01:13 -0000 Subject: [Python-checkins] bpo-34530: Fix distutils find_executable() (GH-9049) Message-ID: https://github.com/python/cpython/commit/39487196c87e28128ea907a0d9b8a88ba53f68d5 commit: 39487196c87e28128ea907a0d9b8a88ba53f68d5 branch: master author: Victor Stinner committer: GitHub date: 2018-09-04T11:01:09+02:00 summary: bpo-34530: Fix distutils find_executable() (GH-9049) distutils.spawn.find_executable() now falls back on os.defpath if the PATH environment variable is not set. files: A Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst M Lib/distutils/spawn.py M Lib/distutils/tests/test_spawn.py diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py index 5dd415a283d1..538768809327 100644 --- a/Lib/distutils/spawn.py +++ b/Lib/distutils/spawn.py @@ -173,7 +173,7 @@ def find_executable(executable, path=None): os.environ['PATH']. Returns the complete filename or None if not found. """ if path is None: - path = os.environ['PATH'] + path = os.environ.get('PATH', os.defpath) paths = path.split(os.pathsep) base, ext = os.path.splitext(executable) diff --git a/Lib/distutils/tests/test_spawn.py b/Lib/distutils/tests/test_spawn.py index 5edc24a3a104..0d455385d8ac 100644 --- a/Lib/distutils/tests/test_spawn.py +++ b/Lib/distutils/tests/test_spawn.py @@ -1,9 +1,13 @@ """Tests for distutils.spawn.""" -import unittest -import sys import os +import stat +import sys +import unittest +from unittest import mock from test.support import run_unittest, unix_shell +from test import support as test_support +from distutils.spawn import find_executable from distutils.spawn import _nt_quote_args from distutils.spawn import spawn from distutils.errors import DistutilsExecError @@ -51,6 +55,47 @@ def test_spawn(self): os.chmod(exe, 0o777) spawn([exe]) # should work without any error + def test_find_executable(self): + with test_support.temp_dir() as tmp_dir: + # use TESTFN to get a pseudo-unique filename + program_noeext = test_support.TESTFN + # Give the temporary program an ".exe" suffix for all. + # It's needed on Windows and not harmful on other platforms. + program = program_noeext + ".exe" + + filename = os.path.join(tmp_dir, program) + with open(filename, "wb"): + pass + os.chmod(filename, stat.S_IXUSR) + + # test path parameter + rv = find_executable(program, path=tmp_dir) + self.assertEqual(rv, filename) + + if sys.platform == 'win32': + # test without ".exe" extension + rv = find_executable(program_noeext, path=tmp_dir) + self.assertEqual(rv, filename) + + # test find in the current directory + with test_support.change_cwd(tmp_dir): + rv = find_executable(program) + self.assertEqual(rv, program) + + # test non-existent program + dont_exist_program = "dontexist_" + program + rv = find_executable(dont_exist_program , path=tmp_dir) + self.assertIsNone(rv) + + # test os.defpath: missing PATH environment variable + with test_support.EnvironmentVarGuard() as env: + with mock.patch('distutils.spawn.os.defpath', tmp_dir): + env.pop('PATH') + + rv = find_executable(program) + self.assertEqual(rv, filename) + + def test_suite(): return unittest.makeSuite(SpawnTestCase) diff --git a/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst b/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst new file mode 100644 index 000000000000..064de73c0ffe --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst @@ -0,0 +1,2 @@ +``distutils.spawn.find_executable()`` now falls back on :data:`os.defpath` +if the ``PATH`` environment variable is not set. From solipsis at pitrou.net Tue Sep 4 05:08:26 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 04 Sep 2018 09:08:26 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=-2 Message-ID: <20180904090826.1.A6991F1C79116EEC@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [-7, 1, 0] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogxyXL6_', '--timeout', '7200'] From webhook-mailer at python.org Tue Sep 4 05:19:20 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 04 Sep 2018 09:19:20 -0000 Subject: [Python-checkins] bpo-34530: Fix distutils find_executable() (GH-9049) Message-ID: https://github.com/python/cpython/commit/7aa3eadca2a50ce651ce603d6100b05bb7106f1c commit: 7aa3eadca2a50ce651ce603d6100b05bb7106f1c branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-04T05:19:13-04:00 summary: bpo-34530: Fix distutils find_executable() (GH-9049) distutils.spawn.find_executable() now falls back on os.defpath if the PATH environment variable is not set. (cherry picked from commit 39487196c87e28128ea907a0d9b8a88ba53f68d5) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst M Lib/distutils/spawn.py M Lib/distutils/tests/test_spawn.py diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py index 5dd415a283d1..538768809327 100644 --- a/Lib/distutils/spawn.py +++ b/Lib/distutils/spawn.py @@ -173,7 +173,7 @@ def find_executable(executable, path=None): os.environ['PATH']. Returns the complete filename or None if not found. """ if path is None: - path = os.environ['PATH'] + path = os.environ.get('PATH', os.defpath) paths = path.split(os.pathsep) base, ext = os.path.splitext(executable) diff --git a/Lib/distutils/tests/test_spawn.py b/Lib/distutils/tests/test_spawn.py index 5edc24a3a104..0d455385d8ac 100644 --- a/Lib/distutils/tests/test_spawn.py +++ b/Lib/distutils/tests/test_spawn.py @@ -1,9 +1,13 @@ """Tests for distutils.spawn.""" -import unittest -import sys import os +import stat +import sys +import unittest +from unittest import mock from test.support import run_unittest, unix_shell +from test import support as test_support +from distutils.spawn import find_executable from distutils.spawn import _nt_quote_args from distutils.spawn import spawn from distutils.errors import DistutilsExecError @@ -51,6 +55,47 @@ def test_spawn(self): os.chmod(exe, 0o777) spawn([exe]) # should work without any error + def test_find_executable(self): + with test_support.temp_dir() as tmp_dir: + # use TESTFN to get a pseudo-unique filename + program_noeext = test_support.TESTFN + # Give the temporary program an ".exe" suffix for all. + # It's needed on Windows and not harmful on other platforms. + program = program_noeext + ".exe" + + filename = os.path.join(tmp_dir, program) + with open(filename, "wb"): + pass + os.chmod(filename, stat.S_IXUSR) + + # test path parameter + rv = find_executable(program, path=tmp_dir) + self.assertEqual(rv, filename) + + if sys.platform == 'win32': + # test without ".exe" extension + rv = find_executable(program_noeext, path=tmp_dir) + self.assertEqual(rv, filename) + + # test find in the current directory + with test_support.change_cwd(tmp_dir): + rv = find_executable(program) + self.assertEqual(rv, program) + + # test non-existent program + dont_exist_program = "dontexist_" + program + rv = find_executable(dont_exist_program , path=tmp_dir) + self.assertIsNone(rv) + + # test os.defpath: missing PATH environment variable + with test_support.EnvironmentVarGuard() as env: + with mock.patch('distutils.spawn.os.defpath', tmp_dir): + env.pop('PATH') + + rv = find_executable(program) + self.assertEqual(rv, filename) + + def test_suite(): return unittest.makeSuite(SpawnTestCase) diff --git a/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst b/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst new file mode 100644 index 000000000000..064de73c0ffe --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst @@ -0,0 +1,2 @@ +``distutils.spawn.find_executable()`` now falls back on :data:`os.defpath` +if the ``PATH`` environment variable is not set. From webhook-mailer at python.org Tue Sep 4 05:40:35 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 04 Sep 2018 09:40:35 -0000 Subject: [Python-checkins] bpo-34530: Fix distutils find_executable() (GH-9049) (GH-9058) Message-ID: https://github.com/python/cpython/commit/7056ca880bf4ff428dfb2c4eee67919dc43ecd34 commit: 7056ca880bf4ff428dfb2c4eee67919dc43ecd34 branch: 2.7 author: Victor Stinner committer: GitHub date: 2018-09-04T11:40:29+02:00 summary: bpo-34530: Fix distutils find_executable() (GH-9049) (GH-9058) distutils.spawn.find_executable() now falls back on os.defpath if the PATH environment variable is not set. (cherry picked from commit 39487196c87e28128ea907a0d9b8a88ba53f68d5) files: A Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst M Lib/distutils/spawn.py M Lib/distutils/tests/test_spawn.py diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py index 321344a3a583..737b293f8db3 100644 --- a/Lib/distutils/spawn.py +++ b/Lib/distutils/spawn.py @@ -208,7 +208,8 @@ def find_executable(executable, path=None): os.environ['PATH']. Returns the complete filename or None if not found. """ if path is None: - path = os.environ['PATH'] + path = os.environ.get('PATH', os.defpath) + paths = path.split(os.pathsep) base, ext = os.path.splitext(executable) diff --git a/Lib/distutils/tests/test_spawn.py b/Lib/distutils/tests/test_spawn.py index defa54d87f12..061a72f1a51d 100644 --- a/Lib/distutils/tests/test_spawn.py +++ b/Lib/distutils/tests/test_spawn.py @@ -1,8 +1,11 @@ """Tests for distutils.spawn.""" -import unittest import os +import stat +import sys import time -from test.test_support import captured_stdout, run_unittest +import unittest +from test.support import captured_stdout, run_unittest +from test import support as test_support from distutils.spawn import _nt_quote_args from distutils.spawn import spawn, find_executable @@ -53,6 +56,48 @@ def test_spawn(self): os.chmod(exe, 0777) spawn([exe]) # should work without any error + def test_find_executable(self): + with test_support.temp_dir() as tmp_dir: + # use TESTFN to get a pseudo-unique filename + program_noeext = test_support.TESTFN + # Give the temporary program an ".exe" suffix for all. + # It's needed on Windows and not harmful on other platforms. + program = program_noeext + ".exe" + + filename = os.path.join(tmp_dir, program) + with open(filename, "wb"): + pass + os.chmod(filename, stat.S_IXUSR) + + # test path parameter + rv = find_executable(program, path=tmp_dir) + self.assertEqual(rv, filename) + + if sys.platform == 'win32': + # test without ".exe" extension + rv = find_executable(program_noeext, path=tmp_dir) + self.assertEqual(rv, filename) + + # test find in the current directory + with test_support.change_cwd(tmp_dir): + rv = find_executable(program) + self.assertEqual(rv, program) + + # test non-existent program + dont_exist_program = "dontexist_" + program + rv = find_executable(dont_exist_program , path=tmp_dir) + self.assertIsNone(rv) + + # test os.defpath: missing PATH environment variable + with test_support.EnvironmentVarGuard() as env: + from distutils import spawn + with test_support.swap_attr(spawn.os, 'defpath', tmp_dir): + env.pop('PATH') + + rv = find_executable(program) + self.assertEqual(rv, filename) + + def test_suite(): return unittest.makeSuite(SpawnTestCase) diff --git a/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst b/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst new file mode 100644 index 000000000000..064de73c0ffe --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst @@ -0,0 +1,2 @@ +``distutils.spawn.find_executable()`` now falls back on :data:`os.defpath` +if the ``PATH`` environment variable is not set. From webhook-mailer at python.org Tue Sep 4 08:04:36 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 04 Sep 2018 12:04:36 -0000 Subject: [Python-checkins] bpo-26544: Add test for platform._comparable_version(). (GH-8973) Message-ID: https://github.com/python/cpython/commit/7917aadb3edb7616d6164c5eaba24df6ac0a5fc6 commit: 7917aadb3edb7616d6164c5eaba24df6ac0a5fc6 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-04T15:04:25+03:00 summary: bpo-26544: Add test for platform._comparable_version(). (GH-8973) files: M Lib/test/test_platform.py diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py index 9ecd5d904e30..fd6da313206d 100644 --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -275,6 +275,42 @@ def test_libc_ver(self): self.assertEqual(platform.libc_ver(support.TESTFN), ('glibc', '1.23.4')) + @support.cpython_only + def test__comparable_version(self): + from platform import _comparable_version as V + self.assertEqual(V('1.2.3'), V('1.2.3')) + self.assertLess(V('1.2.3'), V('1.2.10')) + self.assertEqual(V('1.2.3.4'), V('1_2-3+4')) + self.assertLess(V('1.2spam'), V('1.2dev')) + self.assertLess(V('1.2dev'), V('1.2alpha')) + self.assertLess(V('1.2dev'), V('1.2a')) + self.assertLess(V('1.2alpha'), V('1.2beta')) + self.assertLess(V('1.2a'), V('1.2b')) + self.assertLess(V('1.2beta'), V('1.2c')) + self.assertLess(V('1.2b'), V('1.2c')) + self.assertLess(V('1.2c'), V('1.2RC')) + self.assertLess(V('1.2c'), V('1.2rc')) + self.assertLess(V('1.2RC'), V('1.2.0')) + self.assertLess(V('1.2rc'), V('1.2.0')) + self.assertLess(V('1.2.0'), V('1.2pl')) + self.assertLess(V('1.2.0'), V('1.2p')) + + self.assertLess(V('1.5.1'), V('1.5.2b2')) + self.assertLess(V('3.10a'), V('161')) + self.assertEqual(V('8.02'), V('8.02')) + self.assertLess(V('3.4j'), V('1996.07.12')) + self.assertLess(V('3.1.1.6'), V('3.2.pl0')) + self.assertLess(V('2g6'), V('11g')) + self.assertLess(V('0.9'), V('2.2')) + self.assertLess(V('1.2'), V('1.2.1')) + self.assertLess(V('1.1'), V('1.2.2')) + self.assertLess(V('1.1'), V('1.2')) + self.assertLess(V('1.2.1'), V('1.2.2')) + self.assertLess(V('1.2'), V('1.2.2')) + self.assertLess(V('0.4'), V('0.4.0')) + self.assertLess(V('1.13++'), V('5.5.kw')) + self.assertLess(V('0.960923'), V('2.2beta29')) + def test_popen(self): mswindows = (sys.platform == "win32") From webhook-mailer at python.org Tue Sep 4 10:31:26 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 04 Sep 2018 14:31:26 -0000 Subject: [Python-checkins] [3.7] bpo-26544: Get rid of dependence from distutils in platform. (GH-8356). (GH-8970) Message-ID: https://github.com/python/cpython/commit/20a8392cec2967f15ae81633c1775645b3ca40da commit: 20a8392cec2967f15ae81633c1775645b3ca40da branch: 3.7 author: Serhiy Storchaka committer: GitHub date: 2018-09-04T17:31:18+03:00 summary: [3.7] bpo-26544: Get rid of dependence from distutils in platform. (GH-8356). (GH-8970) (cherry picked from commit 7d81e8f5995df6980a1a02923e224a481375f130) files: M Lib/platform.py M Lib/test/test_platform.py diff --git a/Lib/platform.py b/Lib/platform.py index e22734850786..4482f479bfde 100755 --- a/Lib/platform.py +++ b/Lib/platform.py @@ -136,6 +136,35 @@ # Constant used by test_platform to test linux_distribution(). _UNIXCONFDIR = '/etc' +# Helper for comparing two version number strings. +# Based on the description of the PHP's version_compare(): +# http://php.net/manual/en/function.version-compare.php + +_ver_stages = { + # any string not found in this dict, will get 0 assigned + 'dev': 10, + 'alpha': 20, 'a': 20, + 'beta': 30, 'b': 30, + 'c': 40, + 'RC': 50, 'rc': 50, + # number, will get 100 assigned + 'pl': 200, 'p': 200, +} + +_component_re = re.compile(r'([0-9]+|[._+-])') + +def _comparable_version(version): + result = [] + for v in _component_re.split(version): + if v not in '._+-': + try: + v = int(v, 10) + t = 100 + except ValueError: + t = _ver_stages.get(v, 0) + result.extend((t, v)) + return result + ### Platform specific APIs _libc_search = re.compile(b'(__libc_init)' @@ -159,7 +188,7 @@ def libc_ver(executable=sys.executable, lib='', version='', chunksize=16384): The file is read and scanned in chunks of chunksize bytes. """ - from distutils.version import LooseVersion as V + V = _comparable_version if hasattr(os.path, 'realpath'): # Python 2.2 introduced os.path.realpath(); it is used # here to work around problems with Cygwin not being diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py index eaf14e0f18ad..509d8865c068 100644 --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -285,6 +285,42 @@ def test_libc_ver(self): self.assertEqual(platform.libc_ver(support.TESTFN), ('glibc', '1.23.4')) + @support.cpython_only + def test__comparable_version(self): + from platform import _comparable_version as V + self.assertEqual(V('1.2.3'), V('1.2.3')) + self.assertLess(V('1.2.3'), V('1.2.10')) + self.assertEqual(V('1.2.3.4'), V('1_2-3+4')) + self.assertLess(V('1.2spam'), V('1.2dev')) + self.assertLess(V('1.2dev'), V('1.2alpha')) + self.assertLess(V('1.2dev'), V('1.2a')) + self.assertLess(V('1.2alpha'), V('1.2beta')) + self.assertLess(V('1.2a'), V('1.2b')) + self.assertLess(V('1.2beta'), V('1.2c')) + self.assertLess(V('1.2b'), V('1.2c')) + self.assertLess(V('1.2c'), V('1.2RC')) + self.assertLess(V('1.2c'), V('1.2rc')) + self.assertLess(V('1.2RC'), V('1.2.0')) + self.assertLess(V('1.2rc'), V('1.2.0')) + self.assertLess(V('1.2.0'), V('1.2pl')) + self.assertLess(V('1.2.0'), V('1.2p')) + + self.assertLess(V('1.5.1'), V('1.5.2b2')) + self.assertLess(V('3.10a'), V('161')) + self.assertEqual(V('8.02'), V('8.02')) + self.assertLess(V('3.4j'), V('1996.07.12')) + self.assertLess(V('3.1.1.6'), V('3.2.pl0')) + self.assertLess(V('2g6'), V('11g')) + self.assertLess(V('0.9'), V('2.2')) + self.assertLess(V('1.2'), V('1.2.1')) + self.assertLess(V('1.1'), V('1.2.2')) + self.assertLess(V('1.1'), V('1.2')) + self.assertLess(V('1.2.1'), V('1.2.2')) + self.assertLess(V('1.2'), V('1.2.2')) + self.assertLess(V('0.4'), V('0.4.0')) + self.assertLess(V('1.13++'), V('5.5.kw')) + self.assertLess(V('0.960923'), V('2.2beta29')) + def test_parse_release_file(self): for input, output in ( From webhook-mailer at python.org Tue Sep 4 12:10:37 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 04 Sep 2018 16:10:37 -0000 Subject: [Python-checkins] bpo-34563: Fix for invalid assert on big output of multiprocessing.Process (GH-9027) Message-ID: https://github.com/python/cpython/commit/266f4904a222a784080e29aad0916849e507515d commit: 266f4904a222a784080e29aad0916849e507515d branch: master author: Alexander Buchkovsky committer: Victor Stinner date: 2018-09-04T18:10:28+02:00 summary: bpo-34563: Fix for invalid assert on big output of multiprocessing.Process (GH-9027) Fix for invalid assert on big output of multiprocessing.Process. files: A Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst M Modules/_winapi.c M Modules/clinic/_winapi.c.h diff --git a/Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst b/Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst new file mode 100644 index 000000000000..9127af0d1924 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst @@ -0,0 +1 @@ +On Windows, fix multiprocessing.Connection for very large read: fix _winapi.PeekNamedPipe() and _winapi.ReadFile() for read larger than INT_MAX (usually 2^31-1). \ No newline at end of file diff --git a/Modules/_winapi.c b/Modules/_winapi.c index 30f269fbdccd..44788e5992a0 100644 --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -1338,7 +1338,7 @@ _winapi_PeekNamedPipe_impl(PyObject *module, HANDLE handle, int size) } if (_PyBytes_Resize(&buf, nread)) return NULL; - return Py_BuildValue("Nii", buf, navail, nleft); + return Py_BuildValue("NII", buf, navail, nleft); } else { Py_BEGIN_ALLOW_THREADS @@ -1347,7 +1347,7 @@ _winapi_PeekNamedPipe_impl(PyObject *module, HANDLE handle, int size) if (!ret) { return PyErr_SetExcFromWindowsErr(PyExc_OSError, 0); } - return Py_BuildValue("ii", navail, nleft); + return Py_BuildValue("II", navail, nleft); } } @@ -1355,14 +1355,14 @@ _winapi_PeekNamedPipe_impl(PyObject *module, HANDLE handle, int size) _winapi.ReadFile handle: HANDLE - size: int + size: DWORD overlapped as use_overlapped: bool(accept={int}) = False [clinic start generated code]*/ static PyObject * -_winapi_ReadFile_impl(PyObject *module, HANDLE handle, int size, +_winapi_ReadFile_impl(PyObject *module, HANDLE handle, DWORD size, int use_overlapped) -/*[clinic end generated code: output=492029ca98161d84 input=3f0fde92f74de59a]*/ +/*[clinic end generated code: output=d3d5b44a8201b944 input=08c439d03a11aac5]*/ { DWORD nread; PyObject *buf; diff --git a/Modules/clinic/_winapi.c.h b/Modules/clinic/_winapi.c.h index b14f087732ef..c66522ebab6d 100644 --- a/Modules/clinic/_winapi.c.h +++ b/Modules/clinic/_winapi.c.h @@ -673,7 +673,7 @@ PyDoc_STRVAR(_winapi_ReadFile__doc__, {"ReadFile", (PyCFunction)_winapi_ReadFile, METH_FASTCALL|METH_KEYWORDS, _winapi_ReadFile__doc__}, static PyObject * -_winapi_ReadFile_impl(PyObject *module, HANDLE handle, int size, +_winapi_ReadFile_impl(PyObject *module, HANDLE handle, DWORD size, int use_overlapped); static PyObject * @@ -681,9 +681,9 @@ _winapi_ReadFile(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyOb { PyObject *return_value = NULL; static const char * const _keywords[] = {"handle", "size", "overlapped", NULL}; - static _PyArg_Parser _parser = {"" F_HANDLE "i|i:ReadFile", _keywords, 0}; + static _PyArg_Parser _parser = {"" F_HANDLE "k|i:ReadFile", _keywords, 0}; HANDLE handle; - int size; + DWORD size; int use_overlapped = 0; if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, @@ -941,4 +941,4 @@ _winapi_GetFileType(PyObject *module, PyObject *const *args, Py_ssize_t nargs, P exit: return return_value; } -/*[clinic end generated code: output=ec7f36eb7efc9d00 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=baaf3d379b91be0a input=a9049054013a1b77]*/ From webhook-mailer at python.org Tue Sep 4 12:35:55 2018 From: webhook-mailer at python.org (Steve Dower) Date: Tue, 04 Sep 2018 16:35:55 -0000 Subject: [Python-checkins] bpo-34565: Change a PC/launcher.c comment to accurately describe valid major versions. (GH-9037) Message-ID: https://github.com/python/cpython/commit/3876af4f7c2ef87db6d2d83efc229955968926dd commit: 3876af4f7c2ef87db6d2d83efc229955968926dd branch: master author: Brendan Gerrity committer: Steve Dower date: 2018-09-04T09:35:46-07:00 summary: bpo-34565: Change a PC/launcher.c comment to accurately describe valid major versions. (GH-9037) files: M PC/launcher.c diff --git a/PC/launcher.c b/PC/launcher.c index fcc3abb63de6..75a06c2310c8 100644 --- a/PC/launcher.c +++ b/PC/launcher.c @@ -1051,7 +1051,7 @@ static BOOL validate_version(wchar_t * p) { /* - Version information should start with one of 2 or 3, + Version information should start with the major version, Optionally followed by a period and a minor version, Optionally followed by a minus and one of 32 or 64. Valid examples: @@ -1068,7 +1068,7 @@ validate_version(wchar_t * p) */ BOOL result = (p != NULL); /* Default to False if null pointer. */ - result = result && iswdigit(*p); /* Result = False if fist string element is not a digit. */ + result = result && iswdigit(*p); /* Result = False if first string element is not a digit. */ while (result && iswdigit(*p)) /* Require a major version */ ++p; /* Skip all leading digit(s) */ From webhook-mailer at python.org Tue Sep 4 15:40:08 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 04 Sep 2018 19:40:08 -0000 Subject: [Python-checkins] bpo-34563: Fix for invalid assert on big output of multiprocessing.Process (GH-9027) (GH-9064) Message-ID: https://github.com/python/cpython/commit/e8ca8806a9f47b3bac4ae1c6b8a54c47d1aad8f3 commit: e8ca8806a9f47b3bac4ae1c6b8a54c47d1aad8f3 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Victor Stinner date: 2018-09-04T21:39:54+02:00 summary: bpo-34563: Fix for invalid assert on big output of multiprocessing.Process (GH-9027) (GH-9064) Fix for invalid assert on big output of multiprocessing.Process. (cherry picked from commit 266f4904a222a784080e29aad0916849e507515d) Co-authored-by: Alexander Buchkovsky files: A Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst M Modules/_winapi.c M Modules/clinic/_winapi.c.h diff --git a/Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst b/Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst new file mode 100644 index 000000000000..9127af0d1924 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst @@ -0,0 +1 @@ +On Windows, fix multiprocessing.Connection for very large read: fix _winapi.PeekNamedPipe() and _winapi.ReadFile() for read larger than INT_MAX (usually 2^31-1). \ No newline at end of file diff --git a/Modules/_winapi.c b/Modules/_winapi.c index c596cba3cbc3..9ea5a9266295 100644 --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -1337,7 +1337,7 @@ _winapi_PeekNamedPipe_impl(PyObject *module, HANDLE handle, int size) } if (_PyBytes_Resize(&buf, nread)) return NULL; - return Py_BuildValue("Nii", buf, navail, nleft); + return Py_BuildValue("NII", buf, navail, nleft); } else { Py_BEGIN_ALLOW_THREADS @@ -1346,7 +1346,7 @@ _winapi_PeekNamedPipe_impl(PyObject *module, HANDLE handle, int size) if (!ret) { return PyErr_SetExcFromWindowsErr(PyExc_OSError, 0); } - return Py_BuildValue("ii", navail, nleft); + return Py_BuildValue("II", navail, nleft); } } @@ -1354,14 +1354,14 @@ _winapi_PeekNamedPipe_impl(PyObject *module, HANDLE handle, int size) _winapi.ReadFile handle: HANDLE - size: int + size: DWORD overlapped as use_overlapped: bool(accept={int}) = False [clinic start generated code]*/ static PyObject * -_winapi_ReadFile_impl(PyObject *module, HANDLE handle, int size, +_winapi_ReadFile_impl(PyObject *module, HANDLE handle, DWORD size, int use_overlapped) -/*[clinic end generated code: output=492029ca98161d84 input=3f0fde92f74de59a]*/ +/*[clinic end generated code: output=d3d5b44a8201b944 input=08c439d03a11aac5]*/ { DWORD nread; PyObject *buf; diff --git a/Modules/clinic/_winapi.c.h b/Modules/clinic/_winapi.c.h index b14f087732ef..c66522ebab6d 100644 --- a/Modules/clinic/_winapi.c.h +++ b/Modules/clinic/_winapi.c.h @@ -673,7 +673,7 @@ PyDoc_STRVAR(_winapi_ReadFile__doc__, {"ReadFile", (PyCFunction)_winapi_ReadFile, METH_FASTCALL|METH_KEYWORDS, _winapi_ReadFile__doc__}, static PyObject * -_winapi_ReadFile_impl(PyObject *module, HANDLE handle, int size, +_winapi_ReadFile_impl(PyObject *module, HANDLE handle, DWORD size, int use_overlapped); static PyObject * @@ -681,9 +681,9 @@ _winapi_ReadFile(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyOb { PyObject *return_value = NULL; static const char * const _keywords[] = {"handle", "size", "overlapped", NULL}; - static _PyArg_Parser _parser = {"" F_HANDLE "i|i:ReadFile", _keywords, 0}; + static _PyArg_Parser _parser = {"" F_HANDLE "k|i:ReadFile", _keywords, 0}; HANDLE handle; - int size; + DWORD size; int use_overlapped = 0; if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, @@ -941,4 +941,4 @@ _winapi_GetFileType(PyObject *module, PyObject *const *args, Py_ssize_t nargs, P exit: return return_value; } -/*[clinic end generated code: output=ec7f36eb7efc9d00 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=baaf3d379b91be0a input=a9049054013a1b77]*/ From webhook-mailer at python.org Wed Sep 5 02:29:47 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 05 Sep 2018 06:29:47 -0000 Subject: [Python-checkins] closes bpo-34581 : Conditionalize use of __pragma in Modules/socketmodule.c. (GH-9067) Message-ID: https://github.com/python/cpython/commit/874809ea389e6434787e773a6054a08e0b81f734 commit: 874809ea389e6434787e773a6054a08e0b81f734 branch: master author: Erik Janssens committer: Benjamin Peterson date: 2018-09-04T23:29:42-07:00 summary: closes bpo-34581 : Conditionalize use of __pragma in Modules/socketmodule.c. (GH-9067) files: A Misc/NEWS.d/next/Windows/2018-09-04-23-13-19.bpo-34581.lnbC0k.rst M Modules/socketmodule.c diff --git a/Misc/NEWS.d/next/Windows/2018-09-04-23-13-19.bpo-34581.lnbC0k.rst b/Misc/NEWS.d/next/Windows/2018-09-04-23-13-19.bpo-34581.lnbC0k.rst new file mode 100644 index 000000000000..2dfa1aec9b86 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-04-23-13-19.bpo-34581.lnbC0k.rst @@ -0,0 +1 @@ +Guard MSVC-specific code in socketmodule.c with ``#ifdef _MSC_VER``. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 3a439c4bfadc..014f3ba8b5ea 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -559,15 +559,18 @@ select_error(void) # define SET_SOCK_ERROR(err) WSASetLastError(err) # define SOCK_TIMEOUT_ERR WSAEWOULDBLOCK # define SOCK_INPROGRESS_ERR WSAEWOULDBLOCK -# define SUPPRESS_DEPRECATED_CALL __pragma(warning(suppress: 4996)) #else # define GET_SOCK_ERROR errno # define SET_SOCK_ERROR(err) do { errno = err; } while (0) # define SOCK_TIMEOUT_ERR EWOULDBLOCK # define SOCK_INPROGRESS_ERR EINPROGRESS -# define SUPPRESS_DEPRECATED_CALL #endif +#ifdef _MSC_VER +# define SUPPRESS_DEPRECATED_CALL __pragma(warning(suppress: 4996)) +#else +# define SUPPRESS_DEPRECATED_CALL +#endif #ifdef MS_WINDOWS /* Does WSASocket() support the WSA_FLAG_NO_HANDLE_INHERIT flag? */ From webhook-mailer at python.org Wed Sep 5 02:45:07 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 05 Sep 2018 06:45:07 -0000 Subject: [Python-checkins] closes bpo-34581 : Conditionalize use of __pragma in Modules/socketmodule.c. (GH-9067) Message-ID: https://github.com/python/cpython/commit/5b17d7fccd8f0b4d5030b03924eb00904585ba31 commit: 5b17d7fccd8f0b4d5030b03924eb00904585ba31 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-04T23:45:03-07:00 summary: closes bpo-34581 : Conditionalize use of __pragma in Modules/socketmodule.c. (GH-9067) (cherry picked from commit 874809ea389e6434787e773a6054a08e0b81f734) Co-authored-by: Erik Janssens files: A Misc/NEWS.d/next/Windows/2018-09-04-23-13-19.bpo-34581.lnbC0k.rst M Modules/socketmodule.c diff --git a/Misc/NEWS.d/next/Windows/2018-09-04-23-13-19.bpo-34581.lnbC0k.rst b/Misc/NEWS.d/next/Windows/2018-09-04-23-13-19.bpo-34581.lnbC0k.rst new file mode 100644 index 000000000000..2dfa1aec9b86 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-04-23-13-19.bpo-34581.lnbC0k.rst @@ -0,0 +1 @@ +Guard MSVC-specific code in socketmodule.c with ``#ifdef _MSC_VER``. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index ea01a286874b..f40bd89e577f 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -559,15 +559,18 @@ select_error(void) # define SET_SOCK_ERROR(err) WSASetLastError(err) # define SOCK_TIMEOUT_ERR WSAEWOULDBLOCK # define SOCK_INPROGRESS_ERR WSAEWOULDBLOCK -# define SUPPRESS_DEPRECATED_CALL __pragma(warning(suppress: 4996)) #else # define GET_SOCK_ERROR errno # define SET_SOCK_ERROR(err) do { errno = err; } while (0) # define SOCK_TIMEOUT_ERR EWOULDBLOCK # define SOCK_INPROGRESS_ERR EINPROGRESS -# define SUPPRESS_DEPRECATED_CALL #endif +#ifdef _MSC_VER +# define SUPPRESS_DEPRECATED_CALL __pragma(warning(suppress: 4996)) +#else +# define SUPPRESS_DEPRECATED_CALL +#endif #ifdef MS_WINDOWS /* Does WSASocket() support the WSA_FLAG_NO_HANDLE_INHERIT flag? */ From solipsis at pitrou.net Wed Sep 5 05:10:07 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 05 Sep 2018 09:10:07 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=-3 Message-ID: <20180905091007.1.2F0357B9DCAD4983@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [-7, 1, 0] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [-2, 2, -1] memory blocks, sum=-1 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/refloglPX746', '--timeout', '7200'] From webhook-mailer at python.org Wed Sep 5 10:42:56 2018 From: webhook-mailer at python.org (Zachary Ware) Date: Wed, 05 Sep 2018 14:42:56 -0000 Subject: [Python-checkins] [3.6] bpo-34575: Build with only VS2015 on AppVeyor (GH-9066) Message-ID: https://github.com/python/cpython/commit/635461fca5e90c6e091f1e5b46adafc0d28bf0e2 commit: 635461fca5e90c6e091f1e5b46adafc0d28bf0e2 branch: 3.6 author: Zachary Ware committer: GitHub date: 2018-09-05T09:42:53-05:00 summary: [3.6] bpo-34575: Build with only VS2015 on AppVeyor (GH-9066) Remove VS2017 image from AppVeyor config. We're currently having caching issues between the two images, and VS2017 builds are tested by VSTS. Removing that one here will significantly decrease the time that AppVeyor builds take on this branch. files: M .github/appveyor.yml diff --git a/.github/appveyor.yml b/.github/appveyor.yml index 09eb7db0f0d4..3e0f588a2892 100644 --- a/.github/appveyor.yml +++ b/.github/appveyor.yml @@ -36,4 +36,3 @@ environment: HOST_PYTHON: C:\Python36\python.exe image: - Visual Studio 2015 - - Visual Studio 2017 From webhook-mailer at python.org Wed Sep 5 10:45:11 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 05 Sep 2018 14:45:11 -0000 Subject: [Python-checkins] bpo-34530: Fix distutils find_executable() (GH-9049) (GH-9057) Message-ID: https://github.com/python/cpython/commit/e2c1657dff86decf1e232b66e766d2e51381109c commit: e2c1657dff86decf1e232b66e766d2e51381109c branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Victor Stinner date: 2018-09-05T16:45:08+02:00 summary: bpo-34530: Fix distutils find_executable() (GH-9049) (GH-9057) distutils.spawn.find_executable() now falls back on os.defpath if the PATH environment variable is not set. (cherry picked from commit 39487196c87e28128ea907a0d9b8a88ba53f68d5) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst M Lib/distutils/spawn.py M Lib/distutils/tests/test_spawn.py diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py index 5dd415a283d1..538768809327 100644 --- a/Lib/distutils/spawn.py +++ b/Lib/distutils/spawn.py @@ -173,7 +173,7 @@ def find_executable(executable, path=None): os.environ['PATH']. Returns the complete filename or None if not found. """ if path is None: - path = os.environ['PATH'] + path = os.environ.get('PATH', os.defpath) paths = path.split(os.pathsep) base, ext = os.path.splitext(executable) diff --git a/Lib/distutils/tests/test_spawn.py b/Lib/distutils/tests/test_spawn.py index 5edc24a3a104..0d455385d8ac 100644 --- a/Lib/distutils/tests/test_spawn.py +++ b/Lib/distutils/tests/test_spawn.py @@ -1,9 +1,13 @@ """Tests for distutils.spawn.""" -import unittest -import sys import os +import stat +import sys +import unittest +from unittest import mock from test.support import run_unittest, unix_shell +from test import support as test_support +from distutils.spawn import find_executable from distutils.spawn import _nt_quote_args from distutils.spawn import spawn from distutils.errors import DistutilsExecError @@ -51,6 +55,47 @@ def test_spawn(self): os.chmod(exe, 0o777) spawn([exe]) # should work without any error + def test_find_executable(self): + with test_support.temp_dir() as tmp_dir: + # use TESTFN to get a pseudo-unique filename + program_noeext = test_support.TESTFN + # Give the temporary program an ".exe" suffix for all. + # It's needed on Windows and not harmful on other platforms. + program = program_noeext + ".exe" + + filename = os.path.join(tmp_dir, program) + with open(filename, "wb"): + pass + os.chmod(filename, stat.S_IXUSR) + + # test path parameter + rv = find_executable(program, path=tmp_dir) + self.assertEqual(rv, filename) + + if sys.platform == 'win32': + # test without ".exe" extension + rv = find_executable(program_noeext, path=tmp_dir) + self.assertEqual(rv, filename) + + # test find in the current directory + with test_support.change_cwd(tmp_dir): + rv = find_executable(program) + self.assertEqual(rv, program) + + # test non-existent program + dont_exist_program = "dontexist_" + program + rv = find_executable(dont_exist_program , path=tmp_dir) + self.assertIsNone(rv) + + # test os.defpath: missing PATH environment variable + with test_support.EnvironmentVarGuard() as env: + with mock.patch('distutils.spawn.os.defpath', tmp_dir): + env.pop('PATH') + + rv = find_executable(program) + self.assertEqual(rv, filename) + + def test_suite(): return unittest.makeSuite(SpawnTestCase) diff --git a/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst b/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst new file mode 100644 index 000000000000..064de73c0ffe --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-03-23-23-32.bpo-34530.h_Xsu7.rst @@ -0,0 +1,2 @@ +``distutils.spawn.find_executable()`` now falls back on :data:`os.defpath` +if the ``PATH`` environment variable is not set. From webhook-mailer at python.org Wed Sep 5 10:46:01 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 05 Sep 2018 14:46:01 -0000 Subject: [Python-checkins] [3.7] bpo-26544: Get rid of dependence from distutils in platform. (GH-8356) (GH-8970) (GH-9061) Message-ID: https://github.com/python/cpython/commit/1a3eb125dc07a28a5af62446778ed7cca95ed3da commit: 1a3eb125dc07a28a5af62446778ed7cca95ed3da branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Victor Stinner date: 2018-09-05T16:45:57+02:00 summary: [3.7] bpo-26544: Get rid of dependence from distutils in platform. (GH-8356) (GH-8970) (GH-9061) (cherry picked from commit 7d81e8f5995df6980a1a02923e224a481375f130) (cherry picked from commit 20a8392cec2967f15ae81633c1775645b3ca40da) Co-authored-by: Serhiy Storchaka files: M Lib/platform.py M Lib/test/test_platform.py diff --git a/Lib/platform.py b/Lib/platform.py index c8e0476bfb21..4205abde0388 100755 --- a/Lib/platform.py +++ b/Lib/platform.py @@ -136,6 +136,35 @@ # Constant used by test_platform to test linux_distribution(). _UNIXCONFDIR = '/etc' +# Helper for comparing two version number strings. +# Based on the description of the PHP's version_compare(): +# http://php.net/manual/en/function.version-compare.php + +_ver_stages = { + # any string not found in this dict, will get 0 assigned + 'dev': 10, + 'alpha': 20, 'a': 20, + 'beta': 30, 'b': 30, + 'c': 40, + 'RC': 50, 'rc': 50, + # number, will get 100 assigned + 'pl': 200, 'p': 200, +} + +_component_re = re.compile(r'([0-9]+|[._+-])') + +def _comparable_version(version): + result = [] + for v in _component_re.split(version): + if v not in '._+-': + try: + v = int(v, 10) + t = 100 + except ValueError: + t = _ver_stages.get(v, 0) + result.extend((t, v)) + return result + ### Platform specific APIs _libc_search = re.compile(b'(__libc_init)' @@ -159,7 +188,7 @@ def libc_ver(executable=sys.executable, lib='', version='', chunksize=16384): The file is read and scanned in chunks of chunksize bytes. """ - from distutils.version import LooseVersion as V + V = _comparable_version if hasattr(os.path, 'realpath'): # Python 2.2 introduced os.path.realpath(); it is used # here to work around problems with Cygwin not being diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py index afc4de326907..c8ba7ec35947 100644 --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -285,6 +285,42 @@ def test_libc_ver(self): self.assertEqual(platform.libc_ver(support.TESTFN), ('glibc', '1.23.4')) + @support.cpython_only + def test__comparable_version(self): + from platform import _comparable_version as V + self.assertEqual(V('1.2.3'), V('1.2.3')) + self.assertLess(V('1.2.3'), V('1.2.10')) + self.assertEqual(V('1.2.3.4'), V('1_2-3+4')) + self.assertLess(V('1.2spam'), V('1.2dev')) + self.assertLess(V('1.2dev'), V('1.2alpha')) + self.assertLess(V('1.2dev'), V('1.2a')) + self.assertLess(V('1.2alpha'), V('1.2beta')) + self.assertLess(V('1.2a'), V('1.2b')) + self.assertLess(V('1.2beta'), V('1.2c')) + self.assertLess(V('1.2b'), V('1.2c')) + self.assertLess(V('1.2c'), V('1.2RC')) + self.assertLess(V('1.2c'), V('1.2rc')) + self.assertLess(V('1.2RC'), V('1.2.0')) + self.assertLess(V('1.2rc'), V('1.2.0')) + self.assertLess(V('1.2.0'), V('1.2pl')) + self.assertLess(V('1.2.0'), V('1.2p')) + + self.assertLess(V('1.5.1'), V('1.5.2b2')) + self.assertLess(V('3.10a'), V('161')) + self.assertEqual(V('8.02'), V('8.02')) + self.assertLess(V('3.4j'), V('1996.07.12')) + self.assertLess(V('3.1.1.6'), V('3.2.pl0')) + self.assertLess(V('2g6'), V('11g')) + self.assertLess(V('0.9'), V('2.2')) + self.assertLess(V('1.2'), V('1.2.1')) + self.assertLess(V('1.1'), V('1.2.2')) + self.assertLess(V('1.1'), V('1.2')) + self.assertLess(V('1.2.1'), V('1.2.2')) + self.assertLess(V('1.2'), V('1.2.2')) + self.assertLess(V('0.4'), V('0.4.0')) + self.assertLess(V('1.13++'), V('5.5.kw')) + self.assertLess(V('0.960923'), V('2.2beta29')) + def test_parse_release_file(self): for input, output in ( From webhook-mailer at python.org Wed Sep 5 10:46:29 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 05 Sep 2018 14:46:29 -0000 Subject: [Python-checkins] bpo-26544: Get rid of dependence from distutils in platform. (GH-8356) (GH-8952) Message-ID: https://github.com/python/cpython/commit/9734024ec65311e33936faa83fb1cb249ef0de9d commit: 9734024ec65311e33936faa83fb1cb249ef0de9d branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Victor Stinner date: 2018-09-05T16:46:26+02:00 summary: bpo-26544: Get rid of dependence from distutils in platform. (GH-8356) (GH-8952) (cherry picked from commit 7d81e8f5995df6980a1a02923e224a481375f130) Co-authored-by: Serhiy Storchaka files: M Lib/platform.py diff --git a/Lib/platform.py b/Lib/platform.py index 44a612b28fce..62a5476a8f4d 100755 --- a/Lib/platform.py +++ b/Lib/platform.py @@ -132,6 +132,35 @@ # Standard Unix uses /dev/null DEV_NULL = '/dev/null' +# Helper for comparing two version number strings. +# Based on the description of the PHP's version_compare(): +# http://php.net/manual/en/function.version-compare.php + +_ver_stages = { + # any string not found in this dict, will get 0 assigned + 'dev': 10, + 'alpha': 20, 'a': 20, + 'beta': 30, 'b': 30, + 'c': 40, + 'RC': 50, 'rc': 50, + # number, will get 100 assigned + 'pl': 200, 'p': 200, +} + +_component_re = re.compile(r'([0-9]+|[._+-])') + +def _comparable_version(version): + result = [] + for v in _component_re.split(version): + if v not in '._+-': + try: + v = int(v, 10) + t = 100 + except ValueError: + t = _ver_stages.get(v, 0) + result.extend((t, v)) + return result + ### Platform specific APIs _libc_search = re.compile(r'(__libc_init)' @@ -155,7 +184,7 @@ def libc_ver(executable=sys.executable,lib='',version='', chunksize=2048): The file is read and scanned in chunks of chunksize bytes. """ - from distutils.version import LooseVersion as V + V = _comparable_version if hasattr(os.path, 'realpath'): # Python 2.2 introduced os.path.realpath(); it is used # here to work around problems with Cygwin not being From solipsis at pitrou.net Thu Sep 6 05:10:00 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 06 Sep 2018 09:10:00 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=7 Message-ID: <20180906091000.1.41C98DCDEDCBC8BC@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [-7, 8, 0] memory blocks, sum=1 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_spawn leaked [0, 2, 0] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogqXbjzO', '--timeout', '7200'] From webhook-mailer at python.org Thu Sep 6 07:34:32 2018 From: webhook-mailer at python.org (Tal Einat) Date: Thu, 06 Sep 2018 11:34:32 -0000 Subject: [Python-checkins] bpo-30977: make uuid.UUID use __slots__ (GH-9078) Message-ID: https://github.com/python/cpython/commit/3e2b29dccc3ca9fbc418bfa312ad655782e250f2 commit: 3e2b29dccc3ca9fbc418bfa312ad655782e250f2 branch: master author: Tal Einat committer: GitHub date: 2018-09-06T14:34:25+03:00 summary: bpo-30977: make uuid.UUID use __slots__ (GH-9078) Co-Authored-By: Wouter Bolsterlee. files: A Misc/NEWS.d/next/Library/2018-09-06-10-07-46.bpo-30977.bP661V.rst M Lib/test/test_uuid.py M Lib/uuid.py diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py index 7af1d7aec797..9ec59d519573 100644 --- a/Lib/test/test_uuid.py +++ b/Lib/test/test_uuid.py @@ -4,8 +4,10 @@ import contextlib import io import os +import pickle import shutil import subprocess +import sys py_uuid = support.import_fresh_module('uuid', blocked=['_uuid']) c_uuid = support.import_fresh_module('uuid', fresh=['_uuid']) @@ -311,6 +313,64 @@ def test_getnode(self): node2 = self.uuid.getnode() self.assertEqual(node1, node2, '%012x != %012x' % (node1, node2)) + def _setup_for_pickle(self): + orig_uuid = sys.modules.get('uuid') + sys.modules['uuid'] = self.uuid + + def restore_uuid_module(): + if orig_uuid is not None: + sys.modules['uuid'] = orig_uuid + else: + del sys.modules['uuid'] + self.addCleanup(restore_uuid_module) + + def test_pickle_roundtrip(self): + self._setup_for_pickle() + + u = self.uuid.UUID('12345678123456781234567812345678') + self.assertEqual(u, pickle.loads(pickle.dumps(u))) + + def test_unpickle_previous_python_versions(self): + self._setup_for_pickle() + + u = self.uuid.UUID('12345678123456781234567812345678') + + # Python 2.7 protocol 0-2 pickles of u + py27_pickles = [ + b'ccopy_reg\n_reconstructor\np0\n(cuuid\nUUID\np1\nc__builtin__\nob' + b'ject\np2\nNtp3\nRp4\n(dp5\nS\'int\'\np6\nL24197857161011715162171' + b'839636988778104L\nsb.', + b'ccopy_reg\n_reconstructor\nq\x00(cuuid\nUUID\nq\x01c__builtin__\n' + b'object\nq\x02Ntq\x03Rq\x04}q\x05U\x03intq\x06L2419785716101171516' + b'2171839636988778104L\nsb.', + b'\x80\x02cuuid\nUUID\nq\x00)\x81q\x01}q\x02U\x03intq\x03\x8a\x10xV' + b'4\x12xV4\x12xV4\x12xV4\x12sb.', + ] + # Python 3.6 protocol 0-4 pickles of u + py36_pickles = [ + b'ccopy_reg\n_reconstructor\np0\n(cuuid\nUUID\np1\nc__builtin__\nob' + b'ject\np2\nNtp3\nRp4\n(dp5\nVint\np6\nL241978571610117151621718396' + b'36988778104L\nsb.', + b'ccopy_reg\n_reconstructor\nq\x00(cuuid\nUUID\nq\x01c__builtin__\n' + b'object\nq\x02Ntq\x03Rq\x04}q\x05X\x03\x00\x00\x00intq\x06L2419785' + b'7161011715162171839636988778104L\nsb.', + b'\x80\x02cuuid\nUUID\nq\x00)\x81q\x01}q\x02X\x03\x00\x00\x00intq' + b'\x03\x8a\x10xV4\x12xV4\x12xV4\x12xV4\x12sb.', + b'\x80\x03cuuid\nUUID\nq\x00)\x81q\x01}q\x02X\x03\x00\x00\x00intq' + b'\x03\x8a\x10xV4\x12xV4\x12xV4\x12xV4\x12sb.', + b'\x80\x04\x950\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x94\x8c\x04' + b'UUID\x94\x93\x94)\x81\x94}\x94\x8c\x03int\x94\x8a\x10xV4\x12xV4' + b'\x12xV4\x12xV4\x12sb.', + ] + + for pickled in py27_pickles + py36_pickles: + unpickled = pickle.loads(pickled) + self.assertEqual(unpickled, u) + # is_safe was added in 3.7. When unpickling values from older + # versions, is_safe will be missing, so it should be set to + # SafeUUID.unknown. + self.assertEqual(unpickled.is_safe, self.uuid.SafeUUID.unknown) + # bpo-32502: UUID1 requires a 48-bit identifier, but hardware identifiers # need not necessarily be 48 bits (e.g., EUI-64). def test_uuid1_eui64(self): diff --git a/Lib/uuid.py b/Lib/uuid.py index 66383218e70c..515388221510 100644 --- a/Lib/uuid.py +++ b/Lib/uuid.py @@ -118,6 +118,8 @@ class UUID: uuid_generate_time_safe(3). """ + __slots__ = ('int', 'is_safe') + def __init__(self, hex=None, bytes=None, bytes_le=None, fields=None, int=None, version=None, *, is_safe=SafeUUID.unknown): @@ -201,8 +203,30 @@ def __init__(self, hex=None, bytes=None, bytes_le=None, fields=None, # Set the version number. int &= ~(0xf000 << 64) int |= version << 76 - self.__dict__['int'] = int - self.__dict__['is_safe'] = is_safe + object.__setattr__(self, 'int', int) + object.__setattr__(self, 'is_safe', is_safe) + + def __getstate__(self): + d = {attr: getattr(self, attr) for attr in self.__slots__} + # is_safe is a SafeUUID instance. Return just its value, so that + # it can be unpickled in older Python versions without SafeUUID. + d['is_safe'] = d['is_safe'].value + return d + + def __setstate__(self, state): + # is_safe was added in 3.7 + state.setdefault('is_safe', SafeUUID.unknown.value) + + for attr in self.__slots__: + value = state[attr] + + # for is_safe, restore the SafeUUID from the stored value + if attr == 'is_safe': + try: + value = SafeUUID(value) + except ValueError: + value = SafeUUID.unknown + object.__setattr__(self, attr, value) def __eq__(self, other): if isinstance(other, UUID): diff --git a/Misc/NEWS.d/next/Library/2018-09-06-10-07-46.bpo-30977.bP661V.rst b/Misc/NEWS.d/next/Library/2018-09-06-10-07-46.bpo-30977.bP661V.rst new file mode 100644 index 000000000000..3d547c06beb5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-06-10-07-46.bpo-30977.bP661V.rst @@ -0,0 +1,2 @@ +Make uuid.UUID use ``__slots__`` to reduce its memory footprint. Based on +original patch by Wouter Bolsterlee. From webhook-mailer at python.org Thu Sep 6 09:13:28 2018 From: webhook-mailer at python.org (Christian Heimes) Date: Thu, 06 Sep 2018 13:13:28 -0000 Subject: [Python-checkins] bpo-34542: Update test certs and keys (GH-8997) (GH-9007) Message-ID: https://github.com/python/cpython/commit/2d3f2dc9f8376415a31a6de260ccbe6a86f2816d commit: 2d3f2dc9f8376415a31a6de260ccbe6a86f2816d branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Christian Heimes date: 2018-09-06T15:13:24+02:00 summary: bpo-34542: Update test certs and keys (GH-8997) (GH-9007) Update all test certs and keys to use future proof crypto settings: * 3072 bit RSA keys * SHA-256 signature Signed-off-by: Christian Heimes (cherry picked from commit e6dac0077996b1e1f886f036d6f2606237fa4c85) files: A Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst M Lib/test/allsans.pem M Lib/test/capath/b1930218.0 M Lib/test/capath/ceff1710.0 M Lib/test/idnsans.pem M Lib/test/keycert.passwd.pem M Lib/test/keycert.pem M Lib/test/keycert2.pem M Lib/test/keycert3.pem M Lib/test/keycert4.pem M Lib/test/keycertecc.pem M Lib/test/make_ssl_certs.py M Lib/test/pycacert.pem M Lib/test/pycakey.pem M Lib/test/revocation.crl M Lib/test/ssl_cert.pem M Lib/test/ssl_key.passwd.pem M Lib/test/ssl_key.pem M Lib/test/test_asyncio/utils.py M Lib/test/test_ssl.py diff --git a/Lib/test/allsans.pem b/Lib/test/allsans.pem index bf59f30abaad..6eebde7a57f1 100644 --- a/Lib/test/allsans.pem +++ b/Lib/test/allsans.pem @@ -1,64 +1,81 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQD59JyhPgYe7nhZ -Z2IGhaklNgtRkD+5BVs7lEWovYRBlXpPA6PuaHat25rI8EGYHmlufPherg2Qu6sC -GmZZKo7TgjlmDcwVS4hkebtFH7OZy5Il7Y2ZIdiK7Xp9Z0EPoqwacYowB0a8WhZY -I2Vm4EzCNKl6/htkwjgn2JXGizxvGt/1kNqP/GBAX+vjgeahOsn8jVh96KpFHJbS -g83cX4t8M7FJv7yNoDLvORHnvKCOXbQmr6ZMGcZN8PwS8awQ31khZTpEx+hCe+Pi -GzeOlxpZimXWDAGWA4tZ58Ka/QvO7VQbD5Ci166ODvvs+tEXfBUExtPcS+02IBJV -tzhBna9VAgMBAAECggEAPar9DccIqY76QEyCYcuOPLEFv9zP6+0HYj6lpQkE3U1s -vJvQURyS0zgQCy1Dca1nI6xPdsSIckHq4fzzbWJTlJlXYfdbd5GIGAn0iwxUOkiA -ST0/px0zmKsYgmH8KkhfH7MNfeX9rLCpPJuXA/eo2G03tzGEPqqwQhxsb2ygv2Qs -M7OqJz6RJu87K1Y+psWIv9+VhNVja0kvsg52QMK9mtp8layb54qLI5R5e09sIudq -RHegtnSOBo9kt32H9vWUFaF5PpYt4yks4KYI4ulKGWJGXHMDW4uHUaE/tjNQuYAX -DuDvjN+ECSJvigiUbu2k0xB2KYIb1fpcxlz/YBdADQKBgQD/Z2VtBUjOFnJKz00f -xN0akp7XPgd1yCb1/wZq9PQiGvzIAMDIplioTvjOjhOzPJaWD0GICNeypzQ48+0P -UsPIKbazpIZN6bZncr65plSpg0KANq46hbkPHOo8PHDa7yoxBUSPr8F7P1OCRkn6 -+QdgcnrAly7yfqO2ahAWOX7iCwKBgQD6ifXSCKfRF1GUb3Ws7S1rLxeBasWq+EmC -sUnck0S+AyaMkN+kZ5zejbN+NDuUMQ7+3wUIheTclUhzR0LP3+r5jjHsimJuvOml -wuV37F+Om5lD/Xx27NfbtRKn/bK6o0zDL8JB2eFB0N7Fh7hRYoUMdrpQs5sU91IC -pNYlAcLwHwKBgGvLK9eTf2LbvmksjRR3dgodD8UwfN2NGESC2iaSM+ehFEclajhF -XO3MRt6GwHHJhJTY44OSl9bjEvtmmAr7l34HfQDc04JWvZFzsGOSe/D/YTXT3jz8 -61ohjgrWR5tfjaMa4hDy0Oo/k/NLzzWJnT9rkbtvE3VtVZNLuHZo1dB5AoGBAMHO -wStV6MO1nzUNN+Gqo8zbY/qIJxsH8I26KaIJBk9azpJEa8yZHl+HDEffjgsoHCqL -STB7qzv7+0y53nRCClo8ZmBN+LEjUDcbWjl3z7/YnCpdR9ATjTP3kdQETCNWucXw -Bvy72CX6tqnlQG8soDGxEpXlKl2AqJ9E9icwgqUPAoGAL6xTDdgcYTbk9wxCd41l -NhHTSvLrGXLAzv61PCnlOJEJbuuezb2VW0ibsud5CA4Mi0tf9ET790XSOFd5nCjQ -6rr06AkjQsoFvjL1dO9EzVFPW0JrZ3C9y8ZOjdeAfPEmFL2T6VqmQ+IcCUNhSr39 -NBdKrboEFfnKanfbstekhAs= +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQCg/pM6dP7BTFNc +qe6wIJIBB7HjwL42bp0vjcCVl4Z3MRWFswYpfxy+o+8+PguMp4K6zndA5fwNkK/H +3HmtanncUfPqnV0usN0NHQGh/f9xRoNmB1q2L7kTuO99o0KLQgvonRT2snf8rq9n +tPRzhHUGYhog7zzNxetYV309PHpPr19BcKepDtM5RMk2aBnoN5vtItorjXiDosFm +6o5wQHrcupcVydszba6P75BEbc1XIWvq2Fv8muaw4pCe81QYINyLqgcPNO/nF3Os +5EI4HKjCNRSCOhOcWqYctXLXN9lBdMBBvQc3zDmYzh1eIZewzZXPVEQT33xPkhxz +HNmhcIctpWX4LTRF6FulkcbeuZDga3gkZYJf/M6IpU1WYXr6q8sNxbgmRRX/NuHo +V9oDwBzLG07rKUiqRHfjGqoCRmmVeVYpryvXUNjHGH0nlVzz/8lTUxAnJorO3Fdc +I+6zKLUPICdAlvz51AH6yopgPFhrdgA0pVzPO6L5G8SRQCxKhAUCAwEAAQKCAYAa +2jtOTcNMFGH3G7TfFZ+kolbuaPCQ/aQkEV2k1dAswzgWw8RsWXI+7fLyi8C7Zhks +9VD4tyNyU8at7D0zSoYm1Fh9sl+fcQp9rG/gSBA6IYu7EdD0gEM7YeY4K2nm9k4s +Lz8W4q+WqsBA6PK47cfjF6vKAH1AyRk28+jEtPiln9egf5zHWtyqOanh9D0V+Wh9 +hgmjqAYI1rWxZ7/4Qxj7Bfg7Px7blhi+kzOZ5kKQnNd2JT46hM+jgzah/G3zVE+R +FFW6ksmJgZ+dCuSbE7HEJmKms1CWq/1Cll0A3uy4JTDZOrK4KcZQ9UjjWJWvlXQm +uNXSSAp1k287DLVUm9c22SDeXpb9PyKmzyvJvVmMqqBx6QzHZ/L7WPzpUWAoLcU+ +ZHT7vggDymkIO+fcRbUzv8s5R7RnLbcBga51/5OCUvAWDoJXNw0qwYZOIbfTnQgs +8xbCmbMzllyYM/dK3GxQAwfn8Hzk+DbS/NObMjHLCWLfYeUvutXJSNly6Ny+ZcEC +gcEAzo5Y1UFOfBX4MZLIZ69LfgaXj9URobMwqlEwKil8pWQMa951ga3moLt91nOe +SAQz3meFTBX/VAb2ZHLeIf3FoNkiIx48PkxsR/hhLHpvl26zEg3yXs3tv0IFBx2R +EEnLNpQaAQFR9S1yDOaG2rsb17ZDKyp9isDpAENHAmEnT/XJn+Dc0SOH1EVDjUeM +JqToAF/fjIx/RF4oUJCAgOPBMlRy5ywLQk8uDi6ft0NCzzCi0eCuk1Ty3KzWFGwx +7cYRAoHBAMeIPCzHG3No4JGUFunslVwo5TuC7maO6qYKbq0OyvwWfL4b7gjrMBR9 +d5WyZlp/Vf40O463dg8x8qPNOFWp49f3hxTvvfnt2/m3+CQuDOLfqBbHufZApP1J +U9MubUNnDFHHeJ9l0tg2nhiLw24GHeMARZhA/BimMQPY0OpZPpLVxAUArM2EB7hI +glQpYCtdXhqwl1pl0u3TZ08y3BXYNg9BycdpGRMWSsAwsApJRgNuI/dfDKu0uMYF +/pUhXVPatQKBwGgLpAun3dT7bA3sli5ESo6s22OEPGFrVbQ1OUHDrBnTj742TJKJ ++oY0a2q+ypgUJdx94NM2sWquJybqBaKxpf8j4OI3tLjc3h5SqwAwnE13YZRSmifP +K1cP9mBjMFM4GLjhWUfwVkxeG/kLlhpP7fJ2yNbRjHN8QOH1AavdLGRGts1mA1UF +xMHUMfbUd3Bv2L13ja/KhcD2fPA4GcLS9tpXV5nCwdkg8V4LdkBmDR04rotx1f44 +6Czokt2usmfHQQKBwFkufxbUd2SB/72Rnxw27hse/DY5My0Lu70y9HzNG9TIiEDA +YwgBdp/x5D04W58fQuQ3nFcRkOcBwB2OYBuJr5ibvfiRnyvSMHvQykwBeSj+Jjbo +VinGgvfiimDdY2C48jyrFzLHZBHXd5oo/dRzT3Bicri2cvbhcQ7zHY1hDiK7AL3r +q1DALmMjpXzQcXdwZ9suCrgQwtIhpw8zAEOTO7ZeBT3nr5lkYUy9djFixrRJyjGK +fjNQtzVrAHrPStNr8QKBwQDCC0zhsCnTv4sAJmW7LL6Ayd5rbWhUZ6px1xY0yHMA +hehj+xbaiC6cfVr5Rg0ncvaa8AExu4kXpVsupTyNwvC4NgzLHtfBw6WUdOnd1awE +kSrDtDReBt2wByAcQwttQsrJ1/Pt6zcNJJI4Z9s8G4NTcQWJwUhU20N55JQKR//l +OQJqhq9NVhte/ctDjVwOHs/OhDNvxsAWxdjnf/O2up0os+M2bFkmHuaVW0vQbqTQ +mw7Vbzk2Ff5oT6E3kbC8Ur4= -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIGMDCCBRigAwIBAgIJAJYf8T95ptq5MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV +MIIHMDCCBZigAwIBAgIJALVVA6v9zJS5MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTgwMTE5 -MTkwOTA3WhcNMjgwMTE3MTkwOTA3WjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO +IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTgwODI5 +MTQyMzE3WhcNMjgwODI2MTQyMzE3WjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO Q2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3VuZGF0 -aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB -CgKCAQEA+fScoT4GHu54WWdiBoWpJTYLUZA/uQVbO5RFqL2EQZV6TwOj7mh2rdua -yPBBmB5pbnz4Xq4NkLurAhpmWSqO04I5Zg3MFUuIZHm7RR+zmcuSJe2NmSHYiu16 -fWdBD6KsGnGKMAdGvFoWWCNlZuBMwjSpev4bZMI4J9iVxos8bxrf9ZDaj/xgQF/r -44HmoTrJ/I1YfeiqRRyW0oPN3F+LfDOxSb+8jaAy7zkR57ygjl20Jq+mTBnGTfD8 -EvGsEN9ZIWU6RMfoQnvj4hs3jpcaWYpl1gwBlgOLWefCmv0Lzu1UGw+Qoteujg77 -7PrRF3wVBMbT3EvtNiASVbc4QZ2vVQIDAQABo4IC8TCCAu0wggEwBgNVHREEggEn -MIIBI4IHYWxsc2Fuc6AeBgMqAwSgFwwVc29tZSBvdGhlciBpZGVudGlmaWVyoDUG -BisGAQUCAqArMCmgEBsOS0VSQkVST1MuUkVBTE2hFTAToAMCAQGhDDAKGwh1c2Vy -bmFtZYEQdXNlckBleGFtcGxlLm9yZ4IPd3d3LmV4YW1wbGUub3JnpGcwZTELMAkG -A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo -b24gU29mdHdhcmUgRm91bmRhdGlvbjEYMBYGA1UEAwwPZGlybmFtZSBleGFtcGxl -hhdodHRwczovL3d3dy5weXRob24ub3JnL4cEfwAAAYcQAAAAAAAAAAAAAAAAAAAA -AYgEKgMEBTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG -AQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFH9ye3+WhBnHqNhtFu059bzY -SWM8MIGPBgNVHSMEgYcwgYSAFH9ye3+WhBnHqNhtFu059bzYSWM8oWGkXzBdMQsw -CQYDVQQGEwJYWTEXMBUGA1UEBwwOQ2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5 -dGhvbiBTb2Z0d2FyZSBGb3VuZGF0aW9uMRAwDgYDVQQDDAdhbGxzYW5zggkAlh/x -P3mm2rkwgYMGCCsGAQUFBwEBBHcwdTA8BggrBgEFBQcwAoYwaHR0cDovL3Rlc3Rj -YS5weXRob250ZXN0Lm5ldC90ZXN0Y2EvcHljYWNlcnQuY2VyMDUGCCsGAQUFBzAB -hilodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3RjYS9vY3NwLzBDBgNV -HR8EPDA6MDigNqA0hjJodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3Rj -YS9yZXZvY2F0aW9uLmNybDANBgkqhkiG9w0BAQsFAAOCAQEAYwYJcerUPvnsP7e2 -HGp/It0OZ8Cvpt8Qf7A+NSPvJqkyKakl8zK/50iq/qQKH09CnfEae4rfXLdlYsvV -2PZYK0LDWnyTcHSJWAVJjlSFIFt3ig9FdHv9GYtSWWod66cZ0sEZOoF2IHZUGby+ -Qa+JQpmv5jEuGIZzjcsh6hSOou8ph7LsCsRdVlQqk8rM97vB7DAgh01vedlbolsq -JxsuPRydNFV/eWq3AgAWgZL3LdYYIAgaVOTnnd3xARw8DlT1q6+Lzc71GBXrRZYh -qgd+xC/K1812gMPImTX02bxpkhCuIdVd7cztWi8sdQmSgDEFdYMXo4NzlFTK8dlC -Y4wa3Q== +aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIB +igKCAYEAoP6TOnT+wUxTXKnusCCSAQex48C+Nm6dL43AlZeGdzEVhbMGKX8cvqPv +Pj4LjKeCus53QOX8DZCvx9x5rWp53FHz6p1dLrDdDR0Bof3/cUaDZgdati+5E7jv +faNCi0IL6J0U9rJ3/K6vZ7T0c4R1BmIaIO88zcXrWFd9PTx6T69fQXCnqQ7TOUTJ +NmgZ6Deb7SLaK414g6LBZuqOcEB63LqXFcnbM22uj++QRG3NVyFr6thb/JrmsOKQ +nvNUGCDci6oHDzTv5xdzrORCOByowjUUgjoTnFqmHLVy1zfZQXTAQb0HN8w5mM4d +XiGXsM2Vz1REE998T5IccxzZoXCHLaVl+C00RehbpZHG3rmQ4Gt4JGWCX/zOiKVN +VmF6+qvLDcW4JkUV/zbh6FfaA8AcyxtO6ylIqkR34xqqAkZplXlWKa8r11DYxxh9 +J5Vc8//JU1MQJyaKztxXXCPusyi1DyAnQJb8+dQB+sqKYDxYa3YANKVczzui+RvE +kUAsSoQFAgMBAAGjggLxMIIC7TCCATAGA1UdEQSCAScwggEjggdhbGxzYW5zoB4G +AyoDBKAXDBVzb21lIG90aGVyIGlkZW50aWZpZXKgNQYGKwYBBQICoCswKaAQGw5L +RVJCRVJPUy5SRUFMTaEVMBOgAwIBAaEMMAobCHVzZXJuYW1lgRB1c2VyQGV4YW1w +bGUub3Jngg93d3cuZXhhbXBsZS5vcmekZzBlMQswCQYDVQQGEwJYWTEXMBUGA1UE +BwwOQ2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3Vu +ZGF0aW9uMRgwFgYDVQQDDA9kaXJuYW1lIGV4YW1wbGWGF2h0dHBzOi8vd3d3LnB5 +dGhvbi5vcmcvhwR/AAABhxAAAAAAAAAAAAAAAAAAAAABiAQqAwQFMA4GA1UdDwEB +/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/ +BAIwADAdBgNVHQ4EFgQUoLHAHNTWrHkSCUYkhn5NH0S40CAwgY8GA1UdIwSBhzCB +hIAUoLHAHNTWrHkSCUYkhn5NH0S40CChYaRfMF0xCzAJBgNVBAYTAlhZMRcwFQYD +VQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZv +dW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnOCCQC1VQOr/cyUuTCBgwYIKwYBBQUH +AQEEdzB1MDwGCCsGAQUFBzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0 +L3Rlc3RjYS9weWNhY2VydC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2Eu +cHl0aG9udGVzdC5uZXQvdGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0 +dHA6Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3Js +MA0GCSqGSIb3DQEBCwUAA4IBgQAeKJKycO2DES98gyR2e/GzPYEw87cCS0cEpiiP +3CEUgzfEbF0X89GDKEey4H3Irvosbvt2hEcf2RNpahLUL/fUv53bDmHNmL8qJg5E +UJVMOHvOpSOjqoqeRuSyG0GnnAuUwcxdrZY6UzLdslhuq9F8UjgHr6KSMx56G9uK +LmTy5njMab0in2xL/YRX/0nogK3BHqpUHrfCdEYZkciRxtAa+OPpWn4dcZi+Fpf7 +ZYSgPLNt+djtFDMIAk5Bo+XDaQdW3dhF0w44enrGAOV0xPE+/jOuenNhKBafjuNb +lkeSr45+QZsi1rd18ny8z3uuaGqIAziFgmllZOH2D8giTn6+5jZcCNZCoGKUkPI9 +l/GMWwxg4HQYYlZcsZzTCem9Rb2XcrasAbmhFapMtR+QAwSed5vKE7ZdtQhj74kB +7Q0E7Lkgpp6BaObb2As8/f0K/UlSVSvrYk+i3JT9wK/qqkRGxsTFEF7N9t0rKu8y +4JdQDtZCI552MsFvYW6m+IOYgxg= -----END CERTIFICATE----- diff --git a/Lib/test/capath/b1930218.0 b/Lib/test/capath/b1930218.0 index 07556ff9071a..730e7fd911a5 100644 --- a/Lib/test/capath/b1930218.0 +++ b/Lib/test/capath/b1930218.0 @@ -1,21 +1,26 @@ -----BEGIN CERTIFICATE----- -MIIDbTCCAlWgAwIBAgIJAILtv0HIgJGbMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIEbTCCAtWgAwIBAgIJAMstgJlaaVJbMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yODAxMTcx -OTA5MDZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg -Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCASIwDQYJKoZI -hvcNAQEBBQADggEPADCCAQoCggEBAMMYaWvJRymYjrFWwi76Dl68I4CzB2Ik0kJb -8Uq/qcghdcjj5iwfhzxufBvtOTKVt0CyYEjDmhYI/m1niDQ7d3dwHHBa0R9fBCFU -uQzjQYUdWO4v7fMO79gjofpz+0wo4OXmTQsCUkmGx75+veZWdotwjgqPBjMgHXtb -qtDFG6ubzFQJPL/kQGbx+9b3Fp3EGdTD8v8HvG9aniUbAkql7EKWOnDSbJkrzr7o -0gHv1bqwz5Q+gtAB1ktxgAMKEkWGeYHYS9LotbcsbJpMihAQ5OT1386EkcrRRuCE -cxdm22lDeICDvhRN8T4a1mz13kXzOa+R1T1URL9BzHNoGvwk25ECAwEAAaNQME4w -HQYDVR0OBBYEFJrPz27rcT3bPPGuiGtWcgPLCKdIMB8GA1UdIwQYMBaAFJrPz27r -cT3bPPGuiGtWcgPLCKdIMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEB -ABAlyNwMVVzLg25573fsDY4MBsFLDNb3dVIhuBdKOIids3jEQvu4fBQ4EPus2hEA -W0KHXkWfbU5CpJoYBjkPRaaWiTLWWbPTjuOVtsSiS3QvZ8H7u/lybzdK5/RIM3Hf -uPXmQT/V1S8mCfgOkv9w6varWPuQBNZDLo+x+waradDcqPhbB/LUZh9j+F3BnkFE -u8nofeBG5KfIMl8xYuUcXIndt6JPng0TuF+xhFNMH84Z4QEAXr9BVZSppRPb8lnz -1k65nZ25CtmyGG18sfeWqr32+ZUPSm48fEZb39R47Jrc4uMB5oh3OZOcuipj+SVL -T6wIeTnGe98HNbrAwlC/WoE= +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA4MjYx +NDIzMTZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg +Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCAaIwDQYJKoZI +hvcNAQEBBQADggGPADCCAYoCggGBAJftVUG6NheV23Ec0+FhrFhz48aWzysfuAj1 +nUtLxzD2uAuzUnKgu8lNO47fIo4BV4HJknPMAMbscLA6F0DB3/KMNkzEp4HntiRo +4qB+NQcvoFv5RUb3HvBGEf7KGjxQ8SapX5winPhB4d9PEpUZL1yQARdufj59z+kJ +ryX4+EJ3LW1fNvJ4Hn1Kh2hjbAZxG436Jf7U0/WlF7Hv6hfLVMgnmYDLPEXxLFIc +3R9RRSBQHl6rV3MbQXiW3oSket2PMIU2WHl2oNJhyBuplJljxu74FL+0UlYxl/rr +rFOelc5MxFpKt8oDJ1s1V84C3OzKafiKWjnLFiAVAyRhbPR6/LZI5VkQXEnQI5/7 +cV466WifNHKAJ7Y/TLHZ22N/Z2hKbhH46MD0WhY5Uwto3nf6Ref4kc14zSiUl3FU ++8/wN97JJsXcG56JbQmsyERxy23xlzHVTCAzv3VKoODcaRHtKrRkEBEwiw6wpxDY +isWqG8gmiiXnZp+lahovfF+DxnhPHwIDAQABo1AwTjAdBgNVHQ4EFgQU3b/K2ubR +NLo3dSHKb5oIKPI1tkgwHwYDVR0jBBgwFoAU3b/K2ubRNLo3dSHKb5oIKPI1tkgw +DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAYEAM2pU02vA1wFfnfQFwZNm +kFDQtxjpsB5KoLbadpOvhNutFVQxFRPk3n5ODNUJHDQ1tuVM1m9lfTJf6/ypawf3 +SYLlgX4HgJpj+CzDQLyP1CraPtHuCLdNp4TK9D+hmEW+sQVp59/Xmasb7oswzPf8 +59QLF66Xv+R7/Q+ntIV541niFoe/HylFLCOTdr7Ahx3e7CtCauW7yPQKSggKjFzY +fU3RuL/V9yntktGUBOg1Bld/LCOXh6U1jSbTGkfyFtfZxtQfI0PTJpk5yiD0cSNv +DEp2dvd2H7P+v0ew/CpWgeHS3e4I2PT/WtwlYYqRArmGHPJQc3YlNfy2JSYVy+vE +K2EMHOfuLxeb7PDUoYTn0q/e5BskFKcBh+OrKVhGoNnACuCN11nTG/hUID54paXI +T4sDxJaf7PtHz3YtjWU0J7/6rgEFivOSCt2JbJehx+dgUees60t9LLhlyf5dakhV +juTH+WpA4bhkRem1WSmlX899WH1keeWkCawedmU9lMRo -----END CERTIFICATE----- diff --git a/Lib/test/capath/ceff1710.0 b/Lib/test/capath/ceff1710.0 index 07556ff9071a..730e7fd911a5 100644 --- a/Lib/test/capath/ceff1710.0 +++ b/Lib/test/capath/ceff1710.0 @@ -1,21 +1,26 @@ -----BEGIN CERTIFICATE----- -MIIDbTCCAlWgAwIBAgIJAILtv0HIgJGbMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIEbTCCAtWgAwIBAgIJAMstgJlaaVJbMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yODAxMTcx -OTA5MDZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg -Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCASIwDQYJKoZI -hvcNAQEBBQADggEPADCCAQoCggEBAMMYaWvJRymYjrFWwi76Dl68I4CzB2Ik0kJb -8Uq/qcghdcjj5iwfhzxufBvtOTKVt0CyYEjDmhYI/m1niDQ7d3dwHHBa0R9fBCFU -uQzjQYUdWO4v7fMO79gjofpz+0wo4OXmTQsCUkmGx75+veZWdotwjgqPBjMgHXtb -qtDFG6ubzFQJPL/kQGbx+9b3Fp3EGdTD8v8HvG9aniUbAkql7EKWOnDSbJkrzr7o -0gHv1bqwz5Q+gtAB1ktxgAMKEkWGeYHYS9LotbcsbJpMihAQ5OT1386EkcrRRuCE -cxdm22lDeICDvhRN8T4a1mz13kXzOa+R1T1URL9BzHNoGvwk25ECAwEAAaNQME4w -HQYDVR0OBBYEFJrPz27rcT3bPPGuiGtWcgPLCKdIMB8GA1UdIwQYMBaAFJrPz27r -cT3bPPGuiGtWcgPLCKdIMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEB -ABAlyNwMVVzLg25573fsDY4MBsFLDNb3dVIhuBdKOIids3jEQvu4fBQ4EPus2hEA -W0KHXkWfbU5CpJoYBjkPRaaWiTLWWbPTjuOVtsSiS3QvZ8H7u/lybzdK5/RIM3Hf -uPXmQT/V1S8mCfgOkv9w6varWPuQBNZDLo+x+waradDcqPhbB/LUZh9j+F3BnkFE -u8nofeBG5KfIMl8xYuUcXIndt6JPng0TuF+xhFNMH84Z4QEAXr9BVZSppRPb8lnz -1k65nZ25CtmyGG18sfeWqr32+ZUPSm48fEZb39R47Jrc4uMB5oh3OZOcuipj+SVL -T6wIeTnGe98HNbrAwlC/WoE= +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA4MjYx +NDIzMTZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg +Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCAaIwDQYJKoZI +hvcNAQEBBQADggGPADCCAYoCggGBAJftVUG6NheV23Ec0+FhrFhz48aWzysfuAj1 +nUtLxzD2uAuzUnKgu8lNO47fIo4BV4HJknPMAMbscLA6F0DB3/KMNkzEp4HntiRo +4qB+NQcvoFv5RUb3HvBGEf7KGjxQ8SapX5winPhB4d9PEpUZL1yQARdufj59z+kJ +ryX4+EJ3LW1fNvJ4Hn1Kh2hjbAZxG436Jf7U0/WlF7Hv6hfLVMgnmYDLPEXxLFIc +3R9RRSBQHl6rV3MbQXiW3oSket2PMIU2WHl2oNJhyBuplJljxu74FL+0UlYxl/rr +rFOelc5MxFpKt8oDJ1s1V84C3OzKafiKWjnLFiAVAyRhbPR6/LZI5VkQXEnQI5/7 +cV466WifNHKAJ7Y/TLHZ22N/Z2hKbhH46MD0WhY5Uwto3nf6Ref4kc14zSiUl3FU ++8/wN97JJsXcG56JbQmsyERxy23xlzHVTCAzv3VKoODcaRHtKrRkEBEwiw6wpxDY +isWqG8gmiiXnZp+lahovfF+DxnhPHwIDAQABo1AwTjAdBgNVHQ4EFgQU3b/K2ubR +NLo3dSHKb5oIKPI1tkgwHwYDVR0jBBgwFoAU3b/K2ubRNLo3dSHKb5oIKPI1tkgw +DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAYEAM2pU02vA1wFfnfQFwZNm +kFDQtxjpsB5KoLbadpOvhNutFVQxFRPk3n5ODNUJHDQ1tuVM1m9lfTJf6/ypawf3 +SYLlgX4HgJpj+CzDQLyP1CraPtHuCLdNp4TK9D+hmEW+sQVp59/Xmasb7oswzPf8 +59QLF66Xv+R7/Q+ntIV541niFoe/HylFLCOTdr7Ahx3e7CtCauW7yPQKSggKjFzY +fU3RuL/V9yntktGUBOg1Bld/LCOXh6U1jSbTGkfyFtfZxtQfI0PTJpk5yiD0cSNv +DEp2dvd2H7P+v0ew/CpWgeHS3e4I2PT/WtwlYYqRArmGHPJQc3YlNfy2JSYVy+vE +K2EMHOfuLxeb7PDUoYTn0q/e5BskFKcBh+OrKVhGoNnACuCN11nTG/hUID54paXI +T4sDxJaf7PtHz3YtjWU0J7/6rgEFivOSCt2JbJehx+dgUees60t9LLhlyf5dakhV +juTH+WpA4bhkRem1WSmlX899WH1keeWkCawedmU9lMRo -----END CERTIFICATE----- diff --git a/Lib/test/idnsans.pem b/Lib/test/idnsans.pem index b4a771ce7ec1..e8a41fe5e565 100644 --- a/Lib/test/idnsans.pem +++ b/Lib/test/idnsans.pem @@ -1,64 +1,84 @@ -----BEGIN PRIVATE KEY----- -MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDeU8YQtyeEjPLA -SdMBPTS9QcuAZIJjbJRgr8nsRb767pbmWR9C1JuDy/Bz/AprFC6Om950fLn3pOqR -zDUWZX/qTe+o27i8u0Qzk06bhRkxAdTEoTfRcH/FkJaimJqeTt9rZqc+AGSNKM8o -4GyPW4IELnavmMB30+7rKIJMIpIn1a1k6MybJYdWNSuVqwArAVvRlj5qOiqX7KAS -otFRP8pz+Lgw3qREQzgnZz/bcScKd+5Uy4qMFPNOMjgW6nDV60ekNx0GT+59E/+8 -64GRq34rNVu2SN0XXcQh33R3LwwrvAdymaLyr1YyIRM5gLPxugxCIA0SYjG0YoGB -uUSwtNa7AgMBAAECggEBAJjxUGPXW1wYCja1km5byJgZVwEwI3J6E2igBWyAXm0J -DM3RqWu0DneQKA3h6NjYvV5lY5cG5nex/5vkuvB5SpHIo4GqBV/wA27ne0AJQ9cu -x0utDFUL6xnh6X5ZNKSK5a9gotRIOOPSmxAnswa7kKmHvSX3ExBbvxQOffQaJCk5 -0GHl6I/HltqVzMu4ICAo0NY0gw1n+hVKTo28KkJ9PL7X6v6H5yvZ3L6TkMytSvqf -9iVlYuIN66ToBtxaI4g2RiUJtA2hdT9IP7Wg4YD6Ptyih90zXz2wTzWppFem6UA9 -dePig94R9moj9ucuK0tx3kSATNo0op/XEx1e3OOtcQECgYEA/w7pNOPYgj7VMyYx -p4Lx4BOllzQts8mIBtUVZVQSJ2miun6DTalZVT2V3ayTuE0qhUHd1SHu9F77a9fQ -qaSUUY9elwXyfvcNCfhYVRJxyxirI4Z6ZCBwjpWOGSBB59NTeDhVnbkTlfE6guqS -3KRS1pfIQ6FCvGIrhjRZgHo1TGECgYEA3yXsospbOS7VeBj0UPSB87fp1QM+r48o -RflIsRzdsN9Ka2j6EiYpgKdbgXr80vkctYTK0dT8jrFSk81Y932CZezH2IWo8Meo -40qaFWMboNFBIC4yv6RSRxJMQfYsKnXC2trSnXH+qf55Trey4uZNMX7VJ+RFKExS -ieSWSbTWmJsCgYEAzo3yyoRiiEf+PKgHulLPMtp2VddJ07m30WCrLR5CfWyM/l8K -UtB8qg1v2s+x6aWEc9p9necXLwvkrNdgAqJoAw0KW1/TnILSKmrWjj6brRBTODfl -0kR7It128F4xQV7g0BE/NLX3aIytB+yT9t+Uvni5FBv6gbk26j5m5ScTFsECgYEA -hzrQYQcIqWq8av+Ub8r9Rdlal4BT6Mh0u5MKfmrj3mAzFUyU35LI6/J//cOum5vj -zg0fbHIKa98CEBgNpk4lS+dmZMz7SI92xedb4UIiaB7nvLzCfGj0g6WPGRo6QbED -2OVrZYbDsflJQm8ItYCjny8htf8b+gPmsTIZ8ajps6kCgYBnES8waDDAkL98lK28 -dcgnJXN+1UzeI6//If2uvDZEQ9tG/yMk2JYc84qZJLU5bRplMAjIQUVUcFWa+ZzV -ylnDhagAtiWkHPcElWHym9dH8CRuYM3OTDsApZ7yMB/ArCcZMIA35OvNf6uc4lNV -VD9VkaygPIg6ilv4npeTceqp8A== +MIIG/wIBADANBgkqhkiG9w0BAQEFAASCBukwggblAgEAAoIBgQDAfkvfgnT/+ATq +IiR5ksgGW4EJK900MkBsY9TXQcNdbrGqbyG4EwEvbDWzsAWBqgg76Alto7JgxKl2 +gNskz3T007xycOVAy4z31gzEWMTx+GxNbCiicJfCZjw6Z4Ck/UipooXAAy/S7jwC +z369E4RGa5jsIxxJ+NdDqAheoROIeUKawkfF7xXTyzkwuTuNj2YcV/TvJcsGVGQT +j4tZVSbDVvL38PlQX62VoGmEDK8pl8GL2GUQK3y+gP2VFDHepGxqqXxPLITxpdBF +aIoQcY4YV+nJtNu8vlUzDMizCHKII+YP9JdstxOLCyEbmjLv3rk4tHgIJuOfOyRh +3KjIcdghmNZcZ7vOvpbr341gKBNRLVmsAxtJCqqMfWvNOc6rXfp20bCnL/SDDJeB +/71/KfVeVaffXRtfEWcwJW6YRp7p2MealzAsnUA5xscOKItYhvmJ1FJlUB8LDZ+d +WESld2a4CIdK9lpuf3LDqy+wmcGTdnoVoi2RT68SDYHAb/bbtx0CAwEAAQKCAYEA +qNuhjzYPkKNvHDxLJuKd2QXYEcWKzEjwvqRsh0/HI2UBMXM/bwws0/7mtPPKQM55 +XqPT8XvR9JVP2wZ6NrZN8J71BqcGVGq7BIWeuXbtfR3hU6F3m9eNKRkp6En3TcRq +Q3SwQ5oID86wkf6XPSGrvWKRyUObg0BpBY6XmPtP+T32MXAQKg/rY9k4GAfFuA59 +EowN7uOGT80j5Qzcvsa1tFHH3bxj2B+2VDVdmlrOJXxcamKvdRnQ2gSpbmdCv/4X +HKmRiP7m2qVMxRaNh3kcL/gACGvZw3rUE7K0PwjW+sQ6lRlWF9IqUSDZaswxWrie +EaM2et9qD80g3eUV3gMi6I8BIdmD20p/yqa/l0bQk75Y6KQRUlN5HTodFL1u4/7V +W9N9wVIlx5DB1yJBkCZBDoebr4Y1UOGs4rrkcOd6F+zCeRSwKPFAng5vHpZ8jclS +1X2pV/KnKWIUfcFH9+v6pD9Y+VC5tz5LgaFz4SYFxgMF69t+PCpdoR2+0zwSWKGB +AoHBAPr/bC0L7V/HXeoiNxXjOSAYDgyVvzZC3i8cbW2/7DqdY7DfDmZbabdEX1mu +kaY9S/5abVV5W1eQACh1Cd/MaRkUbm+5nTy/bF9jXfh8408MzrGbAR9KF9GfWgqC +UxRiAkCa2gLuAhfc6uqokf5yZK0F+kZGGYWEeGcFK46i+8JstqknW0LsN1hhd+wE +PFokf+/22NiV0t1T0GERz18fFu0XoQC60D9qaW56EI+6OTADw71XkVh35jbNBa96 +bLPVzQKBwQDEVGMtpARdwoU59fRfDBo9S8iJJ/jTFpJs5nrqGZKah4J2pkTBiDpW +ZNxesVKL6IuvWghIuZGnBw5ovWT9JKQ2NHD4GDm/XJ6veVWOxc7THiBhCvJaG5N7 +3Jbxyn+eJ5yBO9PeR6slZbDrgDSIFUI++XD9A7abod8eTZHh6Q3pYT5mRuvfuDS1 +nlrkvMgGC1ZCQXCVTdq8MmMS9jea4UXhJLn4paGSKQSYXWLWVxTX3Mgk0640XpXN +HvsdCBMgFpECgcEAtESQkAXZ9Yk9tTPftlOcqqU8KeO/EhyScOmM2l8WLb7uY5DA +SdlrotlzVfjf2LJ7ivGtRs9DQC9HPcdZUSgBb6bR7mI0QPYGwreePSKNKzA6nyqB +ctSHKOG3DIcvXhtKHNZar/H7BOKAwgpR2JqWHgKGDsS3/cH3q531+5qpEKl9wx1m +GTmIQmy2cYdVRTSjwlfaxCtO5/ETXzQoaNvuXlgzJVAG0xT9JgB7No78FbUOd0WW +OSDb8g3sAGz/92UdAoHBAJqaOV9/gVC0FWWHIWCXjzIls00OhQr1A2luNeyTJrc5 +bqswU4RzWj0eP8rxqCc0gIsdQtjCCMiW7enc+uG9MPLeaPd1XEQtE2jrC0zax/vI +BYdlr2L1xDYNh2/wHn7UgpHiIv0vQr64TZGl67gdxYUBuNAFVI9J8QFT5EUZ2TKm +uRShVK1fyWIbyUfGwVzBILQcBCQOoukAZLRR34e/Td/1o90B0THfyxaQs3xGJsu1 +uIYHi28B7MUSWHyMF0N1gQKBwB9jtS7k7ZRgb0WzsgutRzJLfirmMeahzJwAQ46o +2xRT1y7vr0Idh9OSTz8Iqva6Frse21HT6agymamqEeC2aYwq+ikA0V7K3AhG3PiN +S5diC+RIZybeTthBTO1yj5pKDniZl+NGe3vIunXMrjZIuk/03VFK2etqdYcXguwH +SkwvuKqC3vvzcJW8dV2qIJ3Bz9LWz0QKqJQZBlCZvfzHf1jN5rpJJPQezys4k723 +ZPGg6Smyks3ElGXevVdidVcPUA== -----END PRIVATE KEY----- Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9f - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5f + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:07 2018 GMT - Not After : Nov 28 19:09:07 2027 GMT + Not Before: Aug 29 14:23:17 2018 GMT + Not After : Jul 7 14:23:17 2028 GMT Subject: C=XY, L=Castle Anthrax, O=Python Software Foundation, CN=idnsans Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:de:53:c6:10:b7:27:84:8c:f2:c0:49:d3:01:3d: - 34:bd:41:cb:80:64:82:63:6c:94:60:af:c9:ec:45: - be:fa:ee:96:e6:59:1f:42:d4:9b:83:cb:f0:73:fc: - 0a:6b:14:2e:8e:9b:de:74:7c:b9:f7:a4:ea:91:cc: - 35:16:65:7f:ea:4d:ef:a8:db:b8:bc:bb:44:33:93: - 4e:9b:85:19:31:01:d4:c4:a1:37:d1:70:7f:c5:90: - 96:a2:98:9a:9e:4e:df:6b:66:a7:3e:00:64:8d:28: - cf:28:e0:6c:8f:5b:82:04:2e:76:af:98:c0:77:d3: - ee:eb:28:82:4c:22:92:27:d5:ad:64:e8:cc:9b:25: - 87:56:35:2b:95:ab:00:2b:01:5b:d1:96:3e:6a:3a: - 2a:97:ec:a0:12:a2:d1:51:3f:ca:73:f8:b8:30:de: - a4:44:43:38:27:67:3f:db:71:27:0a:77:ee:54:cb: - 8a:8c:14:f3:4e:32:38:16:ea:70:d5:eb:47:a4:37: - 1d:06:4f:ee:7d:13:ff:bc:eb:81:91:ab:7e:2b:35: - 5b:b6:48:dd:17:5d:c4:21:df:74:77:2f:0c:2b:bc: - 07:72:99:a2:f2:af:56:32:21:13:39:80:b3:f1:ba: - 0c:42:20:0d:12:62:31:b4:62:81:81:b9:44:b0:b4: - d6:bb + 00:c0:7e:4b:df:82:74:ff:f8:04:ea:22:24:79:92: + c8:06:5b:81:09:2b:dd:34:32:40:6c:63:d4:d7:41: + c3:5d:6e:b1:aa:6f:21:b8:13:01:2f:6c:35:b3:b0: + 05:81:aa:08:3b:e8:09:6d:a3:b2:60:c4:a9:76:80: + db:24:cf:74:f4:d3:bc:72:70:e5:40:cb:8c:f7:d6: + 0c:c4:58:c4:f1:f8:6c:4d:6c:28:a2:70:97:c2:66: + 3c:3a:67:80:a4:fd:48:a9:a2:85:c0:03:2f:d2:ee: + 3c:02:cf:7e:bd:13:84:46:6b:98:ec:23:1c:49:f8: + d7:43:a8:08:5e:a1:13:88:79:42:9a:c2:47:c5:ef: + 15:d3:cb:39:30:b9:3b:8d:8f:66:1c:57:f4:ef:25: + cb:06:54:64:13:8f:8b:59:55:26:c3:56:f2:f7:f0: + f9:50:5f:ad:95:a0:69:84:0c:af:29:97:c1:8b:d8: + 65:10:2b:7c:be:80:fd:95:14:31:de:a4:6c:6a:a9: + 7c:4f:2c:84:f1:a5:d0:45:68:8a:10:71:8e:18:57: + e9:c9:b4:db:bc:be:55:33:0c:c8:b3:08:72:88:23: + e6:0f:f4:97:6c:b7:13:8b:0b:21:1b:9a:32:ef:de: + b9:38:b4:78:08:26:e3:9f:3b:24:61:dc:a8:c8:71: + d8:21:98:d6:5c:67:bb:ce:be:96:eb:df:8d:60:28: + 13:51:2d:59:ac:03:1b:49:0a:aa:8c:7d:6b:cd:39: + ce:ab:5d:fa:76:d1:b0:a7:2f:f4:83:0c:97:81:ff: + bd:7f:29:f5:5e:55:a7:df:5d:1b:5f:11:67:30:25: + 6e:98:46:9e:e9:d8:c7:9a:97:30:2c:9d:40:39:c6: + c7:0e:28:8b:58:86:f9:89:d4:52:65:50:1f:0b:0d: + 9f:9d:58:44:a5:77:66:b8:08:87:4a:f6:5a:6e:7f: + 72:c3:ab:2f:b0:99:c1:93:76:7a:15:a2:2d:91:4f: + af:12:0d:81:c0:6f:f6:db:b7:1d Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Alternative Name: @@ -70,11 +90,11 @@ Certificate: X509v3 Basic Constraints: critical CA:FALSE X509v3 Subject Key Identifier: - 3B:F0:22:A0:1E:9B:CE:2A:7C:AE:B1:32:1B:B0:8E:3E:33:40:E3:FA + 54:53:0C:3C:4C:E3:63:C6:56:08:35:9C:5E:F4:C1:A1:04:3A:C2:C9 X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 DirName:/C=XY/O=Python Software Foundation CA/CN=our-ca-server - serial:82:ED:BF:41:C8:80:91:9B + serial:CB:2D:80:99:5A:69:52:5B Authority Information Access: CA Issuers - URI:http://testca.pythontest.net/testca/pycacert.cer @@ -85,52 +105,65 @@ Certificate: Full Name: URI:http://testca.pythontest.net/testca/revocation.crl - Signature Algorithm: sha1WithRSAEncryption - 8b:1f:d7:e4:0d:15:76:b4:f5:87:33:de:b9:84:9b:f2:c1:9b: - c9:97:50:f7:18:33:ed:b7:60:83:be:bb:94:1c:49:39:ae:54: - 24:43:f7:85:d8:2a:8c:26:17:56:1e:a6:b7:63:c5:05:f1:6e: - f4:79:eb:fd:af:12:84:3c:28:4a:8f:b1:01:97:91:ba:18:2b: - ba:54:25:49:1b:5b:2e:1e:6b:33:2d:f5:07:2e:76:04:e0:a8: - 95:25:3f:cc:c8:26:c0:30:b6:90:d2:2b:e1:e2:13:b0:a8:76: - f0:06:90:b9:d5:28:6b:8a:e9:72:1a:ed:4f:7e:3c:37:2e:00: - aa:9b:f1:29:44:94:f2:dc:c8:31:5f:4c:2d:00:d3:5e:78:6c: - 68:fc:0e:1e:46:be:d8:2e:29:88:78:8e:7e:f5:50:c8:5c:5d: - 5f:4c:09:d5:51:07:40:be:9b:30:ed:a3:29:68:25:6b:88:69: - c7:43:35:54:2f:6e:9a:30:f1:d6:87:54:84:20:ef:a5:aa:33: - df:00:6a:87:a9:b4:d7:89:1f:e7:60:0d:01:60:66:11:61:3f: - d0:9f:86:37:cc:b3:b8:48:7e:1f:d2:7a:0f:02:e7:11:1d:dd: - 34:c4:0b:45:47:2b:05:37:dd:ee:6e:0e:1c:bd:de:24:42:50: - a4:07:af:e5 + Signature Algorithm: sha256WithRSAEncryption + 03:af:93:ab:58:98:74:3c:a1:68:53:18:13:be:56:60:77:d0: + a7:0e:c9:8a:02:1f:92:4d:21:18:61:d7:3e:9a:1d:aa:94:37: + 02:59:31:3b:71:62:d9:cb:04:e8:c7:44:41:f9:0e:0d:90:d1: + 4a:ab:4e:e2:bd:4e:60:1b:3c:eb:2a:b7:8a:ca:24:58:60:64: + d3:a6:20:f6:f5:98:6e:ca:5a:0a:91:63:97:58:f5:18:90:9f: + 17:55:9f:12:cf:22:a7:31:e6:90:da:36:61:3b:d6:42:e6:18: + 19:43:bb:17:52:28:40:a5:11:47:7f:32:fe:83:3d:c3:7d:8b: + 17:1b:5d:f7:20:3e:bd:3b:16:3d:00:01:68:ed:76:7a:a1:af: + ce:cd:dd:52:7f:19:22:db:83:c1:89:b6:90:02:4e:22:bc:b1: + 76:0c:00:ad:c5:3c:33:be:64:92:30:38:1f:b1:04:4f:53:a2: + 4a:fb:63:80:21:8b:03:72:72:4b:df:f1:9d:08:75:f0:94:06: + d3:cb:7c:df:3b:80:19:58:7a:85:ca:2a:bf:b4:8c:5d:f3:b6: + 65:24:37:b2:1d:46:da:1f:39:48:f2:eb:3a:84:98:3a:0c:59: + ff:0a:05:c2:2c:8c:b1:a5:a9:43:a9:8e:47:97:9a:d2:40:9c: + b8:c3:e6:46:1b:db:4b:85:a3:e1:e7:4e:2a:1e:b9:a6:de:ee: + a7:f0:63:3f:0f:e2:90:b6:82:70:4b:93:7f:e9:74:f3:ab:03: + 9e:04:38:f1:46:2d:f6:fe:77:0f:4a:8e:66:23:74:3e:c6:5b: + eb:0e:dd:72:c3:46:1d:a4:f0:2f:b6:18:2e:f1:1c:f9:85:69: + 0e:5a:37:5e:f8:ea:4b:2d:e7:63:ee:a1:e4:b9:f7:fa:a9:11: + 65:64:a1:b7:b5:91:81:5c:4d:b5:27:b4:16:96:4b:df:fc:c4: + 8a:26:b6:87:62:54:88:fb:1d:7d:af:97:25:65:27:38:1e:f7: + 8c:a4:16:46:f2:d9 -----BEGIN CERTIFICATE----- -MIIFvTCCBKWgAwIBAgIJAILtv0HIgJGfMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIGvTCCBSWgAwIBAgIJAMstgJlaaVJfMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDdaFw0yNzExMjgx -OTA5MDdaMF0xCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTdaFw0yODA3MDcx +NDIzMTdaMF0xCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj MCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2lk -bnNhbnMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDeU8YQtyeEjPLA -SdMBPTS9QcuAZIJjbJRgr8nsRb767pbmWR9C1JuDy/Bz/AprFC6Om950fLn3pOqR -zDUWZX/qTe+o27i8u0Qzk06bhRkxAdTEoTfRcH/FkJaimJqeTt9rZqc+AGSNKM8o -4GyPW4IELnavmMB30+7rKIJMIpIn1a1k6MybJYdWNSuVqwArAVvRlj5qOiqX7KAS -otFRP8pz+Lgw3qREQzgnZz/bcScKd+5Uy4qMFPNOMjgW6nDV60ekNx0GT+59E/+8 -64GRq34rNVu2SN0XXcQh33R3LwwrvAdymaLyr1YyIRM5gLPxugxCIA0SYjG0YoGB -uUSwtNa7AgMBAAGjggKOMIICijCB4QYDVR0RBIHZMIHWggdpZG5zYW5zgh94bi0t -a25pZy01cWEuaWRuLnB5dGhvbnRlc3QubmV0gi54bi0ta25pZ3Nnc3NjaGVuLWxj -YjB3LmlkbmEyMDAzLnB5dGhvbnRlc3QubmV0gi54bi0ta25pZ3NnY2hlbi1iNGEz -ZHVuLmlkbmEyMDA4LnB5dGhvbnRlc3QubmV0giR4bi0tbnhhc21xNmIuaWRuYTIw -MDMucHl0aG9udGVzdC5uZXSCJHhuLS1ueGFzbW0xYy5pZG5hMjAwOC5weXRob250 -ZXN0Lm5ldDAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG -AQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFDvwIqAem84qfK6xMhuwjj4z -QOP6MH0GA1UdIwR2MHSAFJrPz27rcT3bPPGuiGtWcgPLCKdIoVGkTzBNMQswCQYD -VQQGEwJYWTEmMCQGA1UECgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0Ex -FjAUBgNVBAMMDW91ci1jYS1zZXJ2ZXKCCQCC7b9ByICRmzCBgwYIKwYBBQUHAQEE -dzB1MDwGCCsGAQUFBzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rl -c3RjYS9weWNhY2VydC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2EucHl0 -aG9udGVzdC5uZXQvdGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6 -Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3JsMA0G -CSqGSIb3DQEBBQUAA4IBAQCLH9fkDRV2tPWHM965hJvywZvJl1D3GDPtt2CDvruU -HEk5rlQkQ/eF2CqMJhdWHqa3Y8UF8W70eev9rxKEPChKj7EBl5G6GCu6VCVJG1su -HmszLfUHLnYE4KiVJT/MyCbAMLaQ0ivh4hOwqHbwBpC51ShriulyGu1Pfjw3LgCq -m/EpRJTy3MgxX0wtANNeeGxo/A4eRr7YLimIeI5+9VDIXF1fTAnVUQdAvpsw7aMp -aCVriGnHQzVUL26aMPHWh1SEIO+lqjPfAGqHqbTXiR/nYA0BYGYRYT/Qn4Y3zLO4 -SH4f0noPAucRHd00xAtFRysFN93ubg4cvd4kQlCkB6/l +bnNhbnMwggGiMA0GCSqGSIb3DQEBAQUAA4IBjwAwggGKAoIBgQDAfkvfgnT/+ATq +IiR5ksgGW4EJK900MkBsY9TXQcNdbrGqbyG4EwEvbDWzsAWBqgg76Alto7JgxKl2 +gNskz3T007xycOVAy4z31gzEWMTx+GxNbCiicJfCZjw6Z4Ck/UipooXAAy/S7jwC +z369E4RGa5jsIxxJ+NdDqAheoROIeUKawkfF7xXTyzkwuTuNj2YcV/TvJcsGVGQT +j4tZVSbDVvL38PlQX62VoGmEDK8pl8GL2GUQK3y+gP2VFDHepGxqqXxPLITxpdBF +aIoQcY4YV+nJtNu8vlUzDMizCHKII+YP9JdstxOLCyEbmjLv3rk4tHgIJuOfOyRh +3KjIcdghmNZcZ7vOvpbr341gKBNRLVmsAxtJCqqMfWvNOc6rXfp20bCnL/SDDJeB +/71/KfVeVaffXRtfEWcwJW6YRp7p2MealzAsnUA5xscOKItYhvmJ1FJlUB8LDZ+d +WESld2a4CIdK9lpuf3LDqy+wmcGTdnoVoi2RT68SDYHAb/bbtx0CAwEAAaOCAo4w +ggKKMIHhBgNVHREEgdkwgdaCB2lkbnNhbnOCH3huLS1rbmlnLTVxYS5pZG4ucHl0 +aG9udGVzdC5uZXSCLnhuLS1rbmlnc2dzc2NoZW4tbGNiMHcuaWRuYTIwMDMucHl0 +aG9udGVzdC5uZXSCLnhuLS1rbmlnc2djaGVuLWI0YTNkdW4uaWRuYTIwMDgucHl0 +aG9udGVzdC5uZXSCJHhuLS1ueGFzbXE2Yi5pZG5hMjAwMy5weXRob250ZXN0Lm5l +dIIkeG4tLW54YXNtbTFjLmlkbmEyMDA4LnB5dGhvbnRlc3QubmV0MA4GA1UdDwEB +/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/ +BAIwADAdBgNVHQ4EFgQUVFMMPEzjY8ZWCDWcXvTBoQQ6wskwfQYDVR0jBHYwdIAU +3b/K2ubRNLo3dSHKb5oIKPI1tkihUaRPME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQK +DB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNh +LXNlcnZlcoIJAMstgJlaaVJbMIGDBggrBgEFBQcBAQR3MHUwPAYIKwYBBQUHMAKG +MGh0dHA6Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3B5Y2FjZXJ0LmNl +cjA1BggrBgEFBQcwAYYpaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0 +Y2Evb2NzcC8wQwYDVR0fBDwwOjA4oDagNIYyaHR0cDovL3Rlc3RjYS5weXRob250 +ZXN0Lm5ldC90ZXN0Y2EvcmV2b2NhdGlvbi5jcmwwDQYJKoZIhvcNAQELBQADggGB +AAOvk6tYmHQ8oWhTGBO+VmB30KcOyYoCH5JNIRhh1z6aHaqUNwJZMTtxYtnLBOjH +REH5Dg2Q0UqrTuK9TmAbPOsqt4rKJFhgZNOmIPb1mG7KWgqRY5dY9RiQnxdVnxLP +Iqcx5pDaNmE71kLmGBlDuxdSKEClEUd/Mv6DPcN9ixcbXfcgPr07Fj0AAWjtdnqh +r87N3VJ/GSLbg8GJtpACTiK8sXYMAK3FPDO+ZJIwOB+xBE9Tokr7Y4AhiwNyckvf +8Z0IdfCUBtPLfN87gBlYeoXKKr+0jF3ztmUkN7IdRtofOUjy6zqEmDoMWf8KBcIs +jLGlqUOpjkeXmtJAnLjD5kYb20uFo+HnTioeuabe7qfwYz8P4pC2gnBLk3/pdPOr +A54EOPFGLfb+dw9KjmYjdD7GW+sO3XLDRh2k8C+2GC7xHPmFaQ5aN1746kst52Pu +oeS59/qpEWVkobe1kYFcTbUntBaWS9/8xIomtodiVIj7HX2vlyVlJzge94ykFkby +2Q== -----END CERTIFICATE----- diff --git a/Lib/test/keycert.passwd.pem b/Lib/test/keycert.passwd.pem index 0ad696055190..cbb3c3bccd2c 100644 --- a/Lib/test/keycert.passwd.pem +++ b/Lib/test/keycert.passwd.pem @@ -1,50 +1,68 @@ -----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED -DEK-Info: DES-EDE3-CBC,E74528136B90D2DD +DEK-Info: DES-EDE3-CBC,D134E931C96D9DEC -WRHVD2PJXPqjFSHg92HURIsUzvsTE4a9oi0SC5yMBFKNWA5Z933gK3XTifp6jul5 -zpNYi8jBXZ2EqJJBxCuVcefmXSxL0q7CMej25TdIC4BVAFJVveeprHPUFkNB0IM1 -go5Lg4YofYqTCg3OE3k7WvfR3Zg1cRYxksDKO+WNZgWyKBex5X4vjOiyUqDl3GKt -kQXnkg1VgPV2Vrx93S9XNdELNRTguwf+XG0fkhtYhp/zCto8uKTgy5elK2P/ulGp -7fe6uj7h/uN9L7EOC6CjRkitywfeBUER739mOcGT4imSFJ9G27TCqPzj2ea3uuaf -/v1xhkQ4M6lNY/gcRfgVpCXhW43aAQV8XXQRMJTqLmz5Y5hYTKn+Ugq5vJ/ngyRM -lu1gUJnYYaemBTb4hbm6aBvnYK9mORa891Pmf+vxU9rYuQIdVAhvvXh4KBreSEBI -1AFy6dFKXl8ZKs6Wrq5wPefmFFkRmZ8OBiiq0fp2ApCRGZw6LsjCgwrRM38JiY7d -3OdsJpKvRYufgUyuuzUE0xA+E4yMvD48M9pPq2fC8O5giuGL1uEekQWXJuq+6ZRI -XYKIeSkuQALbX3RAzCPXTUEMtCYXKm/gxrrwJ+Bet4ob2amf3MX0uvWwOuAq++Fk -J0HFSBxrwvIWOhyQXOPuJdAN8PXA7dWOXfOgOMF0hQYqZCl3T4TiVZJbwVQtg1sN -dO7oAD5ZPHiKzvveZuB6k1FlBG8j0TyAC+44ChxkPDD3jF4dd6zGe62sDf85p4/d -W80gxJeD3xnDxG0ePPns+GuKUpUaWS7WvHtDpeFW1JEhvOqf8p1Li9a7RzWVo8ML -mGTdQgBIYIf6/fk69pFKl0nKtBU75KaunZz4nAmd9bNED4naDurMBg44u5TvODbJ -vgYIYXIYjNvONbskJatVrrTS8zch2NwVIjCi8L/hecwBXbIXzo1pECpc6BU7sQT8 -+i9sDKBeJcRipzfKZNHvnO19mUZaPCY8+a/f9c21DgKXz+bgLcJbohpSaeGM8Gfc -aZd3Vp9n3OJ3g2zQR1++HO9v1vR/wLELu6MeydkvMduHLmOPCn54gZ9z51ZNPAwa -qfFIsH+mLh9ks0H74ssF59uIlstkgB9zmZHv/Q0dK9ZfG/VEH6rSgdETWhZxhoMQ -Z92jXBEFT0zhI3rrIPNY+XS7eJCQIc1wc84Ea3cRk7SP+S1og3JtAxX56ykUwtkM -LQ/Dwwa6h1aqD0l2d5x1/BSdavtTuSegISRWQ4iOmSvEdlFP7H4g6RZk/okbLzMD -Evq5gNc7vlXhVawoQU8JCanJ5ZbbWnIRZfiXxBQS4lpYPKvJt4ML9z/x+82XxcXv -Z93N2Wep7wWW5OwS2LcQcOgZRDSIPompwo/0pMFGOS+5oort0ZDRHdmmGLjvBcCb -1KQmKQ4+8brI/3rjRzts6uDLjTGNxSCieNsnqhwHUv9Mg9WDSWupcGa+x27L89x3 -rObf6+3umcFLSjIzU8wuv1hx/e/y98Kv7BDBNYpAr6kVMrLnzYjAfJbBmqlxkzkQ -IgQzgrk2QZoTdgwR+S374NAMO0AE5IlO+/qa6qp2SORGTDX64I3UNw== +nuGFEej7vIjkYWSMz5OJeVTNntDRQi6ZM4DBm3g8T7i/0odr3WFqGMMKZcIhLYQf +rgRq7RSKtrJ1y5taVucMV+EuCjyfzDo0TsYt+ZrXv/D08eZhjRmkhoHnGVF0TqQm +nQEXM/ERT4J2RM78dnG+homMkI76qOqxgGbRqQqJo6AiVRcAZ45y8s96bru2TAB8 ++pWjO/v0Je7AFVdwSU52N8OOY6uoSAygW+0UY1WVxbVGJF2XfRsNpPX+YQHYl6e+ +3xM5XBVCgr6kmdAyub5qUJ38X3TpdVGoR0i+CVS9GTr2pSRib1zURAeeHnlqiUZM +4m0Gn9s72nJevU1wxED8pwOhR8fnHEmMKGD2HPhKoOCbzDhwwBZO27TNa1uWeM3f +M5oixKDi2PqMn3y2cDx1NjJtP661688EcJ5a2Ih9BgO9xpnhSyzBWEKcAn0tJB0H +/56M0FW6cdOOIzMveGGL7sHW5E+iOdI1n5e7C6KJUzew78Y9qJnhS53EdI6qTz9R +wsIsj1i070Fk6RbPo6zpLlF6w7Zj8GlZaZA7OZZv9wo5VEV/0ST8gmiiBOBc4C6Y +u9hyLIIu4dFEBKyQHRvBnQSLNpKx6or1OGFDVBay2In9Yh2BHh1+vOj/OIz/wq48 +EHOIV27fRJxLu4jeK5LIGDhuPnMJ8AJYQ0bQOUP6fd7p+TxWkAQZPB/Dx/cs3hxr +nFEdzx+eO+IAsObx/b1EGZyEJyETBslu4GwYX7/KK3HsJhDJ1bdZ//28jOCaoir6 +ZOMT72GRwmVoQTJ0XpccfjHfKJDRLT7C1xvzo4Eibth0hpTZkA75IUYUp6qK/PuJ +kH/qdiC7QIkRKtsrawW4vEDna3YtxIYhQqz9+KwO6u/0gzooZtv1RU4U3ifMDB5u +5P5GAzACRqlY8QYBkM869lvWqzQPHvybC4ak9Yx6/heMO9ddjdIW9BaK8BLxvN/6 +UCD936Y4fWltt09jHZIoxWFykouBwmd7bXooNYXmDRNmjTdVhKJuOEOQw8hDzx7e +pWFJ9Z/V4Qm1tvXbCD7QFqMCDoY3qFvVG8DBqXpmxe1yPfz21FWrT7IuqDXAD3ns +vxfN/2a+Cy04U9FBNVCvWqWIs5AgNpdCMJC2FlXKTy+H3/7rIjNyFyvbX0vxIXtK +liOVNXiyVM++KZXqktqMUDlsJENmIHV9B046luqbgW018fHkyEYlL3iRZGbYegwr +XO9VVIKVPw1BEvJ8VNdGFGuZGepd8qX2ezfYADrNR+4t85HDm8inbjTobSjWuljs +ftUNkOeCHqAvWCFQTLCfdykvV08EJfVY79y7yFPtfRV2gxYokXFifjo3su9sVQr1 +UiIS5ZAsIC1hBXWeXoBN7QVTkFi7Yto6E1q2k10LiT3obpUUUQ/oclhrJOCJVjrS +oRcj2QBy8OT4T9slJr5maTWdgd7Lt6+I6cGQXPaDvjGOJl0eBYM14vhx4rRQWytJ +k07hhHFO4+9CGCuHS8AAy2gR6acYFWt2ZiiNZ0z/iPIHNK4YEyy9aLf6uZH/KQjE +jmHToo7XD6QvCAEC5qTHby3o3LfHIhyZi/4L+AhS4FKUHF6M0peeyYt4z3HaK2d2 +N6mHLPdjwNjra7GOmcns4gzcrdfoF+R293KpPal4PjknvR3dZL4kKP/ougTAM5zv +qDIvRbkHzjP8ChTpoLcJsNVXykNcNkjcSi0GHtIpYjh6QX6P2uvR/S4+Bbb9p9rn +hIy/ovu9tWN2hiPxGPe6torF6BulAxsTYlDercC204AyzsrdA0pr6HBgJH9C6ML1 +TchwodbFJqn9rSv91i1liusAGoOvE81AGBdrXY7LxfSNhYY1IK6yR/POJPTd53sA +uX2/j6Rtoksd/2BHPM6AUnI/2B9slhuzWX2aCtWLeuwvXDS6rYuTigaQmLkzTRfM +dlMI3s9KLXxgi5YVumUZleJWXwBNP7KiKajd+VTSD+7WAhyhM5FIG5wVOaxmy4G2 +TyqZ/Ax9d2VEjTQHWvQlLPQ4Mp0EIz0aEl94K/S8CK8bJRH6+PRkar+dJi1xqlL+ +BYb42At9mEJ8odLlFikvNi1+t7jqXk5jRi5C0xFKx3nTtzoH2zNUeuA3R6vSocVK +45jnze9IkKmxMlJ4loR5sgszdpDCD3kXqjtCcbMTmcrGyzJek3HSOTpiEORoTFOe +Rhg6jH5lm+QcC263oipojS0qEQcnsWJP2CylNYMYHR9O/9NQxT3o2lsRHqZTMELV +uQa/SFH+paQNbZOj8MRwPSqqiIxJFuLswKte1R+W7LKn1yBSM7Pp39lNbzGvJD2E +YRfnCwFpJ54voVAuQ4jXJvigCW2qeCjXlxeD6K2j4eGJEEOmIjIW1wjubyBY6OI3 -----END RSA PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIDWTCCAkGgAwIBAgIJAPm6B21bar2bMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV +MIIEWTCCAsGgAwIBAgIJAJinz4jHSjLtMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODAx -MTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH +IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODA4 +MjkxNDIzMTVaFw0yODA4MjYxNDIzMTVaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH DA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5k -YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEP -ADCCAQoCggEBAKvvsX2gEti4shve3iYMc+jE4Se7WHs1Bol2f21H8qNboDOFdeb1 -RKHjmq3exHpajywOUEgne9nKHJY/3f2phR4Y5klqG6liLgiSpVyRlcBGbeT2qEAj -9oLiLFUXLGfGDds2mTwivQDLJBWi51j7ff5k2Pr58fN5ugYMn24T9FNyn0moT+qj -SFoBNm58l9jrdkJSlgWfqPlbiMa+mqDn/SFtrwLF2Trbfzu42Sd9UdIzMaSSrzbN -sGm53pNhCh8KndWUQ8GPP2IsLPoUU4qAtmZuTxCx2S1cXrN9EkmT69tlOH84YfSn -96Ih9bWRc7M5y5bfVdEVM+fKQl3hBRf05qMCAwEAAaMYMBYwFAYDVR0RBA0wC4IJ -bG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQAtQ8f37cCEk7/rAcbYR53ce3iK -Vpihb0U2ni1QjG9Tg9UIExkIGkwTiCm7kwQL+GEStBu9AG/QVrOjeTriRiddhWkk -ze8kRaI3AC/63t6Vh9Q1x6PESgeE4OtAO9JpJCf4GILglA789Y/b/GF8zJZQxR13 -qpB4ZwWw7gCBhdEW59u6CFeBmfDa58hM8lWvuVoRrTi7bjUeC6PAn5HVMzZSykhu -4HaUfBp6bKFjuym2+h/VvM1n8C3chjVSmutsLb6ELdD8IK0vPV/yf5+LN256zSsS -dyUZYd8XwQaioEMKdbhLvnehyzHiWfQIUR3BdhONxoIJhHv/EAo8eCkHHYIF +YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGP +ADCCAYoCggGBALKUqUtopT6E68kN+uJNEt34i2EbmG/bwjcD8IaMsgJPSsMO2Bpd +3S6qWgkCeOyCfmAwBxK2kNbxGb63ouysEv7l8GCTJTWv3hG/HQcejJpnAEGi6K1U +fDbyE/db6yZ12SoHVTGkadN4vYGCPd1Wj9ZO1F877SHQ8rDWX3xgTWkxN2ojBw44 +T8RHSDiG8D/CvG4uEy+VUszL+Uvny5y2poNSqvI3J56sptWSrh8nIIbkPZPBdUne +LYMOHTFK3ZjXSmhlXgziTxK71nnzM3Y9K9gxPnRqoXbvu/wFo55hQCkETiRkYgmm +jXcBMZ0TClQVnQWuLjMthRnWFZs4Lfmwqjs7FZD/61581R2BYehvpWbLvvuOJhwv +DFzexL2sXcAl7SsxbzeQKRHqGbIDfbnQTXfs3/VC6Ye5P82P2ucj+XC32N9piRmO +gCBP8L3ub+YzzdxikZN2gZXXE2jsb3QyE/R2LkWdWyshpKe+RsZP1SBRbHShUyOh +yJ90baoiEwj2mwIDAQABoxgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZI +hvcNAQELBQADggGBAHRUO/UIHl3jXQENewYayHxkIx8t7nu40iO2DXbicSijz5bo +5//xAB6RxhBAlsDBehgQP1uoZg+WJW+nHu3CIVOU3qZNZRaozxiCl2UFKcNqLOmx +R3NKpo1jYf4REQIeG8Yw9+hSWLRbshNteP6bKUUf+vanhg9+axyOEOH/iOQvgk/m +b8wA8wNa4ujWljPbTQnj7ry8RqhTM0GcAN5LSdSvcKcpzLcs3aYwh+Z8e30sQWna +F40sa5u7izgBTOrwpcDm/w5kC46vpRQ5fnbshVw6pne2by0mdMECASid/p25N103 +jMqTFlmO7kpf/jpCSmamp3/JSEE1BJKHwQ6Ql4nzRA2N1mnvWH7Zxcv043gkHeAu +0x8evpvwuhdIyproejNFlBpKmW8OX7yKTCPPMC/VkX8Q1rVkxU0DQ6hmvwZlhoKa +9Wc2uXpw9xF8itV4Uvcdr3dwqByvIqn7iI/gB+4l41e0u8OmH2MKOx4Nxlly5TNW +HcVKQHyOeyvnINuBAQ== -----END CERTIFICATE----- diff --git a/Lib/test/keycert.pem b/Lib/test/keycert.pem index 9545dcf4b94f..0d398633739a 100644 --- a/Lib/test/keycert.pem +++ b/Lib/test/keycert.pem @@ -1,48 +1,66 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCr77F9oBLYuLIb -3t4mDHPoxOEnu1h7NQaJdn9tR/KjW6AzhXXm9USh45qt3sR6Wo8sDlBIJ3vZyhyW -P939qYUeGOZJahupYi4IkqVckZXARm3k9qhAI/aC4ixVFyxnxg3bNpk8Ir0AyyQV -oudY+33+ZNj6+fHzeboGDJ9uE/RTcp9JqE/qo0haATZufJfY63ZCUpYFn6j5W4jG -vpqg5/0hba8Cxdk62387uNknfVHSMzGkkq82zbBpud6TYQofCp3VlEPBjz9iLCz6 -FFOKgLZmbk8QsdktXF6zfRJJk+vbZTh/OGH0p/eiIfW1kXOzOcuW31XRFTPnykJd -4QUX9OajAgMBAAECggEAHppmXDbuw9Z0FVPg9KLIysioTtsgz6VLiZIm8juZK4x2 -glUh/D7xvWL2uDXrgN+3lh7iGUW13LkFx5SMncbbo9TIwI57Z/XKvcnkVwquve+L -RfLFVc1Q5lD9lROv2rS86KTaN4LzYz3FKXi6dvMkpPAsUtfEQhMLkmISypQQq/1z -EJaqo7r85OjN7e0wKazlKZpOzJEa5FQLMVRjTRFhLFNbHXX/tAet2jw+umATKbw8 -hYgiuZ44TwSEd9JeIV/oSYWfI/3HetuYW0ru3caiztRF2NySNu8lcsWgNC7fIku9 -mcHjtSNzs91QN1Qlu7GQvvhpt6OWDirNDCW+49WGaQKBgQDg9SDhfF0jRYslgYbH -cqO4ggaFdHjrAAYpwnAgvanhFZL/zEqm5G1E7l/e2fCkJ9VOSFO0A208chvwMcr+ -dCjHE2tVdE81aQ2v/Eo83VdS1RcOV4Y75yPH48rMhxPaHvxWD/FFDbf0/P2mtPB7 -SZ3kIeZMkE1wxdaO3AKUbQoozwKBgQDDqYgg7kVtygyICE1mB8Hwp6nUxFTczG7y -4XcsDqMIrKmw+PbQluvkoHoStxeVrsTloDhkTjIrpmYLyAiazg+PUJdkd6xrfLSj -VV6X93W0S/1egEb1F1CGFxtk8v/PWH4K76EPL2vxXdxjywz3GWlrL9yDYaB2szzS -DqgwVMqx7QKBgDCD7UF0Bsoyl13RX3XoPXLvZ+SkR+e2q52Z94C4JskKVBeiwX7Y -yNAS8M4pBoMArDoj0xmBm69rlKbqtjLGbnzwrTdSzDpim7cWnBQgUFLm7gAD1Elb -AhZ8BCK0Bw4FnLoa2hfga4oEfdfUMgEE0W5/+SEOBgWKRUmuHUhRc911AoGAY2EN -YmSDYSM5wDIvVb5k9B3EtevOiqNPSw/XnsoEZtiEC/44JnQxdltIBY93bDBrk5IQ -cmoBM4h91kgQjshQwOMXMhFSwvmBKmCm/hrTbvMVytTutXfVD3ZXFKwT4DW7N0TF -ElhsxBh/YzRz7mG62JVjtFt2zDN3ld2Z8YpvtXUCgYEA4EJ4ObS5YyvcXAKHJFo6 -Fxmavyrf8LSm3MFA65uSnFvWukMVqqRMReQc5jvpxHKCis+XvnHzyOfL0gW9ZTi7 -tWGGbBi0TRJCa8BkvgngUZxOxUlMfg/7cVxOIB0TPoUSgxFd/+qVz4GZMvr0dPu7 -eAF7J/8ECVvb0wSPTUI1N3c= +MIIG/wIBADANBgkqhkiG9w0BAQEFAASCBukwggblAgEAAoIBgQCylKlLaKU+hOvJ +DfriTRLd+IthG5hv28I3A/CGjLICT0rDDtgaXd0uqloJAnjsgn5gMAcStpDW8Rm+ +t6LsrBL+5fBgkyU1r94Rvx0HHoyaZwBBouitVHw28hP3W+smddkqB1UxpGnTeL2B +gj3dVo/WTtRfO+0h0PKw1l98YE1pMTdqIwcOOE/ER0g4hvA/wrxuLhMvlVLMy/lL +58uctqaDUqryNyeerKbVkq4fJyCG5D2TwXVJ3i2DDh0xSt2Y10poZV4M4k8Su9Z5 +8zN2PSvYMT50aqF277v8BaOeYUApBE4kZGIJpo13ATGdEwpUFZ0Fri4zLYUZ1hWb +OC35sKo7OxWQ/+tefNUdgWHob6Vmy777jiYcLwxc3sS9rF3AJe0rMW83kCkR6hmy +A3250E137N/1QumHuT/Nj9rnI/lwt9jfaYkZjoAgT/C97m/mM83cYpGTdoGV1xNo +7G90MhP0di5FnVsrIaSnvkbGT9UgUWx0oVMjocifdG2qIhMI9psCAwEAAQKCAYBT +sHmaPmNaZj59jZCqp0YVQlpHWwBYQ5vD3pPE6oCttm0p9nXt/VkfenQRTthOtmT1 +POzDp00/feP7zeGLmqSYUjgRekPw4gdnN7Ip2PY5kdW77NWwDSzdLxuOS8Rq1MW9 +/Yu+ZPe3RBlDbT8C0IM+Atlh/BqIQ3zIxN4g0pzUlF0M33d6AYfYSzOcUhibOO7H +j84r+YXBNkIRgYKZYbutRXuZYaGuqejRpBj3voVu0d3Ntdb6lCWuClpB9HzfGN0c +RTv8g6UYO4sK3qyFn90ibIR/1GB9watvtoWVZqggiWeBzSWVWRsGEf9O+Cx4oJw1 +IphglhmhbgNksbj7bD24on/icldSOiVkoUemUOFmHWhCm4PnB1GmbD8YMfEdSbks +qDr1Ps1zg4mGOinVD/4cY7vuPFO/HCH07wfeaUGzRt4g0/yLr+XjVofOA3oowyxv +JAzr+niHA3lg5ecj4r7M68efwzN1OCyjMrVJw2RAzwvGxE+rm5NiT08SWlKQZnkC +gcEA4wvyLpIur/UB84nV3XVJ89UMNBLm++aTFzld047BLJtMaOhvNqx6Cl5c8VuW +l261KHjiVzpfNM3/A2LBQJcYkhX7avkqEXlj57cl+dCWAVwUzKmLJTPjfaTTZnYJ +xeN3dMYjJz2z2WtgvfvDoJLukVwIMmhTY8wtqqYyQBJ/l06pBsfw5TNvmVIOQHds +8ASOiFt+WRLk2bl9xrGGayqt3VV93KVRzF27cpjOgEcG74F3c0ZW9snERN7vIYwB +JfrlAoHBAMlahPwMP2TYylG8OzHe7EiehTekSO26LGh0Cq3wTGXYsK/q8hQCzL14 +kWW638vpwXL6L9ntvrd7hjzWRO3vX/VxnYEA6f0bpqHq1tZi6lzix5CTUN5McpDg +QnjenSJNrNjS1zEF8WeY9iLEuDI/M/iUW4y9R6s3WpgQhPDXpSvd2g3gMGRUYhxQ +Xna8auiJeYFq0oNaOxvJj+VeOfJ3ZMJttd+Y7gTOYZcbg3SdRb/kdxYki0RMD2hF +4ZvjJ6CTfwKBwQDiMqiZFTJGQwYqp4vWEmAW+I4r4xkUpWatoI2Fk5eI5T9+1PLX +uYXsho56NxEU1UrOg4Cb/p+TcBc8PErkGqR0BkpxDMOInTOXSrQe6lxIBoECVXc3 +HTbrmiay0a5y5GfCgxPKqIJhfcToAceoVjovv0y7S4yoxGZKuUEe7E8JY2iqRNAO +yOvKCCICv/hcN235E44RF+2/rDlOltagNej5tY6rIFkaDdgOF4bD7f9O5eEni1Bg +litfoesDtQP/3rECgcEAkQfvQ7D6tIPmbqsbJBfCr6fmoqZllT4FIJN84b50+OL0 +mTGsfjdqC4tdhx3sdu7/VPbaIqm5NmX10bowWgWSY7MbVME4yQPyqSwC5NbIonEC +d6N0mzoLR0kQ+Ai4u+2g82gicgAq2oj1uSNi3WZi48jQjHYFulCbo246o1NgeFFK +77WshYe2R1ioQfQDOU1URKCR0uTaMHClgfu112yiGd12JAD+aF3TM0kxDXz+sXI5 +SKy311DFxECZeXRLpcC3AoHBAJkNMJWTyPYbeVu+CTQkec8Uun233EkXa2kUNZc/ +5DuXDaK+A3DMgYRufTKSPpDHGaCZ1SYPInX1Uoe2dgVjWssRL2uitR4ENabDoAOA +ICVYXYYNagqQu5wwirF0QeaMXo1fjhuuHQh8GsMdXZvYEaAITZ9/NG5x/oY08+8H +kr78SMBOPy3XQn964uKG+e3JwpOG14GKABdAlrHKFXNWchu/6dgcYXB87mrC/GhO +zNwzC+QhFTZoOomFoqMgFWujng== -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIDWTCCAkGgAwIBAgIJAPm6B21bar2bMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV +MIIEWTCCAsGgAwIBAgIJAJinz4jHSjLtMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODAx -MTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH +IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODA4 +MjkxNDIzMTVaFw0yODA4MjYxNDIzMTVaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH DA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5k -YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEP -ADCCAQoCggEBAKvvsX2gEti4shve3iYMc+jE4Se7WHs1Bol2f21H8qNboDOFdeb1 -RKHjmq3exHpajywOUEgne9nKHJY/3f2phR4Y5klqG6liLgiSpVyRlcBGbeT2qEAj -9oLiLFUXLGfGDds2mTwivQDLJBWi51j7ff5k2Pr58fN5ugYMn24T9FNyn0moT+qj -SFoBNm58l9jrdkJSlgWfqPlbiMa+mqDn/SFtrwLF2Trbfzu42Sd9UdIzMaSSrzbN -sGm53pNhCh8KndWUQ8GPP2IsLPoUU4qAtmZuTxCx2S1cXrN9EkmT69tlOH84YfSn -96Ih9bWRc7M5y5bfVdEVM+fKQl3hBRf05qMCAwEAAaMYMBYwFAYDVR0RBA0wC4IJ -bG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQAtQ8f37cCEk7/rAcbYR53ce3iK -Vpihb0U2ni1QjG9Tg9UIExkIGkwTiCm7kwQL+GEStBu9AG/QVrOjeTriRiddhWkk -ze8kRaI3AC/63t6Vh9Q1x6PESgeE4OtAO9JpJCf4GILglA789Y/b/GF8zJZQxR13 -qpB4ZwWw7gCBhdEW59u6CFeBmfDa58hM8lWvuVoRrTi7bjUeC6PAn5HVMzZSykhu -4HaUfBp6bKFjuym2+h/VvM1n8C3chjVSmutsLb6ELdD8IK0vPV/yf5+LN256zSsS -dyUZYd8XwQaioEMKdbhLvnehyzHiWfQIUR3BdhONxoIJhHv/EAo8eCkHHYIF +YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGP +ADCCAYoCggGBALKUqUtopT6E68kN+uJNEt34i2EbmG/bwjcD8IaMsgJPSsMO2Bpd +3S6qWgkCeOyCfmAwBxK2kNbxGb63ouysEv7l8GCTJTWv3hG/HQcejJpnAEGi6K1U +fDbyE/db6yZ12SoHVTGkadN4vYGCPd1Wj9ZO1F877SHQ8rDWX3xgTWkxN2ojBw44 +T8RHSDiG8D/CvG4uEy+VUszL+Uvny5y2poNSqvI3J56sptWSrh8nIIbkPZPBdUne +LYMOHTFK3ZjXSmhlXgziTxK71nnzM3Y9K9gxPnRqoXbvu/wFo55hQCkETiRkYgmm +jXcBMZ0TClQVnQWuLjMthRnWFZs4Lfmwqjs7FZD/61581R2BYehvpWbLvvuOJhwv +DFzexL2sXcAl7SsxbzeQKRHqGbIDfbnQTXfs3/VC6Ye5P82P2ucj+XC32N9piRmO +gCBP8L3ub+YzzdxikZN2gZXXE2jsb3QyE/R2LkWdWyshpKe+RsZP1SBRbHShUyOh +yJ90baoiEwj2mwIDAQABoxgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZI +hvcNAQELBQADggGBAHRUO/UIHl3jXQENewYayHxkIx8t7nu40iO2DXbicSijz5bo +5//xAB6RxhBAlsDBehgQP1uoZg+WJW+nHu3CIVOU3qZNZRaozxiCl2UFKcNqLOmx +R3NKpo1jYf4REQIeG8Yw9+hSWLRbshNteP6bKUUf+vanhg9+axyOEOH/iOQvgk/m +b8wA8wNa4ujWljPbTQnj7ry8RqhTM0GcAN5LSdSvcKcpzLcs3aYwh+Z8e30sQWna +F40sa5u7izgBTOrwpcDm/w5kC46vpRQ5fnbshVw6pne2by0mdMECASid/p25N103 +jMqTFlmO7kpf/jpCSmamp3/JSEE1BJKHwQ6Ql4nzRA2N1mnvWH7Zxcv043gkHeAu +0x8evpvwuhdIyproejNFlBpKmW8OX7yKTCPPMC/VkX8Q1rVkxU0DQ6hmvwZlhoKa +9Wc2uXpw9xF8itV4Uvcdr3dwqByvIqn7iI/gB+4l41e0u8OmH2MKOx4Nxlly5TNW +HcVKQHyOeyvnINuBAQ== -----END CERTIFICATE----- diff --git a/Lib/test/keycert2.pem b/Lib/test/keycert2.pem index bb5fa65a8aca..ed6ae85a4649 100644 --- a/Lib/test/keycert2.pem +++ b/Lib/test/keycert2.pem @@ -1,49 +1,66 @@ -----BEGIN PRIVATE KEY----- -MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQC3ulRNfhbOAey/ -B+wIVYx+d5az7EV4riR6yi/qE6G+bxbTvay2pqySHtDweuaYSh2cVmcasBKKIFJm -rCD1zR8UmLb5i2XFIina1t3eePCuBZMrvZZwkzlQUSM1AZtjGOO/W0I3FwO6y645 -9xA5PduKI7SMYkH/VL3zE5W1JwMovv6bvNiT+GU5l6mB9ylCTgLpmUqoQhRqz/35 -zCzVyoh+ppDvVcpWYfvXywsXsgQwbAF0QJm8SSFi0TZm5ykv4WE16afQp08yuZS0 -3U4K3MJCa4rxO58edcxBopWYfQ29K3iINM8enRfr5q+u5mAAbALAEEvyFjgLWl/u -7arxn7bJAgMBAAECggEBAJfMt8KfHzBunrDnVrk8FayYGkfmOzAOkc1yKEx6k/TH -zFB+Mqlm5MaF95P5t3S0J+r36JBAUdEWC38RUNpF9BwMYYGlDxzlsTdCuGYL/q+J -o6NMLXQt7/jQUQqGnWAvPFzqhbcGqOo5R2ZVH25sEWv9PDuRI35XAepIkDTwWsfa -P6UcJJoP+4v9B++fb3sSL4zNwp1BqS4wxR8YTR0t1zQqOxJ5BGPw1J8aBMs1sq5t -qyosAQAT63kLrdqWotHaM26QxjqEQUMlh12XMWb5GdBXUxbvyGtEabsqskGa/f8B -RdHE437J8D8l+jxb2mZLzrlaH3dq2tbFGCe1rT8qLRECgYEA5CWIvoD/YnQydLGA -OlEhCSocqURuqcotg9Ev0nt/C60jkr/NHFLGppz9lhqjIDjixt3sIMGZMFzxRtwM -pSYal3XiR7rZuHau9iM35yDhpuytEiGbYy1ADakJRzY5jq/Qa8RfPP9Atua5xAeP -q6DiSnq9vhHv9G+O4MxzHBmrw9sCgYEAziiJWFthcwvuXn3Jv9xFYKEb/06puZAx -EgQCz/3rPzv5fmGD/sKVo1U/K4z/eA82DNeKG8QRTFJCxT8TCNRxOmGV7HdCYo/B -4BTNNvbKcdi3l0j75kKoADg+nt5CD5lz6gLG0GrUEnVO1y5HVfCTb3BEAfa36C85 -9i0sfQGiwysCgYEAuus9k8cgdct5oz3iLuVVSark/JGCkT2B+OOkaLChsDFUWeEm -7TOsaclpwldkmvvAYOplkZjMJ2GelE2pVo1XcAw3LkmaI5WpVyQXoxe/iQGT8qzy -IFlsh0Scw2lb0tmcyw6CcPk4TiHOxRrkzNrtS9QwLM+JZx0XVHptPPKTVc0CgYAu -j/VFYY5G/8Dc0qhIjyWUR48dQNUQtkJ/ASzpcT46z/7vznKTjbtiYpSb74KbyUO5 -7sygrM4DYOj3x+Eys1jHiNbly6HQxQtS4x/edCsRP5NntfI+9XsgYZOzKhvdjhki -F3J0DEzNxnUCIM+311hVaRPTJbgv1srOkTFlIoNydQKBgQC6/OHGaC/OewQqRlRK -Mg5KZm01/pk4iKrpA5nG7OTAeoa70NzXNtG8J3WnaJ4mWanNwNUOyRMAMrsUAy9q -EeGqHM5mMFpY4TeVuNLL21lu/x3KYw6mKL3Ctinn+JLAoYoqEy8deZnEA5/tjYlz -YhFBchnUicjoUN1chdpM6SpV2Q== +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQDKjrjWZlfOs1Ch +qt1RoyLfqyXbHVXIAW0fTzAxfJnxvFOiWqAAKgC2qVQM8Y080kRUuRaXP/w9ywXT ++MzX6tByy5VbTYJYyTjHOH46EWLNdcqEJs4+FCVqOIYrQPQ6pGAhCXmgBy4Vb42J +ABLwb+Kt+y2Dk15tggVcAHP2Khri+lRXWvda+kZAe2F1IojmuWyCTy3FEYHic5qN +BsXcf6u1oyFV8MybOuz1zGj3vd2C+dEKO4Ohw9rRwnvHSatjM+CfwiXf8kTXzDBF +Z/8W3+6yA49pHxRbG7FE3K1TAPhkrp+BVTIUOcdI74wEA6UEkWFF5sQcmmAth59M +EQrl2CXorftEPhsKZE59dUP1+nYAPvR/mTySNCSw7/rvdf+csRSZ5ollMu/lxsht +ulJYJI03+IiDTn47FI5D+IF25REK7d4LzGIo6T73ktsT+qpSXHuTWC+IABm8AMF9 +7ljxHSwMRU/z+O5uiONRItDAgKH/OItFG54PtY2vAhaO0YiZrZcCAwEAAQKCAYB2 +hTo8IVghlySH5B1p5kXCkDcvVaPaypLaLhCp9Blzq9lX9yUF043lU4Ddrf0RaIsY +88/3IjZqxb+cP0lE0Z20fdDfwqORZfQ2BaU+PuwMAm9EEhy9kDYwR/ChoHkHUyT4 +T7392BWr70Dmt8ddLmp5mK4R/gnTk6+lHJK9p/dhdk4haxWvAyBWHJty2Yk3T6nh +OYkzdUIFidUVza+6jG2hc1lPGv3tmnYKgNeulkblm10oWphz79C6ycx5WG7TNgef +CQ3z7//Nn89YTiaUBjLvoLvxRTMwO96r7E/FaslSl/fWnF3HP3lut26Z/mNfhiwj +qn7AhUwpSNPV0qcxFWXr/rXUjdk745wv8wOODK8atjjE/vt/MRBK0rAOIPSm3ecx +37PKNtR4i+sNeDEcY1IyTHE6wFvJSy5y8AFpn5y8tbqYfhlEVWZ4pcnlrKxhWm7j +oBkB/4GBjKQgbQ7ttym9eNG1wIbZ8v9N06+yeLs/NCc4bFZEgcWjFqBH1bLtAYEC +gcEA8tt8iYNqbsDH2ognjEmbbBxrDBmyYcEKRpg1i1SUopcZl8i93IHpG7EgJIaj +l7aWSbASAxjnK02t0VZ3nNS60acibzRwY/+e8OrSqlQdMXlAB2ggBA86drDJpfBl +WGJG8TJVY9bc1TU2uuwtZR1LAMSsRHVp+3IvKLpHrne5exPd3x6KGYcuaM+Uk/rE +u6tLsFNwaCdh+iBFFDT2bnYIw7jAsokJUkwxMVxSC0/21s2blhO/q5LsN1gFC1kN +TbpXAoHBANWE7TmG2szPvujPwrK18v6iJlHCA2n50AgAQXrsetj2JcF3HYHYdHnq +z36MQ6FpBKOiQumozWvb32WTjEwdG2kix7GEfam4DAUBdqYuCHzPcR12K5Tc8hsX +NG7JXUAeS8ZJEiOdu95X59JHyBxUQtNfte5rcbaV17SVw6K6bsWVJnj60YjtJrpa +xHvv1ZRnT2WEzJGpA+ii1h3I52N7ipGBiw172qcW+bKJukMi8eHxx5CC9e5tBpnu +C+Ou/eYewQKBwHxNa0jXQrq9YY2w8s0TP8HuKbxfyrXOIHxRm9ZczFcMD8VosgUT +WUUbO+B2KXWVtwawYAfFz0ySzcy//SkAmT6F1VIl/QCx7aBSENGti+Ous98WpIxv +XvUxN4T/rl+2raj2ok4fw5g9TG4QRIvkmmciQyonDr/sicbG0bmy/fTJDl8NOpIm +ZtKurNWxHNERtAPkMTyeK7/ilHjrQtb3AzVqcvbuvR6qcONa5YN0wlrfkisWoJwo +707EdpCAXBbUsQKBwQCnpzcpu2Sj+t9ZKIElF87T93gFLETH+ppJHgJMRdDz+NqO +fTwTD2XtsNz57aLQ44f8AFVv6NZbQYq41FEOFrDGLcQE9BZDpDrz10FVnMGXVr7n +tjjkK1SCxwapkr0AsoknCYsPojO4kud46loLPHI4TGeq7HyeNCvqJMo3RRHjXIiX +58GNNUD6hHjRI/FdFH14Jf0GxmJGUU20l2Jwb7nPJJuNm9mE53pqoNA7FP4+Pj1H +kD0Q2FSdmxeE0IuWHEECgcBgw6ogJ/FRRGLcym+aApqP9BChK+W8FDfDc9Mi4p/J +g+XmetWNFGCGTlOefGqUDIkwSG+QVOEN3hxziXbsjnvfpGApqoaulAI5oRvrwIcj +QIvD2mt0PB52k5ZL9QL2K9sgBa43BJDyCKooMAlTy2XMM+NyXVxQKmzf3r3jQ5sl +Rptk7ro38a9G8Rs99RFDyOmP1haOM0KXZvPksN4nsXuTlE01cnwnI29XKAlEZaoA +pQPLXD8W/KK4mwDbmokYXmo= -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIDYjCCAkqgAwIBAgIJALJXRr8qF6oIMA0GCSqGSIb3DQEBCwUAMGIxCzAJBgNV +MIIEYjCCAsqgAwIBAgIJAJm2YulYpr+6MA0GCSqGSIb3DQEBCwUAMGIxCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u IFNvZnR3YXJlIEZvdW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTAeFw0x -ODAxMTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMGIxCzAJBgNVBAYTAlhZMRcwFQYD +ODA4MjkxNDIzMTZaFw0yODA4MjYxNDIzMTZaMGIxCzAJBgNVBAYTAlhZMRcwFQYD VQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZv -dW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTCCASIwDQYJKoZIhvcNAQEB -BQADggEPADCCAQoCggEBALe6VE1+Fs4B7L8H7AhVjH53lrPsRXiuJHrKL+oTob5v -FtO9rLamrJIe0PB65phKHZxWZxqwEoogUmasIPXNHxSYtvmLZcUiKdrW3d548K4F -kyu9lnCTOVBRIzUBm2MY479bQjcXA7rLrjn3EDk924ojtIxiQf9UvfMTlbUnAyi+ -/pu82JP4ZTmXqYH3KUJOAumZSqhCFGrP/fnMLNXKiH6mkO9VylZh+9fLCxeyBDBs -AXRAmbxJIWLRNmbnKS/hYTXpp9CnTzK5lLTdTgrcwkJrivE7nx51zEGilZh9Db0r -eIg0zx6dF+vmr67mYABsAsAQS/IWOAtaX+7tqvGftskCAwEAAaMbMBkwFwYDVR0R -BBAwDoIMZmFrZWhvc3RuYW1lMA0GCSqGSIb3DQEBCwUAA4IBAQCZhHhGItpkqhEq -ntMRd6Hv0GoOJixNvgeMwK4NJSRT/no3OirtUTzccn46h+SWibSa2eVssAV+pAVJ -HbzkN/DH27A1mMx1zJL1ekcOKA1AF6MXhUnrUGXMqW36YNtzHfXJLrwvpLJ13OQg -/Kxo4Nw68bGzM+PyRtKU/mpgYyfcvwR+ZSeIDh1fvUZK/IEVCf8ub42GPVs5wPfv -M+k5aHxWTxeif3K1byTRzxHupYNG2yWO4XEdnBGOuOwzzN4/iQyNcsuQKeuKHGrt -YvIlG/ri04CQ7xISZCj74yjTZ+/A2bXre2mQXAHqKPumHL7cl34+erzbUaxYxbTE -u5FcOmLQ +dW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTCCAaIwDQYJKoZIhvcNAQEB +BQADggGPADCCAYoCggGBAMqOuNZmV86zUKGq3VGjIt+rJdsdVcgBbR9PMDF8mfG8 +U6JaoAAqALapVAzxjTzSRFS5Fpc//D3LBdP4zNfq0HLLlVtNgljJOMc4fjoRYs11 +yoQmzj4UJWo4hitA9DqkYCEJeaAHLhVvjYkAEvBv4q37LYOTXm2CBVwAc/YqGuL6 +VFda91r6RkB7YXUiiOa5bIJPLcURgeJzmo0Gxdx/q7WjIVXwzJs67PXMaPe93YL5 +0Qo7g6HD2tHCe8dJq2Mz4J/CJd/yRNfMMEVn/xbf7rIDj2kfFFsbsUTcrVMA+GSu +n4FVMhQ5x0jvjAQDpQSRYUXmxByaYC2Hn0wRCuXYJeit+0Q+GwpkTn11Q/X6dgA+ +9H+ZPJI0JLDv+u91/5yxFJnmiWUy7+XGyG26UlgkjTf4iINOfjsUjkP4gXblEQrt +3gvMYijpPveS2xP6qlJce5NYL4gAGbwAwX3uWPEdLAxFT/P47m6I41Ei0MCAof84 +i0Ubng+1ja8CFo7RiJmtlwIDAQABoxswGTAXBgNVHREEEDAOggxmYWtlaG9zdG5h +bWUwDQYJKoZIhvcNAQELBQADggGBAMIVLp6e6saH2NQSg8iFg8Ewg/K/etI++jHo +gCJ697AY02wtfrBox1XtljlmI2xpJtVAYZWHhrNqwrEG43aB7YEV6RqTcG6QUVqa +NbD8iNCnMKm7fP89hZizmqA1l4aHnieI3ucOqpgooM7FQwLX6qk+rSue6lD5N/5f +avsublnj8rNKyDfHpQ3AWduLoj8QqctpzI3CqoDZNLNzaDnzVWpxT1SKDQ88q7VI +W5zb+lndpdQlCu3v5HM4w5UpwL/k1htl/z6PnPseS2UdlXv6A8KITnCLg5PLP4tz +2oTAg9gjOtRP/0uwkhvicwoFzFJNVT813lzTLE1jlobMPiZhsS1mjaJGPD9GQZDK +ny3j8ogrIRGjnI4xpOMNNDVphcvwtV8fRbvURSHCj9Y4kCLpD5ODuoyEyLYicJIv +GZP456GP0iSCK5GKO0ij/YzGCkPWD5zA+mYFpMMGZPTwajenMw7TVaPXcc9CZBtr +oOjwwiLEqdkpxUj13mJYTlt5wsS/Kw== -----END CERTIFICATE----- diff --git a/Lib/test/keycert3.pem b/Lib/test/keycert3.pem index 621eb08bb0ce..e0a8205a660e 100644 --- a/Lib/test/keycert3.pem +++ b/Lib/test/keycert3.pem @@ -1,64 +1,84 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDgV4G+Zzf2DT5n -oAisIGFhn/bz7Vn5WiXUqbDsxROJOh/7BtOlduZka0pPhFylGbnxS8l1kEWHRI2T -6hOoWzumB6ItKiN+T5J30lAvSyo7iwdFoAQ/S5nPXQfhNARQe/NEOhRtpcuNdyx4 -BWdPdPuJQA1ASNJCLwcLOoRxaLbKLvb2V5T5FCAkeNPtRvPuT4gKQItMmiHfAhoV -C8MZWF/GC0RukHINys5MwqeFexam8CznmQPMYrLdhmKTj3DTivCPoh97EDIFGlgZ -SCaaYDVQA+aqlo/q2pi52PFwC1KzhNEA7EeOqSwC1NQjwjHuhcnf9WbxrgTq2zh3 -rv5YEW2ZAgMBAAECggEAPfSMtTumPcJskIumuXp7yk02EyliZrWZqwBuBwVqHsS5 -nkbFXnXWrLbgn9MrDsFrE5NdgKUmPnQVMVs8sIr5jyGejSCNCs4I4iRn1pfIgwcj -K/xEEALd6GGF0pDd/CgvB5GOoLVf4KKf2kmLvWrOKJpSzoUN5A8+v8AaYYOMr4sC -czbvfGomzEIewEG+Rw9zOVUDlmwyEKPQZ47E7PQ+EEA7oeFdR+1Zj6eT9ndegf8B -54frySYCLRUCk/sHCpWhaJBtBrcpht7Y8CfY7hiH/7x866fvuLnYPz4YALtUb0wN -7zUCNS9ol3n4LbjFFKfZtiRjKaCBRzMjK0rz6ydFcQKBgQDyLI3oGbnW73vqcDe/ -6eR0w++fiCAVhfMs3AO/gOaJi2la2JHlJ5u+cIHQIOFwEhn6Zq0AtdmnFx1TS5IQ -C0MdXI0XoQQw7rEF8EJcvfe85Z0QxENVhzydtdb8QpJfnQGfBfLyQlaaRYzRRHB6 -VdYUHF3EIPVIhbjbghal+Qep/QKBgQDtJlRPHkWwTMevu0J0fYbWN1ywtVTFUR// -k7VyORSf8yuuSnaQRop4cbcqONxmDKH6Or1fl3NYBsAxtXkkOK1E2OZNo2sfQdRa -wpA7o7mPHRhztQFpT5vflp+8P6+PEFat8D04eBOhNwrwwfhiPjD4gv5KvN4XutRW -VWv/2pnmzQKBgHPvHGg2mJ7quvm6ixXW1MWJX1eSBToIjCe3lBvDi5nhIaiZ8Q4w -7gA3QA3xD7tlDwauzLeAVxgEmsdbcCs6GQEfY3QiYy1Bt4FOSZa4YrcNfSmfq1Rw -j3Y4rRjKjeQz96i3YlzToT3tecJc7zPBj+DEy6au2H3Fdn+vQURneWHJAoGBANG7 -XES8mRVaUh/wlM1BVsaNH8SIGfiHzqzRjV7/bGYpQTBbWpAuUrhCmaMVtpXqBjav -TFwGLVRkZAWSYRjPpy2ERenT5SE3rv61o6mbGrifGsj6A82HQmtzYsGx8SmtYXtj -REF0sKebbmmOooUAS379GrguYJzL9o6D7YfRZNrhAoGAVfb/tiFU4S67DSpYpQey -ULhgfsFpDByICY6Potsg67gVFf9jIaB83NPTx3u/r6sHFgxFw7lQsuZcgSuWMu7t -glzOXVIP11Y5sl5CJ5OsfeK1/0umMZF5MWPyAQCx/qrPlZL86vXjt24Y/VaOxsAi -CZYdyJsjgOrJrWoMbo5ta54= +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQCfKC83Qe9/ZGMW +YhbpARRiKco6mJI9CNNeaf7A89TE+w5Y3GSwS8uzqp5C6QebZzPNueg8HYoTwN85 +Z3xM036/Qw9KhQVth+XDAqM+19e5KHkYcxg3d3ZI1HgY170eakaLBvMDN5ULoFOw +Is2PtwM2o9cjd5mfSuWttI6+fCqop8/l8cerG9iX2GH39p3iWwWoTZuYndAA9qYv +07YWajuQ1ESWKPjHYGTnMvu4xIzibC1mXd2M6u/IjNO6g426SKFaRDWQkx01gIV/ +CyKs9DgZoeMHkKZuPqZVOxOK+A/NrmrqHFsPIsrs5wk7QAVju5/X1skpn/UGQlmM +RwBaQULOs1FagA+54RXU6qUPW0YmhJ4xOB4gHHD1vjAKEsRZ7/6zcxMyOm+M1DbK +RTH4NWjVWpnY8XaVGdRhtTpH9MjycpKhF+D2Zdy2tQXtqu2GdcMnUedt13fn9xDu +P4PophE0ip/IMgn+kb4m9e+S+K9lldQl0B+4BcGWAqHelh2KuU0CAwEAAQKCAYEA +lKiWIYjmyRjdLKUGPTES9vWNvNmRjozV0RQ0LcoSbMMLDZkeO0UwyWqOVHUQ8+ib +jIcfEjeNJxI57oZopeHOO5vJhpNlFH+g7ltiW2qERqA1K88lSXm99Bzw6FNqhCRE +K8ub5N9fyfJA+P4o/xm0WK8EXk5yIUV17p/9zJJxzgKgv2jsVTi3QG2OZGvn4Oug +ByomMZEGHkBDzdxz8c/cP1Tlk1RFuwSgews178k2xq7AYSM/s0YmHi7b/RSvptX6 +1v8P8kXNUe4AwTaNyrlvF2lwIadZ8h1hA7tCE2n44b7a7KfhAkwcbr1T59ioYh6P +zxsyPT678uD51dbtD/DXJCcoeeFOb8uzkR2KNcrnQzZpCJnRq4Gp5ybxwsxxuzpr +gz0gbNlhuWtE7EoSzmIK9t+WTS7IM2CvZymd6/OAh1Fuw6AQhSp64XRp3OfMMAAC +Ie2EPtKj4islWGT8VoUjuRYGmdRh4duAH1dkiAXOWA3R7y5a1/y/iE8KE8BtxocB +AoHBAM8aiURgpu1Fs0Oqz6izec7KSLL3l8hmW+MKUOfk/Ybng6FrTFsL5YtzR+Ap +wW4wwWnnIKEc1JLiZ7g8agRETK8hr5PwFXUn/GSWC0SMsazLJToySQS5LOV0tLzK +kJ3jtNU7tnlDGNkCHTHSoVL2T/8t+IkZI/h5Z6wjlYPvU2Iu0nVIXtiG+alv4A6M +Hrh9l5or4mjB6rGnVXeYohLkCm6s/W97ahVxLMcEdbsBo1prm2JqGnSoiR/tEFC/ +QHQnbQKBwQDEu7kW0Yg9sZ89QtYtVQ1YpixFZORaUeRIRLnpEs1w7L1mCbOZ2Lj9 +JHxsH05cYAc7HJfPwwxv3+3aGAIC/dfu4VSwEFtatAzUpzlhzKS5+HQCWB4JUNNU +MQ3+FwK2xQX4Ph8t+OzrFiYcK2g0An5UxWMa2HWIAWUOhnTOydAVsoH6yP31cVm4 +0hxoABCwflaNLNGjRUyfBpLTAcNu/YtcE+KREy7YAAgXXrhRSO4XpLsSXwLnLT7/ +YOkoBWDcTWECgcBPWnSUDZCIQ3efithMZJBciqd2Y2X19Dpq8O31HImD4jtOY0V7 +cUB/wSkeHAGwjd/eCyA2e0x8B2IEdqmMfvr+86JJxekC3dJYXCFvH5WIhsH53YCa +3bT1KlWCLP9ib/g+58VQC0R/Cc9T4sfLePNH7D5ZkZd1wlbV30CPr+i8KwKay6MD +xhvtLx+jk07GE+E9wmjbCMo7TclyrLoVEOlqZMAqshgApT+p9eyCPetwXuDHwa3n +WxhHclcZCV7R4rUCgcAkdGSnxcvpIrDPOUNWwxvmAWTStw9ZbTNP8OxCNCm9cyDl +d4bAS1h8D/a+Uk7C70hnu7Sl2w7C7Eu2zhwRUdhhe3+l4GINPK/j99i6NqGPlGpq +xMlMEJ4YS768BqeKFpg0l85PRoEgTsphDeoROSUPsEPdBZ9BxIBlYKTkbKESZDGR +twzYHljx1n1NCDYPflmrb1KpXn4EOcObNghw2KqqNUUWfOeBPwBA1FxzM4BrAStp +DBINpGS4Dc0mjViVegECgcA3hTtm82XdxQXj9LQmb/E3lKx/7H87XIOeNMmvjYuZ +iS9wKrkF+u42vyoDxcKMCnxP5056wpdST4p56r+SBwVTHcc3lGBSGcMTIfwRXrj3 +thOA2our2n4ouNIsYyTlcsQSzifwmpRmVMRPxl9fYVdEWUgB83FgHT0D9avvZnF9 +t9OccnGJXShAIZIBADhVj/JwG4FbaX42NijD5PNpVLk1Y17OV0I576T9SfaQoBjJ +aH1M/zC4aVaS0DYB/Gxq7v8= -----END PRIVATE KEY----- Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9c - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5c + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Nov 28 19:09:06 2027 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Jul 7 14:23:16 2028 GMT Subject: C=XY, L=Castle Anthrax, O=Python Software Foundation, CN=localhost Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:e0:57:81:be:67:37:f6:0d:3e:67:a0:08:ac:20: - 61:61:9f:f6:f3:ed:59:f9:5a:25:d4:a9:b0:ec:c5: - 13:89:3a:1f:fb:06:d3:a5:76:e6:64:6b:4a:4f:84: - 5c:a5:19:b9:f1:4b:c9:75:90:45:87:44:8d:93:ea: - 13:a8:5b:3b:a6:07:a2:2d:2a:23:7e:4f:92:77:d2: - 50:2f:4b:2a:3b:8b:07:45:a0:04:3f:4b:99:cf:5d: - 07:e1:34:04:50:7b:f3:44:3a:14:6d:a5:cb:8d:77: - 2c:78:05:67:4f:74:fb:89:40:0d:40:48:d2:42:2f: - 07:0b:3a:84:71:68:b6:ca:2e:f6:f6:57:94:f9:14: - 20:24:78:d3:ed:46:f3:ee:4f:88:0a:40:8b:4c:9a: - 21:df:02:1a:15:0b:c3:19:58:5f:c6:0b:44:6e:90: - 72:0d:ca:ce:4c:c2:a7:85:7b:16:a6:f0:2c:e7:99: - 03:cc:62:b2:dd:86:62:93:8f:70:d3:8a:f0:8f:a2: - 1f:7b:10:32:05:1a:58:19:48:26:9a:60:35:50:03: - e6:aa:96:8f:ea:da:98:b9:d8:f1:70:0b:52:b3:84: - d1:00:ec:47:8e:a9:2c:02:d4:d4:23:c2:31:ee:85: - c9:df:f5:66:f1:ae:04:ea:db:38:77:ae:fe:58:11: - 6d:99 + 00:9f:28:2f:37:41:ef:7f:64:63:16:62:16:e9:01: + 14:62:29:ca:3a:98:92:3d:08:d3:5e:69:fe:c0:f3: + d4:c4:fb:0e:58:dc:64:b0:4b:cb:b3:aa:9e:42:e9: + 07:9b:67:33:cd:b9:e8:3c:1d:8a:13:c0:df:39:67: + 7c:4c:d3:7e:bf:43:0f:4a:85:05:6d:87:e5:c3:02: + a3:3e:d7:d7:b9:28:79:18:73:18:37:77:76:48:d4: + 78:18:d7:bd:1e:6a:46:8b:06:f3:03:37:95:0b:a0: + 53:b0:22:cd:8f:b7:03:36:a3:d7:23:77:99:9f:4a: + e5:ad:b4:8e:be:7c:2a:a8:a7:cf:e5:f1:c7:ab:1b: + d8:97:d8:61:f7:f6:9d:e2:5b:05:a8:4d:9b:98:9d: + d0:00:f6:a6:2f:d3:b6:16:6a:3b:90:d4:44:96:28: + f8:c7:60:64:e7:32:fb:b8:c4:8c:e2:6c:2d:66:5d: + dd:8c:ea:ef:c8:8c:d3:ba:83:8d:ba:48:a1:5a:44: + 35:90:93:1d:35:80:85:7f:0b:22:ac:f4:38:19:a1: + e3:07:90:a6:6e:3e:a6:55:3b:13:8a:f8:0f:cd:ae: + 6a:ea:1c:5b:0f:22:ca:ec:e7:09:3b:40:05:63:bb: + 9f:d7:d6:c9:29:9f:f5:06:42:59:8c:47:00:5a:41: + 42:ce:b3:51:5a:80:0f:b9:e1:15:d4:ea:a5:0f:5b: + 46:26:84:9e:31:38:1e:20:1c:70:f5:be:30:0a:12: + c4:59:ef:fe:b3:73:13:32:3a:6f:8c:d4:36:ca:45: + 31:f8:35:68:d5:5a:99:d8:f1:76:95:19:d4:61:b5: + 3a:47:f4:c8:f2:72:92:a1:17:e0:f6:65:dc:b6:b5: + 05:ed:aa:ed:86:75:c3:27:51:e7:6d:d7:77:e7:f7: + 10:ee:3f:83:e8:a6:11:34:8a:9f:c8:32:09:fe:91: + be:26:f5:ef:92:f8:af:65:95:d4:25:d0:1f:b8:05: + c1:96:02:a1:de:96:1d:8a:b9:4d Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Alternative Name: @@ -70,11 +90,11 @@ Certificate: X509v3 Basic Constraints: critical CA:FALSE X509v3 Subject Key Identifier: - 85:11:BE:16:47:04:D1:30:EE:86:8A:18:70:BE:A8:28:6F:82:3D:CE + 8F:EA:1D:E3:33:5C:00:16:B3:8B:6F:6B:6F:D3:4C:CB:B5:CB:7C:55 X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 DirName:/C=XY/O=Python Software Foundation CA/CN=our-ca-server - serial:82:ED:BF:41:C8:80:91:9B + serial:CB:2D:80:99:5A:69:52:5B Authority Information Access: CA Issuers - URI:http://testca.pythontest.net/testca/pycacert.cer @@ -85,48 +105,60 @@ Certificate: Full Name: URI:http://testca.pythontest.net/testca/revocation.crl - Signature Algorithm: sha1WithRSAEncryption - 7f:a1:7e:3e:68:01:b0:32:b8:57:b8:03:68:13:13:b3:e3:f4: - 70:2f:15:e5:0f:87:b9:fd:e0:12:e3:16:f2:91:53:c7:4e:25: - af:ca:cb:a7:d9:9d:57:4d:bf:a2:80:d4:78:aa:04:31:fd:6d: - cc:6d:82:43:e9:62:16:0d:0e:26:8b:e7:f1:3d:57:5c:68:02: - 9c:2b:b6:c9:fd:62:2f:10:85:88:cc:44:a5:e7:a2:3e:89:f2: - 1f:02:6a:3f:d0:3c:6c:24:2d:bc:51:62:7a:ec:25:c5:86:87: - 77:35:8f:f9:7e:d0:17:3d:77:56:bf:1a:0c:be:09:78:ee:ea: - 73:97:65:60:94:91:35:b3:5c:46:8a:5e:6d:94:52:de:48:b7: - 1f:6c:28:79:7f:ff:08:8d:e4:7d:d0:b9:0b:7c:ae:c4:1d:2a: - a1:b3:50:11:82:03:5e:6c:e7:26:fa:05:32:39:07:83:49:b9: - a2:fa:04:da:0d:e5:ff:4c:db:97:d0:c3:a7:43:37:4c:16:de: - 3c:b5:e9:7e:82:d4:b3:10:df:d1:c1:66:72:9c:15:67:19:3b: - 7b:91:0a:82:07:67:c5:06:03:5f:80:54:08:81:8a:b1:5c:7c: - 4c:d2:07:38:92:eb:12:f5:71:ae:de:05:15:c8:e1:33:f0:e4: - 96:0f:0f:1e + Signature Algorithm: sha256WithRSAEncryption + 27:f5:8c:59:10:f4:c6:e7:28:00:bf:ba:8d:7b:13:03:f1:1c: + a6:5f:b3:06:55:a4:22:b9:db:b2:d5:46:bd:f7:0c:dd:43:6e: + b4:79:65:67:21:0c:2a:55:ee:40:8e:85:9f:9f:47:bb:0a:2a: + 4d:b6:64:74:98:a0:7f:ae:dc:f1:2e:db:42:77:18:e0:75:8b: + 26:35:68:c3:41:ed:6b:c8:77:72:6f:6a:9a:5d:55:69:02:fd: + 5a:54:c8:57:cb:b0:65:03:16:e2:0f:00:39:99:66:a0:9b:88: + 93:17:e2:5a:2d:79:35:5f:97:57:78:c4:af:f5:99:5e:86:ab: + d3:11:ad:1a:a2:0d:fa:52:10:b9:fe:bf:9d:ce:33:d9:86:b2: + 9c:16:f8:d6:75:08:8a:db:0a:e5:b4:2b:16:7f:b4:f9:2a:9f: + c3:d2:77:d7:cd:65:1e:f4:6c:1e:eb:59:b9:f0:ae:5f:a4:1f: + cc:4a:c4:b9:7a:a9:d9:6b:32:68:3b:e1:65:b0:84:b7:90:c4: + ae:fe:f4:37:4f:21:a0:de:9f:3a:b1:e5:cc:16:04:66:3f:0b: + 41:dc:42:3d:20:3e:ec:b7:95:2b:35:57:fa:be:7f:b6:3a:ba: + ca:4f:58:fe:75:3e:08:89:2c:8c:b0:5d:2e:f9:89:10:2b:f9: + 41:46:4f:3c:00:b7:27:d3:65:24:28:17:23:26:31:42:ea:7e: + 4e:93:e4:7b:68:54:ca:9f:46:f3:ef:2b:e9:85:0c:b5:84:b2: + d5:35:34:80:75:2b:f0:91:23:b8:08:01:8e:b9:0a:54:d4:fb: + 34:52:fe:d9:45:f0:80:3b:b6:c1:6f:82:d1:1f:f2:3b:08:f6: + 46:a6:96:27:61:4b:58:32:7a:0e:1d:59:c5:44:ad:5e:1a:79: + 33:c1:d4:05:2f:4a:d3:d8:42:42:8d:33:e3:63:ca:d5:87:97: + 9b:4d:b8:1a:03:34:bb:1c:d2:02:3f:59:23:e2:23:80:88:63: + c2:f0:a2:63:a8:8b -----BEGIN CERTIFICATE----- -MIIE8TCCA9mgAwIBAgIJAILtv0HIgJGcMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIF8TCCBFmgAwIBAgIJAMstgJlaaVJcMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yNzExMjgx -OTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA3MDcx +NDIzMTZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj MCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxv -Y2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOBXgb5nN/YN -PmegCKwgYWGf9vPtWflaJdSpsOzFE4k6H/sG06V25mRrSk+EXKUZufFLyXWQRYdE -jZPqE6hbO6YHoi0qI35PknfSUC9LKjuLB0WgBD9Lmc9dB+E0BFB780Q6FG2ly413 -LHgFZ090+4lADUBI0kIvBws6hHFotsou9vZXlPkUICR40+1G8+5PiApAi0yaId8C -GhULwxlYX8YLRG6Qcg3KzkzCp4V7FqbwLOeZA8xist2GYpOPcNOK8I+iH3sQMgUa -WBlIJppgNVAD5qqWj+ramLnY8XALUrOE0QDsR46pLALU1CPCMe6Fyd/1ZvGuBOrb -OHeu/lgRbZkCAwEAAaOCAcAwggG8MBQGA1UdEQQNMAuCCWxvY2FsaG9zdDAOBgNV -HQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1Ud -EwEB/wQCMAAwHQYDVR0OBBYEFIURvhZHBNEw7oaKGHC+qChvgj3OMH0GA1UdIwR2 -MHSAFJrPz27rcT3bPPGuiGtWcgPLCKdIoVGkTzBNMQswCQYDVQQGEwJYWTEmMCQG -A1UECgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0ExFjAUBgNVBAMMDW91 -ci1jYS1zZXJ2ZXKCCQCC7b9ByICRmzCBgwYIKwYBBQUHAQEEdzB1MDwGCCsGAQUF -BzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3RjYS9weWNhY2Vy -dC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQv -dGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly90ZXN0Y2EucHl0 -aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3JsMA0GCSqGSIb3DQEBBQUA -A4IBAQB/oX4+aAGwMrhXuANoExOz4/RwLxXlD4e5/eAS4xbykVPHTiWvysun2Z1X -Tb+igNR4qgQx/W3MbYJD6WIWDQ4mi+fxPVdcaAKcK7bJ/WIvEIWIzESl56I+ifIf -Amo/0DxsJC28UWJ67CXFhod3NY/5ftAXPXdWvxoMvgl47upzl2VglJE1s1xGil5t -lFLeSLcfbCh5f/8IjeR90LkLfK7EHSqhs1ARggNebOcm+gUyOQeDSbmi+gTaDeX/ -TNuX0MOnQzdMFt48tel+gtSzEN/RwWZynBVnGTt7kQqCB2fFBgNfgFQIgYqxXHxM -0gc4kusS9XGu3gUVyOEz8OSWDw8e +Y2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAJ8oLzdB739k +YxZiFukBFGIpyjqYkj0I015p/sDz1MT7DljcZLBLy7OqnkLpB5tnM8256DwdihPA +3zlnfEzTfr9DD0qFBW2H5cMCoz7X17koeRhzGDd3dkjUeBjXvR5qRosG8wM3lQug +U7AizY+3Azaj1yN3mZ9K5a20jr58Kqinz+Xxx6sb2JfYYff2neJbBahNm5id0AD2 +pi/TthZqO5DURJYo+MdgZOcy+7jEjOJsLWZd3Yzq78iM07qDjbpIoVpENZCTHTWA +hX8LIqz0OBmh4weQpm4+plU7E4r4D82uauocWw8iyuznCTtABWO7n9fWySmf9QZC +WYxHAFpBQs6zUVqAD7nhFdTqpQ9bRiaEnjE4HiAccPW+MAoSxFnv/rNzEzI6b4zU +NspFMfg1aNVamdjxdpUZ1GG1Okf0yPJykqEX4PZl3La1Be2q7YZ1wydR523Xd+f3 +EO4/g+imETSKn8gyCf6Rvib175L4r2WV1CXQH7gFwZYCod6WHYq5TQIDAQABo4IB +wDCCAbwwFAYDVR0RBA0wC4IJbG9jYWxob3N0MA4GA1UdDwEB/wQEAwIFoDAdBgNV +HSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4E +FgQUj+od4zNcABazi29rb9NMy7XLfFUwfQYDVR0jBHYwdIAU3b/K2ubRNLo3dSHK +b5oIKPI1tkihUaRPME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29m +dHdhcmUgRm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcoIJAMst +gJlaaVJbMIGDBggrBgEFBQcBAQR3MHUwPAYIKwYBBQUHMAKGMGh0dHA6Ly90ZXN0 +Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3B5Y2FjZXJ0LmNlcjA1BggrBgEFBQcw +AYYpaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0Y2Evb2NzcC8wQwYD +VR0fBDwwOjA4oDagNIYyaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0 +Y2EvcmV2b2NhdGlvbi5jcmwwDQYJKoZIhvcNAQELBQADggGBACf1jFkQ9MbnKAC/ +uo17EwPxHKZfswZVpCK527LVRr33DN1DbrR5ZWchDCpV7kCOhZ+fR7sKKk22ZHSY +oH+u3PEu20J3GOB1iyY1aMNB7WvId3JvappdVWkC/VpUyFfLsGUDFuIPADmZZqCb +iJMX4loteTVfl1d4xK/1mV6Gq9MRrRqiDfpSELn+v53OM9mGspwW+NZ1CIrbCuW0 +KxZ/tPkqn8PSd9fNZR70bB7rWbnwrl+kH8xKxLl6qdlrMmg74WWwhLeQxK7+9DdP +IaDenzqx5cwWBGY/C0HcQj0gPuy3lSs1V/q+f7Y6uspPWP51PgiJLIywXS75iRAr ++UFGTzwAtyfTZSQoFyMmMULqfk6T5HtoVMqfRvPvK+mFDLWEstU1NIB1K/CRI7gI +AY65ClTU+zRS/tlF8IA7tsFvgtEf8jsI9kamlidhS1gyeg4dWcVErV4aeTPB1AUv +StPYQkKNM+NjytWHl5tNuBoDNLsc0gI/WSPiI4CIY8LwomOoiw== -----END CERTIFICATE----- diff --git a/Lib/test/keycert4.pem b/Lib/test/keycert4.pem index b7df7f3f2c71..d1ebb82486de 100644 --- a/Lib/test/keycert4.pem +++ b/Lib/test/keycert4.pem @@ -1,64 +1,84 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDH/76hZAZH4cSV -CmVZa5HEqKCjCKrcPwBECs9BS+3ibwN4x9NnFNP+tCeFGgJXl7WGFoeXgg3oK+1p -FsOWpsRHuF3BdqkCnShSydmT8bLaGHwKeL0cPxJP5T/uW7ezPKW2VWXGMwmwRaRJ -9dj2VCUu20vDZWSGFr9zjnjoJczBtH3RsVUgpK7euEHuQ5pIM9QSOaCo+5FPR7s7 -1nU7YqbFWtd+NhC8Og1G497B31DQlHciF6BRm6/cNGAmHaAErKUGBFdkGtFPHBn4 -vktoEg9fwxJAZLvGpoTZWrB4HRsRwVTmFdGvK+JXK225xF23AXRXp/snhSuSFeLj -E5cpyJJ7AgMBAAECggEAQOv527X2e/sDr0XSpHZQuT/r9UBpBlnFIlFH+fBF5k0X -GWv0ae/O6U1dzs0kmX57xG0n0ry6+vTXeleTYiH8cTOd66EzN9AAOO+hG29IGZf9 -HAEZkkO/FARc/mjzdtFnEYsjIHWM3ZWdwQx3Q28JKu6w51rQiN51g3NqOCGdF/uF -rE5XPKsKndn+nLHvsNuApFgUYZEwdrozgUueEgRaPTUCNhzotcA9eWoBdA24XNhk -x8Cm/bZWabXm7gBO75zl3Cu2F21ay+EuwyOZTsx6lZi6YX9/zo1mkO81Zi3tQk50 -NMEI0feLNwsdxTbmOcVJadjOgd+QVghlFyr5HGBWMQKBgQD3AH3rhnAo6tOyNkGN -+IzIU1MhUS452O7IavykUYO9sM24BVChpRtlI9Dpev4yE/q3BAO3+oWT3cJrN7/3 -iyo1dzAkpGvI65XWfElXFM4nLjEiZzx4W9fiPN91Oucpr0ED6+BZXTtz4gVm0TP/ -TUc2xvTB6EKvIyWmKOYEi0snxQKBgQDPSOjbz9jWOrC9XY7PmtLB6QJDDz7XSGVK -wzD+gDAPpAwhk58BEokdOhBx2Lwl8zMJi0CRHgH2vNvkRyhvUQ4UFzisrqann/Tw -klp5sw3iWC6ERC8z9zL7GfHs7sK3mOVeAdK6ffowPM3JrZ2vPusVBdr0MN3oZwki -CtNXqbY1PwKBgGheQNbAW6wubX0kB9chavtKmhm937Z5v4vYCSC1gOEqUAKt3EAx -L74wwBmn6rjmUE382EVpCgBM99WuHONQXmlxD1qsTw763LlgkuzE0cckcYaD8L06 -saHa7uDuHrcyYlpx1L5t8q0ol/e19i6uTKUMtGcq6OJwC3yGU4sgAIWxAoGBAMVq -qiQXm2vFL+jafxYoXUvDMJ1PmskMsTP4HOR2j8+FrOwZnVk3HxGP6HOVOPRn4JbZ -YiAT1Uj6a+7I+rCyINdvmlGUcTK6fFzW9oZryvBkjcD483/pkktmVWwTpa2YV/Ml -h16IdsyUTGYlDUYHhXtbPUJOfDpIT4F1j/0wrFGfAoGAO82BcUsehEUQE0xvQLIn -7QaFtUI5z19WW730jVuEobiYlh9Ka4DPbKMvka8MwyOxEwhk39gZQavmfG6+wZm+ -kjERU23LhHziJGWS2Um4yIhC7myKbWaLzjHEq72dszLpQku4BzE5fT60fxI7cURD -WGm/Z3Q2weS3ZGIoMj1RNPI= +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQDGjpiHzq7ghxhM +ZzrnRsGBC/cmw8EREIdbqlrz/l8BFaWeipvO5Hb/MyU8xs2zLUrqIr2JNf+Eii8Y +m4bYmZclFra4jomaiSlxTZOe3dMV8m4vAq4eT2mSfZZC1+XAutqdz7WhHxhMVEm3 +AyTWvTC3qCbnlbX5VIoQUwFrsSWqDiHyaGdK3rrOTKFUKM8YPiq/BZkm6A4eiFci +5wd/SPD+w0pIscZbQW1MUr5bs54uylWaUmtfI8KJt6BDZQ/uA06c6i863sSCEI6L +gq+wyikeJGNMxZMfgu3dzfv4BiZBQX0ZhiRvqseDSdPcuVa2Ifb6CFlg298neweY +4EAIE1O+uqo5h8FF1aUOMZpQEZuzsp9R/TAMBHX1YmVjG/kRdBeaHe3whzB1Pfue +PIX2ZTMmLNYbYbfnmxhk1nn8aAvoT98pNw8y3/2k2KNsu24n9uSkkxAoqJ19WKwm +mL8MpJKAzLv45tRvhN+QLtnRdu+LJ9m29npQHFmYLbdqRfmidnMCAwEAAQKCAYBd +w1C8MRnb5W/QBJ+IP515NxFLOP2e9VM2MkgpGGH8vSAssf/Jv5GCCcD35lmU1zqd +PjKK7PjwueBrmmYfOshpN0Sp+oV4eHUdkCi5yL65inYFtRpMLewIxU2D2zgfvx0l +kMSQhYKP6O22gsGOtmCfGcTlb4kzaHyaINh25nyGxY26TxsX+/3zFbTJbUv+grzk +39vmx4aDXJbpYHfl36gOZmJZ2bl1tnvKovhJjZSRO/MYoPsbPmPLbO89ZCgVmXFc +GVkb5Cram6i3iyutSDjxWN7Fb8uy8pFLPGAXZgF7pQoXPSEHZe8GEWBnWSC9KaDa +uM9Ir847/Muy1ceCmxKcI2WrSjoH2AhPcmHgvbPE9Mynr6+uzReSP3q7Wh9PHm23 +oFx3DwdCfmjysnpAMBawNmJdWyxVDbZ6eyrhp17ADpsMaDTynZ+fJjgMr+MmWtbU +YSRD0wWtl/DrzsaePZsOjCpKYJyulC+rh9/Zz1aiwrGWPbgEAzDrD6Q1Zp0mUCEC +gcEA+XskmGIB9rRPy+YQmRgzQ555PsjLWsnQsNktP6KBhlQjFKJZXRZ0DxDTS7h8 +NrJrUDBmwfsgzggVbeO55VP5FGwD6DNeO/Bz++Fdevh8uKQFHDfk4sbIUPS91qw4 +s7OW7PR7C7Jf7Dnjmsn42o2lO4FsbcEn2F+PHOvoLl/OrSx73lS/RkdOEItW8d8/ +ExRohylnba/I2vCE9bNZd4DGjMW87j/THKPadDZWEqWggcrjY8x6ibSQGm2n2Tka +8B+vAoHBAMu+zl8kqFlYDG24dGfVpMaOYj5Osj0cV5f7O2pZ15sCevjiqoKGHH7G +O8EiI5pRBZ893+Fsx6YWmcKue88lfGvaoQjV0LUbfMfX/FoD39w/ZLx31yKEiFuc +KvMiRV5mO3kQiHBVX9vamzr5NeaErccXY9LnhaKOMX9blgiDQZH7fc3RhodcFWrC +9yfX6ryfidpPnRvK7Ops7hVnFKyyS4FaAarnzH1B2WcVcD4lYYxhMc8VXeU3eKOh +j1fI/F5ifQKBwQDpCjB670HqUzAexL9IYqSwSz3yedoK6m24ZIWx5XicI8fJJIXZ +QHoVAKB/IMtWxH8dnri+Bnj0O/TYe1pQb4pBm0xjAGjMEKYm6LNLhQXr67qiS0vQ +0eKYTKVv+9vTcLRQj2bI3Exh+wkys+tzK9DmrtS8CSvRICIs3+g4OWJzvRPP8NXj +LgQrzBzhPqpKhkvFxdVJTmSOrxFj+a5exLmzEZqT6qanIB+VYpQwQuqVkxGpTX5B +V5ssNLYPYRpapx0CgcByCtQixzcAA1u5knR9pkT76ris3YnA0Ptqk3I3XiBjoGjK +pL0CICUVBMpvmTdKai12a8DDwgqiOaZJJTchxH63NAHNGzkeFkuq5IdYrzB/bHBr +WbzukjZs6KXVv4oKg7ioVAu6rN7iBaO7x8BWzk8i0EHMzFCto1+rRM1e6HEsUBOj +v7LIU0+dmZGUGLRIbhhQPR3Yb6ZatSwyiKc23vmKZqHmUqbQOaqBm6te7beDRugF +XJVY9sqs9IJyhYpVHlUCgcAPoslwYKeAXagsxdQrH3D9VJDXVOHWKMBqQZDio5dB +Q80uWpuxtt6nhZkQO1JIWnYb6v+zbDbcgjByBIDuxCdBW9d+QQnanKmVyrXguK91 +C3OcHHOmSduFdWC3/zYW1mW97Tz1sXyam2hly1u3L5kW+GnE1hr9VVPjQNrO9+Ge +qW0coaJqKY78q3Rm2dtyZeJSFFI1o/DQ3blyItsFpg/QrR+a5XrS6Nw2ZLIL4Azo +J1CTgMwjhwlMNCI4t4dkHd0= -----END PRIVATE KEY----- Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9d - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5d + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Nov 28 19:09:06 2027 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Jul 7 14:23:16 2028 GMT Subject: C=XY, L=Castle Anthrax, O=Python Software Foundation, CN=fakehostname Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:c7:ff:be:a1:64:06:47:e1:c4:95:0a:65:59:6b: - 91:c4:a8:a0:a3:08:aa:dc:3f:00:44:0a:cf:41:4b: - ed:e2:6f:03:78:c7:d3:67:14:d3:fe:b4:27:85:1a: - 02:57:97:b5:86:16:87:97:82:0d:e8:2b:ed:69:16: - c3:96:a6:c4:47:b8:5d:c1:76:a9:02:9d:28:52:c9: - d9:93:f1:b2:da:18:7c:0a:78:bd:1c:3f:12:4f:e5: - 3f:ee:5b:b7:b3:3c:a5:b6:55:65:c6:33:09:b0:45: - a4:49:f5:d8:f6:54:25:2e:db:4b:c3:65:64:86:16: - bf:73:8e:78:e8:25:cc:c1:b4:7d:d1:b1:55:20:a4: - ae:de:b8:41:ee:43:9a:48:33:d4:12:39:a0:a8:fb: - 91:4f:47:bb:3b:d6:75:3b:62:a6:c5:5a:d7:7e:36: - 10:bc:3a:0d:46:e3:de:c1:df:50:d0:94:77:22:17: - a0:51:9b:af:dc:34:60:26:1d:a0:04:ac:a5:06:04: - 57:64:1a:d1:4f:1c:19:f8:be:4b:68:12:0f:5f:c3: - 12:40:64:bb:c6:a6:84:d9:5a:b0:78:1d:1b:11:c1: - 54:e6:15:d1:af:2b:e2:57:2b:6d:b9:c4:5d:b7:01: - 74:57:a7:fb:27:85:2b:92:15:e2:e3:13:97:29:c8: - 92:7b + 00:c6:8e:98:87:ce:ae:e0:87:18:4c:67:3a:e7:46: + c1:81:0b:f7:26:c3:c1:11:10:87:5b:aa:5a:f3:fe: + 5f:01:15:a5:9e:8a:9b:ce:e4:76:ff:33:25:3c:c6: + cd:b3:2d:4a:ea:22:bd:89:35:ff:84:8a:2f:18:9b: + 86:d8:99:97:25:16:b6:b8:8e:89:9a:89:29:71:4d: + 93:9e:dd:d3:15:f2:6e:2f:02:ae:1e:4f:69:92:7d: + 96:42:d7:e5:c0:ba:da:9d:cf:b5:a1:1f:18:4c:54: + 49:b7:03:24:d6:bd:30:b7:a8:26:e7:95:b5:f9:54: + 8a:10:53:01:6b:b1:25:aa:0e:21:f2:68:67:4a:de: + ba:ce:4c:a1:54:28:cf:18:3e:2a:bf:05:99:26:e8: + 0e:1e:88:57:22:e7:07:7f:48:f0:fe:c3:4a:48:b1: + c6:5b:41:6d:4c:52:be:5b:b3:9e:2e:ca:55:9a:52: + 6b:5f:23:c2:89:b7:a0:43:65:0f:ee:03:4e:9c:ea: + 2f:3a:de:c4:82:10:8e:8b:82:af:b0:ca:29:1e:24: + 63:4c:c5:93:1f:82:ed:dd:cd:fb:f8:06:26:41:41: + 7d:19:86:24:6f:aa:c7:83:49:d3:dc:b9:56:b6:21: + f6:fa:08:59:60:db:df:27:7b:07:98:e0:40:08:13: + 53:be:ba:aa:39:87:c1:45:d5:a5:0e:31:9a:50:11: + 9b:b3:b2:9f:51:fd:30:0c:04:75:f5:62:65:63:1b: + f9:11:74:17:9a:1d:ed:f0:87:30:75:3d:fb:9e:3c: + 85:f6:65:33:26:2c:d6:1b:61:b7:e7:9b:18:64:d6: + 79:fc:68:0b:e8:4f:df:29:37:0f:32:df:fd:a4:d8: + a3:6c:bb:6e:27:f6:e4:a4:93:10:28:a8:9d:7d:58: + ac:26:98:bf:0c:a4:92:80:cc:bb:f8:e6:d4:6f:84: + df:90:2e:d9:d1:76:ef:8b:27:d9:b6:f6:7a:50:1c: + 59:98:2d:b7:6a:45:f9:a2:76:73 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Alternative Name: @@ -70,11 +90,11 @@ Certificate: X509v3 Basic Constraints: critical CA:FALSE X509v3 Subject Key Identifier: - F8:76:79:CB:11:85:F0:46:E5:95:E6:7E:69:CB:12:5E:4E:AA:EC:4D + 52:E0:93:AA:52:55:B7:BB:E7:A8:E0:8C:DE:41:2E:F4:07:F0:36:FB X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 DirName:/C=XY/O=Python Software Foundation CA/CN=our-ca-server - serial:82:ED:BF:41:C8:80:91:9B + serial:CB:2D:80:99:5A:69:52:5B Authority Information Access: CA Issuers - URI:http://testca.pythontest.net/testca/pycacert.cer @@ -85,48 +105,60 @@ Certificate: Full Name: URI:http://testca.pythontest.net/testca/revocation.crl - Signature Algorithm: sha1WithRSAEncryption - 6d:50:8d:fb:ee:4e:93:8b:eb:47:56:ba:38:cc:80:e1:9d:c7: - e1:9e:1f:9c:22:0c:d2:08:9b:ed:bf:31:d9:00:ee:af:8c:56: - 78:92:d1:7c:ba:4e:81:7f:82:1f:f4:68:99:86:91:c6:cb:57: - d3:b9:41:12:fa:75:53:fd:22:32:21:50:af:6b:4c:b1:34:36: - d1:a8:25:0a:d0:f0:f8:81:7d:69:58:6e:af:e3:d2:c4:32:87: - 79:d7:cd:ad:0c:56:f3:15:27:10:0c:f9:57:59:53:00:ed:af: - 5d:4d:07:86:7a:e5:f3:97:88:bc:86:b4:f1:17:46:33:55:28: - 66:7b:70:d3:a5:12:b9:4f:c7:ed:e6:13:20:2d:f0:9e:ec:17: - 64:cf:fd:13:14:1b:76:ba:64:ac:c5:51:b6:cd:13:0a:93:b1: - fd:43:09:a0:0b:44:6c:77:45:43:0b:e5:ed:70:b2:76:dc:08: - 4a:5b:73:5f:c1:fc:7f:63:70:f8:b9:ca:3c:98:06:5f:fd:98: - d1:e4:e6:61:5f:09:8f:6c:18:86:98:9c:cb:3f:73:7b:3f:38: - f5:a7:09:20:ee:a5:63:1c:ff:8b:a6:d1:8c:e8:f4:84:3d:99: - 38:0f:cc:e0:52:03:f9:18:05:23:76:39:de:52:ce:8e:fb:a6: - 6e:f5:4f:c3 + Signature Algorithm: sha256WithRSAEncryption + 29:d2:3f:82:3f:c1:38:35:a6:bd:81:10:fe:64:ec:ff:7e:e1: + c6:6f:7f:86:65:f9:31:6f:fb:ef:32:4e:2f:87:c8:42:de:6c: + 8d:b8:06:08:8f:37:70:95:7d:e1:40:d4:82:2b:8d:b3:4a:fd: + 34:c5:9e:df:ff:01:53:4a:4f:08:f4:58:e1:74:fc:78:e3:3e: + 71:a7:5e:66:07:ea:d2:04:31:e2:75:a8:4c:80:17:86:92:20: + d2:32:a7:9a:65:8b:1a:5f:f1:4c:c8:50:6d:00:fc:99:bf:69: + b3:48:f3:45:5a:ee:35:50:14:b8:f3:92:92:c6:9f:0e:5d:eb: + 0d:e8:ec:f2:a4:09:6b:dc:66:2b:fc:df:4c:fc:65:a1:ae:d3: + b5:88:6a:a4:e7:08:1c:94:49:e0:b8:c1:04:8c:21:09:6c:55: + 4b:2c:97:10:f1:8c:6c:d0:bb:ba:8d:93:e8:47:8b:4d:8e:7d: + 7d:85:53:18:c8:f8:f4:8f:67:3a:b1:aa:3e:18:34:6c:3a:e6: + a6:c7:2f:be:83:38:f5:d5:e5:d2:17:28:61:6c:b6:49:99:21: + 69:a4:a8:b6:94:76:fd:18:ad:35:52:45:64:fb:f1:5d:8e:bb: + c0:21:2e:59:55:24:af:bb:8f:b2:0a:7b:17:f0:34:97:8e:68: + a9:f2:d0:3e:f6:73:82:f8:7c:4e:9a:70:7d:d6:b3:8c:cc:85: + 04:5c:02:8f:74:da:88:3a:20:a8:7e:c2:9e:b0:dd:56:1f:ce: + cd:42:16:c6:14:91:ad:30:e0:dc:76:f2:2c:56:ea:38:45:d8: + c0:3e:b8:90:fa:f3:38:99:ec:44:07:35:8f:69:62:0c:f9:ef: + b7:9d:e5:15:42:6e:fb:fe:4c:ff:e8:94:5a:1a:b0:80:b2:0e: + 17:3d:e1:87:a8:08:84:93:74:68:8d:29:df:ca:0b:6a:44:32: + 8a:51:3b:d6:38:db:bd:e3:2a:1b:5e:20:48:81:82:19:91:c6: + 87:8c:0f:cd:51:ea -----BEGIN CERTIFICATE----- -MIIE9zCCA9+gAwIBAgIJAILtv0HIgJGdMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIF9zCCBF+gAwIBAgIJAMstgJlaaVJdMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yNzExMjgx -OTA5MDZaMGIxCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA3MDcx +NDIzMTZaMGIxCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj MCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24xFTATBgNVBAMMDGZh -a2Vob3N0bmFtZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMf/vqFk -BkfhxJUKZVlrkcSooKMIqtw/AEQKz0FL7eJvA3jH02cU0/60J4UaAleXtYYWh5eC -Degr7WkWw5amxEe4XcF2qQKdKFLJ2ZPxstoYfAp4vRw/Ek/lP+5bt7M8pbZVZcYz -CbBFpEn12PZUJS7bS8NlZIYWv3OOeOglzMG0fdGxVSCkrt64Qe5Dmkgz1BI5oKj7 -kU9HuzvWdTtipsVa1342ELw6DUbj3sHfUNCUdyIXoFGbr9w0YCYdoASspQYEV2Qa -0U8cGfi+S2gSD1/DEkBku8amhNlasHgdGxHBVOYV0a8r4lcrbbnEXbcBdFen+yeF -K5IV4uMTlynIknsCAwEAAaOCAcMwggG/MBcGA1UdEQQQMA6CDGZha2Vob3N0bmFt -ZTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC -MAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFPh2ecsRhfBG5ZXmfmnLEl5OquxNMH0G -A1UdIwR2MHSAFJrPz27rcT3bPPGuiGtWcgPLCKdIoVGkTzBNMQswCQYDVQQGEwJY -WTEmMCQGA1UECgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0ExFjAUBgNV -BAMMDW91ci1jYS1zZXJ2ZXKCCQCC7b9ByICRmzCBgwYIKwYBBQUHAQEEdzB1MDwG -CCsGAQUFBzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3RjYS9w -eWNhY2VydC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2EucHl0aG9udGVz -dC5uZXQvdGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly90ZXN0 -Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3JsMA0GCSqGSIb3 -DQEBBQUAA4IBAQBtUI377k6Ti+tHVro4zIDhncfhnh+cIgzSCJvtvzHZAO6vjFZ4 -ktF8uk6Bf4If9GiZhpHGy1fTuUES+nVT/SIyIVCva0yxNDbRqCUK0PD4gX1pWG6v -49LEMod5182tDFbzFScQDPlXWVMA7a9dTQeGeuXzl4i8hrTxF0YzVShme3DTpRK5 -T8ft5hMgLfCe7Bdkz/0TFBt2umSsxVG2zRMKk7H9QwmgC0Rsd0VDC+XtcLJ23AhK -W3Nfwfx/Y3D4uco8mAZf/ZjR5OZhXwmPbBiGmJzLP3N7Pzj1pwkg7qVjHP+LptGM -6PSEPZk4D8zgUgP5GAUjdjneUs6O+6Zu9U/D +a2Vob3N0bmFtZTCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAMaOmIfO +ruCHGExnOudGwYEL9ybDwREQh1uqWvP+XwEVpZ6Km87kdv8zJTzGzbMtSuoivYk1 +/4SKLxibhtiZlyUWtriOiZqJKXFNk57d0xXybi8Crh5PaZJ9lkLX5cC62p3PtaEf +GExUSbcDJNa9MLeoJueVtflUihBTAWuxJaoOIfJoZ0reus5MoVQozxg+Kr8FmSbo +Dh6IVyLnB39I8P7DSkixxltBbUxSvluzni7KVZpSa18jwom3oENlD+4DTpzqLzre +xIIQjouCr7DKKR4kY0zFkx+C7d3N+/gGJkFBfRmGJG+qx4NJ09y5VrYh9voIWWDb +3yd7B5jgQAgTU766qjmHwUXVpQ4xmlARm7Oyn1H9MAwEdfViZWMb+RF0F5od7fCH +MHU9+548hfZlMyYs1htht+ebGGTWefxoC+hP3yk3DzLf/aTYo2y7bif25KSTECio +nX1YrCaYvwykkoDMu/jm1G+E35Au2dF274sn2bb2elAcWZgtt2pF+aJ2cwIDAQAB +o4IBwzCCAb8wFwYDVR0RBBAwDoIMZmFrZWhvc3RuYW1lMA4GA1UdDwEB/wQEAwIF +oDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAd +BgNVHQ4EFgQUUuCTqlJVt7vnqOCM3kEu9AfwNvswfQYDVR0jBHYwdIAU3b/K2ubR +NLo3dSHKb5oIKPI1tkihUaRPME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZl +coIJAMstgJlaaVJbMIGDBggrBgEFBQcBAQR3MHUwPAYIKwYBBQUHMAKGMGh0dHA6 +Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3B5Y2FjZXJ0LmNlcjA1Bggr +BgEFBQcwAYYpaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0Y2Evb2Nz +cC8wQwYDVR0fBDwwOjA4oDagNIYyaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5l +dC90ZXN0Y2EvcmV2b2NhdGlvbi5jcmwwDQYJKoZIhvcNAQELBQADggGBACnSP4I/ +wTg1pr2BEP5k7P9+4cZvf4Zl+TFv++8yTi+HyELebI24BgiPN3CVfeFA1IIrjbNK +/TTFnt//AVNKTwj0WOF0/HjjPnGnXmYH6tIEMeJ1qEyAF4aSINIyp5plixpf8UzI +UG0A/Jm/abNI80Va7jVQFLjzkpLGnw5d6w3o7PKkCWvcZiv830z8ZaGu07WIaqTn +CByUSeC4wQSMIQlsVUsslxDxjGzQu7qNk+hHi02OfX2FUxjI+PSPZzqxqj4YNGw6 +5qbHL76DOPXV5dIXKGFstkmZIWmkqLaUdv0YrTVSRWT78V2Ou8AhLllVJK+7j7IK +exfwNJeOaKny0D72c4L4fE6acH3Ws4zMhQRcAo902og6IKh+wp6w3VYfzs1CFsYU +ka0w4Nx28ixW6jhF2MA+uJD68ziZ7EQHNY9pYgz577ed5RVCbvv+TP/olFoasICy +Dhc94YeoCISTdGiNKd/KC2pEMopRO9Y4273jKhteIEiBghmRxoeMD81R6g== -----END CERTIFICATE----- diff --git a/Lib/test/keycertecc.pem b/Lib/test/keycertecc.pem index deb484f99207..02eda2835f07 100644 --- a/Lib/test/keycertecc.pem +++ b/Lib/test/keycertecc.pem @@ -1,31 +1,31 @@ -----BEGIN PRIVATE KEY----- -MIG2AgEAMBAGByqGSM49AgEGBSuBBAAiBIGeMIGbAgEBBDDe3QWmhZX07HZbntz4 -CFqAOaoYMdYwD7Z3WPNIc2zR7p4D6BMOa7NAWjLV5A7CUw6hZANiAAQ5IVKzLLz4 -LCfcpy6fMOp+jk5KwywsU3upPtjA6E3UetxPcfnnv+gghRyDAYLN2OVqZgLMEmUo -F1j1SM1QrbhHIuNcVxI9gPPMdumcNFSz/hqxrBRtA/8Z2gywczdNLjc= +MIG2AgEAMBAGByqGSM49AgEGBSuBBAAiBIGeMIGbAgEBBDBL2Y5JfpzbgHw+t4Q+ +c5SHhsZcD9ylEtUMg7OyF9xW6j+3VIVORGaokcOtE0Z2Y5ehZANiAASzz/rInKUz +onpxP5bLxmq8fmrtgRSS0jRPUOU16XKX+KtifnLbmLHQtPrctdkRRROCxnURz2fB +ihQTJkXyBMSswNTRCs+4DUKbMAfihigMVYgdWbZPFBDleo5aeFw4/FM= -----END PRIVATE KEY----- Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9e - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5e + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Nov 28 19:09:06 2027 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Jul 7 14:23:16 2028 GMT Subject: C=XY, L=Castle Anthrax, O=Python Software Foundation, CN=localhost-ecc Subject Public Key Info: Public Key Algorithm: id-ecPublicKey Public-Key: (384 bit) pub: - 04:39:21:52:b3:2c:bc:f8:2c:27:dc:a7:2e:9f:30: - ea:7e:8e:4e:4a:c3:2c:2c:53:7b:a9:3e:d8:c0:e8: - 4d:d4:7a:dc:4f:71:f9:e7:bf:e8:20:85:1c:83:01: - 82:cd:d8:e5:6a:66:02:cc:12:65:28:17:58:f5:48: - cd:50:ad:b8:47:22:e3:5c:57:12:3d:80:f3:cc:76: - e9:9c:34:54:b3:fe:1a:b1:ac:14:6d:03:ff:19:da: - 0c:b0:73:37:4d:2e:37 + 04:b3:cf:fa:c8:9c:a5:33:a2:7a:71:3f:96:cb:c6: + 6a:bc:7e:6a:ed:81:14:92:d2:34:4f:50:e5:35:e9: + 72:97:f8:ab:62:7e:72:db:98:b1:d0:b4:fa:dc:b5: + d9:11:45:13:82:c6:75:11:cf:67:c1:8a:14:13:26: + 45:f2:04:c4:ac:c0:d4:d1:0a:cf:b8:0d:42:9b:30: + 07:e2:86:28:0c:55:88:1d:59:b6:4f:14:10:e5:7a: + 8e:5a:78:5c:38:fc:53 ASN1 OID: secp384r1 NIST CURVE: P-384 X509v3 extensions: @@ -38,11 +38,11 @@ Certificate: X509v3 Basic Constraints: critical CA:FALSE X509v3 Subject Key Identifier: - 33:23:0E:15:04:83:2E:3D:BF:DA:81:6D:10:38:80:C3:C2:B0:A4:74 + C6:82:22:BF:4F:3D:40:AD:9B:16:AD:E7:C5:ED:C4:82:EB:35:97:98 X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 DirName:/C=XY/O=Python Software Foundation CA/CN=our-ca-server - serial:82:ED:BF:41:C8:80:91:9B + serial:CB:2D:80:99:5A:69:52:5B Authority Information Access: CA Issuers - URI:http://testca.pythontest.net/testca/pycacert.cer @@ -53,44 +53,54 @@ Certificate: Full Name: URI:http://testca.pythontest.net/testca/revocation.crl - Signature Algorithm: sha1WithRSAEncryption - 3b:6f:97:af:7e:5f:e0:14:34:ed:57:7e:de:ce:c4:85:1e:aa: - 84:52:94:7c:e5:ce:e9:9c:88:8b:ad:b5:4d:16:ac:af:81:ea: - b8:a2:e2:50:2e:cb:e9:11:bd:1b:a6:3f:0c:a2:d7:7b:67:72: - b3:43:16:ad:c6:87:ac:6e:ac:47:78:ef:2f:8c:86:e8:9b:d1: - 43:8c:c1:7a:91:30:e9:14:d6:9f:41:8b:9b:0b:24:9b:78:86: - 11:8a:fc:2b:cd:c9:13:ee:90:4f:14:33:51:a3:c4:9e:d6:06: - 48:f5:41:12:af:f0:f2:71:40:78:f5:96:c2:5d:cf:e1:38:ff: - bf:10:eb:74:2f:c2:23:21:3e:27:f5:f1:f2:af:2c:62:82:31: - 00:c8:96:1b:c3:7e:8d:71:89:e7:40:b5:67:1a:33:fb:c0:8b: - 96:0c:36:78:25:27:82:d8:27:27:52:0f:f7:69:cd:ff:2b:92: - 10:d3:d2:0a:db:65:ed:af:90:eb:db:76:f3:8a:7a:13:9e:c6: - 33:57:15:42:06:13:d6:54:49:fa:84:a7:0e:1d:14:72:ca:19: - 8e:2b:aa:a4:02:54:3c:f6:1c:23:81:7a:59:54:b0:92:65:72: - c8:e5:ba:9f:03:4e:30:f2:4d:45:85:e3:35:a8:b1:68:58:b9: - 3b:20:a3:eb + Signature Algorithm: sha256WithRSAEncryption + 76:e3:19:4d:34:78:50:3e:fa:63:53:d6:3f:01:87:e8:f4:a3: + a9:81:5b:31:d6:de:3a:98:f3:bb:70:4d:29:35:1f:b0:6a:b3: + 9d:bf:03:2b:79:c4:f2:0b:32:f8:fc:f6:cb:13:47:28:81:fa: + 96:b3:1a:1d:bd:4b:f6:35:df:87:ef:6e:74:63:87:3d:7e:2b: + c6:78:d4:8e:ef:03:e6:01:11:22:4e:1b:ef:2c:c1:c5:4e:3f: + 4a:07:ae:92:ef:d3:ac:79:59:7c:60:89:4b:3d:39:08:ef:c4: + 9a:dc:b0:8b:ee:5f:30:40:d3:c2:f3:f8:90:77:9d:8c:a7:07: + b9:5f:62:83:4d:37:fa:36:e1:1d:26:2b:cc:8f:7c:6f:f1:23: + 87:71:48:40:ad:6b:30:16:47:4c:d7:98:bb:f5:9b:63:c8:66: + 47:65:58:d2:c1:07:81:14:0c:25:20:87:b9:1d:ab:0b:56:db: + 2c:ab:36:db:7f:c7:42:52:af:91:d6:fb:18:cf:94:f7:1e:25: + 99:ce:20:78:c6:f8:69:6e:9c:53:f3:fe:90:3e:4d:ca:d5:d6: + ac:6e:02:17:be:4a:0f:fe:e6:14:d4:ce:25:df:17:8f:6f:b9: + d3:28:dc:b4:98:ef:05:6f:eb:20:14:1c:c1:e9:9d:02:7b:0e: + 0f:e4:a8:bc:3b:62:e0:42:0c:b0:f7:a1:63:fe:98:d7:aa:b0: + f6:ed:ff:ab:4f:1a:9a:8f:eb:f0:86:61:d2:d3:a5:08:d0:db: + e4:d6:a9:0e:ec:08:6f:af:fb:ef:73:3f:47:69:97:90:b2:5a: + 6f:31:66:a7:4c:32:0c:e9:ea:18:ce:a9:79:9c:f5:c4:42:f5: + 68:53:b2:a4:8c:98:3f:97:34:62:61:41:0a:54:d7:0b:cd:33: + c8:62:62:da:f7:07:c6:c6:3b:fa:68:ca:5f:62:3e:57:db:bd: + cb:16:94:07:9a:b5:31:55:b8:f8:cb:b0:7f:a0:d1:82:df:71: + c8:90:60:b3:88:b0 -----BEGIN CERTIFICATE----- -MIIESzCCAzOgAwIBAgIJAILtv0HIgJGeMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIEyzCCAzOgAwIBAgIJAMstgJlaaVJeMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yNzExMjgx -OTA5MDZaMGMxCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA3MDcx +NDIzMTZaMGMxCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj MCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24xFjAUBgNVBAMMDWxv -Y2FsaG9zdC1lY2MwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAAQ5IVKzLLz4LCfcpy6f -MOp+jk5KwywsU3upPtjA6E3UetxPcfnnv+gghRyDAYLN2OVqZgLMEmUoF1j1SM1Q -rbhHIuNcVxI9gPPMdumcNFSz/hqxrBRtA/8Z2gywczdNLjejggHEMIIBwDAYBgNV +Y2FsaG9zdC1lY2MwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASzz/rInKUzonpxP5bL +xmq8fmrtgRSS0jRPUOU16XKX+KtifnLbmLHQtPrctdkRRROCxnURz2fBihQTJkXy +BMSswNTRCs+4DUKbMAfihigMVYgdWbZPFBDleo5aeFw4/FOjggHEMIIBwDAYBgNV HREEETAPgg1sb2NhbGhvc3QtZWNjMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAU -BggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUMyMO -FQSDLj2/2oFtEDiAw8KwpHQwfQYDVR0jBHYwdIAUms/PbutxPds88a6Ia1ZyA8sI -p0ihUaRPME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg -Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcoIJAILtv0HIgJGb +BggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUxoIi +v089QK2bFq3nxe3Egus1l5gwfQYDVR0jBHYwdIAU3b/K2ubRNLo3dSHKb5oIKPI1 +tkihUaRPME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg +Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcoIJAMstgJlaaVJb MIGDBggrBgEFBQcBAQR3MHUwPAYIKwYBBQUHMAKGMGh0dHA6Ly90ZXN0Y2EucHl0 aG9udGVzdC5uZXQvdGVzdGNhL3B5Y2FjZXJ0LmNlcjA1BggrBgEFBQcwAYYpaHR0 cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0Y2Evb2NzcC8wQwYDVR0fBDww OjA4oDagNIYyaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0Y2EvcmV2 -b2NhdGlvbi5jcmwwDQYJKoZIhvcNAQEFBQADggEBADtvl69+X+AUNO1Xft7OxIUe -qoRSlHzlzumciIuttU0WrK+B6rii4lAuy+kRvRumPwyi13tncrNDFq3Gh6xurEd4 -7y+Mhuib0UOMwXqRMOkU1p9Bi5sLJJt4hhGK/CvNyRPukE8UM1GjxJ7WBkj1QRKv -8PJxQHj1lsJdz+E4/78Q63QvwiMhPif18fKvLGKCMQDIlhvDfo1xiedAtWcaM/vA -i5YMNnglJ4LYJydSD/dpzf8rkhDT0grbZe2vkOvbdvOKehOexjNXFUIGE9ZUSfqE -pw4dFHLKGY4rqqQCVDz2HCOBellUsJJlcsjlup8DTjDyTUWF4zWosWhYuTsgo+s= +b2NhdGlvbi5jcmwwDQYJKoZIhvcNAQELBQADggGBAHbjGU00eFA++mNT1j8Bh+j0 +o6mBWzHW3jqY87twTSk1H7Bqs52/Ayt5xPILMvj89ssTRyiB+pazGh29S/Y134fv +bnRjhz1+K8Z41I7vA+YBESJOG+8swcVOP0oHrpLv06x5WXxgiUs9OQjvxJrcsIvu +XzBA08Lz+JB3nYynB7lfYoNNN/o24R0mK8yPfG/xI4dxSECtazAWR0zXmLv1m2PI +ZkdlWNLBB4EUDCUgh7kdqwtW2yyrNtt/x0JSr5HW+xjPlPceJZnOIHjG+GlunFPz +/pA+TcrV1qxuAhe+Sg/+5hTUziXfF49vudMo3LSY7wVv6yAUHMHpnQJ7Dg/kqLw7 +YuBCDLD3oWP+mNeqsPbt/6tPGpqP6/CGYdLTpQjQ2+TWqQ7sCG+v++9zP0dpl5Cy +Wm8xZqdMMgzp6hjOqXmc9cRC9WhTsqSMmD+XNGJhQQpU1wvNM8hiYtr3B8bGO/po +yl9iPlfbvcsWlAeatTFVuPjLsH+g0YLfcciQYLOIsA== -----END CERTIFICATE----- diff --git a/Lib/test/make_ssl_certs.py b/Lib/test/make_ssl_certs.py index b908c40c11ef..362276583f50 100644 --- a/Lib/test/make_ssl_certs.py +++ b/Lib/test/make_ssl_certs.py @@ -69,7 +69,7 @@ dir = cadir database = $dir/index.txt crlnumber = $dir/crl.txt - default_md = sha1 + default_md = sha256 default_days = 3600 default_crl_days = 3600 certificate = pycacert.pem @@ -108,7 +108,7 @@ def make_cert_key(hostname, sign=False, extra_san='', - ext='req_x509_extensions_full', key='rsa:2048'): + ext='req_x509_extensions_full', key='rsa:3072'): print("creating cert for " + hostname) tempnames = [] for i in range(3): @@ -174,7 +174,7 @@ def make_ca(): t.flush() with tempfile.NamedTemporaryFile() as f: args = ['req', '-new', '-days', '3650', '-extensions', 'v3_ca', '-nodes', - '-newkey', 'rsa:2048', '-keyout', 'pycakey.pem', + '-newkey', 'rsa:3072', '-keyout', 'pycakey.pem', '-out', f.name, '-subj', '/C=XY/L=Castle Anthrax/O=Python Software Foundation CA/CN=our-ca-server'] check_call(['openssl'] + args) diff --git a/Lib/test/pycacert.pem b/Lib/test/pycacert.pem index 850fa32aef7e..73150c960f35 100644 --- a/Lib/test/pycacert.pem +++ b/Lib/test/pycacert.pem @@ -2,78 +2,98 @@ Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9b - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5b + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Jan 17 19:09:06 2028 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Aug 26 14:23:16 2028 GMT Subject: C=XY, O=Python Software Foundation CA, CN=our-ca-server Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:c3:18:69:6b:c9:47:29:98:8e:b1:56:c2:2e:fa: - 0e:5e:bc:23:80:b3:07:62:24:d2:42:5b:f1:4a:bf: - a9:c8:21:75:c8:e3:e6:2c:1f:87:3c:6e:7c:1b:ed: - 39:32:95:b7:40:b2:60:48:c3:9a:16:08:fe:6d:67: - 88:34:3b:77:77:70:1c:70:5a:d1:1f:5f:04:21:54: - b9:0c:e3:41:85:1d:58:ee:2f:ed:f3:0e:ef:d8:23: - a1:fa:73:fb:4c:28:e0:e5:e6:4d:0b:02:52:49:86: - c7:be:7e:bd:e6:56:76:8b:70:8e:0a:8f:06:33:20: - 1d:7b:5b:aa:d0:c5:1b:ab:9b:cc:54:09:3c:bf:e4: - 40:66:f1:fb:d6:f7:16:9d:c4:19:d4:c3:f2:ff:07: - bc:6f:5a:9e:25:1b:02:4a:a5:ec:42:96:3a:70:d2: - 6c:99:2b:ce:be:e8:d2:01:ef:d5:ba:b0:cf:94:3e: - 82:d0:01:d6:4b:71:80:03:0a:12:45:86:79:81:d8: - 4b:d2:e8:b5:b7:2c:6c:9a:4c:8a:10:10:e4:e4:f5: - df:ce:84:91:ca:d1:46:e0:84:73:17:66:db:69:43: - 78:80:83:be:14:4d:f1:3e:1a:d6:6c:f5:de:45:f3: - 39:af:91:d5:3d:54:44:bf:41:cc:73:68:1a:fc:24: - db:91 + 00:97:ed:55:41:ba:36:17:95:db:71:1c:d3:e1:61: + ac:58:73:e3:c6:96:cf:2b:1f:b8:08:f5:9d:4b:4b: + c7:30:f6:b8:0b:b3:52:72:a0:bb:c9:4d:3b:8e:df: + 22:8e:01:57:81:c9:92:73:cc:00:c6:ec:70:b0:3a: + 17:40:c1:df:f2:8c:36:4c:c4:a7:81:e7:b6:24:68: + e2:a0:7e:35:07:2f:a0:5b:f9:45:46:f7:1e:f0:46: + 11:fe:ca:1a:3c:50:f1:26:a9:5f:9c:22:9c:f8:41: + e1:df:4f:12:95:19:2f:5c:90:01:17:6e:7e:3e:7d: + cf:e9:09:af:25:f8:f8:42:77:2d:6d:5f:36:f2:78: + 1e:7d:4a:87:68:63:6c:06:71:1b:8d:fa:25:fe:d4: + d3:f5:a5:17:b1:ef:ea:17:cb:54:c8:27:99:80:cb: + 3c:45:f1:2c:52:1c:dd:1f:51:45:20:50:1e:5e:ab: + 57:73:1b:41:78:96:de:84:a4:7a:dd:8f:30:85:36: + 58:79:76:a0:d2:61:c8:1b:a9:94:99:63:c6:ee:f8: + 14:bf:b4:52:56:31:97:fa:eb:ac:53:9e:95:ce:4c: + c4:5a:4a:b7:ca:03:27:5b:35:57:ce:02:dc:ec:ca: + 69:f8:8a:5a:39:cb:16:20:15:03:24:61:6c:f4:7a: + fc:b6:48:e5:59:10:5c:49:d0:23:9f:fb:71:5e:3a: + e9:68:9f:34:72:80:27:b6:3f:4c:b1:d9:db:63:7f: + 67:68:4a:6e:11:f8:e8:c0:f4:5a:16:39:53:0b:68: + de:77:fa:45:e7:f8:91:cd:78:cd:28:94:97:71:54: + fb:cf:f0:37:de:c9:26:c5:dc:1b:9e:89:6d:09:ac: + c8:44:71:cb:6d:f1:97:31:d5:4c:20:33:bf:75:4a: + a0:e0:dc:69:11:ed:2a:b4:64:10:11:30:8b:0e:b0: + a7:10:d8:8a:c5:aa:1b:c8:26:8a:25:e7:66:9f:a5: + 6a:1a:2f:7c:5f:83:c6:78:4f:1f Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Key Identifier: - 9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 X509v3 Basic Constraints: CA:TRUE - Signature Algorithm: sha1WithRSAEncryption - 10:25:c8:dc:0c:55:5c:cb:83:6e:79:ef:77:ec:0d:8e:0c:06: - c1:4b:0c:d6:f7:75:52:21:b8:17:4a:38:88:9d:b3:78:c4:42: - fb:b8:7c:14:38:10:fb:ac:da:11:00:5b:42:87:5e:45:9f:6d: - 4e:42:a4:9a:18:06:39:0f:45:a6:96:89:32:d6:59:b3:d3:8e: - e3:95:b6:c4:a2:4b:74:2f:67:c1:fb:bb:f9:72:6f:37:4a:e7: - f4:48:33:71:df:b8:f5:e6:41:3f:d5:d5:2f:26:09:f8:0e:92: - ff:70:ea:f6:ab:58:fb:90:04:d6:43:2e:8f:b1:fb:06:ab:69: - d0:dc:a8:f8:5b:07:f2:d4:66:1f:63:f8:5d:c1:9e:41:44:bb: - c9:e8:7d:e0:46:e4:a7:c8:32:5f:31:62:e5:1c:5c:89:dd:b7: - a2:4f:9e:0d:13:b8:5f:b1:84:53:4c:1f:ce:19:e1:01:00:5e: - bf:41:55:94:a9:a5:13:db:f2:59:f3:d6:4e:b9:9d:9d:b9:0a: - d9:b2:18:6d:7c:b1:f7:96:aa:bd:f6:f9:95:0f:4a:6e:3c:7c: - 46:5b:df:d4:78:ec:9a:dc:e2:e3:01:e6:88:77:39:93:9c:ba: - 2a:63:f9:25:4b:4f:ac:08:79:39:c6:7b:df:07:35:ba:c0:c2: - 50:bf:5a:81 + Signature Algorithm: sha256WithRSAEncryption + 33:6a:54:d3:6b:c0:d7:01:5f:9d:f4:05:c1:93:66:90:50:d0: + b7:18:e9:b0:1e:4a:a0:b6:da:76:93:af:84:db:ad:15:54:31: + 15:13:e4:de:7e:4e:0c:d5:09:1c:34:35:b6:e5:4c:d6:6f:65: + 7d:32:5f:eb:fc:a9:6b:07:f7:49:82:e5:81:7e:07:80:9a:63: + f8:2c:c3:40:bc:8f:d4:2a:da:3e:d1:ee:08:b7:4d:a7:84:ca: + f4:3f:a1:98:45:be:b1:05:69:e7:df:d7:99:ab:1b:ee:8b:30: + cc:f7:fc:e7:d4:0b:17:ae:97:bf:e4:7b:fd:0f:a7:b4:85:79: + e3:59:e2:16:87:bf:1f:29:45:2c:23:93:76:be:c0:87:1d:de: + ec:2b:42:6a:e5:bb:c8:f4:0a:4a:08:0a:8c:5c:d8:7d:4d:d1: + b8:bf:d5:f7:29:ed:92:d1:94:04:e8:35:06:57:7f:2c:23:97: + 87:a5:35:8d:26:d3:1a:47:f2:16:d7:d9:c6:d4:1f:23:43:d3: + 26:99:39:ca:20:f4:71:23:6f:0c:4a:76:76:f7:76:1f:b3:fe: + bf:47:b0:fc:2a:56:81:e1:d2:dd:ee:08:d8:f4:ff:5a:dc:25: + 61:8a:91:02:b9:86:1c:f2:50:73:76:25:35:fc:b6:25:26:15: + cb:eb:c4:2b:61:0c:1c:e7:ee:2f:17:9b:ec:f0:d4:a1:84:e7: + d2:af:de:e4:1b:24:14:a7:01:87:e3:ab:29:58:46:a0:d9:c0: + 0a:e0:8d:d7:59:d3:1b:f8:54:20:3e:78:a5:a5:c8:4f:8b:03: + c4:96:9f:ec:fb:47:cf:76:2d:8d:65:34:27:bf:fa:ae:01:05: + 8a:f3:92:0a:dd:89:6c:97:a1:c7:e7:60:51:e7:ac:eb:4b:7d: + 2c:b8:65:c9:fe:5d:6a:48:55:8e:e4:c7:f9:6a:40:e1:b8:64: + 45:e9:b5:59:29:a5:5f:cf:7d:58:7d:64:79:e5:a4:09:ac:1e: + 76:65:3d:94:c4:68 -----BEGIN CERTIFICATE----- -MIIDbTCCAlWgAwIBAgIJAILtv0HIgJGbMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIEbTCCAtWgAwIBAgIJAMstgJlaaVJbMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yODAxMTcx -OTA5MDZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg -Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCASIwDQYJKoZI -hvcNAQEBBQADggEPADCCAQoCggEBAMMYaWvJRymYjrFWwi76Dl68I4CzB2Ik0kJb -8Uq/qcghdcjj5iwfhzxufBvtOTKVt0CyYEjDmhYI/m1niDQ7d3dwHHBa0R9fBCFU -uQzjQYUdWO4v7fMO79gjofpz+0wo4OXmTQsCUkmGx75+veZWdotwjgqPBjMgHXtb -qtDFG6ubzFQJPL/kQGbx+9b3Fp3EGdTD8v8HvG9aniUbAkql7EKWOnDSbJkrzr7o -0gHv1bqwz5Q+gtAB1ktxgAMKEkWGeYHYS9LotbcsbJpMihAQ5OT1386EkcrRRuCE -cxdm22lDeICDvhRN8T4a1mz13kXzOa+R1T1URL9BzHNoGvwk25ECAwEAAaNQME4w -HQYDVR0OBBYEFJrPz27rcT3bPPGuiGtWcgPLCKdIMB8GA1UdIwQYMBaAFJrPz27r -cT3bPPGuiGtWcgPLCKdIMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEB -ABAlyNwMVVzLg25573fsDY4MBsFLDNb3dVIhuBdKOIids3jEQvu4fBQ4EPus2hEA -W0KHXkWfbU5CpJoYBjkPRaaWiTLWWbPTjuOVtsSiS3QvZ8H7u/lybzdK5/RIM3Hf -uPXmQT/V1S8mCfgOkv9w6varWPuQBNZDLo+x+waradDcqPhbB/LUZh9j+F3BnkFE -u8nofeBG5KfIMl8xYuUcXIndt6JPng0TuF+xhFNMH84Z4QEAXr9BVZSppRPb8lnz -1k65nZ25CtmyGG18sfeWqr32+ZUPSm48fEZb39R47Jrc4uMB5oh3OZOcuipj+SVL -T6wIeTnGe98HNbrAwlC/WoE= +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA4MjYx +NDIzMTZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg +Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCAaIwDQYJKoZI +hvcNAQEBBQADggGPADCCAYoCggGBAJftVUG6NheV23Ec0+FhrFhz48aWzysfuAj1 +nUtLxzD2uAuzUnKgu8lNO47fIo4BV4HJknPMAMbscLA6F0DB3/KMNkzEp4HntiRo +4qB+NQcvoFv5RUb3HvBGEf7KGjxQ8SapX5winPhB4d9PEpUZL1yQARdufj59z+kJ +ryX4+EJ3LW1fNvJ4Hn1Kh2hjbAZxG436Jf7U0/WlF7Hv6hfLVMgnmYDLPEXxLFIc +3R9RRSBQHl6rV3MbQXiW3oSket2PMIU2WHl2oNJhyBuplJljxu74FL+0UlYxl/rr +rFOelc5MxFpKt8oDJ1s1V84C3OzKafiKWjnLFiAVAyRhbPR6/LZI5VkQXEnQI5/7 +cV466WifNHKAJ7Y/TLHZ22N/Z2hKbhH46MD0WhY5Uwto3nf6Ref4kc14zSiUl3FU ++8/wN97JJsXcG56JbQmsyERxy23xlzHVTCAzv3VKoODcaRHtKrRkEBEwiw6wpxDY +isWqG8gmiiXnZp+lahovfF+DxnhPHwIDAQABo1AwTjAdBgNVHQ4EFgQU3b/K2ubR +NLo3dSHKb5oIKPI1tkgwHwYDVR0jBBgwFoAU3b/K2ubRNLo3dSHKb5oIKPI1tkgw +DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAYEAM2pU02vA1wFfnfQFwZNm +kFDQtxjpsB5KoLbadpOvhNutFVQxFRPk3n5ODNUJHDQ1tuVM1m9lfTJf6/ypawf3 +SYLlgX4HgJpj+CzDQLyP1CraPtHuCLdNp4TK9D+hmEW+sQVp59/Xmasb7oswzPf8 +59QLF66Xv+R7/Q+ntIV541niFoe/HylFLCOTdr7Ahx3e7CtCauW7yPQKSggKjFzY +fU3RuL/V9yntktGUBOg1Bld/LCOXh6U1jSbTGkfyFtfZxtQfI0PTJpk5yiD0cSNv +DEp2dvd2H7P+v0ew/CpWgeHS3e4I2PT/WtwlYYqRArmGHPJQc3YlNfy2JSYVy+vE +K2EMHOfuLxeb7PDUoYTn0q/e5BskFKcBh+OrKVhGoNnACuCN11nTG/hUID54paXI +T4sDxJaf7PtHz3YtjWU0J7/6rgEFivOSCt2JbJehx+dgUees60t9LLhlyf5dakhV +juTH+WpA4bhkRem1WSmlX899WH1keeWkCawedmU9lMRo -----END CERTIFICATE----- diff --git a/Lib/test/pycakey.pem b/Lib/test/pycakey.pem index 16b75879d09b..c283f8909868 100644 --- a/Lib/test/pycakey.pem +++ b/Lib/test/pycakey.pem @@ -1,28 +1,40 @@ -----BEGIN PRIVATE KEY----- -MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDDGGlryUcpmI6x -VsIu+g5evCOAswdiJNJCW/FKv6nIIXXI4+YsH4c8bnwb7TkylbdAsmBIw5oWCP5t -Z4g0O3d3cBxwWtEfXwQhVLkM40GFHVjuL+3zDu/YI6H6c/tMKODl5k0LAlJJhse+ -fr3mVnaLcI4KjwYzIB17W6rQxRurm8xUCTy/5EBm8fvW9xadxBnUw/L/B7xvWp4l -GwJKpexCljpw0myZK86+6NIB79W6sM+UPoLQAdZLcYADChJFhnmB2EvS6LW3LGya -TIoQEOTk9d/OhJHK0UbghHMXZttpQ3iAg74UTfE+GtZs9d5F8zmvkdU9VES/Qcxz -aBr8JNuRAgMBAAECggEAZHgv4hg3k45C/cSmH7caq2LMDb0kskAwH4hlzI7DipLg -q2Hh6Rsbc92aAG+8IvbC9ohl2VMSCQL8s667j9qH/XQ40QuT4kn2QIv2+FIYLcsd -Pxxjt+YbUf2XrvkHkwMCPqLJTkAVzFOijdGLThF83vZJz9oz4SRKyno8j2LSix68 -WEfnjdyWqYb0eS0luKrLHw+IL7bD5vfc/P0q6u31zJ9h8zEyN5EBCj5OxM/hD0VO -nObrp6r9Bs+xx+yRx+8J5Db6LPXggl5nBqsqrDKVDe6uTysYVgstqkfaDv1L78Vu -3BNdKPAdJ+ucPJrQufzFHBDIIN+Xwckf/09gdQagGQKBgQDnvFaOjZfqc6wL/kNK -tszQtedbdwP20L+EWdNEVsVWK1TOw36Pmkrp2AYLXMd7W1QQu0KukM89EFb84wKo -s4C9V/ch162mUhEAveaLioi7bMwMPIib2V6pHmYGG8nQVRvgkZVYx6ZtPEvWye1v -wmCzzxxK0gC6PQGxp8MSv9yXDwKBgQDXhe57ufc52pgJ+Agyl4PLkllIbG2DKQHG -LwY06v73jllirTpWBOBvN0NvEsI2Pj4aK/BXRNYN1PS7xi/3C6MVWxnOpBtbq3H5 -DwFb5mpfgJmhV6DZ6jMw7h3Yvy35ViKoiI9UK3eTmhkerH3DsILEje7jE9dGmIOJ -4oLa50JjXwKBgQDdTfyveMNasIrejTzATmC89Or0a22KuQIdKBddjSw5xXnhV8s2 -4temCJqFIV6UDLz0mZDt2vc+zqr0KOtyJrLMoAQv+qQoUPlR5wkTvAImU5luGiUw -CN+gzJoMPV93KMBNr1qcBVaHvWyDvCWXdF8beLABOBpfwUEr4xWlgzrruwKBgCvf -cr2zDJW1Xu/gkuKhn02ofA5XLC/gACF03yGUmNSSILYKp25tTba2HD8XJXvfTcsM -GL/bHmvwZuV2obr7nnYxdl5vX7ZYfzoBCPjJPew1BJEognD50PPr9R1zRYuVMjb2 -nZ63vn7IhsaMvIlCfExAzFljZ5ZSY6yE9LhVDVmnAoGBALOwMwpkm1drx5UNSJO7 -70Q8kYzg0oQhCo/7b6DWbAglDPSWQS5IA4rHYOwL3sE+69G2Exe+1454rVDisojW -XdSyA3svI/YQeom8R2LIM/ayCPxCc3/Dxy9+aQQT4lW3F0XQIxod/QsQJxpZIOnF -jOSPclypgV2X6dDOwDkd2Tgh +MIIG/AIBADANBgkqhkiG9w0BAQEFAASCBuYwggbiAgEAAoIBgQCX7VVBujYXldtx +HNPhYaxYc+PGls8rH7gI9Z1LS8cw9rgLs1JyoLvJTTuO3yKOAVeByZJzzADG7HCw +OhdAwd/yjDZMxKeB57YkaOKgfjUHL6Bb+UVG9x7wRhH+yho8UPEmqV+cIpz4QeHf +TxKVGS9ckAEXbn4+fc/pCa8l+PhCdy1tXzbyeB59SodoY2wGcRuN+iX+1NP1pRex +7+oXy1TIJ5mAyzxF8SxSHN0fUUUgUB5eq1dzG0F4lt6EpHrdjzCFNlh5dqDSYcgb +qZSZY8bu+BS/tFJWMZf666xTnpXOTMRaSrfKAydbNVfOAtzsymn4ilo5yxYgFQMk +YWz0evy2SOVZEFxJ0COf+3FeOulonzRygCe2P0yx2dtjf2doSm4R+OjA9FoWOVML +aN53+kXn+JHNeM0olJdxVPvP8DfeySbF3BueiW0JrMhEcctt8Zcx1UwgM791SqDg +3GkR7Sq0ZBARMIsOsKcQ2IrFqhvIJool52afpWoaL3xfg8Z4Tx8CAwEAAQKCAYB6 +1g1kwyYRE70FS4WUaOBr8+dqHW0LNO4bcFrpEi/PSuurqiUzQPoT3DoXXhoWLseN +zGh476yBKZJDKfS7CwYCmZMdprK4uZvu/E6f7Or7EGrbckOtCQkew8iw9L8ZnWgd +FjyThPjdUIdLgidIHcDJWjVHuLKh3B9KD+ZpEU/IjYtRLvbCPJSKQMQShrBE1Rau +SF6IF5P4vK7X0162NlQqMLpQBAKLml93VJcERzVY1u53JJnkG1loIrNvE32zvZ0C +ZnGVpWwamixVrO9K66F+Ml3Y3bkguF8aPUitc+l+yPmUSXmcDcKmhw9DZA0k5t39 +FxVYGn1uJlvHll8QvV9aZtzuTSkAN8VWNufGwivLLsoRlRb1LtGWqHY583FlUWtz +99ABCBehZ2EpAP+MrRj+pyKuMrkQH61bbOhjqifBM8MhHdn9uNmMpvnKyGPMIdRY +cLH4i2S5aQVvmsQbOa98DLOFGXdf+z5HuwdxHtG1S3J7jkT+FkIyYDehJA3X8UEC +gcEAyCpD8rMFfR5qLwrajhy8vbV7TIkNfFHEkm9tCWDBHwuOJqA0DFuMDAKl7cMv +Uo7Z6R2Fqe2OyWvxYkOi/CSjvtT+PTiA0ux1tXUZcxcRSIsLqQZV+elsKcv+QJPy +vf02vNvHjaMaRwl+NYtqpfr1s/3EdJnWCNC3nV1dD+mWVJoO3kGAK5grLAPM1/uP +stARN5Tnh3Doh8e1Yl4V4UKcVqyVqDykX1OLSmPqNH86T4Um0B4h+jf4UBBdDBz1 +rD3JAoHBAMJOZ3M7LqX+F2haSrF/hnG1y9qAqDWGsvy+8YgjFwPFWu7LvqLuXLuz +S3+5GGhplMuM0itqA9PyPotlgtG5O9hAU8SyMitrx1uTW+Q2U3iYPZQ9O327l1cO +F2jKljq0aJrXp+5iWUq8t/FG6DAqYYUCY/X1SheqEXCgCh4ldRhXig3TBYbVZNs9 +7azN0lk408AO/Hx7WYreFQVS7A/EJhk/M1yyIqnJESuxkDefjV4hTVkRXhh+MrCe +vF/jHqh5pwKBwHxXPQRbzvINXbrBTEjxcxGJ1gESNg1fIfQxQZOMxgrJ+9DkvdBb +YiDn2DldgV0Qni8ghrKrfoKDClyXVXy6KfnWh+Rx4BymhOxmxJto3fSpY2HpLKll +JirErLli7my1CjbBdDH4+s7cB8mtRF+9CLp5znr8QSgSt60KnU/QM/F0Df5kxADQ +syjRZ4NXoslaVQeo+TZ6nggSuAtWFNNstH9nEESE/zq0RBe+/3MDAa76MMUhosuz +zw21TIfEyZvoeQKBwDpszNpvPzWWU3+DNtZsXAaw/Vz0Np/xorwwxfuDYZY2r4MC +LI5dUfD2losPIvGyXZVfAIshU4lVW80adt2M7xu1K/sHAeLgg49bndPfKfYnAM0k +JFFIKNd6WzudPtLkEFgO5GXfmK3KVRztjz98XtpZv6jjWqYG8zuEQ8aQyMbK+63w +d8b1P2BVHLRLJybA2Zr0ZqMfi+sfn/570pNjDXml8VG8FoQq+0jCGXVAOofFR7ay +bDK9L4zADjBe4IcUHQKBwFwj8TEVlWxtG+IWO5d+vyuP0OPjSYOmYq4dCMyZ2+Xy +Y+XDYEhlgGTVxafRMTwt57VV3hJTtRxUZziDA++atr8+gPio+QHBYg1JgCKsqKYL +TGoSVrM1jTfdl1orwkpgQmq2q5j7ExpNA3Spsm5kyCaJ1S/8Ivusqaod8S4t7UhW +BRec3gQ+UYv/V3Pc9hS1Zdzf5+G+PDOYoNmveY16hcu0DKc/PlqGtJuBoQjjH7ir +1YVK9GAgLk0VqJvePnF57A== -----END PRIVATE KEY----- diff --git a/Lib/test/revocation.crl b/Lib/test/revocation.crl index 53cb4b3721d9..c05461ca7f93 100644 --- a/Lib/test/revocation.crl +++ b/Lib/test/revocation.crl @@ -1,11 +1,14 @@ -----BEGIN X509 CRL----- -MIIBpjCBjwIBATANBgkqhkiG9w0BAQUFADBNMQswCQYDVQQGEwJYWTEmMCQGA1UE +MIICJjCBjwIBATANBgkqhkiG9w0BAQsFADBNMQswCQYDVQQGEwJYWTEmMCQGA1UE CgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0ExFjAUBgNVBAMMDW91ci1j -YS1zZXJ2ZXIXDTE4MDExOTE5MDkwNloXDTI3MTEyODE5MDkwNlqgDjAMMAoGA1Ud -FAQDAgEAMA0GCSqGSIb3DQEBBQUAA4IBAQBYVzH8n2LdyJJ/t8CpWjz652hZJ0sY -DeNYcwuTPzR9CbSwEwNbf0kY9bgWXGfoRD2SnnCnuNNJXO2MuXtxf6qYx2ZjhJm8 -qgxXs0Bz4agRswNMbumjHCmqIv1t88vbrO0+ItEZDK7RJVIMBtVJ0XYOHvD/IG/i -zqa1Fl3uCTvQbTJ2TrqzJeP/Vl40hOD+VdBBZK3j0r4AkCKU3tAiHYTGmHKhPxy1 -f8Yet+4SRMGp1BdDezTI1bICpSZhRJ4geW0UzuCZnXPW8IZzioUmdUBAmAMHPWFr -B0sTTc/ntD4jHG1/T5b0oiDMbXIbh5Iz9iQNcY0IbotkCw39h+K90wY6 +YS1zZXJ2ZXIXDTE4MDgyOTE0MjMxNloXDTI4MDcwNzE0MjMxNlqgDjAMMAoGA1Ud +FAQDAgEAMA0GCSqGSIb3DQEBCwUAA4IBgQCPhrtGSbuvxPAI3YWQFDB4iOWdBnVk +ugW1lsifmCsE86FfID0EwUut1SRHlksltMtcoULMEIdu8yMLWci++4ve22EEuMKT +HUc3T/wBIuQUhA7U4deFG8CZPAxRpNoK470y7dkD4OVf0Gxa6WYDl9z8mXKmWCB9 +hvzqVfLWNSLTAVPsHtkD5PXdi5yRkQr6wYD7poWaIvkpsn7EKCY6Tw5V3rsbRuZq +AGVCq5TH3mctcmwLloCJ4Xr/1q0DsRrYxeeLYxE+UpvvCbVBKgtjBK7zINS7AbcJ +CYCYKUwGWv1fYKJ+KQQHf75mT3jQ9lWuzOj/YWK4k1EBnYmVGuKKt73lLFxC6h3y +MUnaBZc1KZSyJj0IxfHg/o6qx8NgKOl9XRIQ5g5B30cwpPOskGhEhodbTTY3bPtm +RQ36JvQZngzmkhyhr+MDEV5yUTOShfUiclzQOx26CmLmLHWxOZgXtFZob/oKrvbm +Gen/+7K7YTw6hfY52U7J2FuQRGOyzBXfBYQ= -----END X509 CRL----- diff --git a/Lib/test/ssl_cert.pem b/Lib/test/ssl_cert.pem index b1dd3f387f7c..de596717bd85 100644 --- a/Lib/test/ssl_cert.pem +++ b/Lib/test/ssl_cert.pem @@ -1,20 +1,26 @@ -----BEGIN CERTIFICATE----- -MIIDWTCCAkGgAwIBAgIJAPm6B21bar2bMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV +MIIEWTCCAsGgAwIBAgIJAJinz4jHSjLtMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODAx -MTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH +IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODA4 +MjkxNDIzMTVaFw0yODA4MjYxNDIzMTVaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH DA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5k -YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEP -ADCCAQoCggEBAKvvsX2gEti4shve3iYMc+jE4Se7WHs1Bol2f21H8qNboDOFdeb1 -RKHjmq3exHpajywOUEgne9nKHJY/3f2phR4Y5klqG6liLgiSpVyRlcBGbeT2qEAj -9oLiLFUXLGfGDds2mTwivQDLJBWi51j7ff5k2Pr58fN5ugYMn24T9FNyn0moT+qj -SFoBNm58l9jrdkJSlgWfqPlbiMa+mqDn/SFtrwLF2Trbfzu42Sd9UdIzMaSSrzbN -sGm53pNhCh8KndWUQ8GPP2IsLPoUU4qAtmZuTxCx2S1cXrN9EkmT69tlOH84YfSn -96Ih9bWRc7M5y5bfVdEVM+fKQl3hBRf05qMCAwEAAaMYMBYwFAYDVR0RBA0wC4IJ -bG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQAtQ8f37cCEk7/rAcbYR53ce3iK -Vpihb0U2ni1QjG9Tg9UIExkIGkwTiCm7kwQL+GEStBu9AG/QVrOjeTriRiddhWkk -ze8kRaI3AC/63t6Vh9Q1x6PESgeE4OtAO9JpJCf4GILglA789Y/b/GF8zJZQxR13 -qpB4ZwWw7gCBhdEW59u6CFeBmfDa58hM8lWvuVoRrTi7bjUeC6PAn5HVMzZSykhu -4HaUfBp6bKFjuym2+h/VvM1n8C3chjVSmutsLb6ELdD8IK0vPV/yf5+LN256zSsS -dyUZYd8XwQaioEMKdbhLvnehyzHiWfQIUR3BdhONxoIJhHv/EAo8eCkHHYIF +YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGP +ADCCAYoCggGBALKUqUtopT6E68kN+uJNEt34i2EbmG/bwjcD8IaMsgJPSsMO2Bpd +3S6qWgkCeOyCfmAwBxK2kNbxGb63ouysEv7l8GCTJTWv3hG/HQcejJpnAEGi6K1U +fDbyE/db6yZ12SoHVTGkadN4vYGCPd1Wj9ZO1F877SHQ8rDWX3xgTWkxN2ojBw44 +T8RHSDiG8D/CvG4uEy+VUszL+Uvny5y2poNSqvI3J56sptWSrh8nIIbkPZPBdUne +LYMOHTFK3ZjXSmhlXgziTxK71nnzM3Y9K9gxPnRqoXbvu/wFo55hQCkETiRkYgmm +jXcBMZ0TClQVnQWuLjMthRnWFZs4Lfmwqjs7FZD/61581R2BYehvpWbLvvuOJhwv +DFzexL2sXcAl7SsxbzeQKRHqGbIDfbnQTXfs3/VC6Ye5P82P2ucj+XC32N9piRmO +gCBP8L3ub+YzzdxikZN2gZXXE2jsb3QyE/R2LkWdWyshpKe+RsZP1SBRbHShUyOh +yJ90baoiEwj2mwIDAQABoxgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZI +hvcNAQELBQADggGBAHRUO/UIHl3jXQENewYayHxkIx8t7nu40iO2DXbicSijz5bo +5//xAB6RxhBAlsDBehgQP1uoZg+WJW+nHu3CIVOU3qZNZRaozxiCl2UFKcNqLOmx +R3NKpo1jYf4REQIeG8Yw9+hSWLRbshNteP6bKUUf+vanhg9+axyOEOH/iOQvgk/m +b8wA8wNa4ujWljPbTQnj7ry8RqhTM0GcAN5LSdSvcKcpzLcs3aYwh+Z8e30sQWna +F40sa5u7izgBTOrwpcDm/w5kC46vpRQ5fnbshVw6pne2by0mdMECASid/p25N103 +jMqTFlmO7kpf/jpCSmamp3/JSEE1BJKHwQ6Ql4nzRA2N1mnvWH7Zxcv043gkHeAu +0x8evpvwuhdIyproejNFlBpKmW8OX7yKTCPPMC/VkX8Q1rVkxU0DQ6hmvwZlhoKa +9Wc2uXpw9xF8itV4Uvcdr3dwqByvIqn7iI/gB+4l41e0u8OmH2MKOx4Nxlly5TNW +HcVKQHyOeyvnINuBAQ== -----END CERTIFICATE----- diff --git a/Lib/test/ssl_key.passwd.pem b/Lib/test/ssl_key.passwd.pem index 669c7ce3c815..e4f1370ab270 100644 --- a/Lib/test/ssl_key.passwd.pem +++ b/Lib/test/ssl_key.passwd.pem @@ -1,30 +1,42 @@ -----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED -DEK-Info: DES-EDE3-CBC,2D5DD30B9D440DBB +DEK-Info: DES-EDE3-CBC,8064BE1494B24B13 -01gIwpy3XxPGsY0PoK59vAxdLhkVj3odO0Z1ULamUzIte6ThKL1HqnZiUlXpYKfK -XqHVVeQ1xouxiDRNFLJ4CqBG4HbRtqTkl+sfaNTVveL18lOOMAZy6W3dCGAnWOTZ -Z0RJyZlQxxjNQLuko4tIvFkrShXIgdiVFjwAhRU0KTUb7UQ2xfFA9R0Kfde30pzz -zSjb/OmYqAIhkdvafGXvJxzZAorQkU9akDh+uJ6cht5B/RGZsbKACYDSv2WSV5yW -r+fKVYcTup33r0Jj8hAD6fVY15K8BJknpkF9HfSlZnmmr2WDaffLokOOnCV/I1ie -WD7ENA7K//48km5D3Ogh2b2/0Iwuzjq8Mvd8aR39N9nINbGR+HNT85pawoo1S0W9 -pQTU4XTmxfXjtR2287C6XZyQ/tBwvNDMFPVhlxsGOdLYwoV5e/L1t1qIfkTlbuvd -JaMzOhSSLjiC156IFoH7PTPe+g75hw2b32XJURFGlaYknHF7P4BmCiwXOQYo5CCo -MQGGlw5qBCqODrIsc03wpL2jUzgvyPqLyaw395ITuSoGX+WO7vUQaGW0Tz/sOoTs -3pK+bTi2QmqZMe7xBOj07CYMMOo4QPrM6NpbObt+Jja2UXaxvKa9BwqCEQzA4pQZ -8ZHHfEWIaDffKTGkAlqm+S8qCtsrEZJhIn3aI/ikzK8v+YkWw6w+8t/tR1V8ET/s -CoYGIR7I8WhdfKAwgx2QT5bt1jkYKJyKPm4Iacp2mNh9gNFVq+JSKF318e7BrR3+ -wyqMkDxRYnov3ybtf6kiICxPREDqa6UG1xRq3SbWz6NnIF/1hoHs79YlSYbMfXNU -ffIxBaXNCcH6jM9duP2YRnO29jLwfLM/mmokTBBjyOBaKZia9GPa4naXoATXW3z+ -Xx4EKIUkKdb53kiV6NtEKMPialAnkeoHTEjyLPgaV8mCHLvGQbnxbYwvpPJH0e2f -CWgiw6ci4ROOzcZ7HJHIDUprwK0xRKn43hoI44fivlSHOFX6da6o3wIqhEUqMKwL -JQDS1GORRk1ndRXP+7Ub1dO+Vo/DqO1VcTr2o5RwZ1LWPnzLqbCG50mvTLH4djB+ -+hf6vlmnFC30N3yUFXWE5vS10nJHYP88dD9CB2RsaWzpxD9Zxl+PKcRsppen6HyO -u3b71a/TBOkJcI+lkOatEFvbuqzBAqhMceMctO+Dl55RFsbxfIw/IXZjdP0PYZ0C -t20DrIdBsvl9F/mfYpmkV4DF7yci78DqnRBcxylVNF2vwX7o+2fq/TsEwsHn3KnT -kvcF5Cq8Vr5C8ugWX8JfveNym0BjLu6Lr58qS4a6qCNGEGPFKyB+xkm4KEScbarQ -aLbEbfulMM7q9//sEOOLexIx7mNoLd29Xzn5hsLCAZLWX6wMq6JVJ/zbBOAHDbBT -yhi03yd5Kvw3swSt4QZj+uR3qTFwxkXUFiVvrSfxRZoyKsxsLr9Z7D8aoH9Rkb2L -6KjZ31nt9Drh7NJfh6ReANBW6INdDW0Y2mbzoDozLszAYjVfuUUEE76iJqXY0N4W -kNr0OQQTUtDpVk0AZZZvy17xV+rkqGgwlOqTvHbwFYEQvgwVz4EKUw== +KJrffOMbo8M0I3PzcYxRZGMpKD1yB3Ii4+bT5XoanxjIJ+4fdx6LfZ0Rsx+riyzs +tymsQu/iYY9j+4rCvN9+eetsL1X6iZpiimKsLexcid9M3fb0vxED5Sgw0dvunCUA +xhqjLIKR92MKbODHf6KrDKCpsiPbjq4gZ7P+uCGXAMHL3MXIJSC0hW9rK7Ce6oyO +CjpIcgB8x+GUWZZZhAFdlzIHMZrteNP2P5HK6QcaT71P034Dz1hhqoj4Q0t+Fta2 +4tfsM/bnTR/l6hwlhPa1e3Uj322tDTDWBScgWANn5+sEWldLmozMaWhZsn22pfk2 +KjRMGXG024JVheV882nbdOBvG7oq+lxkZ/ZP+vvqJqnvYtf7WtM8UivzYpe5Hz5b +kVvWzPjBLUSZ9whM9rDLqSSqMPyPvDTuEmLkuq+xm7pYJmsLqIMP2klZLqRxLX6K +uqwplb8UG440qauxgnQ905PId1l2fJEnRtV+7vXprA0L0QotgXLVHBhLmTFM+3PH +9H3onf31dionUAPrn3nfVE36HhvVgRyvDBnBzJSIMighgq21Qx/d1dk0DRYi1hUI +nCHl0YJPXheVcXR7JiSF2XQCAaFuS1Mr7NCXfWZOZQC/0dkvmHnl9DUAhuqq9BNZ +1cKhZXcKHadg2/r0Zup/oDzmHPUEfTAXT0xbqoWlhkdwbF2veWQ96A/ncx3ISTb4 +PkXBlX9rdia8nmtyQDQRn4NuvchbaGkj4WKFC8pF8Hn7naHqwjpHaDUimBc0CoQW +edNJqruKWwtSVLuwKHCC2gZFX9AXSKJXJz/QRSUlhFGOhuF/J6yKaXj6n5lxWNiQ +54J+OP/hz2aS95CD2+Zf1SKpxdWiLZSIQqESpmmUrXROixNJZ/Z7gI74Dd9dSJOH +W+3AU03vrrFZVrJVZhjcINHoH1Skh6JKscH18L6x4U868nSr4SrRLX8BhHllOQyD +bmU+PZAjF8ZBIaCtTGulDXD29F73MeAZeTSsgQjFu0iKLj1wPiphbx8i/SUtR4YP +X6PVA04g66r1NBw+3RQASVorZ3g1MSFvITHXcbKkBDeJH2z1+c6t/VVyTONnQhM5 +lLgRSk6HCbetvT9PKxWrWutA12pdBYEHdZhMHVf2+xclky7l09w8hg2/qqcdGRGe +oAOZ72t0l5ObNyaruDKUS6f4AjOyWq/Xj5xuFtf1n3tQHyslSyCTPcAbQhDfTHUx +vixb/V9qvYPt7OCn8py7v1M69NH42QVFAvwveDIFjZdqfIKBoJK2V4qPoevJI6uj +Q5ByMt8OXOjSXNpHXpYQWUiWeCwOEBXJX8rzCHdMtg37jJ0zCmeErR1NTdg+EujM +TWYgd06jlT67tURST0aB2kg4ijKgUJefD313LW1zC6gVsTbjSZxYyRbPfSP6flQB +yCi1C19E2OsgleqbkBVC5GlYUzaJT7SGjCRmGx1eqtbrALu+LVH24Wceexlpjydl ++s2nf/DZlKun/tlPh6YioifPCJjByZMQOCEfIox6BkemZETz8uYA4TTWimG13Z03 +gyDGC2jdpEW414J2qcQDvrdUgJ+HlhrAAHaWpMQDbXYxBGoZ+3+ORvQV4kAsCwL8 +k3EIrVpePdik+1xgOWsyLj6QxFXlTMvL6Wc5pnArFPORsgHEolJvxSPTf9aAHNPn +V2WBvxiLBtYpGrujAUM40Syx/aN2RPtcXYPAusHUBw+S8/p+/8Kg8GZmnIXG3F89 +45Eepl2quZYIrou7a1fwIpIIZ0hFiBQ1mlHVMFtxwVHS1bQb3SU2GeO+JcGjdVXc +04qeGuQ5M164eQ5C0T7ZQ1ULiUlFWKD30m+cjqmZzt3d7Q0mKpMKuESIuZJo/wpD +Nas432aLKUhcNx/pOYLkKJRpGZKOupQoD5iUj/j44o8JoFkDK33v2S57XB5QGz28 +9Zuhx49b3W8mbM6EBanlQKLWJGCxXqc/jhYhFWn+b0MhidynFgA0oeWvf6ZDyt6H +Yi5Etxsar09xp0Do3NxtQXLuSUu0ji2pQzSIKuoqQWKqldm6VrpwojiqJhy4WQBQ +aVVyFeWBC7G3Zj76dO+yp2sfJ0itJUQ8AIB9Cg0f34rEZu+r9luPmqBoUeL95Tk7 +YvCOU3Jl8Iqysv8aNpVXT8sa8rrSbruWCByEePZ37RIdHLMVBwVY0eVaFQjrjU7E +mXmM9eaoYLfXOllsQ+M2+qPFUITr/GU3Qig13DhK/+yC1R6V2a0l0WRhMltIPYKW +Ztvvr4hK5LcYCeS113BLiMbDIMMZZYGDZGMdC8DnnVbT2loF0Rfmp80Af31KmMQ4 +6XvMatW9UDjBoY5a/YMpdm7SRwm+MgV2KNPpc2kST87/yi9oprGAb8qiarHiHTM0 -----END RSA PRIVATE KEY----- diff --git a/Lib/test/ssl_key.pem b/Lib/test/ssl_key.pem index b63f38bc5cf2..1ea4578d81ec 100644 --- a/Lib/test/ssl_key.pem +++ b/Lib/test/ssl_key.pem @@ -1,28 +1,40 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCr77F9oBLYuLIb -3t4mDHPoxOEnu1h7NQaJdn9tR/KjW6AzhXXm9USh45qt3sR6Wo8sDlBIJ3vZyhyW -P939qYUeGOZJahupYi4IkqVckZXARm3k9qhAI/aC4ixVFyxnxg3bNpk8Ir0AyyQV -oudY+33+ZNj6+fHzeboGDJ9uE/RTcp9JqE/qo0haATZufJfY63ZCUpYFn6j5W4jG -vpqg5/0hba8Cxdk62387uNknfVHSMzGkkq82zbBpud6TYQofCp3VlEPBjz9iLCz6 -FFOKgLZmbk8QsdktXF6zfRJJk+vbZTh/OGH0p/eiIfW1kXOzOcuW31XRFTPnykJd -4QUX9OajAgMBAAECggEAHppmXDbuw9Z0FVPg9KLIysioTtsgz6VLiZIm8juZK4x2 -glUh/D7xvWL2uDXrgN+3lh7iGUW13LkFx5SMncbbo9TIwI57Z/XKvcnkVwquve+L -RfLFVc1Q5lD9lROv2rS86KTaN4LzYz3FKXi6dvMkpPAsUtfEQhMLkmISypQQq/1z -EJaqo7r85OjN7e0wKazlKZpOzJEa5FQLMVRjTRFhLFNbHXX/tAet2jw+umATKbw8 -hYgiuZ44TwSEd9JeIV/oSYWfI/3HetuYW0ru3caiztRF2NySNu8lcsWgNC7fIku9 -mcHjtSNzs91QN1Qlu7GQvvhpt6OWDirNDCW+49WGaQKBgQDg9SDhfF0jRYslgYbH -cqO4ggaFdHjrAAYpwnAgvanhFZL/zEqm5G1E7l/e2fCkJ9VOSFO0A208chvwMcr+ -dCjHE2tVdE81aQ2v/Eo83VdS1RcOV4Y75yPH48rMhxPaHvxWD/FFDbf0/P2mtPB7 -SZ3kIeZMkE1wxdaO3AKUbQoozwKBgQDDqYgg7kVtygyICE1mB8Hwp6nUxFTczG7y -4XcsDqMIrKmw+PbQluvkoHoStxeVrsTloDhkTjIrpmYLyAiazg+PUJdkd6xrfLSj -VV6X93W0S/1egEb1F1CGFxtk8v/PWH4K76EPL2vxXdxjywz3GWlrL9yDYaB2szzS -DqgwVMqx7QKBgDCD7UF0Bsoyl13RX3XoPXLvZ+SkR+e2q52Z94C4JskKVBeiwX7Y -yNAS8M4pBoMArDoj0xmBm69rlKbqtjLGbnzwrTdSzDpim7cWnBQgUFLm7gAD1Elb -AhZ8BCK0Bw4FnLoa2hfga4oEfdfUMgEE0W5/+SEOBgWKRUmuHUhRc911AoGAY2EN -YmSDYSM5wDIvVb5k9B3EtevOiqNPSw/XnsoEZtiEC/44JnQxdltIBY93bDBrk5IQ -cmoBM4h91kgQjshQwOMXMhFSwvmBKmCm/hrTbvMVytTutXfVD3ZXFKwT4DW7N0TF -ElhsxBh/YzRz7mG62JVjtFt2zDN3ld2Z8YpvtXUCgYEA4EJ4ObS5YyvcXAKHJFo6 -Fxmavyrf8LSm3MFA65uSnFvWukMVqqRMReQc5jvpxHKCis+XvnHzyOfL0gW9ZTi7 -tWGGbBi0TRJCa8BkvgngUZxOxUlMfg/7cVxOIB0TPoUSgxFd/+qVz4GZMvr0dPu7 -eAF7J/8ECVvb0wSPTUI1N3c= +MIIG/wIBADANBgkqhkiG9w0BAQEFAASCBukwggblAgEAAoIBgQCylKlLaKU+hOvJ +DfriTRLd+IthG5hv28I3A/CGjLICT0rDDtgaXd0uqloJAnjsgn5gMAcStpDW8Rm+ +t6LsrBL+5fBgkyU1r94Rvx0HHoyaZwBBouitVHw28hP3W+smddkqB1UxpGnTeL2B +gj3dVo/WTtRfO+0h0PKw1l98YE1pMTdqIwcOOE/ER0g4hvA/wrxuLhMvlVLMy/lL +58uctqaDUqryNyeerKbVkq4fJyCG5D2TwXVJ3i2DDh0xSt2Y10poZV4M4k8Su9Z5 +8zN2PSvYMT50aqF277v8BaOeYUApBE4kZGIJpo13ATGdEwpUFZ0Fri4zLYUZ1hWb +OC35sKo7OxWQ/+tefNUdgWHob6Vmy777jiYcLwxc3sS9rF3AJe0rMW83kCkR6hmy +A3250E137N/1QumHuT/Nj9rnI/lwt9jfaYkZjoAgT/C97m/mM83cYpGTdoGV1xNo +7G90MhP0di5FnVsrIaSnvkbGT9UgUWx0oVMjocifdG2qIhMI9psCAwEAAQKCAYBT +sHmaPmNaZj59jZCqp0YVQlpHWwBYQ5vD3pPE6oCttm0p9nXt/VkfenQRTthOtmT1 +POzDp00/feP7zeGLmqSYUjgRekPw4gdnN7Ip2PY5kdW77NWwDSzdLxuOS8Rq1MW9 +/Yu+ZPe3RBlDbT8C0IM+Atlh/BqIQ3zIxN4g0pzUlF0M33d6AYfYSzOcUhibOO7H +j84r+YXBNkIRgYKZYbutRXuZYaGuqejRpBj3voVu0d3Ntdb6lCWuClpB9HzfGN0c +RTv8g6UYO4sK3qyFn90ibIR/1GB9watvtoWVZqggiWeBzSWVWRsGEf9O+Cx4oJw1 +IphglhmhbgNksbj7bD24on/icldSOiVkoUemUOFmHWhCm4PnB1GmbD8YMfEdSbks +qDr1Ps1zg4mGOinVD/4cY7vuPFO/HCH07wfeaUGzRt4g0/yLr+XjVofOA3oowyxv +JAzr+niHA3lg5ecj4r7M68efwzN1OCyjMrVJw2RAzwvGxE+rm5NiT08SWlKQZnkC +gcEA4wvyLpIur/UB84nV3XVJ89UMNBLm++aTFzld047BLJtMaOhvNqx6Cl5c8VuW +l261KHjiVzpfNM3/A2LBQJcYkhX7avkqEXlj57cl+dCWAVwUzKmLJTPjfaTTZnYJ +xeN3dMYjJz2z2WtgvfvDoJLukVwIMmhTY8wtqqYyQBJ/l06pBsfw5TNvmVIOQHds +8ASOiFt+WRLk2bl9xrGGayqt3VV93KVRzF27cpjOgEcG74F3c0ZW9snERN7vIYwB +JfrlAoHBAMlahPwMP2TYylG8OzHe7EiehTekSO26LGh0Cq3wTGXYsK/q8hQCzL14 +kWW638vpwXL6L9ntvrd7hjzWRO3vX/VxnYEA6f0bpqHq1tZi6lzix5CTUN5McpDg +QnjenSJNrNjS1zEF8WeY9iLEuDI/M/iUW4y9R6s3WpgQhPDXpSvd2g3gMGRUYhxQ +Xna8auiJeYFq0oNaOxvJj+VeOfJ3ZMJttd+Y7gTOYZcbg3SdRb/kdxYki0RMD2hF +4ZvjJ6CTfwKBwQDiMqiZFTJGQwYqp4vWEmAW+I4r4xkUpWatoI2Fk5eI5T9+1PLX +uYXsho56NxEU1UrOg4Cb/p+TcBc8PErkGqR0BkpxDMOInTOXSrQe6lxIBoECVXc3 +HTbrmiay0a5y5GfCgxPKqIJhfcToAceoVjovv0y7S4yoxGZKuUEe7E8JY2iqRNAO +yOvKCCICv/hcN235E44RF+2/rDlOltagNej5tY6rIFkaDdgOF4bD7f9O5eEni1Bg +litfoesDtQP/3rECgcEAkQfvQ7D6tIPmbqsbJBfCr6fmoqZllT4FIJN84b50+OL0 +mTGsfjdqC4tdhx3sdu7/VPbaIqm5NmX10bowWgWSY7MbVME4yQPyqSwC5NbIonEC +d6N0mzoLR0kQ+Ai4u+2g82gicgAq2oj1uSNi3WZi48jQjHYFulCbo246o1NgeFFK +77WshYe2R1ioQfQDOU1URKCR0uTaMHClgfu112yiGd12JAD+aF3TM0kxDXz+sXI5 +SKy311DFxECZeXRLpcC3AoHBAJkNMJWTyPYbeVu+CTQkec8Uun233EkXa2kUNZc/ +5DuXDaK+A3DMgYRufTKSPpDHGaCZ1SYPInX1Uoe2dgVjWssRL2uitR4ENabDoAOA +ICVYXYYNagqQu5wwirF0QeaMXo1fjhuuHQh8GsMdXZvYEaAITZ9/NG5x/oY08+8H +kr78SMBOPy3XQn964uKG+e3JwpOG14GKABdAlrHKFXNWchu/6dgcYXB87mrC/GhO +zNwzC+QhFTZoOomFoqMgFWujng== -----END PRIVATE KEY----- diff --git a/Lib/test/test_asyncio/utils.py b/Lib/test/test_asyncio/utils.py index 5362591b5d73..e7438d40a441 100644 --- a/Lib/test/test_asyncio/utils.py +++ b/Lib/test/test_asyncio/utils.py @@ -57,9 +57,9 @@ def data_file(filename): 'issuer': ((('countryName', 'XY'),), (('organizationName', 'Python Software Foundation CA'),), (('commonName', 'our-ca-server'),)), - 'notAfter': 'Nov 28 19:09:06 2027 GMT', - 'notBefore': 'Jan 19 19:09:06 2018 GMT', - 'serialNumber': '82EDBF41C880919C', + 'notAfter': 'Jul 7 14:23:16 2028 GMT', + 'notBefore': 'Aug 29 14:23:16 2018 GMT', + 'serialNumber': 'CB2D80995A69525C', 'subject': ((('countryName', 'XY'),), (('localityName', 'Castle Anthrax'),), (('organizationName', 'Python Software Foundation'),), diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index 5c22630d190c..49a4d7295422 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -61,9 +61,9 @@ def data_file(*name): (('localityName', 'Castle Anthrax'),), (('organizationName', 'Python Software Foundation'),), (('commonName', 'localhost'),)), - 'notAfter': 'Jan 17 19:09:06 2028 GMT', - 'notBefore': 'Jan 19 19:09:06 2018 GMT', - 'serialNumber': 'F9BA076D5B6ABD9B', + 'notAfter': 'Aug 26 14:23:15 2028 GMT', + 'notBefore': 'Aug 29 14:23:15 2018 GMT', + 'serialNumber': '98A7CF88C74A32ED', 'subject': ((('countryName', 'XY'),), (('localityName', 'Castle Anthrax'),), (('organizationName', 'Python Software Foundation'),), @@ -86,9 +86,9 @@ def data_file(*name): 'issuer': ((('countryName', 'XY'),), (('organizationName', 'Python Software Foundation CA'),), (('commonName', 'our-ca-server'),)), - 'notAfter': 'Nov 28 19:09:06 2027 GMT', - 'notBefore': 'Jan 19 19:09:06 2018 GMT', - 'serialNumber': '82EDBF41C880919C', + 'notAfter': 'Jul 7 14:23:16 2028 GMT', + 'notBefore': 'Aug 29 14:23:16 2018 GMT', + 'serialNumber': 'CB2D80995A69525C', 'subject': ((('countryName', 'XY'),), (('localityName', 'Castle Anthrax'),), (('organizationName', 'Python Software Foundation'),), @@ -334,6 +334,8 @@ def test_random_fork(self): self.assertNotEqual(child_random, parent_random) + maxDiff = None + def test_parse_cert(self): # note that this uses an 'unofficial' function in _ssl.c, # provided solely for this test, to exercise the certificate diff --git a/Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst b/Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst new file mode 100644 index 000000000000..1ca3c7d7996c --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst @@ -0,0 +1 @@ +Use 3072 RSA keys and SHA-256 signature for test certs and keys. From webhook-mailer at python.org Thu Sep 6 14:43:41 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 06 Sep 2018 18:43:41 -0000 Subject: [Python-checkins] closes bpo-34594: Don't hardcode errno values in the tests. (GH-9076) Message-ID: https://github.com/python/cpython/commit/b03c2c51909e3b5b5966d86a2829b5ddf2d496aa commit: b03c2c51909e3b5b5966d86a2829b5ddf2d496aa branch: master author: Zackery Spytz committer: Benjamin Peterson date: 2018-09-06T11:43:30-07:00 summary: closes bpo-34594: Don't hardcode errno values in the tests. (GH-9076) files: A Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst M Lib/test/test_spwd.py M Lib/test/test_tabnanny.py diff --git a/Lib/test/test_spwd.py b/Lib/test/test_spwd.py index e893f3a847fd..07793c84c8e9 100644 --- a/Lib/test/test_spwd.py +++ b/Lib/test/test_spwd.py @@ -67,8 +67,6 @@ def test_getspnam_exception(self): spwd.getspnam(name) except KeyError as exc: self.skipTest("spwd entry %r doesn't exist: %s" % (name, exc)) - else: - self.assertEqual(str(cm.exception), '[Errno 13] Permission denied') if __name__ == "__main__": diff --git a/Lib/test/test_tabnanny.py b/Lib/test/test_tabnanny.py index ec887361730b..845096e63c26 100644 --- a/Lib/test/test_tabnanny.py +++ b/Lib/test/test_tabnanny.py @@ -5,6 +5,7 @@ """ from unittest import TestCase, mock from unittest import mock +import errno import tabnanny import tokenize import tempfile @@ -232,7 +233,8 @@ def test_when_nannynag_error(self): def test_when_no_file(self): """A python file which does not exist actually in system.""" path = 'no_file.py' - err = f"{path!r}: I/O Error: [Errno 2] No such file or directory: {path!r}\n" + err = f"{path!r}: I/O Error: [Errno {errno.ENOENT}] " \ + f"No such file or directory: {path!r}\n" self.verify_tabnanny_check(path, err=err) def test_errored_directory(self): diff --git a/Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst b/Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst new file mode 100644 index 000000000000..7a7b1f055561 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst @@ -0,0 +1 @@ +Fix usage of hardcoded ``errno`` values in the tests. From webhook-mailer at python.org Fri Sep 7 00:54:53 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 07 Sep 2018 04:54:53 -0000 Subject: [Python-checkins] closes bpo-34599: Improve performance of _Py_bytes_capitalize(). (GH-9083) Message-ID: https://github.com/python/cpython/commit/593bb30e82eded7f2ec02f7d1aa49742e6962113 commit: 593bb30e82eded7f2ec02f7d1aa49742e6962113 branch: master author: Sergey Fedoseev committer: Benjamin Peterson date: 2018-09-06T21:54:49-07:00 summary: closes bpo-34599: Improve performance of _Py_bytes_capitalize(). (GH-9083) files: M Objects/bytes_methods.c diff --git a/Objects/bytes_methods.c b/Objects/bytes_methods.c index 05679e31c9d6..37c5f7dbc804 100644 --- a/Objects/bytes_methods.c +++ b/Objects/bytes_methods.c @@ -361,23 +361,9 @@ and the rest lower-cased."); void _Py_bytes_capitalize(char *result, const char *s, Py_ssize_t len) { - Py_ssize_t i; - - if (0 < len) { - int c = Py_CHARMASK(*s++); - if (Py_ISLOWER(c)) - *result = Py_TOUPPER(c); - else - *result = c; - result++; - } - for (i = 1; i < len; i++) { - int c = Py_CHARMASK(*s++); - if (Py_ISUPPER(c)) - *result = Py_TOLOWER(c); - else - *result = c; - result++; + if (len > 0) { + *result = Py_TOUPPER(*s); + _Py_bytes_lower(result + 1, s + 1, len - 1); } } From webhook-mailer at python.org Fri Sep 7 03:37:05 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 07:37:05 -0000 Subject: [Python-checkins] bpo-25750: fix refcounts in type_getattro() (GH-6118) Message-ID: https://github.com/python/cpython/commit/8f735485acf2e35a75d2fa019feb8f905598c4e5 commit: 8f735485acf2e35a75d2fa019feb8f905598c4e5 branch: master author: jdemeyer committer: Victor Stinner date: 2018-09-07T09:37:00+02:00 summary: bpo-25750: fix refcounts in type_getattro() (GH-6118) When calling tp_descr_get(self, obj, type), make sure that we own a strong reference to "self". files: A Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst b/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst new file mode 100644 index 000000000000..09ffb368b7c6 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst @@ -0,0 +1,2 @@ +Fix rare Python crash due to bad refcounting in ``type_getattro()`` if a +descriptor deletes itself from the class. Patch by Jeroen Demeyer. diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 52fcfeb22871..b9e69bf1bd1d 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3145,6 +3145,7 @@ type_getattro(PyTypeObject *type, PyObject *name) PyTypeObject *metatype = Py_TYPE(type); PyObject *meta_attribute, *attribute; descrgetfunc meta_get; + PyObject* res; if (!PyUnicode_Check(name)) { PyErr_Format(PyExc_TypeError, @@ -3166,6 +3167,7 @@ type_getattro(PyTypeObject *type, PyObject *name) meta_attribute = _PyType_Lookup(metatype, name); if (meta_attribute != NULL) { + Py_INCREF(meta_attribute); meta_get = Py_TYPE(meta_attribute)->tp_descr_get; if (meta_get != NULL && PyDescr_IsData(meta_attribute)) { @@ -3173,10 +3175,11 @@ type_getattro(PyTypeObject *type, PyObject *name) * writes. Assume the attribute is not overridden in * type's tp_dict (and bases): call the descriptor now. */ - return meta_get(meta_attribute, (PyObject *)type, - (PyObject *)metatype); + res = meta_get(meta_attribute, (PyObject *)type, + (PyObject *)metatype); + Py_DECREF(meta_attribute); + return res; } - Py_INCREF(meta_attribute); } /* No data descriptor found on metatype. Look in tp_dict of this @@ -3184,6 +3187,7 @@ type_getattro(PyTypeObject *type, PyObject *name) attribute = _PyType_Lookup(type, name); if (attribute != NULL) { /* Implement descriptor functionality, if any */ + Py_INCREF(attribute); descrgetfunc local_get = Py_TYPE(attribute)->tp_descr_get; Py_XDECREF(meta_attribute); @@ -3191,11 +3195,12 @@ type_getattro(PyTypeObject *type, PyObject *name) if (local_get != NULL) { /* NULL 2nd argument indicates the descriptor was * found on the target object itself (or a base) */ - return local_get(attribute, (PyObject *)NULL, - (PyObject *)type); + res = local_get(attribute, (PyObject *)NULL, + (PyObject *)type); + Py_DECREF(attribute); + return res; } - Py_INCREF(attribute); return attribute; } From webhook-mailer at python.org Fri Sep 7 03:50:44 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 07 Sep 2018 07:50:44 -0000 Subject: [Python-checkins] bpo-25750: fix refcounts in type_getattro() (GH-6118) Message-ID: https://github.com/python/cpython/commit/f862f3abaed59b83763707ae529f0fe487961ba9 commit: f862f3abaed59b83763707ae529f0fe487961ba9 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-07T00:50:35-07:00 summary: bpo-25750: fix refcounts in type_getattro() (GH-6118) When calling tp_descr_get(self, obj, type), make sure that we own a strong reference to "self". (cherry picked from commit 8f735485acf2e35a75d2fa019feb8f905598c4e5) Co-authored-by: jdemeyer files: A Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst b/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst new file mode 100644 index 000000000000..09ffb368b7c6 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst @@ -0,0 +1,2 @@ +Fix rare Python crash due to bad refcounting in ``type_getattro()`` if a +descriptor deletes itself from the class. Patch by Jeroen Demeyer. diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 52fcfeb22871..b9e69bf1bd1d 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3145,6 +3145,7 @@ type_getattro(PyTypeObject *type, PyObject *name) PyTypeObject *metatype = Py_TYPE(type); PyObject *meta_attribute, *attribute; descrgetfunc meta_get; + PyObject* res; if (!PyUnicode_Check(name)) { PyErr_Format(PyExc_TypeError, @@ -3166,6 +3167,7 @@ type_getattro(PyTypeObject *type, PyObject *name) meta_attribute = _PyType_Lookup(metatype, name); if (meta_attribute != NULL) { + Py_INCREF(meta_attribute); meta_get = Py_TYPE(meta_attribute)->tp_descr_get; if (meta_get != NULL && PyDescr_IsData(meta_attribute)) { @@ -3173,10 +3175,11 @@ type_getattro(PyTypeObject *type, PyObject *name) * writes. Assume the attribute is not overridden in * type's tp_dict (and bases): call the descriptor now. */ - return meta_get(meta_attribute, (PyObject *)type, - (PyObject *)metatype); + res = meta_get(meta_attribute, (PyObject *)type, + (PyObject *)metatype); + Py_DECREF(meta_attribute); + return res; } - Py_INCREF(meta_attribute); } /* No data descriptor found on metatype. Look in tp_dict of this @@ -3184,6 +3187,7 @@ type_getattro(PyTypeObject *type, PyObject *name) attribute = _PyType_Lookup(type, name); if (attribute != NULL) { /* Implement descriptor functionality, if any */ + Py_INCREF(attribute); descrgetfunc local_get = Py_TYPE(attribute)->tp_descr_get; Py_XDECREF(meta_attribute); @@ -3191,11 +3195,12 @@ type_getattro(PyTypeObject *type, PyObject *name) if (local_get != NULL) { /* NULL 2nd argument indicates the descriptor was * found on the target object itself (or a base) */ - return local_get(attribute, (PyObject *)NULL, - (PyObject *)type); + res = local_get(attribute, (PyObject *)NULL, + (PyObject *)type); + Py_DECREF(attribute); + return res; } - Py_INCREF(attribute); return attribute; } From webhook-mailer at python.org Fri Sep 7 03:57:48 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 07:57:48 -0000 Subject: [Python-checkins] bpo-25750: fix refcounts in type_getattro() (GH-6118) (GH-9088) Message-ID: https://github.com/python/cpython/commit/3ee07432f2e607cc6e7e6ea2d3695b672ceb1cea commit: 3ee07432f2e607cc6e7e6ea2d3695b672ceb1cea branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Victor Stinner date: 2018-09-07T09:57:45+02:00 summary: bpo-25750: fix refcounts in type_getattro() (GH-6118) (GH-9088) When calling tp_descr_get(self, obj, type), make sure that we own a strong reference to "self". (cherry picked from commit 8f735485acf2e35a75d2fa019feb8f905598c4e5) Co-authored-by: jdemeyer files: A Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst b/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst new file mode 100644 index 000000000000..09ffb368b7c6 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst @@ -0,0 +1,2 @@ +Fix rare Python crash due to bad refcounting in ``type_getattro()`` if a +descriptor deletes itself from the class. Patch by Jeroen Demeyer. diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 3468c64060df..ed2d40064adc 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3004,6 +3004,7 @@ type_getattro(PyTypeObject *type, PyObject *name) PyTypeObject *metatype = Py_TYPE(type); PyObject *meta_attribute, *attribute; descrgetfunc meta_get; + PyObject* res; if (!PyUnicode_Check(name)) { PyErr_Format(PyExc_TypeError, @@ -3025,6 +3026,7 @@ type_getattro(PyTypeObject *type, PyObject *name) meta_attribute = _PyType_Lookup(metatype, name); if (meta_attribute != NULL) { + Py_INCREF(meta_attribute); meta_get = Py_TYPE(meta_attribute)->tp_descr_get; if (meta_get != NULL && PyDescr_IsData(meta_attribute)) { @@ -3032,10 +3034,11 @@ type_getattro(PyTypeObject *type, PyObject *name) * writes. Assume the attribute is not overridden in * type's tp_dict (and bases): call the descriptor now. */ - return meta_get(meta_attribute, (PyObject *)type, - (PyObject *)metatype); + res = meta_get(meta_attribute, (PyObject *)type, + (PyObject *)metatype); + Py_DECREF(meta_attribute); + return res; } - Py_INCREF(meta_attribute); } /* No data descriptor found on metatype. Look in tp_dict of this @@ -3043,6 +3046,7 @@ type_getattro(PyTypeObject *type, PyObject *name) attribute = _PyType_Lookup(type, name); if (attribute != NULL) { /* Implement descriptor functionality, if any */ + Py_INCREF(attribute); descrgetfunc local_get = Py_TYPE(attribute)->tp_descr_get; Py_XDECREF(meta_attribute); @@ -3050,11 +3054,12 @@ type_getattro(PyTypeObject *type, PyObject *name) if (local_get != NULL) { /* NULL 2nd argument indicates the descriptor was * found on the target object itself (or a base) */ - return local_get(attribute, (PyObject *)NULL, - (PyObject *)type); + res = local_get(attribute, (PyObject *)NULL, + (PyObject *)type); + Py_DECREF(attribute); + return res; } - Py_INCREF(attribute); return attribute; } From webhook-mailer at python.org Fri Sep 7 04:15:42 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 08:15:42 -0000 Subject: [Python-checkins] bpo-25750: fix refcounts in type_getattro() (GH-6118) (GH-9091) Message-ID: https://github.com/python/cpython/commit/bf2bd8f8a1d88de60c114de957f50fe2433e3937 commit: bf2bd8f8a1d88de60c114de957f50fe2433e3937 branch: 2.7 author: Victor Stinner committer: GitHub date: 2018-09-07T10:15:31+02:00 summary: bpo-25750: fix refcounts in type_getattro() (GH-6118) (GH-9091) When calling tp_descr_get(self, obj, type), make sure that we own a strong reference to "self". (cherry picked from commit 8f735485acf2e35a75d2fa019feb8f905598c4e5) files: A Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst b/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst new file mode 100644 index 000000000000..09ffb368b7c6 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-03-14-21-42-17.bpo-25750.lxgkQz.rst @@ -0,0 +1,2 @@ +Fix rare Python crash due to bad refcounting in ``type_getattro()`` if a +descriptor deletes itself from the class. Patch by Jeroen Demeyer. diff --git a/Objects/typeobject.c b/Objects/typeobject.c index d576b8250d01..844fb0074920 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -2611,6 +2611,7 @@ type_getattro(PyTypeObject *type, PyObject *name) PyTypeObject *metatype = Py_TYPE(type); PyObject *meta_attribute, *attribute; descrgetfunc meta_get; + PyObject* res; if (!PyString_Check(name)) { PyErr_Format(PyExc_TypeError, @@ -2632,6 +2633,7 @@ type_getattro(PyTypeObject *type, PyObject *name) meta_attribute = _PyType_Lookup(metatype, name); if (meta_attribute != NULL) { + Py_INCREF(meta_attribute); meta_get = Py_TYPE(meta_attribute)->tp_descr_get; if (meta_get != NULL && PyDescr_IsData(meta_attribute)) { @@ -2639,10 +2641,11 @@ type_getattro(PyTypeObject *type, PyObject *name) * writes. Assume the attribute is not overridden in * type's tp_dict (and bases): call the descriptor now. */ - return meta_get(meta_attribute, (PyObject *)type, - (PyObject *)metatype); + res = meta_get(meta_attribute, (PyObject *)type, + (PyObject *)metatype); + Py_DECREF(meta_attribute); + return res; } - Py_INCREF(meta_attribute); } /* No data descriptor found on metatype. Look in tp_dict of this @@ -2650,18 +2653,21 @@ type_getattro(PyTypeObject *type, PyObject *name) attribute = _PyType_Lookup(type, name); if (attribute != NULL) { /* Implement descriptor functionality, if any */ - descrgetfunc local_get = Py_TYPE(attribute)->tp_descr_get; + descrgetfunc local_get; + Py_INCREF(attribute); + local_get = Py_TYPE(attribute)->tp_descr_get; Py_XDECREF(meta_attribute); if (local_get != NULL) { /* NULL 2nd argument indicates the descriptor was * found on the target object itself (or a base) */ - return local_get(attribute, (PyObject *)NULL, - (PyObject *)type); + res = local_get(attribute, (PyObject *)NULL, + (PyObject *)type); + Py_DECREF(attribute); + return res; } - Py_INCREF(attribute); return attribute; } From webhook-mailer at python.org Fri Sep 7 04:56:17 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 08:56:17 -0000 Subject: [Python-checkins] _sre.c: Removed unused SRE_IS_ALNUM macro (GH-9090) Message-ID: https://github.com/python/cpython/commit/f9925d86c9d7a44b612576131cd5b84cfee36c4e commit: f9925d86c9d7a44b612576131cd5b84cfee36c4e branch: master author: Sergey Fedoseev committer: Victor Stinner date: 2018-09-07T10:56:09+02:00 summary: _sre.c: Removed unused SRE_IS_ALNUM macro (GH-9090) files: M Modules/_sre.c diff --git a/Modules/_sre.c b/Modules/_sre.c index b6be6f6ffa60..d67083037e51 100644 --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -92,8 +92,6 @@ static const char copyright[] = ((ch) < 128 && Py_ISSPACE(ch)) #define SRE_IS_LINEBREAK(ch)\ ((ch) == '\n') -#define SRE_IS_ALNUM(ch)\ - ((ch) < 128 && Py_ISALNUM(ch)) #define SRE_IS_WORD(ch)\ ((ch) < 128 && (Py_ISALNUM(ch) || (ch) == '_')) From solipsis at pitrou.net Fri Sep 7 05:08:53 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 07 Sep 2018 09:08:53 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=3 Message-ID: <20180907090853.1.E9F414B1C474F2B0@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [7, -7, 1] memory blocks, sum=1 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [0, -2, 0] memory blocks, sum=-2 test_multiprocessing_spawn leaked [-2, 1, 1] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog1l3pN0', '--timeout', '7200'] From webhook-mailer at python.org Fri Sep 7 05:31:52 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 09:31:52 -0000 Subject: [Python-checkins] Doc: Missing 'f' in an f-string. (GH-9074) Message-ID: https://github.com/python/cpython/commit/25fa141487e61b94f15289619cb3af764cf65e58 commit: 25fa141487e61b94f15289619cb3af764cf65e58 branch: master author: Julien Palard committer: Victor Stinner date: 2018-09-07T11:31:47+02:00 summary: Doc: Missing 'f' in an f-string. (GH-9074) files: M Doc/tutorial/inputoutput.rst diff --git a/Doc/tutorial/inputoutput.rst b/Doc/tutorial/inputoutput.rst index dfe4c3849cbe..a92c26681596 100644 --- a/Doc/tutorial/inputoutput.rst +++ b/Doc/tutorial/inputoutput.rst @@ -127,7 +127,7 @@ applies :func:`repr`:: >>> animals = 'eels' >>> print(f'My hovercraft is full of {animals}.') My hovercraft is full of eels. - >>> print('My hovercraft is full of {animals !r}.') + >>> print(f'My hovercraft is full of {animals !r}.') My hovercraft is full of 'eels'. For a reference on these format specifications, see From webhook-mailer at python.org Fri Sep 7 05:59:05 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 09:59:05 -0000 Subject: [Python-checkins] Doc: Missing 'f' in an f-string. (GH-9074) (GH-9095) Message-ID: https://github.com/python/cpython/commit/854b740910f20441b6a42c2047d5f0ad0451404e commit: 854b740910f20441b6a42c2047d5f0ad0451404e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Victor Stinner date: 2018-09-07T11:59:00+02:00 summary: Doc: Missing 'f' in an f-string. (GH-9074) (GH-9095) (cherry picked from commit 25fa141487e61b94f15289619cb3af764cf65e58) Co-authored-by: Julien Palard files: M Doc/tutorial/inputoutput.rst diff --git a/Doc/tutorial/inputoutput.rst b/Doc/tutorial/inputoutput.rst index dfe4c3849cbe..a92c26681596 100644 --- a/Doc/tutorial/inputoutput.rst +++ b/Doc/tutorial/inputoutput.rst @@ -127,7 +127,7 @@ applies :func:`repr`:: >>> animals = 'eels' >>> print(f'My hovercraft is full of {animals}.') My hovercraft is full of eels. - >>> print('My hovercraft is full of {animals !r}.') + >>> print(f'My hovercraft is full of {animals !r}.') My hovercraft is full of 'eels'. For a reference on these format specifications, see From webhook-mailer at python.org Fri Sep 7 08:06:24 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 12:06:24 -0000 Subject: [Python-checkins] bpo-33625: Release GIL for grp.getgr{nam, gid} and pwd.getpw{nam, uid} (GH-7081) Message-ID: https://github.com/python/cpython/commit/23e65b25557f957af840cf8fe68e80659ce28629 commit: 23e65b25557f957af840cf8fe68e80659ce28629 branch: master author: William Grzybowski committer: Victor Stinner date: 2018-09-07T14:06:15+02:00 summary: bpo-33625: Release GIL for grp.getgr{nam,gid} and pwd.getpw{nam,uid} (GH-7081) Release GIL on grp.getgrnam(), grp.getgrgid(), pwd.getpwnam() and pwd.getpwuid() if reentrant variants of these functions are available. Patch by William Grzybowski. files: A Misc/NEWS.d/next/Library/2018-05-23-17-07-54.bpo-33625.nzQgD8.rst M Modules/grpmodule.c M Modules/pwdmodule.c M configure M configure.ac M pyconfig.h.in diff --git a/Misc/NEWS.d/next/Library/2018-05-23-17-07-54.bpo-33625.nzQgD8.rst b/Misc/NEWS.d/next/Library/2018-05-23-17-07-54.bpo-33625.nzQgD8.rst new file mode 100644 index 000000000000..265fab2289c2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-05-23-17-07-54.bpo-33625.nzQgD8.rst @@ -0,0 +1,3 @@ +Release GIL on `grp.getgrnam`, `grp.getgrgid`, `pwd.getpwnam` and +`pwd.getpwuid` if reentrant variants of these functions are available. +Patch by William Grzybowski. diff --git a/Modules/grpmodule.c b/Modules/grpmodule.c index f577fd3ab4ec..ffdfa1b6f9fb 100644 --- a/Modules/grpmodule.c +++ b/Modules/grpmodule.c @@ -37,6 +37,8 @@ static PyStructSequence_Desc struct_group_type_desc = { static int initialized; static PyTypeObject StructGrpType; +#define DEFAULT_BUFFER_SIZE 1024 + static PyObject * mkgrent(struct group *p) { @@ -96,7 +98,9 @@ static PyObject * grp_getgrgid_impl(PyObject *module, PyObject *id) /*[clinic end generated code: output=30797c289504a1ba input=15fa0e2ccf5cda25]*/ { - PyObject *py_int_id; + PyObject *py_int_id, *retval = NULL; + int nomem = 0; + char *buf = NULL, *buf2 = NULL; gid_t gid; struct group *p; @@ -119,8 +123,48 @@ grp_getgrgid_impl(PyObject *module, PyObject *id) } Py_DECREF(py_int_id); } +#ifdef HAVE_GETGRGID_R + Py_BEGIN_ALLOW_THREADS + int status; + Py_ssize_t bufsize; + struct group grp; + + bufsize = sysconf(_SC_GETGR_R_SIZE_MAX); + if (bufsize == -1) { + bufsize = DEFAULT_BUFFER_SIZE; + } + + while (1) { + buf2 = PyMem_RawRealloc(buf, bufsize); + if (buf2 == NULL) { + p = NULL; + nomem = 1; + break; + } + buf = buf2; + status = getgrgid_r(gid, &grp, buf, bufsize, &p); + if (status != 0) { + p = NULL; + } + if (p != NULL || status != ERANGE) { + break; + } + if (bufsize > (PY_SSIZE_T_MAX >> 1)) { + nomem = 1; + break; + } + bufsize <<= 1; + } - if ((p = getgrgid(gid)) == NULL) { + Py_END_ALLOW_THREADS +#else + p = getgrgid(gid); +#endif + if (p == NULL) { + PyMem_RawFree(buf); + if (nomem == 1) { + return PyErr_NoMemory(); + } PyObject *gid_obj = _PyLong_FromGid(gid); if (gid_obj == NULL) return NULL; @@ -128,7 +172,11 @@ grp_getgrgid_impl(PyObject *module, PyObject *id) Py_DECREF(gid_obj); return NULL; } - return mkgrent(p); + retval = mkgrent(p); +#ifdef HAVE_GETGRGID_R + PyMem_RawFree(buf); +#endif + return retval; } /*[clinic input] @@ -145,7 +193,8 @@ static PyObject * grp_getgrnam_impl(PyObject *module, PyObject *name) /*[clinic end generated code: output=67905086f403c21c input=08ded29affa3c863]*/ { - char *name_chars; + char *buf = NULL, *buf2 = NULL, *name_chars; + int nomem = 0; struct group *p; PyObject *bytes, *retval = NULL; @@ -154,13 +203,55 @@ grp_getgrnam_impl(PyObject *module, PyObject *name) /* check for embedded null bytes */ if (PyBytes_AsStringAndSize(bytes, &name_chars, NULL) == -1) goto out; +#ifdef HAVE_GETGRNAM_R + Py_BEGIN_ALLOW_THREADS + int status; + Py_ssize_t bufsize; + struct group grp; + + bufsize = sysconf(_SC_GETGR_R_SIZE_MAX); + if (bufsize == -1) { + bufsize = DEFAULT_BUFFER_SIZE; + } + + while(1) { + buf2 = PyMem_RawRealloc(buf, bufsize); + if (buf2 == NULL) { + p = NULL; + nomem = 1; + break; + } + buf = buf2; + status = getgrnam_r(name_chars, &grp, buf, bufsize, &p); + if (status != 0) { + p = NULL; + } + if (p != NULL || status != ERANGE) { + break; + } + if (bufsize > (PY_SSIZE_T_MAX >> 1)) { + nomem = 1; + break; + } + bufsize <<= 1; + } - if ((p = getgrnam(name_chars)) == NULL) { - PyErr_Format(PyExc_KeyError, "getgrnam(): name not found: %s", name_chars); + Py_END_ALLOW_THREADS +#else + p = getgrnam(name_chars); +#endif + if (p == NULL) { + if (nomem == 1) { + PyErr_NoMemory(); + } + else { + PyErr_Format(PyExc_KeyError, "getgrnam(): name not found: %s", name_chars); + } goto out; } retval = mkgrent(p); out: + PyMem_RawFree(buf); Py_DECREF(bytes); return retval; } diff --git a/Modules/pwdmodule.c b/Modules/pwdmodule.c index bbef2de9c522..36192a570433 100644 --- a/Modules/pwdmodule.c +++ b/Modules/pwdmodule.c @@ -50,6 +50,8 @@ exception is raised if the entry asked for cannot be found."); static int initialized; static PyTypeObject StructPwdType; +#define DEFAULT_BUFFER_SIZE 1024 + static void sets(PyObject *v, int i, const char* val) { @@ -116,8 +118,11 @@ static PyObject * pwd_getpwuid(PyObject *module, PyObject *uidobj) /*[clinic end generated code: output=c4ee1d4d429b86c4 input=ae64d507a1c6d3e8]*/ { + PyObject *retval = NULL; uid_t uid; + int nomem = 0; struct passwd *p; + char *buf = NULL, *buf2 = NULL; if (!_Py_Uid_Converter(uidobj, &uid)) { if (PyErr_ExceptionMatches(PyExc_OverflowError)) @@ -125,7 +130,47 @@ pwd_getpwuid(PyObject *module, PyObject *uidobj) "getpwuid(): uid not found"); return NULL; } - if ((p = getpwuid(uid)) == NULL) { +#ifdef HAVE_GETPWUID_R + Py_BEGIN_ALLOW_THREADS + int status; + Py_ssize_t bufsize; + struct passwd pwd; + + bufsize = sysconf(_SC_GETPW_R_SIZE_MAX); + if (bufsize == -1) { + bufsize = DEFAULT_BUFFER_SIZE; + } + + while(1) { + buf2 = PyMem_RawRealloc(buf, bufsize); + if (buf2 == NULL) { + nomem = 1; + break; + } + buf = buf2; + status = getpwuid_r(uid, &pwd, buf, bufsize, &p); + if (status != 0) { + p = NULL; + } + if (p != NULL || status != ERANGE) { + break; + } + if (bufsize > (PY_SSIZE_T_MAX >> 1)) { + nomem = 1; + break; + } + bufsize <<= 1; + } + + Py_END_ALLOW_THREADS +#else + p = getpwuid(uid); +#endif + if (p == NULL) { + PyMem_RawFree(buf); + if (nomem == 1) { + return PyErr_NoMemory(); + } PyObject *uid_obj = _PyLong_FromUid(uid); if (uid_obj == NULL) return NULL; @@ -134,7 +179,11 @@ pwd_getpwuid(PyObject *module, PyObject *uidobj) Py_DECREF(uid_obj); return NULL; } - return mkpwent(p); + retval = mkpwent(p); +#ifdef HAVE_GETPWUID_R + PyMem_RawFree(buf); +#endif + return retval; } /*[clinic input] @@ -152,7 +201,8 @@ static PyObject * pwd_getpwnam_impl(PyObject *module, PyObject *arg) /*[clinic end generated code: output=6abeee92430e43d2 input=d5f7e700919b02d3]*/ { - char *name; + char *buf = NULL, *buf2 = NULL, *name; + int nomem = 0; struct passwd *p; PyObject *bytes, *retval = NULL; @@ -161,13 +211,55 @@ pwd_getpwnam_impl(PyObject *module, PyObject *arg) /* check for embedded null bytes */ if (PyBytes_AsStringAndSize(bytes, &name, NULL) == -1) goto out; - if ((p = getpwnam(name)) == NULL) { - PyErr_Format(PyExc_KeyError, - "getpwnam(): name not found: %s", name); +#ifdef HAVE_GETPWNAM_R + Py_BEGIN_ALLOW_THREADS + int status; + Py_ssize_t bufsize; + struct passwd pwd; + + bufsize = sysconf(_SC_GETPW_R_SIZE_MAX); + if (bufsize == -1) { + bufsize = DEFAULT_BUFFER_SIZE; + } + + while(1) { + buf2 = PyMem_RawRealloc(buf, bufsize); + if (buf2 == NULL) { + nomem = 1; + break; + } + buf = buf2; + status = getpwnam_r(name, &pwd, buf, bufsize, &p); + if (status != 0) { + p = NULL; + } + if (p != NULL || status != ERANGE) { + break; + } + if (bufsize > (PY_SSIZE_T_MAX >> 1)) { + nomem = 1; + break; + } + bufsize <<= 1; + } + + Py_END_ALLOW_THREADS +#else + p = getpwnam(name); +#endif + if (p == NULL) { + if (nomem == 1) { + PyErr_NoMemory(); + } + else { + PyErr_Format(PyExc_KeyError, + "getpwnam(): name not found: %s", name); + } goto out; } retval = mkpwent(p); out: + PyMem_RawFree(buf); Py_DECREF(bytes); return retval; } diff --git a/configure b/configure index 0b07f0b97bf1..b4a70e6736bb 100755 --- a/configure +++ b/configure @@ -781,6 +781,7 @@ infodir docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -890,6 +891,7 @@ datadir='${datarootdir}' sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1142,6 +1144,15 @@ do | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1279,7 +1290,7 @@ fi for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1432,6 +1443,7 @@ Fine tuning of the installation directories: --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -11242,8 +11254,9 @@ for ac_func in alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ clock confstr ctermid dup3 execv faccessat fchmod fchmodat fchown fchownat \ fexecve fdopendir fork fpathconf fstatat ftime ftruncate futimesat \ futimens futimes gai_strerror getentropy \ + getgrgid_r getgrnam_r \ getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \ - getpriority getresuid getresgid getpwent getspnam getspent getsid getwd \ + getpriority getresuid getresgid getpwent getpwnam_r getpwuid_r getspnam getspent getsid getwd \ if_nameindex \ initgroups kill killpg lchmod lchown lockf linkat lstat lutimes mmap \ memrchr mbrtowc mkdirat mkfifo \ @@ -13675,6 +13688,7 @@ fi + # checks for system services # (none yet) diff --git a/configure.ac b/configure.ac index 5b66624d7819..e47586bd403c 100644 --- a/configure.ac +++ b/configure.ac @@ -3434,8 +3434,9 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ clock confstr ctermid dup3 execv faccessat fchmod fchmodat fchown fchownat \ fexecve fdopendir fork fpathconf fstatat ftime ftruncate futimesat \ futimens futimes gai_strerror getentropy \ + getgrgid_r getgrnam_r \ getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \ - getpriority getresuid getresgid getpwent getspnam getspent getsid getwd \ + getpriority getresuid getresgid getpwent getpwnam_r getpwuid_r getspnam getspent getsid getwd \ if_nameindex \ initgroups kill killpg lchmod lchown lockf linkat lstat lutimes mmap \ memrchr mbrtowc mkdirat mkfifo \ diff --git a/pyconfig.h.in b/pyconfig.h.in index a82c3737c3c4..4d3a4b995fe3 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -403,6 +403,12 @@ /* Define to 1 if you have the `getentropy' function. */ #undef HAVE_GETENTROPY +/* Define to 1 if you have the `getgrgid_r' function. */ +#undef HAVE_GETGRGID_R + +/* Define to 1 if you have the `getgrnam_r' function. */ +#undef HAVE_GETGRNAM_R + /* Define to 1 if you have the `getgrouplist' function. */ #undef HAVE_GETGROUPLIST @@ -457,6 +463,12 @@ /* Define to 1 if you have the `getpwent' function. */ #undef HAVE_GETPWENT +/* Define to 1 if you have the `getpwnam_r' function. */ +#undef HAVE_GETPWNAM_R + +/* Define to 1 if you have the `getpwuid_r' function. */ +#undef HAVE_GETPWUID_R + /* Define to 1 if the getrandom() function is available */ #undef HAVE_GETRANDOM From webhook-mailer at python.org Fri Sep 7 08:17:24 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 12:17:24 -0000 Subject: [Python-checkins] [3.6] bpo-34563: Fix for invalid assert on big output of multiprocessing.Process (GH-9027) (GH-9069) Message-ID: https://github.com/python/cpython/commit/23f427a0fdb888212136cf8745a9f5f832a3f374 commit: 23f427a0fdb888212136cf8745a9f5f832a3f374 branch: 3.6 author: Alexander Buchkovsky committer: Victor Stinner date: 2018-09-07T14:17:18+02:00 summary: [3.6] bpo-34563: Fix for invalid assert on big output of multiprocessing.Process (GH-9027) (GH-9069) Fix for invalid assert on big output of multiprocessing.Process. (cherry picked from commit 266f4904a222a784080e29aad0916849e507515d) files: A Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst M Modules/_winapi.c M Modules/clinic/_winapi.c.h diff --git a/Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst b/Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst new file mode 100644 index 000000000000..9127af0d1924 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-01-20-43-10.bpo-34563.7NQK7B.rst @@ -0,0 +1 @@ +On Windows, fix multiprocessing.Connection for very large read: fix _winapi.PeekNamedPipe() and _winapi.ReadFile() for read larger than INT_MAX (usually 2^31-1). \ No newline at end of file diff --git a/Modules/_winapi.c b/Modules/_winapi.c index b98e7789b787..0647f6bedf1b 100644 --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -1167,7 +1167,7 @@ _winapi_PeekNamedPipe_impl(PyObject *module, HANDLE handle, int size) } if (_PyBytes_Resize(&buf, nread)) return NULL; - return Py_BuildValue("Nii", buf, navail, nleft); + return Py_BuildValue("NII", buf, navail, nleft); } else { Py_BEGIN_ALLOW_THREADS @@ -1176,7 +1176,7 @@ _winapi_PeekNamedPipe_impl(PyObject *module, HANDLE handle, int size) if (!ret) { return PyErr_SetExcFromWindowsErr(PyExc_IOError, 0); } - return Py_BuildValue("ii", navail, nleft); + return Py_BuildValue("II", navail, nleft); } } @@ -1184,14 +1184,14 @@ _winapi_PeekNamedPipe_impl(PyObject *module, HANDLE handle, int size) _winapi.ReadFile handle: HANDLE - size: int + size: DWORD overlapped as use_overlapped: int(c_default='0') = False [clinic start generated code]*/ static PyObject * -_winapi_ReadFile_impl(PyObject *module, HANDLE handle, int size, +_winapi_ReadFile_impl(PyObject *module, HANDLE handle, DWORD size, int use_overlapped) -/*[clinic end generated code: output=492029ca98161d84 input=8dd810194e86ac7d]*/ +/*[clinic end generated code: output=d3d5b44a8201b944 input=1b7d0ed0de1e50bc]*/ { DWORD nread; PyObject *buf; diff --git a/Modules/clinic/_winapi.c.h b/Modules/clinic/_winapi.c.h index 5bfbaf0d5634..feb98bc797d1 100644 --- a/Modules/clinic/_winapi.c.h +++ b/Modules/clinic/_winapi.c.h @@ -673,7 +673,7 @@ PyDoc_STRVAR(_winapi_ReadFile__doc__, {"ReadFile", (PyCFunction)_winapi_ReadFile, METH_FASTCALL, _winapi_ReadFile__doc__}, static PyObject * -_winapi_ReadFile_impl(PyObject *module, HANDLE handle, int size, +_winapi_ReadFile_impl(PyObject *module, HANDLE handle, DWORD size, int use_overlapped); static PyObject * @@ -681,9 +681,9 @@ _winapi_ReadFile(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject * { PyObject *return_value = NULL; static const char * const _keywords[] = {"handle", "size", "overlapped", NULL}; - static _PyArg_Parser _parser = {"" F_HANDLE "i|i:ReadFile", _keywords, 0}; + static _PyArg_Parser _parser = {"" F_HANDLE "k|i:ReadFile", _keywords, 0}; HANDLE handle; - int size; + DWORD size; int use_overlapped = 0; if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, @@ -889,4 +889,4 @@ _winapi_WriteFile(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject exit: return return_value; } -/*[clinic end generated code: output=46d6382a6662c4a9 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=6c5cf8865d381c70 input=a9049054013a1b77]*/ From webhook-mailer at python.org Fri Sep 7 11:20:46 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 15:20:46 -0000 Subject: [Python-checkins] bpo-34605, libregrtest: Rename --slaveargs to --worker-args (GH-9099) Message-ID: https://github.com/python/cpython/commit/012f5b968a738b15ae9b40c499a1c0778b0615a9 commit: 012f5b968a738b15ae9b40c499a1c0778b0615a9 branch: master author: Victor Stinner committer: GitHub date: 2018-09-07T17:20:42+02:00 summary: bpo-34605, libregrtest: Rename --slaveargs to --worker-args (GH-9099) Rename also run_tests_slave() to run_tests_worker(). files: M Lib/test/libregrtest/cmdline.py M Lib/test/libregrtest/main.py M Lib/test/libregrtest/runtest_mp.py M Lib/test/test_regrtest.py diff --git a/Lib/test/libregrtest/cmdline.py b/Lib/test/libregrtest/cmdline.py index dab17c3edf32..bd126b33c9bd 100644 --- a/Lib/test/libregrtest/cmdline.py +++ b/Lib/test/libregrtest/cmdline.py @@ -170,7 +170,7 @@ def _create_parser(): group.add_argument('--wait', action='store_true', help='wait for user input, e.g., allow a debugger ' 'to be attached') - group.add_argument('--slaveargs', metavar='ARGS') + group.add_argument('--worker-args', metavar='ARGS') group.add_argument('-S', '--start', metavar='START', help='the name of the test at which to start.' + more_details) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index e262a7a172b9..a176db59c0fb 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -542,9 +542,9 @@ def _main(self, tests, kwargs): print(msg, file=sys.stderr, flush=True) sys.exit(2) - if self.ns.slaveargs is not None: - from test.libregrtest.runtest_mp import run_tests_slave - run_tests_slave(self.ns.slaveargs) + if self.ns.worker_args is not None: + from test.libregrtest.runtest_mp import run_tests_worker + run_tests_worker(self.ns.worker_args) if self.ns.wait: input("Press any key to continue...") diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py index 907451cf6311..1f07cfbd8cbe 100644 --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -24,23 +24,23 @@ def run_test_in_subprocess(testname, ns): - """Run the given test in a subprocess with --slaveargs. + """Run the given test in a subprocess with --worker-args. ns is the option Namespace parsed from command-line arguments. regrtest - is invoked in a subprocess with the --slaveargs argument; when the + is invoked in a subprocess with the --worker-args argument; when the subprocess exits, its return code, stdout and stderr are returned as a 3-tuple. """ from subprocess import Popen, PIPE ns_dict = vars(ns) - slaveargs = (ns_dict, testname) - slaveargs = json.dumps(slaveargs) + worker_args = (ns_dict, testname) + worker_args = json.dumps(worker_args) cmd = [sys.executable, *support.args_from_interpreter_flags(), '-u', # Unbuffered stdout and stderr '-m', 'test.regrtest', - '--slaveargs', slaveargs] + '--worker-args', worker_args] if ns.pgo: cmd += ['--pgo'] @@ -58,8 +58,8 @@ def run_test_in_subprocess(testname, ns): return retcode, stdout, stderr -def run_tests_slave(slaveargs): - ns_dict, testname = json.loads(slaveargs) +def run_tests_worker(worker_args): + ns_dict, testname = json.loads(worker_args) ns = types.SimpleNamespace(**ns_dict) setup_tests(ns) diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py index af332ad15d92..f923311f3f20 100644 --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -67,10 +67,10 @@ def test_wait(self): ns = libregrtest._parse_args(['--wait']) self.assertTrue(ns.wait) - def test_slaveargs(self): - ns = libregrtest._parse_args(['--slaveargs', '[[], {}]']) - self.assertEqual(ns.slaveargs, '[[], {}]') - self.checkError(['--slaveargs'], 'expected one argument') + def test_worker_args(self): + ns = libregrtest._parse_args(['--worker-args', '[[], {}]']) + self.assertEqual(ns.worker_args, '[[], {}]') + self.checkError(['--worker-args'], 'expected one argument') def test_start(self): for opt in '-S', '--start': From webhook-mailer at python.org Fri Sep 7 11:30:36 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 15:30:36 -0000 Subject: [Python-checkins] bpo-34605: Avoid master/slave terms (GH-9101) Message-ID: https://github.com/python/cpython/commit/5e922658fb55734bf8b4c6246033ea93af172ff7 commit: 5e922658fb55734bf8b4c6246033ea93af172ff7 branch: master author: Victor Stinner committer: GitHub date: 2018-09-07T17:30:33+02:00 summary: bpo-34605: Avoid master/slave terms (GH-9101) * Replace "master process" with "parent process" * Replace "master option mappings" with "main option mappings" * Replace "master pattern object" with "main pattern object" * ssl: replace "master" with "server" * And some other similar changes files: M Doc/library/gc.rst M Doc/library/multiprocessing.rst M Lib/distutils/command/install.py M Lib/optparse.py M Lib/sre_parse.py M Lib/test/test_ssl.py M Tools/README diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst index 153d8fb70456..722a0e804314 100644 --- a/Doc/library/gc.rst +++ b/Doc/library/gc.rst @@ -181,7 +181,7 @@ The :mod:`gc` module provides the following functions: fork() call to make the gc copy-on-write friendly or to speed up collection. Also collection before a POSIX fork() call may free pages for future allocation which can cause copy-on-write too so it's advised to disable gc - in master process and freeze before fork and enable gc in child process. + in parent process and freeze before fork and enable gc in child process. .. versionadded:: 3.7 diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 504f3a1c3c33..a7d26f9b4554 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -435,7 +435,7 @@ process which created it. (If you try this it will actually output three full tracebacks interleaved in a semi-random fashion, and then you may have to - stop the master process somehow.) + stop the parent process somehow.) Reference diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py index 0258d3deae94..41bf4bb9fb22 100644 --- a/Lib/distutils/command/install.py +++ b/Lib/distutils/command/install.py @@ -223,7 +223,7 @@ def initialize_options(self): def finalize_options(self): """Finalizes options.""" - # This method (and its pliant slaves, like 'finalize_unix()', + # This method (and its pliant childs, like 'finalize_unix()', # 'finalize_other()', and 'select_scheme()') is where the default # installation directories for modules, extension modules, and # anything else we care to install from a Python module diff --git a/Lib/optparse.py b/Lib/optparse.py index e8ac1e156a2b..1c450c6fcbe3 100644 --- a/Lib/optparse.py +++ b/Lib/optparse.py @@ -929,7 +929,7 @@ def __init__(self, option_class, conflict_handler, description): self.set_description(description) def _create_option_mappings(self): - # For use by OptionParser constructor -- create the master + # For use by OptionParser constructor -- create the main # option mappings used by this OptionParser and all # OptionGroups that it owns. self._short_opt = {} # single letter -> Option instance diff --git a/Lib/sre_parse.py b/Lib/sre_parse.py index 7a172ff2fb14..f414402f9379 100644 --- a/Lib/sre_parse.py +++ b/Lib/sre_parse.py @@ -72,7 +72,7 @@ class Verbose(Exception): pass class Pattern: - # master pattern object. keeps track of global attributes + # main pattern object. keeps track of global attributes def __init__(self): self.flags = 0 self.groupdict = {} diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index 2cb4ad4a36d4..e120a2f9adf2 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -3494,7 +3494,7 @@ def test_no_shared_ciphers(self): client_context, server_context, hostname = testing_context() # OpenSSL enables all TLS 1.3 ciphers, enforce TLS 1.2 for test client_context.options |= ssl.OP_NO_TLSv1_3 - # Force different suites on client and master + # Force different suites on client and server client_context.set_ciphers("AES128") server_context.set_ciphers("AES256") with ThreadedEchoServer(context=server_context) as server: diff --git a/Tools/README b/Tools/README index 35528811fee9..6c5fb2081812 100644 --- a/Tools/README +++ b/Tools/README @@ -1,7 +1,7 @@ This directory contains a number of Python programs that are useful while building or extending Python. -buildbot Batchfiles for running on Windows buildslaves. +buildbot Batchfiles for running on Windows buildbot workers. ccbench A Python threads-based concurrency benchmark. (*) From webhook-mailer at python.org Fri Sep 7 11:44:27 2018 From: webhook-mailer at python.org (Pablo Galindo) Date: Fri, 07 Sep 2018 15:44:27 -0000 Subject: [Python-checkins] bpo-20104: Add flag capabilities to posix_spawn (GH-6693) Message-ID: https://github.com/python/cpython/commit/254a4663d8c5970ae2928185c50ebaa6c7e62c80 commit: 254a4663d8c5970ae2928185c50ebaa6c7e62c80 branch: master author: Pablo Galindo committer: GitHub date: 2018-09-07T16:44:24+01:00 summary: bpo-20104: Add flag capabilities to posix_spawn (GH-6693) Implement the "attributes objects" parameter of `os.posix_spawn` to complete the implementation and fully cover the underlying API. files: A Misc/NEWS.d/next/Core and Builtins/2018-05-05-23-26-58.bpo-20104.tDBciE.rst M Doc/library/os.rst M Lib/test/test_posix.py M Modules/clinic/posixmodule.c.h M Modules/posixmodule.c diff --git a/Doc/library/os.rst b/Doc/library/os.rst index df136da02cb1..b8d6fffb303b 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -3394,7 +3394,9 @@ written in Python, such as a mail server's external command delivery program. subprocesses. -.. function:: posix_spawn(path, argv, env, file_actions=None) +.. function:: posix_spawn(path, argv, env, file_actions=None, /, *, \ + setpgroup=None, resetids=False, setsigmask=(), \ + setsigdef=(), scheduler=None) Wraps the :c:func:`posix_spawn` C library API for use from Python. @@ -3432,6 +3434,36 @@ written in Python, such as a mail server's external command delivery program. :c:func:`posix_spawn_file_actions_adddup2` API calls used to prepare for the :c:func:`posix_spawn` call itself. + The *setpgroup* argument will set the process group of the child to the value + specified. If the value specified is 0, the child's process group ID will be + made the same as its process ID. If the value of *setpgroup* is not set, the + child will inherit the parent's process group ID. This argument corresponds + to the C library :c:data:`POSIX_SPAWN_SETPGROUP` flag. + + If the *resetids* argument is ``True`` it will reset the effective UID and + GID of the child to the real UID and GID of the parent process. If the + argument is ``False``, then the child retains the effective UID and GID of + the parent. In either case, if the set-user-ID and set-group-ID permission + bits are enabled on the executable file, their effect will override the + setting of the effective UID and GID. This argument corresponds to the C + library :c:data:`POSIX_SPAWN_RESETIDS` flag. + + The *setsigmask* argument will set the signal mask to the signal set + specified. If the parameter is not used, then the child inherits the + parent's signal mask. This argument corresponds to the C library + :c:data:`POSIX_SPAWN_SETSIGMASK` flag. + + The *sigdef* argument will reset the disposition of all signals in the set + specified. This argument corresponds to the C library + :c:data:`POSIX_SPAWN_SETSIGDEF` flag. + + The *scheduler* argument must be a tuple containing the (optional) scheduler + policy and an instance of :class:`sched_param` with the scheduler parameters. + A value of ``None`` in the place of the scheduler policy indicates that is + not being provided. This argument is a combination of the C library + :c:data:`POSIX_SPAWN_SETSCHEDPARAM` and :c:data:`POSIX_SPAWN_SETSCHEDULER` + flags. + .. versionadded:: 3.7 diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py index ac18aa32528d..dafe9c1350e4 100644 --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -8,6 +8,7 @@ import errno import sys +import signal import time import os import platform @@ -16,6 +17,7 @@ import tempfile import unittest import warnings +import textwrap _DUMMY_SYMLINK = os.path.join(tempfile.gettempdir(), support.TESTFN + '-dummy-symlink') @@ -1540,6 +1542,147 @@ def test_empty_file_actions(self): ) self.assertEqual(os.waitpid(pid, 0), (pid, 0)) + def test_resetids_explicit_default(self): + pid = posix.posix_spawn( + sys.executable, + [sys.executable, '-c', 'pass'], + os.environ, + resetids=False + ) + self.assertEqual(os.waitpid(pid, 0), (pid, 0)) + + def test_resetids(self): + pid = posix.posix_spawn( + sys.executable, + [sys.executable, '-c', 'pass'], + os.environ, + resetids=True + ) + self.assertEqual(os.waitpid(pid, 0), (pid, 0)) + + def test_resetids_wrong_type(self): + with self.assertRaises(TypeError): + posix.posix_spawn(sys.executable, + [sys.executable, "-c", "pass"], + os.environ, resetids=None) + + def test_setpgroup(self): + pid = posix.posix_spawn( + sys.executable, + [sys.executable, '-c', 'pass'], + os.environ, + setpgroup=os.getpgrp() + ) + self.assertEqual(os.waitpid(pid, 0), (pid, 0)) + + def test_setpgroup_wrong_type(self): + with self.assertRaises(TypeError): + posix.posix_spawn(sys.executable, + [sys.executable, "-c", "pass"], + os.environ, setpgroup="023") + + @unittest.skipUnless(hasattr(signal, 'pthread_sigmask'), + 'need signal.pthread_sigmask()') + def test_setsigmask(self): + code = textwrap.dedent("""\ + import _testcapi, signal + _testcapi.raise_signal(signal.SIGUSR1)""") + + pid = posix.posix_spawn( + sys.executable, + [sys.executable, '-c', code], + os.environ, + setsigmask=[signal.SIGUSR1] + ) + self.assertEqual(os.waitpid(pid, 0), (pid, 0)) + + def test_setsigmask_wrong_type(self): + with self.assertRaises(TypeError): + posix.posix_spawn(sys.executable, + [sys.executable, "-c", "pass"], + os.environ, setsigmask=34) + with self.assertRaises(TypeError): + posix.posix_spawn(sys.executable, + [sys.executable, "-c", "pass"], + os.environ, setsigmask=["j"]) + with self.assertRaises(ValueError): + posix.posix_spawn(sys.executable, + [sys.executable, "-c", "pass"], + os.environ, setsigmask=[signal.NSIG, + signal.NSIG+1]) + + @unittest.skipUnless(hasattr(signal, 'pthread_sigmask'), + 'need signal.pthread_sigmask()') + def test_setsigdef(self): + original_handler = signal.signal(signal.SIGUSR1, signal.SIG_IGN) + code = textwrap.dedent("""\ + import _testcapi, signal + _testcapi.raise_signal(signal.SIGUSR1)""") + try: + pid = posix.posix_spawn( + sys.executable, + [sys.executable, '-c', code], + os.environ, + setsigdef=[signal.SIGUSR1] + ) + finally: + signal.signal(signal.SIGUSR1, original_handler) + + pid2, status = os.waitpid(pid, 0) + self.assertEqual(pid2, pid) + self.assertTrue(os.WIFSIGNALED(status), status) + self.assertEqual(os.WTERMSIG(status), signal.SIGUSR1) + + def test_setsigdef_wrong_type(self): + with self.assertRaises(TypeError): + posix.posix_spawn(sys.executable, + [sys.executable, "-c", "pass"], + os.environ, setsigdef=34) + with self.assertRaises(TypeError): + posix.posix_spawn(sys.executable, + [sys.executable, "-c", "pass"], + os.environ, setsigdef=["j"]) + with self.assertRaises(ValueError): + posix.posix_spawn(sys.executable, + [sys.executable, "-c", "pass"], + os.environ, setsigdef=[signal.NSIG, signal.NSIG+1]) + + @unittest.skipUnless(hasattr(posix, 'sched_setscheduler'), "can't change scheduler") + def test_setscheduler_only_param(self): + policy = os.sched_getscheduler(0) + priority = os.sched_get_priority_min(policy) + code = textwrap.dedent(f"""\ + import os + if os.sched_getscheduler(0) != {policy}: + os.exit(101) + if os.sched_getparam(0).sched_priority != {priority}: + os.exit(102)""") + pid = posix.posix_spawn( + sys.executable, + [sys.executable, '-c', code], + os.environ, + scheduler=(None, os.sched_param(priority)) + ) + self.assertEqual(os.waitpid(pid, 0), (pid, 0)) + + @unittest.skipUnless(hasattr(posix, 'sched_setscheduler'), "can't change scheduler") + def test_setscheduler_with_policy(self): + policy = os.sched_getscheduler(0) + priority = os.sched_get_priority_min(policy) + code = textwrap.dedent(f"""\ + import os + if os.sched_getscheduler(0) != {policy}: + os.exit(101) + if os.sched_getparam(0).sched_priority != {priority}: + os.exit(102)""") + pid = posix.posix_spawn( + sys.executable, + [sys.executable, '-c', code], + os.environ, + scheduler=(policy, os.sched_param(priority)) + ) + self.assertEqual(os.waitpid(pid, 0), (pid, 0)) + def test_multiple_file_actions(self): file_actions = [ (os.POSIX_SPAWN_OPEN, 3, os.path.realpath(__file__), os.O_RDONLY, 0), diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-05-05-23-26-58.bpo-20104.tDBciE.rst b/Misc/NEWS.d/next/Core and Builtins/2018-05-05-23-26-58.bpo-20104.tDBciE.rst new file mode 100644 index 000000000000..1d725ba7eb71 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-05-05-23-26-58.bpo-20104.tDBciE.rst @@ -0,0 +1,2 @@ +Added support for the `setpgroup`, `resetids`, `setsigmask`, `setsigdef` and +`scheduler` parameters of `posix_spawn`. Patch by Pablo Galindo. diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h index 7a2188504ac5..f7767c4af0bb 100644 --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -1730,7 +1730,9 @@ os_execve(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *k #if defined(HAVE_POSIX_SPAWN) PyDoc_STRVAR(os_posix_spawn__doc__, -"posix_spawn($module, path, argv, env, file_actions=None, /)\n" +"posix_spawn($module, path, argv, env, file_actions=None, /, *,\n" +" setpgroup=None, resetids=False, setsigmask=(),\n" +" setsigdef=(), scheduler=None)\n" "--\n" "\n" "Execute the program specified by path in a new process.\n" @@ -1742,29 +1744,48 @@ PyDoc_STRVAR(os_posix_spawn__doc__, " env\n" " Dictionary of strings mapping to strings.\n" " file_actions\n" -" A sequence of file action tuples."); +" A sequence of file action tuples.\n" +" setpgroup\n" +" The pgroup to use with the POSIX_SPAWN_SETPGROUP flag.\n" +" resetids\n" +" If the value is `True` the POSIX_SPAWN_RESETIDS will be activated.\n" +" setsigmask\n" +" The sigmask to use with the POSIX_SPAWN_SETSIGMASK flag.\n" +" setsigdef\n" +" The sigmask to use with the POSIX_SPAWN_SETSIGDEF flag.\n" +" scheduler\n" +" A tuple with the scheduler policy (optional) and parameters."); #define OS_POSIX_SPAWN_METHODDEF \ - {"posix_spawn", (PyCFunction)os_posix_spawn, METH_FASTCALL, os_posix_spawn__doc__}, + {"posix_spawn", (PyCFunction)os_posix_spawn, METH_FASTCALL|METH_KEYWORDS, os_posix_spawn__doc__}, static PyObject * os_posix_spawn_impl(PyObject *module, path_t *path, PyObject *argv, - PyObject *env, PyObject *file_actions); + PyObject *env, PyObject *file_actions, + PyObject *setpgroup, int resetids, PyObject *setsigmask, + PyObject *setsigdef, PyObject *scheduler); static PyObject * -os_posix_spawn(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +os_posix_spawn(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", "", "", "", "setpgroup", "resetids", "setsigmask", "setsigdef", "scheduler", NULL}; + static _PyArg_Parser _parser = {"O&OO|O$OiOOO:posix_spawn", _keywords, 0}; path_t path = PATH_T_INITIALIZE("posix_spawn", "path", 0, 0); PyObject *argv; PyObject *env; PyObject *file_actions = Py_None; + PyObject *setpgroup = NULL; + int resetids = 0; + PyObject *setsigmask = NULL; + PyObject *setsigdef = NULL; + PyObject *scheduler = NULL; - if (!_PyArg_ParseStack(args, nargs, "O&OO|O:posix_spawn", - path_converter, &path, &argv, &env, &file_actions)) { + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + path_converter, &path, &argv, &env, &file_actions, &setpgroup, &resetids, &setsigmask, &setsigdef, &scheduler)) { goto exit; } - return_value = os_posix_spawn_impl(module, &path, argv, env, file_actions); + return_value = os_posix_spawn_impl(module, &path, argv, env, file_actions, setpgroup, resetids, setsigmask, setsigdef, scheduler); exit: /* Cleanup for path */ @@ -6627,4 +6648,4 @@ os_getrandom(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject #ifndef OS_GETRANDOM_METHODDEF #define OS_GETRANDOM_METHODDEF #endif /* !defined(OS_GETRANDOM_METHODDEF) */ -/*[clinic end generated code: output=47fb6a3e88cba6d9 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=ef78384ae88712e1 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 7f13735eadc8..4a8a8d70ae42 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -5177,6 +5177,114 @@ enum posix_spawn_file_actions_identifier { POSIX_SPAWN_DUP2 }; +static int +convert_sched_param(PyObject *param, struct sched_param *res); + +static int +parse_posix_spawn_flags(PyObject *setpgroup, int resetids, PyObject *setsigmask, + PyObject *setsigdef, PyObject *scheduler, + posix_spawnattr_t *attrp) +{ + long all_flags = 0; + + errno = posix_spawnattr_init(attrp); + if (errno) { + posix_error(); + return -1; + } + + if (setpgroup) { + pid_t pgid = PyLong_AsPid(setpgroup); + if (pgid == (pid_t)-1 && PyErr_Occurred()) { + goto fail; + } + errno = posix_spawnattr_setpgroup(attrp, pgid); + if (errno) { + posix_error(); + goto fail; + } + all_flags |= POSIX_SPAWN_SETPGROUP; + } + + if (resetids) { + all_flags |= POSIX_SPAWN_RESETIDS; + } + + if (setsigmask) { + sigset_t set; + if (!_Py_Sigset_Converter(setsigmask, &set)) { + goto fail; + } + errno = posix_spawnattr_setsigmask(attrp, &set); + if (errno) { + posix_error(); + goto fail; + } + all_flags |= POSIX_SPAWN_SETSIGMASK; + } + + if (setsigdef) { + sigset_t set; + if (!_Py_Sigset_Converter(setsigdef, &set)) { + goto fail; + } + errno = posix_spawnattr_setsigdefault(attrp, &set); + if (errno) { + posix_error(); + goto fail; + } + all_flags |= POSIX_SPAWN_SETSIGDEF; + } + + if (scheduler) { +#ifdef POSIX_SPAWN_SETSCHEDULER + PyObject *py_schedpolicy; + struct sched_param schedparam; + + if (!PyArg_ParseTuple(scheduler, "OO&" + ";A scheduler tuple must have two elements", + &py_schedpolicy, convert_sched_param, &schedparam)) { + goto fail; + } + if (py_schedpolicy != Py_None) { + int schedpolicy = _PyLong_AsInt(py_schedpolicy); + + if (schedpolicy == -1 && PyErr_Occurred()) { + goto fail; + } + errno = posix_spawnattr_setschedpolicy(attrp, schedpolicy); + if (errno) { + posix_error(); + goto fail; + } + all_flags |= POSIX_SPAWN_SETSCHEDULER; + } + errno = posix_spawnattr_setschedparam(attrp, &schedparam); + if (errno) { + posix_error(); + goto fail; + } + all_flags |= POSIX_SPAWN_SETSCHEDPARAM; +#else + PyErr_SetString(PyExc_NotImplementedError, + "The scheduler option is not supported in this system."); + goto fail; +#endif + } + + errno = posix_spawnattr_setflags(attrp, all_flags); + if (errno) { + posix_error(); + goto fail; + } + + return 0; + +fail: + (void)posix_spawnattr_destroy(attrp); + return -1; +} + static int parse_file_actions(PyObject *file_actions, posix_spawn_file_actions_t *file_actionsp, @@ -5277,6 +5385,7 @@ parse_file_actions(PyObject *file_actions, } Py_DECREF(file_action); } + Py_DECREF(seq); return 0; @@ -5299,19 +5408,33 @@ os.posix_spawn file_actions: object = None A sequence of file action tuples. / - + * + setpgroup: object = NULL + The pgroup to use with the POSIX_SPAWN_SETPGROUP flag. + resetids: bool(accept={int}) = False + If the value is `True` the POSIX_SPAWN_RESETIDS will be activated. + setsigmask: object(c_default='NULL') = () + The sigmask to use with the POSIX_SPAWN_SETSIGMASK flag. + setsigdef: object(c_default='NULL') = () + The sigmask to use with the POSIX_SPAWN_SETSIGDEF flag. + scheduler: object = NULL + A tuple with the scheduler policy (optional) and parameters. Execute the program specified by path in a new process. [clinic start generated code]*/ static PyObject * os_posix_spawn_impl(PyObject *module, path_t *path, PyObject *argv, - PyObject *env, PyObject *file_actions) -/*[clinic end generated code: output=d023521f541c709c input=a3db1021d33230dc]*/ + PyObject *env, PyObject *file_actions, + PyObject *setpgroup, int resetids, PyObject *setsigmask, + PyObject *setsigdef, PyObject *scheduler) +/*[clinic end generated code: output=45dfa4c515d09f2c input=2d7a7578430a90f0]*/ { EXECV_CHAR **argvlist = NULL; EXECV_CHAR **envlist = NULL; posix_spawn_file_actions_t file_actions_buf; posix_spawn_file_actions_t *file_actionsp = NULL; + posix_spawnattr_t attr; + posix_spawnattr_t *attrp = NULL; Py_ssize_t argc, envc; PyObject *result = NULL; PyObject *temp_buffer = NULL; @@ -5373,9 +5496,15 @@ os_posix_spawn_impl(PyObject *module, path_t *path, PyObject *argv, file_actionsp = &file_actions_buf; } + if (parse_posix_spawn_flags(setpgroup, resetids, setsigmask, + setsigdef, scheduler, &attr)) { + goto exit; + } + attrp = &attr; + _Py_BEGIN_SUPPRESS_IPH err_code = posix_spawn(&pid, path->narrow, - file_actionsp, NULL, argvlist, envlist); + file_actionsp, attrp, argvlist, envlist); _Py_END_SUPPRESS_IPH if (err_code) { errno = err_code; @@ -5388,6 +5517,9 @@ os_posix_spawn_impl(PyObject *module, path_t *path, PyObject *argv, if (file_actionsp) { (void)posix_spawn_file_actions_destroy(file_actionsp); } + if (attrp) { + (void)posix_spawnattr_destroy(attrp); + } if (envlist) { free_string_array(envlist, envc); } From webhook-mailer at python.org Fri Sep 7 12:01:04 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 16:01:04 -0000 Subject: [Python-checkins] bpo-34595: Add %T format to PyUnicode_FromFormatV() (GH-9080) Message-ID: https://github.com/python/cpython/commit/886483e2b9bbabf60ab769683269b873381dd5ee commit: 886483e2b9bbabf60ab769683269b873381dd5ee branch: master author: Victor Stinner committer: GitHub date: 2018-09-07T18:00:58+02:00 summary: bpo-34595: Add %T format to PyUnicode_FromFormatV() (GH-9080) * Add %T format to PyUnicode_FromFormatV(), and so to PyUnicode_FromFormat() and PyErr_Format(), to format an object type name: equivalent to "%s" with Py_TYPE(obj)->tp_name. * Replace Py_TYPE(obj)->tp_name with %T format in unicodeobject.c. * Add unit test on %T format. * Rename unicode_fromformat_write_cstr() to unicode_fromformat_write_utf8(), to make the intent more explicit. files: A Misc/NEWS.d/next/C API/2018-09-06-11-17-49.bpo-34595.Hkz62y.rst M Doc/c-api/unicode.rst M Lib/test/test_unicode.py M Objects/unicodeobject.c diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 92e22b16a4ef..66b1efc60fde 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -519,6 +519,9 @@ APIs: | :attr:`%R` | PyObject\* | The result of calling | | | | :c:func:`PyObject_Repr`. | +-------------------+---------------------+--------------------------------+ + | :attr:`%T` | PyObject\* | Object type name, equivalent | + | | | to ``Py_TYPE(op)->tp_name``. | + +-------------------+---------------------+--------------------------------+ An unrecognized format character causes all the rest of the format string to be copied as-is to the result string, and any extra arguments discarded. @@ -543,6 +546,9 @@ APIs: Support width and precision formatter for ``"%s"``, ``"%A"``, ``"%U"``, ``"%V"``, ``"%S"``, ``"%R"`` added. + .. versionchanged:: 3.7 + Support for ``"%T"`` (object type name) added. + .. c:function:: PyObject* PyUnicode_FromFormatV(const char *format, va_list vargs) diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py index fb7bb2d523fe..73111f1c24a2 100644 --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -2655,6 +2655,10 @@ def check_format(expected, format, *args): check_format(r"%A:'abc\xe9\uabcd\U0010ffff'", b'%%A:%A', 'abc\xe9\uabcd\U0010ffff') + # test %T (object type name) + check_format(r"type name: str", + b'type name: %T', 'text') + # test %V check_format('repr=abc', b'repr=%V', 'abc', b'xyz') diff --git a/Misc/NEWS.d/next/C API/2018-09-06-11-17-49.bpo-34595.Hkz62y.rst b/Misc/NEWS.d/next/C API/2018-09-06-11-17-49.bpo-34595.Hkz62y.rst new file mode 100644 index 000000000000..c054a8e229f5 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2018-09-06-11-17-49.bpo-34595.Hkz62y.rst @@ -0,0 +1,4 @@ +:c:func:`PyUnicode_FromFormatV`: add ``%T`` format to +:c:func:`PyUnicode_FromFormatV`, and so to :c:func:`PyUnicode_FromFormat` +and :c:func:`PyErr_Format`, to format an object type name: equivalent to +"%s" with ``Py_TYPE(obj)->tp_name``. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index a797f838eb41..3e61c9c37010 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -768,8 +768,7 @@ ensure_unicode(PyObject *obj) { if (!PyUnicode_Check(obj)) { PyErr_Format(PyExc_TypeError, - "must be str, not %.100s", - Py_TYPE(obj)->tp_name); + "must be str, not %T", obj); return -1; } return PyUnicode_READY(obj); @@ -2530,7 +2529,7 @@ unicode_fromformat_write_str(_PyUnicodeWriter *writer, PyObject *str, } static int -unicode_fromformat_write_cstr(_PyUnicodeWriter *writer, const char *str, +unicode_fromformat_write_utf8(_PyUnicodeWriter *writer, const char *str, Py_ssize_t width, Py_ssize_t precision) { /* UTF-8 */ @@ -2747,7 +2746,7 @@ unicode_fromformat_arg(_PyUnicodeWriter *writer, { /* UTF-8 */ const char *s = va_arg(*vargs, const char*); - if (unicode_fromformat_write_cstr(writer, s, width, precision) < 0) + if (unicode_fromformat_write_utf8(writer, s, width, precision) < 0) return NULL; break; } @@ -2773,7 +2772,7 @@ unicode_fromformat_arg(_PyUnicodeWriter *writer, } else { assert(str != NULL); - if (unicode_fromformat_write_cstr(writer, str, width, precision) < 0) + if (unicode_fromformat_write_utf8(writer, str, width, precision) < 0) return NULL; } break; @@ -2827,6 +2826,17 @@ unicode_fromformat_arg(_PyUnicodeWriter *writer, break; } + case 'T': + { + /* Object type name (tp_name) */ + PyObject *obj = va_arg(*vargs, PyObject *); + PyTypeObject *type = Py_TYPE(obj); + const char *type_name = type->tp_name; + if (unicode_fromformat_write_utf8(writer, type_name, -1, -1) < 0) { + return NULL; + } + break; + } case '%': if (_PyUnicodeWriter_WriteCharInline(writer, '%') < 0) return NULL; @@ -3024,8 +3034,7 @@ PyUnicode_FromObject(PyObject *obj) return _PyUnicode_Copy(obj); } PyErr_Format(PyExc_TypeError, - "Can't convert '%.100s' object to str implicitly", - Py_TYPE(obj)->tp_name); + "Can't convert '%T' object to str implicitly", obj); return NULL; } @@ -3061,8 +3070,8 @@ PyUnicode_FromEncodedObject(PyObject *obj, /* Retrieve a bytes buffer view through the PEP 3118 buffer interface */ if (PyObject_GetBuffer(obj, &buffer, PyBUF_SIMPLE) < 0) { PyErr_Format(PyExc_TypeError, - "decoding to str: need a bytes-like object, %.80s found", - Py_TYPE(obj)->tp_name); + "decoding to str: need a bytes-like object, %T found", + obj); return NULL; } @@ -3192,10 +3201,9 @@ PyUnicode_Decode(const char *s, goto onError; if (!PyUnicode_Check(unicode)) { PyErr_Format(PyExc_TypeError, - "'%.400s' decoder returned '%.400s' instead of 'str'; " + "'%.400s' decoder returned '%T' instead of 'str'; " "use codecs.decode() to decode to arbitrary types", - encoding, - Py_TYPE(unicode)->tp_name); + encoding, unicode); Py_DECREF(unicode); goto onError; } @@ -3255,10 +3263,9 @@ PyUnicode_AsDecodedUnicode(PyObject *unicode, goto onError; if (!PyUnicode_Check(v)) { PyErr_Format(PyExc_TypeError, - "'%.400s' decoder returned '%.400s' instead of 'str'; " + "'%.400s' decoder returned '%T' instead of 'str'; " "use codecs.decode() to decode to arbitrary types", - encoding, - Py_TYPE(unicode)->tp_name); + encoding, unicode); Py_DECREF(v); goto onError; } @@ -3489,10 +3496,9 @@ PyUnicode_AsEncodedString(PyObject *unicode, } PyErr_Format(PyExc_TypeError, - "'%.400s' encoder returned '%.400s' instead of 'bytes'; " + "'%.400s' encoder returned '%T' instead of 'bytes'; " "use codecs.encode() to encode to arbitrary types", - encoding, - Py_TYPE(v)->tp_name); + encoding, v); Py_DECREF(v); return NULL; } @@ -3523,10 +3529,9 @@ PyUnicode_AsEncodedUnicode(PyObject *unicode, goto onError; if (!PyUnicode_Check(v)) { PyErr_Format(PyExc_TypeError, - "'%.400s' encoder returned '%.400s' instead of 'str'; " + "'%.400s' encoder returned '%T' instead of 'str'; " "use codecs.encode() to encode to arbitrary types", - encoding, - Py_TYPE(v)->tp_name); + encoding, v); Py_DECREF(v); goto onError; } @@ -3698,9 +3703,11 @@ PyUnicode_FSDecoder(PyObject* arg, void* addr) if (!PyBytes_Check(path) && PyErr_WarnFormat(PyExc_DeprecationWarning, 1, - "path should be string, bytes, or os.PathLike, not %.200s", - Py_TYPE(arg)->tp_name)) { - Py_DECREF(path); + "path should be string, bytes, " + "or os.PathLike, not %T", + arg)) + { + Py_DECREF(path); return 0; } path_bytes = PyBytes_FromObject(path); @@ -3717,8 +3724,8 @@ PyUnicode_FSDecoder(PyObject* arg, void* addr) } else { PyErr_Format(PyExc_TypeError, - "path should be string, bytes, or os.PathLike, not %.200s", - Py_TYPE(arg)->tp_name); + "path should be string, bytes, or os.PathLike, not %T", + arg); Py_DECREF(path); return 0; } @@ -9886,9 +9893,8 @@ _PyUnicode_JoinArray(PyObject *separator, PyObject *const *items, Py_ssize_t seq else { if (!PyUnicode_Check(separator)) { PyErr_Format(PyExc_TypeError, - "separator: expected str instance," - " %.80s found", - Py_TYPE(separator)->tp_name); + "separator: expected str instance, %T found", + separator); goto onError; } if (PyUnicode_READY(separator)) @@ -9919,9 +9925,8 @@ _PyUnicode_JoinArray(PyObject *separator, PyObject *const *items, Py_ssize_t seq item = items[i]; if (!PyUnicode_Check(item)) { PyErr_Format(PyExc_TypeError, - "sequence item %zd: expected str instance," - " %.80s found", - i, Py_TYPE(item)->tp_name); + "sequence item %zd: expected str instance, %T found", + i, item); goto onError; } if (PyUnicode_READY(item) == -1) @@ -10736,7 +10741,7 @@ convert_uc(PyObject *obj, void *addr) if (!PyUnicode_Check(obj)) { PyErr_Format(PyExc_TypeError, "The fill character must be a unicode character, " - "not %.100s", Py_TYPE(obj)->tp_name); + "not %T", obj); return 0; } if (PyUnicode_READY(obj) < 0) @@ -11142,8 +11147,8 @@ PyUnicode_Contains(PyObject *str, PyObject *substr) if (!PyUnicode_Check(substr)) { PyErr_Format(PyExc_TypeError, - "'in ' requires string as left operand, not %.100s", - Py_TYPE(substr)->tp_name); + "'in ' requires string as left operand, not %T", + substr); return -1; } if (PyUnicode_READY(substr) == -1) @@ -12848,9 +12853,7 @@ unicode_split_impl(PyObject *self, PyObject *sep, Py_ssize_t maxsplit) if (PyUnicode_Check(sep)) return split(self, sep, maxsplit); - PyErr_Format(PyExc_TypeError, - "must be str or None, not %.100s", - Py_TYPE(sep)->tp_name); + PyErr_Format(PyExc_TypeError, "must be str or None, not %T", sep); return NULL; } @@ -13036,9 +13039,7 @@ unicode_rsplit_impl(PyObject *self, PyObject *sep, Py_ssize_t maxsplit) if (PyUnicode_Check(sep)) return rsplit(self, sep, maxsplit); - PyErr_Format(PyExc_TypeError, - "must be str or None, not %.100s", - Py_TYPE(sep)->tp_name); + PyErr_Format(PyExc_TypeError, "must be str or None, not %T", sep); return NULL; } @@ -13333,8 +13334,8 @@ unicode_startswith(PyObject *self, if (!PyUnicode_Check(substring)) { PyErr_Format(PyExc_TypeError, "tuple for startswith must only contain str, " - "not %.100s", - Py_TYPE(substring)->tp_name); + "not %T", + substring); return NULL; } result = tailmatch(self, substring, start, end, -1); @@ -13350,7 +13351,7 @@ unicode_startswith(PyObject *self, if (!PyUnicode_Check(subobj)) { PyErr_Format(PyExc_TypeError, "startswith first arg must be str or " - "a tuple of str, not %.100s", Py_TYPE(subobj)->tp_name); + "a tuple of str, not %T", subobj); return NULL; } result = tailmatch(self, subobj, start, end, -1); @@ -13387,8 +13388,8 @@ unicode_endswith(PyObject *self, if (!PyUnicode_Check(substring)) { PyErr_Format(PyExc_TypeError, "tuple for endswith must only contain str, " - "not %.100s", - Py_TYPE(substring)->tp_name); + "not %T", + substring); return NULL; } result = tailmatch(self, substring, start, end, +1); @@ -13403,7 +13404,7 @@ unicode_endswith(PyObject *self, if (!PyUnicode_Check(subobj)) { PyErr_Format(PyExc_TypeError, "endswith first arg must be str or " - "a tuple of str, not %.100s", Py_TYPE(subobj)->tp_name); + "a tuple of str, not %T", subobj); return NULL; } result = tailmatch(self, subobj, start, end, +1); @@ -14313,15 +14314,13 @@ mainformatlong(PyObject *v, case 'x': case 'X': PyErr_Format(PyExc_TypeError, - "%%%c format: an integer is required, " - "not %.200s", - type, Py_TYPE(v)->tp_name); + "%%%c format: an integer is required, not %T", + type, v); break; default: PyErr_Format(PyExc_TypeError, - "%%%c format: a number is required, " - "not %.200s", - type, Py_TYPE(v)->tp_name); + "%%%c format: a number is required, not %T", + type, v); break; } return -1; From webhook-mailer at python.org Fri Sep 7 12:13:14 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 16:13:14 -0000 Subject: [Python-checkins] bpo-34605: childs => children (GH-9102) Message-ID: https://github.com/python/cpython/commit/7e610bcdf128f61b925654e4fa80fbac83537d0e commit: 7e610bcdf128f61b925654e4fa80fbac83537d0e branch: master author: Victor Stinner committer: GitHub date: 2018-09-07T18:13:10+02:00 summary: bpo-34605: childs => children (GH-9102) files: M Lib/distutils/command/install.py diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py index 41bf4bb9fb22..a1d1a1ea37ad 100644 --- a/Lib/distutils/command/install.py +++ b/Lib/distutils/command/install.py @@ -223,7 +223,7 @@ def initialize_options(self): def finalize_options(self): """Finalizes options.""" - # This method (and its pliant childs, like 'finalize_unix()', + # This method (and its pliant children, like 'finalize_unix()', # 'finalize_other()', and 'select_scheme()') is where the default # installation directories for modules, extension modules, and # anything else we care to install from a Python module From webhook-mailer at python.org Fri Sep 7 12:17:13 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 07 Sep 2018 16:17:13 -0000 Subject: [Python-checkins] [3.6] bpo-34594: Don't hardcode errno values in the tests. (GH-9096) Message-ID: https://github.com/python/cpython/commit/9eeecfd5d03cfd6d5cef71de31eed7f65f0e658b commit: 9eeecfd5d03cfd6d5cef71de31eed7f65f0e658b branch: 3.6 author: Zackery Spytz committer: Benjamin Peterson date: 2018-09-07T09:17:08-07:00 summary: [3.6] bpo-34594: Don't hardcode errno values in the tests. (GH-9096) (cherry picked from commit b03c2c51909e3b5b5966d86a2829b5ddf2d496aa) files: A Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst M Lib/test/test_spwd.py diff --git a/Lib/test/test_spwd.py b/Lib/test/test_spwd.py index e893f3a847fd..07793c84c8e9 100644 --- a/Lib/test/test_spwd.py +++ b/Lib/test/test_spwd.py @@ -67,8 +67,6 @@ def test_getspnam_exception(self): spwd.getspnam(name) except KeyError as exc: self.skipTest("spwd entry %r doesn't exist: %s" % (name, exc)) - else: - self.assertEqual(str(cm.exception), '[Errno 13] Permission denied') if __name__ == "__main__": diff --git a/Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst b/Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst new file mode 100644 index 000000000000..c6026b0f717b --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst @@ -0,0 +1 @@ +Fix usage of hardcoded ``errno`` values in the tests. \ No newline at end of file From webhook-mailer at python.org Fri Sep 7 12:17:36 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 07 Sep 2018 16:17:36 -0000 Subject: [Python-checkins] [3.7] bpo-34594: Don't hardcode errno values in the tests. (GH-9094) Message-ID: https://github.com/python/cpython/commit/18d7dff1bb6f5ca7060b0b2e2a2e74493619178f commit: 18d7dff1bb6f5ca7060b0b2e2a2e74493619178f branch: 3.7 author: Zackery Spytz committer: Benjamin Peterson date: 2018-09-07T09:17:32-07:00 summary: [3.7] bpo-34594: Don't hardcode errno values in the tests. (GH-9094) (cherry picked from commit b03c2c51909e3b5b5966d86a2829b5ddf2d496aa) files: A Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst M Lib/test/test_spwd.py diff --git a/Lib/test/test_spwd.py b/Lib/test/test_spwd.py index e893f3a847fd..07793c84c8e9 100644 --- a/Lib/test/test_spwd.py +++ b/Lib/test/test_spwd.py @@ -67,8 +67,6 @@ def test_getspnam_exception(self): spwd.getspnam(name) except KeyError as exc: self.skipTest("spwd entry %r doesn't exist: %s" % (name, exc)) - else: - self.assertEqual(str(cm.exception), '[Errno 13] Permission denied') if __name__ == "__main__": diff --git a/Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst b/Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst new file mode 100644 index 000000000000..c6026b0f717b --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-05-23-50-21.bpo-34594.tqL-GS.rst @@ -0,0 +1 @@ +Fix usage of hardcoded ``errno`` values in the tests. \ No newline at end of file From webhook-mailer at python.org Fri Sep 7 13:10:43 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 07 Sep 2018 17:10:43 -0000 Subject: [Python-checkins] bpo-34604: Fix possible mojibake in pwd.getpwnam() and grp.getgrnam() (GH-9098) Message-ID: https://github.com/python/cpython/commit/28658485a54ad5f9df52ecc12d9046269f1654ec commit: 28658485a54ad5f9df52ecc12d9046269f1654ec branch: master author: William Grzybowski committer: Victor Stinner date: 2018-09-07T19:10:39+02:00 summary: bpo-34604: Fix possible mojibake in pwd.getpwnam() and grp.getgrnam() (GH-9098) Pass the user/group name as Unicode to the formatting function, instead of always decoding a bytes string from UTF-8. files: A Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst M Modules/clinic/pwdmodule.c.h M Modules/grpmodule.c M Modules/pwdmodule.c diff --git a/Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst b/Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst new file mode 100644 index 000000000000..562a69124b3d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst @@ -0,0 +1,2 @@ +Fix possible mojibake in the error message of `pwd.getpwnam` and +`grp.getgrnam`. Patch by William Grzybowski. diff --git a/Modules/clinic/pwdmodule.c.h b/Modules/clinic/pwdmodule.c.h index f9e0644f264d..979dfdb07ee2 100644 --- a/Modules/clinic/pwdmodule.c.h +++ b/Modules/clinic/pwdmodule.c.h @@ -14,7 +14,7 @@ PyDoc_STRVAR(pwd_getpwuid__doc__, {"getpwuid", (PyCFunction)pwd_getpwuid, METH_O, pwd_getpwuid__doc__}, PyDoc_STRVAR(pwd_getpwnam__doc__, -"getpwnam($module, arg, /)\n" +"getpwnam($module, name, /)\n" "--\n" "\n" "Return the password database entry for the given user name.\n" @@ -25,18 +25,18 @@ PyDoc_STRVAR(pwd_getpwnam__doc__, {"getpwnam", (PyCFunction)pwd_getpwnam, METH_O, pwd_getpwnam__doc__}, static PyObject * -pwd_getpwnam_impl(PyObject *module, PyObject *arg); +pwd_getpwnam_impl(PyObject *module, PyObject *name); static PyObject * -pwd_getpwnam(PyObject *module, PyObject *arg_) +pwd_getpwnam(PyObject *module, PyObject *arg) { PyObject *return_value = NULL; - PyObject *arg; + PyObject *name; - if (!PyArg_Parse(arg_, "U:getpwnam", &arg)) { + if (!PyArg_Parse(arg, "U:getpwnam", &name)) { goto exit; } - return_value = pwd_getpwnam_impl(module, arg); + return_value = pwd_getpwnam_impl(module, name); exit: return return_value; @@ -69,4 +69,4 @@ pwd_getpwall(PyObject *module, PyObject *Py_UNUSED(ignored)) #ifndef PWD_GETPWALL_METHODDEF #define PWD_GETPWALL_METHODDEF #endif /* !defined(PWD_GETPWALL_METHODDEF) */ -/*[clinic end generated code: output=fc41d8d88ec206d8 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=3c93120d6dd86905 input=a9049054013a1b77]*/ diff --git a/Modules/grpmodule.c b/Modules/grpmodule.c index ffdfa1b6f9fb..74286ab3974d 100644 --- a/Modules/grpmodule.c +++ b/Modules/grpmodule.c @@ -245,7 +245,7 @@ grp_getgrnam_impl(PyObject *module, PyObject *name) PyErr_NoMemory(); } else { - PyErr_Format(PyExc_KeyError, "getgrnam(): name not found: %s", name_chars); + PyErr_Format(PyExc_KeyError, "getgrnam(): name not found: %S", name); } goto out; } diff --git a/Modules/pwdmodule.c b/Modules/pwdmodule.c index 36192a570433..d15286dc10fc 100644 --- a/Modules/pwdmodule.c +++ b/Modules/pwdmodule.c @@ -189,7 +189,7 @@ pwd_getpwuid(PyObject *module, PyObject *uidobj) /*[clinic input] pwd.getpwnam - arg: unicode + name: unicode / Return the password database entry for the given user name. @@ -198,18 +198,18 @@ See `help(pwd)` for more on password database entries. [clinic start generated code]*/ static PyObject * -pwd_getpwnam_impl(PyObject *module, PyObject *arg) -/*[clinic end generated code: output=6abeee92430e43d2 input=d5f7e700919b02d3]*/ +pwd_getpwnam_impl(PyObject *module, PyObject *name) +/*[clinic end generated code: output=359ce1ddeb7a824f input=a6aeb5e3447fb9e0]*/ { - char *buf = NULL, *buf2 = NULL, *name; + char *buf = NULL, *buf2 = NULL, *name_chars; int nomem = 0; struct passwd *p; PyObject *bytes, *retval = NULL; - if ((bytes = PyUnicode_EncodeFSDefault(arg)) == NULL) + if ((bytes = PyUnicode_EncodeFSDefault(name)) == NULL) return NULL; /* check for embedded null bytes */ - if (PyBytes_AsStringAndSize(bytes, &name, NULL) == -1) + if (PyBytes_AsStringAndSize(bytes, &name_chars, NULL) == -1) goto out; #ifdef HAVE_GETPWNAM_R Py_BEGIN_ALLOW_THREADS @@ -229,7 +229,7 @@ pwd_getpwnam_impl(PyObject *module, PyObject *arg) break; } buf = buf2; - status = getpwnam_r(name, &pwd, buf, bufsize, &p); + status = getpwnam_r(name_chars, &pwd, buf, bufsize, &p); if (status != 0) { p = NULL; } @@ -245,7 +245,7 @@ pwd_getpwnam_impl(PyObject *module, PyObject *arg) Py_END_ALLOW_THREADS #else - p = getpwnam(name); + p = getpwnam(name_chars); #endif if (p == NULL) { if (nomem == 1) { @@ -253,7 +253,7 @@ pwd_getpwnam_impl(PyObject *module, PyObject *arg) } else { PyErr_Format(PyExc_KeyError, - "getpwnam(): name not found: %s", name); + "getpwnam(): name not found: %S", name); } goto out; } From webhook-mailer at python.org Fri Sep 7 18:02:59 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Fri, 07 Sep 2018 22:02:59 -0000 Subject: [Python-checkins] bpo-23855: Add missing NULL checks for malloc() in _msi.c (GH-9038) Message-ID: https://github.com/python/cpython/commit/4e519377b1b84c9414a360961276993d24198825 commit: 4e519377b1b84c9414a360961276993d24198825 branch: master author: Zackery Spytz committer: Berker Peksag date: 2018-09-08T01:02:56+03:00 summary: bpo-23855: Add missing NULL checks for malloc() in _msi.c (GH-9038) files: M PC/_msi.c diff --git a/PC/_msi.c b/PC/_msi.c index 000d81f139f3..024b2d3c9fd3 100644 --- a/PC/_msi.c +++ b/PC/_msi.c @@ -330,6 +330,10 @@ msierror(int status) code = MsiRecordGetInteger(err, 1); /* XXX code */ if (MsiFormatRecord(0, err, res, &size) == ERROR_MORE_DATA) { res = malloc(size+1); + if (res == NULL) { + MsiCloseHandle(err); + return PyErr_NoMemory(); + } MsiFormatRecord(0, err, res, &size); res[size]='\0'; } @@ -560,6 +564,9 @@ summary_getproperty(msiobj* si, PyObject *args) &fval, sval, &ssize); if (status == ERROR_MORE_DATA) { sval = malloc(ssize); + if (sval == NULL) { + return PyErr_NoMemory(); + } status = MsiSummaryInfoGetProperty(si->h, field, &type, &ival, &fval, sval, &ssize); } From webhook-mailer at python.org Fri Sep 7 18:04:51 2018 From: webhook-mailer at python.org (Pablo Galindo) Date: Fri, 07 Sep 2018 22:04:51 -0000 Subject: [Python-checkins] bpo-34246: Use no mutable default args in smtplib (GH-8554) Message-ID: https://github.com/python/cpython/commit/d5fbe9b1a3d65ceeb9159c5ba999ee966a945f76 commit: d5fbe9b1a3d65ceeb9159c5ba999ee966a945f76 branch: master author: Pablo Aguiar committer: Pablo Galindo date: 2018-09-07T23:04:48+01:00 summary: bpo-34246: Use no mutable default args in smtplib (GH-8554) Some methods of the SMTP class use mutable default arguments. Specially `send_message` is affected as it mutates one of the args by appending items to it, which has side effects on further calls. files: A Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst M Doc/library/smtplib.rst M Lib/smtplib.py M Lib/test/test_smtplib.py M Misc/ACKS diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index e5effd0306a4..805217252ae8 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -419,7 +419,7 @@ An :class:`SMTP` instance has the following methods: :exc:`SMTPException`. -.. method:: SMTP.sendmail(from_addr, to_addrs, msg, mail_options=[], rcpt_options=[]) +.. method:: SMTP.sendmail(from_addr, to_addrs, msg, mail_options=(), rcpt_options=()) Send mail. The required arguments are an :rfc:`822` from-address string, a list of :rfc:`822` to-address strings (a bare string will be treated as a list with 1 @@ -491,7 +491,7 @@ An :class:`SMTP` instance has the following methods: .. method:: SMTP.send_message(msg, from_addr=None, to_addrs=None, \ - mail_options=[], rcpt_options=[]) + mail_options=(), rcpt_options=()) This is a convenience method for calling :meth:`sendmail` with the message represented by an :class:`email.message.Message` object. The arguments have diff --git a/Lib/smtplib.py b/Lib/smtplib.py index b679875fd2c5..048c6bfb0671 100755 --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -513,7 +513,7 @@ def noop(self): """SMTP 'noop' command -- doesn't do anything :>""" return self.docmd("noop") - def mail(self, sender, options=[]): + def mail(self, sender, options=()): """SMTP 'mail' command -- begins mail xfer session. This method may raise the following exceptions: @@ -534,7 +534,7 @@ def mail(self, sender, options=[]): self.putcmd("mail", "FROM:%s%s" % (quoteaddr(sender), optionlist)) return self.getreply() - def rcpt(self, recip, options=[]): + def rcpt(self, recip, options=()): """SMTP 'rcpt' command -- indicates 1 recipient for this mail.""" optionlist = '' if options and self.does_esmtp: @@ -785,8 +785,8 @@ def starttls(self, keyfile=None, certfile=None, context=None): raise SMTPResponseException(resp, reply) return (resp, reply) - def sendmail(self, from_addr, to_addrs, msg, mail_options=[], - rcpt_options=[]): + def sendmail(self, from_addr, to_addrs, msg, mail_options=(), + rcpt_options=()): """This command performs an entire mail transaction. The arguments are: @@ -890,7 +890,7 @@ def sendmail(self, from_addr, to_addrs, msg, mail_options=[], return senderrs def send_message(self, msg, from_addr=None, to_addrs=None, - mail_options=[], rcpt_options={}): + mail_options=(), rcpt_options=()): """Converts message to a bytestring and passes it to sendmail. The arguments are as for sendmail, except that msg is an @@ -958,7 +958,7 @@ def send_message(self, msg, from_addr=None, to_addrs=None, if international: g = email.generator.BytesGenerator( bytesmsg, policy=msg.policy.clone(utf8=True)) - mail_options += ['SMTPUTF8', 'BODY=8BITMIME'] + mail_options = (*mail_options, 'SMTPUTF8', 'BODY=8BITMIME') else: g = email.generator.BytesGenerator(bytesmsg) g.flatten(msg_copy, linesep='\r\n') diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py index 495764f9aca9..8a29e98a4f30 100644 --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -20,6 +20,7 @@ import unittest from test import support, mock_socket from test.support import HOST, HOSTv4, HOSTv6 +from unittest.mock import Mock if sys.platform == 'darwin': @@ -578,6 +579,33 @@ def testNonnumericPort(self): "localhost:bogus") +class DefaultArgumentsTests(unittest.TestCase): + + def setUp(self): + self.msg = EmailMessage() + self.msg['From'] = 'P?olo ' + self.smtp = smtplib.SMTP() + self.smtp.ehlo = Mock(return_value=(200, 'OK')) + self.smtp.has_extn, self.smtp.sendmail = Mock(), Mock() + + def testSendMessage(self): + expected_mail_options = ('SMTPUTF8', 'BODY=8BITMIME') + self.smtp.send_message(self.msg) + self.smtp.send_message(self.msg) + self.assertEqual(self.smtp.sendmail.call_args_list[0][0][3], + expected_mail_options) + self.assertEqual(self.smtp.sendmail.call_args_list[1][0][3], + expected_mail_options) + + def testSendMessageWithMailOptions(self): + mail_options = ['STARTTLS'] + expected_mail_options = ('STARTTLS', 'SMTPUTF8', 'BODY=8BITMIME') + self.smtp.send_message(self.msg, None, None, mail_options) + self.assertEqual(mail_options, ['STARTTLS']) + self.assertEqual(self.smtp.sendmail.call_args_list[0][0][3], + expected_mail_options) + + # test response of client to a non-successful HELO message class BadHELOServerTests(unittest.TestCase): diff --git a/Misc/ACKS b/Misc/ACKS index 82fbc921feaa..75047d89010e 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -22,6 +22,7 @@ Eitan Adler Anton Afanasyev Ali Afshar Nitika Agarwal +Pablo S. Blum de Aguiar Jim Ahlstrom Farhan Ahmad Matthew Ahrens diff --git a/Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst b/Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst new file mode 100644 index 000000000000..50c91ece07ef --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst @@ -0,0 +1,2 @@ +:meth:`smtplib.SMTP.send_message` no longer modifies the content of the +*mail_options* argument. Patch by Pablo S. Blum de Aguiar. From webhook-mailer at python.org Fri Sep 7 18:10:32 2018 From: webhook-mailer at python.org (Pablo Galindo) Date: Fri, 07 Sep 2018 22:10:32 -0000 Subject: [Python-checkins] [3.6] bpo-34007: Skip traceback tests if the Program Counter is not available. (GH-9022) Message-ID: https://github.com/python/cpython/commit/963bcc8b71e4ab8c9ee9a91ed1300b6e39219821 commit: 963bcc8b71e4ab8c9ee9a91ed1300b6e39219821 branch: 3.6 author: Pablo Galindo committer: GitHub date: 2018-09-07T23:10:28+01:00 summary: [3.6] bpo-34007: Skip traceback tests if the Program Counter is not available. (GH-9022) Sometimes some versions of the shared libraries that are part of the traceback are compiled in optimised mode and the Program Counter (PC) is not present, not allowing gdb to walk the frames back. When this happens, the Python bindings of gdb raise an exception, making the test impossible to succeed. (cherry picked from commit f2ef51f8bec525b21e52988880c8a029642795ed) Co-authored-by: Pablo Galindo files: M Lib/test/test_gdb.py diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py index b7554d698c9d..bedec1fb493a 100644 --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -212,6 +212,15 @@ def get_stack_trace(self, source=None, script=None, for line in errlines: if not line: continue + # bpo34007: Sometimes some versions of the shared libraries that + # are part of the traceback are compiled in optimised mode and the + # Program Counter (PC) is not present, not allowing gdb to walk the + # frames back. When this happens, the Python bindings of gdb raise + # an exception, making the test impossible to succeed. + if "PC not saved" in line: + raise unittest.SkipTest("gdb cannot walk the frame object" + " because the Program Counter is" + " not present") if not line.startswith(ignore_patterns): unexpected_errlines.append(line) From webhook-mailer at python.org Fri Sep 7 18:15:33 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 07 Sep 2018 22:15:33 -0000 Subject: [Python-checkins] bpo-23855: Add missing NULL checks for malloc() in _msi.c (GH-9038) Message-ID: https://github.com/python/cpython/commit/73994077250bd70385cb8e7a92f24874129369d1 commit: 73994077250bd70385cb8e7a92f24874129369d1 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-07T15:15:30-07:00 summary: bpo-23855: Add missing NULL checks for malloc() in _msi.c (GH-9038) (cherry picked from commit 4e519377b1b84c9414a360961276993d24198825) Co-authored-by: Zackery Spytz files: M PC/_msi.c diff --git a/PC/_msi.c b/PC/_msi.c index 000d81f139f3..024b2d3c9fd3 100644 --- a/PC/_msi.c +++ b/PC/_msi.c @@ -330,6 +330,10 @@ msierror(int status) code = MsiRecordGetInteger(err, 1); /* XXX code */ if (MsiFormatRecord(0, err, res, &size) == ERROR_MORE_DATA) { res = malloc(size+1); + if (res == NULL) { + MsiCloseHandle(err); + return PyErr_NoMemory(); + } MsiFormatRecord(0, err, res, &size); res[size]='\0'; } @@ -560,6 +564,9 @@ summary_getproperty(msiobj* si, PyObject *args) &fval, sval, &ssize); if (status == ERROR_MORE_DATA) { sval = malloc(ssize); + if (sval == NULL) { + return PyErr_NoMemory(); + } status = MsiSummaryInfoGetProperty(si->h, field, &type, &ival, &fval, sval, &ssize); } From webhook-mailer at python.org Fri Sep 7 18:29:34 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 07 Sep 2018 22:29:34 -0000 Subject: [Python-checkins] bpo-34246: Use no mutable default args in smtplib (GH-8554) Message-ID: https://github.com/python/cpython/commit/9835696ec4c57a9a30f1c11cfb4c5d3e121bf97c commit: 9835696ec4c57a9a30f1c11cfb4c5d3e121bf97c branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-07T15:29:27-07:00 summary: bpo-34246: Use no mutable default args in smtplib (GH-8554) Some methods of the SMTP class use mutable default arguments. Specially `send_message` is affected as it mutates one of the args by appending items to it, which has side effects on further calls. (cherry picked from commit d5fbe9b1a3d65ceeb9159c5ba999ee966a945f76) Co-authored-by: Pablo Aguiar files: A Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst M Doc/library/smtplib.rst M Lib/smtplib.py M Lib/test/test_smtplib.py M Misc/ACKS diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index e5effd0306a4..805217252ae8 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -419,7 +419,7 @@ An :class:`SMTP` instance has the following methods: :exc:`SMTPException`. -.. method:: SMTP.sendmail(from_addr, to_addrs, msg, mail_options=[], rcpt_options=[]) +.. method:: SMTP.sendmail(from_addr, to_addrs, msg, mail_options=(), rcpt_options=()) Send mail. The required arguments are an :rfc:`822` from-address string, a list of :rfc:`822` to-address strings (a bare string will be treated as a list with 1 @@ -491,7 +491,7 @@ An :class:`SMTP` instance has the following methods: .. method:: SMTP.send_message(msg, from_addr=None, to_addrs=None, \ - mail_options=[], rcpt_options=[]) + mail_options=(), rcpt_options=()) This is a convenience method for calling :meth:`sendmail` with the message represented by an :class:`email.message.Message` object. The arguments have diff --git a/Lib/smtplib.py b/Lib/smtplib.py index b679875fd2c5..048c6bfb0671 100755 --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -513,7 +513,7 @@ def noop(self): """SMTP 'noop' command -- doesn't do anything :>""" return self.docmd("noop") - def mail(self, sender, options=[]): + def mail(self, sender, options=()): """SMTP 'mail' command -- begins mail xfer session. This method may raise the following exceptions: @@ -534,7 +534,7 @@ def mail(self, sender, options=[]): self.putcmd("mail", "FROM:%s%s" % (quoteaddr(sender), optionlist)) return self.getreply() - def rcpt(self, recip, options=[]): + def rcpt(self, recip, options=()): """SMTP 'rcpt' command -- indicates 1 recipient for this mail.""" optionlist = '' if options and self.does_esmtp: @@ -785,8 +785,8 @@ def starttls(self, keyfile=None, certfile=None, context=None): raise SMTPResponseException(resp, reply) return (resp, reply) - def sendmail(self, from_addr, to_addrs, msg, mail_options=[], - rcpt_options=[]): + def sendmail(self, from_addr, to_addrs, msg, mail_options=(), + rcpt_options=()): """This command performs an entire mail transaction. The arguments are: @@ -890,7 +890,7 @@ def sendmail(self, from_addr, to_addrs, msg, mail_options=[], return senderrs def send_message(self, msg, from_addr=None, to_addrs=None, - mail_options=[], rcpt_options={}): + mail_options=(), rcpt_options=()): """Converts message to a bytestring and passes it to sendmail. The arguments are as for sendmail, except that msg is an @@ -958,7 +958,7 @@ def send_message(self, msg, from_addr=None, to_addrs=None, if international: g = email.generator.BytesGenerator( bytesmsg, policy=msg.policy.clone(utf8=True)) - mail_options += ['SMTPUTF8', 'BODY=8BITMIME'] + mail_options = (*mail_options, 'SMTPUTF8', 'BODY=8BITMIME') else: g = email.generator.BytesGenerator(bytesmsg) g.flatten(msg_copy, linesep='\r\n') diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py index 495764f9aca9..8a29e98a4f30 100644 --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -20,6 +20,7 @@ import unittest from test import support, mock_socket from test.support import HOST, HOSTv4, HOSTv6 +from unittest.mock import Mock if sys.platform == 'darwin': @@ -578,6 +579,33 @@ def testNonnumericPort(self): "localhost:bogus") +class DefaultArgumentsTests(unittest.TestCase): + + def setUp(self): + self.msg = EmailMessage() + self.msg['From'] = 'P?olo ' + self.smtp = smtplib.SMTP() + self.smtp.ehlo = Mock(return_value=(200, 'OK')) + self.smtp.has_extn, self.smtp.sendmail = Mock(), Mock() + + def testSendMessage(self): + expected_mail_options = ('SMTPUTF8', 'BODY=8BITMIME') + self.smtp.send_message(self.msg) + self.smtp.send_message(self.msg) + self.assertEqual(self.smtp.sendmail.call_args_list[0][0][3], + expected_mail_options) + self.assertEqual(self.smtp.sendmail.call_args_list[1][0][3], + expected_mail_options) + + def testSendMessageWithMailOptions(self): + mail_options = ['STARTTLS'] + expected_mail_options = ('STARTTLS', 'SMTPUTF8', 'BODY=8BITMIME') + self.smtp.send_message(self.msg, None, None, mail_options) + self.assertEqual(mail_options, ['STARTTLS']) + self.assertEqual(self.smtp.sendmail.call_args_list[0][0][3], + expected_mail_options) + + # test response of client to a non-successful HELO message class BadHELOServerTests(unittest.TestCase): diff --git a/Misc/ACKS b/Misc/ACKS index bcf5604b6e21..1c53672e1cf0 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -22,6 +22,7 @@ Eitan Adler Anton Afanasyev Ali Afshar Nitika Agarwal +Pablo S. Blum de Aguiar Jim Ahlstrom Farhan Ahmad Matthew Ahrens diff --git a/Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst b/Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst new file mode 100644 index 000000000000..50c91ece07ef --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst @@ -0,0 +1,2 @@ +:meth:`smtplib.SMTP.send_message` no longer modifies the content of the +*mail_options* argument. Patch by Pablo S. Blum de Aguiar. From webhook-mailer at python.org Fri Sep 7 19:15:25 2018 From: webhook-mailer at python.org (Pablo Galindo) Date: Fri, 07 Sep 2018 23:15:25 -0000 Subject: [Python-checkins] bpo-34246: Make sure test_smtplib always cleans resources when finished (GH-9108) Message-ID: https://github.com/python/cpython/commit/5b7a2cb5caeb7df68e637f45a98632cbc84a51bf commit: 5b7a2cb5caeb7df68e637f45a98632cbc84a51bf branch: master author: Pablo Galindo committer: GitHub date: 2018-09-08T00:15:22+01:00 summary: bpo-34246: Make sure test_smtplib always cleans resources when finished (GH-9108) * Make sure that when some of the tests in test_smtplib fail, the allocated threads and sockets are not leaked. * Use support.join_thread() instead of thread.join() to avoid infinite blocks. files: M Lib/test/test_smtplib.py diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py index 8a29e98a4f30..0c863ed7e203 100644 --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -20,6 +20,7 @@ import unittest from test import support, mock_socket from test.support import HOST, HOSTv4, HOSTv6 +from test.support import threading_setup, threading_cleanup, join_thread from unittest.mock import Mock @@ -193,6 +194,7 @@ class DebuggingServerTests(unittest.TestCase): maxDiff = None def setUp(self): + self.thread_key = threading_setup() self.real_getfqdn = socket.getfqdn socket.getfqdn = mock_socket.getfqdn # temporarily replace sys.stdout to capture DebuggingServer output @@ -224,12 +226,15 @@ def tearDown(self): self.client_evt.set() # wait for the server thread to terminate self.serv_evt.wait() - self.thread.join() + join_thread(self.thread) # restore sys.stdout sys.stdout = self.old_stdout # restore DEBUGSTREAM smtpd.DEBUGSTREAM.close() smtpd.DEBUGSTREAM = self.old_DEBUGSTREAM + del self.thread + self.doCleanups() + threading_cleanup(*self.thread_key) def get_output_without_xpeer(self): test_output = self.output.getvalue() @@ -247,6 +252,7 @@ def testSourceAddress(self): try: smtp = smtplib.SMTP(self.host, self.port, local_hostname='localhost', timeout=3, source_address=(self.host, src_port)) + self.addCleanup(smtp.close) self.assertEqual(smtp.source_address, (self.host, src_port)) self.assertEqual(smtp.local_hostname, 'localhost') smtp.quit() @@ -257,12 +263,14 @@ def testSourceAddress(self): def testNOOP(self): smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) expected = (250, b'OK') self.assertEqual(smtp.noop(), expected) smtp.quit() def testRSET(self): smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) expected = (250, b'OK') self.assertEqual(smtp.rset(), expected) smtp.quit() @@ -270,6 +278,7 @@ def testRSET(self): def testELHO(self): # EHLO isn't implemented in DebuggingServer smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) expected = (250, b'\nSIZE 33554432\nHELP') self.assertEqual(smtp.ehlo(), expected) smtp.quit() @@ -277,6 +286,7 @@ def testELHO(self): def testEXPNNotImplemented(self): # EXPN isn't implemented in DebuggingServer smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) expected = (502, b'EXPN not implemented') smtp.putcmd('EXPN') self.assertEqual(smtp.getreply(), expected) @@ -284,6 +294,7 @@ def testEXPNNotImplemented(self): def testVRFY(self): smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) expected = (252, b'Cannot VRFY user, but will accept message ' + \ b'and attempt delivery') self.assertEqual(smtp.vrfy('nobody at nowhere.com'), expected) @@ -294,6 +305,7 @@ def testSecondHELO(self): # check that a second HELO returns a message that it's a duplicate # (this behavior is specific to smtpd.SMTPChannel) smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.helo() expected = (503, b'Duplicate HELO/EHLO') self.assertEqual(smtp.helo(), expected) @@ -301,6 +313,7 @@ def testSecondHELO(self): def testHELP(self): smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) self.assertEqual(smtp.help(), b'Supported commands: EHLO HELO MAIL ' + \ b'RCPT DATA RSET NOOP QUIT VRFY') smtp.quit() @@ -309,6 +322,7 @@ def testSend(self): # connect and send mail m = 'A test message' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.sendmail('John', 'Sally', m) # XXX(nnorwitz): this test is flaky and dies with a bad file descriptor # in asyncore. This sleep might help, but should really be fixed @@ -325,6 +339,7 @@ def testSend(self): def testSendBinary(self): m = b'A test message' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.sendmail('John', 'Sally', m) # XXX (see comment in testSend) time.sleep(0.01) @@ -340,6 +355,7 @@ def testSendNeedingDotQuote(self): # Issue 12283 m = '.A test\n.mes.sage.' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.sendmail('John', 'Sally', m) # XXX (see comment in testSend) time.sleep(0.01) @@ -354,6 +370,7 @@ def testSendNeedingDotQuote(self): def testSendNullSender(self): m = 'A test message' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.sendmail('<>', 'Sally', m) # XXX (see comment in testSend) time.sleep(0.01) @@ -371,6 +388,7 @@ def testSendNullSender(self): def testSendMessage(self): m = email.mime.text.MIMEText('A test message') smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.send_message(m, from_addr='John', to_addrs='Sally') # XXX (see comment in testSend) time.sleep(0.01) @@ -395,6 +413,7 @@ def testSendMessageWithAddresses(self): m['CC'] = 'Sally, Fred' m['Bcc'] = 'John Root , "Dinsdale" ' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.send_message(m) # XXX (see comment in testSend) time.sleep(0.01) @@ -428,6 +447,7 @@ def testSendMessageWithSomeAddresses(self): m['From'] = 'foo at bar.com' m['To'] = 'John, Dinsdale' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.send_message(m) # XXX (see comment in testSend) time.sleep(0.01) @@ -455,6 +475,7 @@ def testSendMessageWithSpecifiedAddresses(self): m['From'] = 'foo at bar.com' m['To'] = 'John, Dinsdale' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.send_message(m, from_addr='joe at example.com', to_addrs='foo at example.net') # XXX (see comment in testSend) time.sleep(0.01) @@ -485,6 +506,7 @@ def testSendMessageWithMultipleFrom(self): m['Sender'] = 'the_rescuers at Rescue-Aid-Society.com' m['To'] = 'John, Dinsdale' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.send_message(m) # XXX (see comment in testSend) time.sleep(0.01) @@ -517,6 +539,7 @@ def testSendMessageResent(self): m['Resent-To'] = 'Martha , Jeff' m['Resent-Bcc'] = 'doe at losthope.net' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) smtp.send_message(m) # XXX (see comment in testSend) time.sleep(0.01) @@ -555,6 +578,7 @@ def testSendMessageMultipleResentRaises(self): m['Resent-To'] = 'holy at grail.net' m['Resent-From'] = 'Martha , Jeff' smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.addCleanup(smtp.close) with self.assertRaises(ValueError): smtp.send_message(m) smtp.close() @@ -630,6 +654,7 @@ class TooLongLineTests(unittest.TestCase): respdata = b'250 OK' + (b'.' * smtplib._MAXLINE * 2) + b'\n' def setUp(self): + self.thread_key = threading_setup() self.old_stdout = sys.stdout self.output = io.StringIO() sys.stdout = self.output @@ -639,15 +664,18 @@ def setUp(self): self.sock.settimeout(15) self.port = support.bind_port(self.sock) servargs = (self.evt, self.respdata, self.sock) - thread = threading.Thread(target=server, args=servargs) - thread.start() - self.addCleanup(thread.join) + self.thread = threading.Thread(target=server, args=servargs) + self.thread.start() self.evt.wait() self.evt.clear() def tearDown(self): self.evt.wait() sys.stdout = self.old_stdout + join_thread(self.thread) + del self.thread + self.doCleanups() + threading_cleanup(*self.thread_key) def testLineTooLong(self): self.assertRaises(smtplib.SMTPResponseException, smtplib.SMTP, @@ -877,6 +905,7 @@ def handle_error(self): class SMTPSimTests(unittest.TestCase): def setUp(self): + self.thread_key = threading_setup() self.real_getfqdn = socket.getfqdn socket.getfqdn = mock_socket.getfqdn self.serv_evt = threading.Event() @@ -899,7 +928,10 @@ def tearDown(self): self.client_evt.set() # wait for the server thread to terminate self.serv_evt.wait() - self.thread.join() + join_thread(self.thread) + del self.thread + self.doCleanups() + threading_cleanup(*self.thread_key) def testBasic(self): # smoke test @@ -1162,6 +1194,7 @@ class SMTPUTF8SimTests(unittest.TestCase): maxDiff = None def setUp(self): + self.thread_key = threading_setup() self.real_getfqdn = socket.getfqdn socket.getfqdn = mock_socket.getfqdn self.serv_evt = threading.Event() @@ -1186,7 +1219,10 @@ def tearDown(self): self.client_evt.set() # wait for the server thread to terminate self.serv_evt.wait() - self.thread.join() + join_thread(self.thread) + del self.thread + self.doCleanups() + threading_cleanup(*self.thread_key) def test_test_server_supports_extensions(self): smtp = smtplib.SMTP( @@ -1283,6 +1319,7 @@ class SimSMTPAUTHInitialResponseServer(SimSMTPServer): class SMTPAUTHInitialResponseSimTests(unittest.TestCase): def setUp(self): + self.thread_key = threading_setup() self.real_getfqdn = socket.getfqdn socket.getfqdn = mock_socket.getfqdn self.serv_evt = threading.Event() @@ -1306,7 +1343,10 @@ def tearDown(self): self.client_evt.set() # wait for the server thread to terminate self.serv_evt.wait() - self.thread.join() + join_thread(self.thread) + del self.thread + self.doCleanups() + threading_cleanup(*self.thread_key) def testAUTH_PLAIN_initial_response_login(self): self.serv.add_feature('AUTH PLAIN') From webhook-mailer at python.org Fri Sep 7 19:16:20 2018 From: webhook-mailer at python.org (Pablo Galindo) Date: Fri, 07 Sep 2018 23:16:20 -0000 Subject: [Python-checkins] bpo-33083: Update "What's new" with math.factorial changes (GH-9109) Message-ID: https://github.com/python/cpython/commit/fa221d804f1bc07d992f820069bad24f176ed66d commit: fa221d804f1bc07d992f820069bad24f176ed66d branch: master author: Pablo Galindo committer: GitHub date: 2018-09-08T00:16:17+01:00 summary: bpo-33083: Update "What's new" with math.factorial changes (GH-9109) * Add elimination of non-int-like parameters in math.factorial to "What's new". files: M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index d07896b71f23..2de7a50a471d 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -272,6 +272,9 @@ Changes in the Python API success; an exception was raised on error under Unix. (Contributed by Berker Peksag in :issue:`2122`.) +* The function :func:`math.factorial` no longer accepts arguments that are not + int-like. (Contributed by Pablo Galindo in :issue:`33083`.) + CPython bytecode changes ------------------------ From webhook-mailer at python.org Fri Sep 7 21:20:31 2018 From: webhook-mailer at python.org (Pablo Galindo) Date: Sat, 08 Sep 2018 01:20:31 -0000 Subject: [Python-checkins] [3.6] bpo-34246: Use no mutable default args in smtplib (GH-8554) (#9112) Message-ID: https://github.com/python/cpython/commit/eb6ab73f93c8b883a8d75a83560e2b4c59170d95 commit: eb6ab73f93c8b883a8d75a83560e2b4c59170d95 branch: 3.6 author: Pablo Galindo committer: GitHub date: 2018-09-08T02:20:27+01:00 summary: [3.6] bpo-34246: Use no mutable default args in smtplib (GH-8554) (#9112) Some methods of the SMTP class use mutable default arguments. Specially `send_message` is affected as it mutates one of the args by appending items to it, which has side effects on further calls.. (cherry picked from commit d5fbe9b1a3d65ceeb9159c5ba999ee966a945f76) Co-authored-by: Pablo Aguiar files: A Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst M Doc/library/smtplib.rst M Lib/smtplib.py M Lib/test/test_smtplib.py M Misc/ACKS diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index e5effd0306a4..805217252ae8 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -419,7 +419,7 @@ An :class:`SMTP` instance has the following methods: :exc:`SMTPException`. -.. method:: SMTP.sendmail(from_addr, to_addrs, msg, mail_options=[], rcpt_options=[]) +.. method:: SMTP.sendmail(from_addr, to_addrs, msg, mail_options=(), rcpt_options=()) Send mail. The required arguments are an :rfc:`822` from-address string, a list of :rfc:`822` to-address strings (a bare string will be treated as a list with 1 @@ -491,7 +491,7 @@ An :class:`SMTP` instance has the following methods: .. method:: SMTP.send_message(msg, from_addr=None, to_addrs=None, \ - mail_options=[], rcpt_options=[]) + mail_options=(), rcpt_options=()) This is a convenience method for calling :meth:`sendmail` with the message represented by an :class:`email.message.Message` object. The arguments have diff --git a/Lib/smtplib.py b/Lib/smtplib.py index b679875fd2c5..048c6bfb0671 100755 --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -513,7 +513,7 @@ def noop(self): """SMTP 'noop' command -- doesn't do anything :>""" return self.docmd("noop") - def mail(self, sender, options=[]): + def mail(self, sender, options=()): """SMTP 'mail' command -- begins mail xfer session. This method may raise the following exceptions: @@ -534,7 +534,7 @@ def mail(self, sender, options=[]): self.putcmd("mail", "FROM:%s%s" % (quoteaddr(sender), optionlist)) return self.getreply() - def rcpt(self, recip, options=[]): + def rcpt(self, recip, options=()): """SMTP 'rcpt' command -- indicates 1 recipient for this mail.""" optionlist = '' if options and self.does_esmtp: @@ -785,8 +785,8 @@ def starttls(self, keyfile=None, certfile=None, context=None): raise SMTPResponseException(resp, reply) return (resp, reply) - def sendmail(self, from_addr, to_addrs, msg, mail_options=[], - rcpt_options=[]): + def sendmail(self, from_addr, to_addrs, msg, mail_options=(), + rcpt_options=()): """This command performs an entire mail transaction. The arguments are: @@ -890,7 +890,7 @@ def sendmail(self, from_addr, to_addrs, msg, mail_options=[], return senderrs def send_message(self, msg, from_addr=None, to_addrs=None, - mail_options=[], rcpt_options={}): + mail_options=(), rcpt_options=()): """Converts message to a bytestring and passes it to sendmail. The arguments are as for sendmail, except that msg is an @@ -958,7 +958,7 @@ def send_message(self, msg, from_addr=None, to_addrs=None, if international: g = email.generator.BytesGenerator( bytesmsg, policy=msg.policy.clone(utf8=True)) - mail_options += ['SMTPUTF8', 'BODY=8BITMIME'] + mail_options = (*mail_options, 'SMTPUTF8', 'BODY=8BITMIME') else: g = email.generator.BytesGenerator(bytesmsg) g.flatten(msg_copy, linesep='\r\n') diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py index 79b3bd436876..18110191dc3b 100644 --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -18,6 +18,11 @@ import unittest from test import support, mock_socket +from unittest.mock import Mock + +HOST = "localhost" +HOSTv4 = "127.0.0.1" +HOSTv6 = "::1" try: import threading @@ -569,6 +574,33 @@ def testNonnumericPort(self): "localhost:bogus") +class DefaultArgumentsTests(unittest.TestCase): + + def setUp(self): + self.msg = EmailMessage() + self.msg['From'] = 'P?olo ' + self.smtp = smtplib.SMTP() + self.smtp.ehlo = Mock(return_value=(200, 'OK')) + self.smtp.has_extn, self.smtp.sendmail = Mock(), Mock() + + def testSendMessage(self): + expected_mail_options = ('SMTPUTF8', 'BODY=8BITMIME') + self.smtp.send_message(self.msg) + self.smtp.send_message(self.msg) + self.assertEqual(self.smtp.sendmail.call_args_list[0][0][3], + expected_mail_options) + self.assertEqual(self.smtp.sendmail.call_args_list[1][0][3], + expected_mail_options) + + def testSendMessageWithMailOptions(self): + mail_options = ['STARTTLS'] + expected_mail_options = ('STARTTLS', 'SMTPUTF8', 'BODY=8BITMIME') + self.smtp.send_message(self.msg, None, None, mail_options) + self.assertEqual(mail_options, ['STARTTLS']) + self.assertEqual(self.smtp.sendmail.call_args_list[0][0][3], + expected_mail_options) + + # test response of client to a non-successful HELO message @unittest.skipUnless(threading, 'Threading required for this test.') class BadHELOServerTests(unittest.TestCase): diff --git a/Misc/ACKS b/Misc/ACKS index 64a007236b65..bd8a5fb81f0f 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -21,6 +21,7 @@ Ron Adam Anton Afanasyev Ali Afshar Nitika Agarwal +Pablo S. Blum de Aguiar Jim Ahlstrom Farhan Ahmad Matthew Ahrens diff --git a/Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst b/Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst new file mode 100644 index 000000000000..50c91ece07ef --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-29-15-25-15.bpo-34246.xiKq-Q.rst @@ -0,0 +1,2 @@ +:meth:`smtplib.SMTP.send_message` no longer modifies the content of the +*mail_options* argument. Patch by Pablo S. Blum de Aguiar. From solipsis at pitrou.net Sat Sep 8 05:08:44 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 08 Sep 2018 09:08:44 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=7 Message-ID: <20180908090844.1.110F749308E071C7@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 3] memory blocks, sum=3 test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogM7g8dC', '--timeout', '7200'] From webhook-mailer at python.org Sat Sep 8 07:48:22 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sat, 08 Sep 2018 11:48:22 -0000 Subject: [Python-checkins] bpo-20104: Change the file_actions parameter of os.posix_spawn(). (GH-6725) Message-ID: https://github.com/python/cpython/commit/d700f97b627989d41cd4629dc02969f9a6b56d2f commit: d700f97b627989d41cd4629dc02969f9a6b56d2f branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-08T14:48:18+03:00 summary: bpo-20104: Change the file_actions parameter of os.posix_spawn(). (GH-6725) * Make its default value an empty tuple instead of None. * Make it a keyword-only parameter. files: M Doc/library/os.rst M Lib/test/test_posix.py M Modules/clinic/posixmodule.c.h M Modules/posixmodule.c diff --git a/Doc/library/os.rst b/Doc/library/os.rst index b8d6fffb303b..bc8d5a8abbf3 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -3394,7 +3394,7 @@ written in Python, such as a mail server's external command delivery program. subprocesses. -.. function:: posix_spawn(path, argv, env, file_actions=None, /, *, \ +.. function:: posix_spawn(path, argv, env, *, file_actions=None, \ setpgroup=None, resetids=False, setsigmask=(), \ setsigdef=(), scheduler=None) @@ -3402,7 +3402,8 @@ written in Python, such as a mail server's external command delivery program. Most users should use :func:`subprocess.run` instead of :func:`posix_spawn`. - The *path*, *args*, and *env* arguments are similar to :func:`execve`. + The positional-only arguments *path*, *args*, and *env* are similar to + :func:`execve`. The *file_actions* argument may be a sequence of tuples describing actions to take on specific file descriptors in the child process between the C diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py index dafe9c1350e4..7a2fc263cb27 100644 --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -1499,8 +1499,7 @@ def test_returns_pid(self): pidfile.write(str(os.getpid())) """ args = self.python_args('-c', script) - pid = posix.posix_spawn(args[0], args, - os.environ) + pid = posix.posix_spawn(args[0], args, os.environ) self.assertEqual(os.waitpid(pid, 0), (pid, 0)) with open(pidfile) as f: self.assertEqual(f.read(), str(pid)) @@ -1538,7 +1537,7 @@ def test_empty_file_actions(self): self.NOOP_PROGRAM[0], self.NOOP_PROGRAM, os.environ, - [] + file_actions=[] ) self.assertEqual(os.waitpid(pid, 0), (pid, 0)) @@ -1691,37 +1690,38 @@ def test_multiple_file_actions(self): ] pid = posix.posix_spawn(self.NOOP_PROGRAM[0], self.NOOP_PROGRAM, - os.environ, file_actions) + os.environ, + file_actions=file_actions) self.assertEqual(os.waitpid(pid, 0), (pid, 0)) def test_bad_file_actions(self): args = self.NOOP_PROGRAM with self.assertRaises(TypeError): - posix.posix_spawn(args[0], args, - os.environ, [None]) + posix.posix_spawn(args[0], args, os.environ, + file_actions=[None]) with self.assertRaises(TypeError): - posix.posix_spawn(args[0], args, - os.environ, [()]) + posix.posix_spawn(args[0], args, os.environ, + file_actions=[()]) with self.assertRaises(TypeError): - posix.posix_spawn(args[0], args, - os.environ, [(None,)]) + posix.posix_spawn(args[0], args, os.environ, + file_actions=[(None,)]) with self.assertRaises(TypeError): - posix.posix_spawn(args[0], args, - os.environ, [(12345,)]) + posix.posix_spawn(args[0], args, os.environ, + file_actions=[(12345,)]) with self.assertRaises(TypeError): - posix.posix_spawn(args[0], args, - os.environ, [(os.POSIX_SPAWN_CLOSE,)]) + posix.posix_spawn(args[0], args, os.environ, + file_actions=[(os.POSIX_SPAWN_CLOSE,)]) with self.assertRaises(TypeError): - posix.posix_spawn(args[0], args, - os.environ, [(os.POSIX_SPAWN_CLOSE, 1, 2)]) + posix.posix_spawn(args[0], args, os.environ, + file_actions=[(os.POSIX_SPAWN_CLOSE, 1, 2)]) with self.assertRaises(TypeError): - posix.posix_spawn(args[0], args, - os.environ, [(os.POSIX_SPAWN_CLOSE, None)]) + posix.posix_spawn(args[0], args, os.environ, + file_actions=[(os.POSIX_SPAWN_CLOSE, None)]) with self.assertRaises(ValueError): - posix.posix_spawn(args[0], args, - os.environ, - [(os.POSIX_SPAWN_OPEN, 3, __file__ + '\0', - os.O_RDONLY, 0)]) + posix.posix_spawn(args[0], args, os.environ, + file_actions=[(os.POSIX_SPAWN_OPEN, + 3, __file__ + '\0', + os.O_RDONLY, 0)]) def test_open_file(self): outfile = support.TESTFN @@ -1736,8 +1736,8 @@ def test_open_file(self): stat.S_IRUSR | stat.S_IWUSR), ] args = self.python_args('-c', script) - pid = posix.posix_spawn(args[0], args, - os.environ, file_actions) + pid = posix.posix_spawn(args[0], args, os.environ, + file_actions=file_actions) self.assertEqual(os.waitpid(pid, 0), (pid, 0)) with open(outfile) as f: self.assertEqual(f.read(), 'hello') @@ -1754,9 +1754,8 @@ def test_close_file(self): closefile.write('is closed %d' % e.errno) """ args = self.python_args('-c', script) - pid = posix.posix_spawn(args[0], args, - os.environ, - [(os.POSIX_SPAWN_CLOSE, 0),]) + pid = posix.posix_spawn(args[0], args, os.environ, + file_actions=[(os.POSIX_SPAWN_CLOSE, 0),]) self.assertEqual(os.waitpid(pid, 0), (pid, 0)) with open(closefile) as f: self.assertEqual(f.read(), 'is closed %d' % errno.EBADF) @@ -1773,8 +1772,8 @@ def test_dup2(self): (os.POSIX_SPAWN_DUP2, childfile.fileno(), 1), ] args = self.python_args('-c', script) - pid = posix.posix_spawn(args[0], args, - os.environ, file_actions) + pid = posix.posix_spawn(args[0], args, os.environ, + file_actions=file_actions) self.assertEqual(os.waitpid(pid, 0), (pid, 0)) with open(dupfile) as f: self.assertEqual(f.read(), 'hello') diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h index f7767c4af0bb..c3849a93c0c2 100644 --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -1730,7 +1730,7 @@ os_execve(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *k #if defined(HAVE_POSIX_SPAWN) PyDoc_STRVAR(os_posix_spawn__doc__, -"posix_spawn($module, path, argv, env, file_actions=None, /, *,\n" +"posix_spawn($module, path, argv, env, /, *, file_actions=(),\n" " setpgroup=None, resetids=False, setsigmask=(),\n" " setsigdef=(), scheduler=None)\n" "--\n" @@ -1769,12 +1769,12 @@ static PyObject * os_posix_spawn(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; - static const char * const _keywords[] = {"", "", "", "", "setpgroup", "resetids", "setsigmask", "setsigdef", "scheduler", NULL}; - static _PyArg_Parser _parser = {"O&OO|O$OiOOO:posix_spawn", _keywords, 0}; + static const char * const _keywords[] = {"", "", "", "file_actions", "setpgroup", "resetids", "setsigmask", "setsigdef", "scheduler", NULL}; + static _PyArg_Parser _parser = {"O&OO|$OOiOOO:posix_spawn", _keywords, 0}; path_t path = PATH_T_INITIALIZE("posix_spawn", "path", 0, 0); PyObject *argv; PyObject *env; - PyObject *file_actions = Py_None; + PyObject *file_actions = NULL; PyObject *setpgroup = NULL; int resetids = 0; PyObject *setsigmask = NULL; @@ -6648,4 +6648,4 @@ os_getrandom(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject #ifndef OS_GETRANDOM_METHODDEF #define OS_GETRANDOM_METHODDEF #endif /* !defined(OS_GETRANDOM_METHODDEF) */ -/*[clinic end generated code: output=ef78384ae88712e1 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=758ee0434fb03d90 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 4a8a8d70ae42..c5d9a17b002e 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -5405,10 +5405,10 @@ os.posix_spawn Tuple or list of strings. env: object Dictionary of strings mapping to strings. - file_actions: object = None - A sequence of file action tuples. / * + file_actions: object(c_default='NULL') = () + A sequence of file action tuples. setpgroup: object = NULL The pgroup to use with the POSIX_SPAWN_SETPGROUP flag. resetids: bool(accept={int}) = False @@ -5419,6 +5419,7 @@ os.posix_spawn The sigmask to use with the POSIX_SPAWN_SETSIGDEF flag. scheduler: object = NULL A tuple with the scheduler policy (optional) and parameters. + Execute the program specified by path in a new process. [clinic start generated code]*/ @@ -5427,7 +5428,7 @@ os_posix_spawn_impl(PyObject *module, path_t *path, PyObject *argv, PyObject *env, PyObject *file_actions, PyObject *setpgroup, int resetids, PyObject *setsigmask, PyObject *setsigdef, PyObject *scheduler) -/*[clinic end generated code: output=45dfa4c515d09f2c input=2d7a7578430a90f0]*/ +/*[clinic end generated code: output=45dfa4c515d09f2c input=2891c2f1d457e39b]*/ { EXECV_CHAR **argvlist = NULL; EXECV_CHAR **envlist = NULL; @@ -5477,7 +5478,7 @@ os_posix_spawn_impl(PyObject *module, path_t *path, PyObject *argv, goto exit; } - if (file_actions != Py_None) { + if (file_actions != NULL) { /* There is a bug in old versions of glibc that makes some of the * helper functions for manipulating file actions not copy the provided * buffers. The problem is that posix_spawn_file_actions_addopen does not From webhook-mailer at python.org Sat Sep 8 16:31:31 2018 From: webhook-mailer at python.org (=?utf-8?q?=C3=89ric?= Araujo) Date: Sat, 08 Sep 2018 20:31:31 -0000 Subject: [Python-checkins] bpo-34421 avoid unicode error in distutils logging (GH-8799) Message-ID: https://github.com/python/cpython/commit/0afada163c7ef25c3a9d46ed445481fb69f2ecaf commit: 0afada163c7ef25c3a9d46ed445481fb69f2ecaf branch: master author: Julien Malard committer: ?ric Araujo date: 2018-09-08T16:31:26-04:00 summary: bpo-34421 avoid unicode error in distutils logging (GH-8799) This caused installation errors in some cases on Windows. Patch by Julien Malard. files: A Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst M Lib/distutils/log.py diff --git a/Lib/distutils/log.py b/Lib/distutils/log.py index b301a8338c20..3a6602bc8b8e 100644 --- a/Lib/distutils/log.py +++ b/Lib/distutils/log.py @@ -31,7 +31,10 @@ def _log(self, level, msg, args): # emulate backslashreplace error handler encoding = stream.encoding msg = msg.encode(encoding, "backslashreplace").decode(encoding) - stream.write('%s\n' % msg) + try: + stream.write('%s\n' % msg) + except UnicodeEncodeError: + stream.write('%s\n' % msg.encode('unicode-escape').decode('ascii')) stream.flush() def log(self, level, msg, *args): diff --git a/Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst b/Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst new file mode 100644 index 000000000000..cc1db086f0c4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst @@ -0,0 +1 @@ +Fix distutils logging for non-ASCII strings. This caused installation issues on Windows. From webhook-mailer at python.org Sat Sep 8 16:44:24 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 08 Sep 2018 20:44:24 -0000 Subject: [Python-checkins] bpo-34421 avoid unicode error in distutils logging (GH-8799) Message-ID: https://github.com/python/cpython/commit/3b36642924a51e6bceb7033916c3049764817166 commit: 3b36642924a51e6bceb7033916c3049764817166 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-08T13:44:20-07:00 summary: bpo-34421 avoid unicode error in distutils logging (GH-8799) This caused installation errors in some cases on Windows. Patch by Julien Malard. (cherry picked from commit 0afada163c7ef25c3a9d46ed445481fb69f2ecaf) Co-authored-by: Julien Malard files: A Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst M Lib/distutils/log.py diff --git a/Lib/distutils/log.py b/Lib/distutils/log.py index b301a8338c20..3a6602bc8b8e 100644 --- a/Lib/distutils/log.py +++ b/Lib/distutils/log.py @@ -31,7 +31,10 @@ def _log(self, level, msg, args): # emulate backslashreplace error handler encoding = stream.encoding msg = msg.encode(encoding, "backslashreplace").decode(encoding) - stream.write('%s\n' % msg) + try: + stream.write('%s\n' % msg) + except UnicodeEncodeError: + stream.write('%s\n' % msg.encode('unicode-escape').decode('ascii')) stream.flush() def log(self, level, msg, *args): diff --git a/Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst b/Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst new file mode 100644 index 000000000000..cc1db086f0c4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst @@ -0,0 +1 @@ +Fix distutils logging for non-ASCII strings. This caused installation issues on Windows. From webhook-mailer at python.org Sat Sep 8 16:53:02 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 08 Sep 2018 20:53:02 -0000 Subject: [Python-checkins] bpo-34421 avoid unicode error in distutils logging (GH-8799) Message-ID: https://github.com/python/cpython/commit/77b92b15a5e5c84b91d3fd9d02f63db432fa8903 commit: 77b92b15a5e5c84b91d3fd9d02f63db432fa8903 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-08T13:52:59-07:00 summary: bpo-34421 avoid unicode error in distutils logging (GH-8799) This caused installation errors in some cases on Windows. Patch by Julien Malard. (cherry picked from commit 0afada163c7ef25c3a9d46ed445481fb69f2ecaf) Co-authored-by: Julien Malard files: A Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst M Lib/distutils/log.py diff --git a/Lib/distutils/log.py b/Lib/distutils/log.py index b301a8338c20..3a6602bc8b8e 100644 --- a/Lib/distutils/log.py +++ b/Lib/distutils/log.py @@ -31,7 +31,10 @@ def _log(self, level, msg, args): # emulate backslashreplace error handler encoding = stream.encoding msg = msg.encode(encoding, "backslashreplace").decode(encoding) - stream.write('%s\n' % msg) + try: + stream.write('%s\n' % msg) + except UnicodeEncodeError: + stream.write('%s\n' % msg.encode('unicode-escape').decode('ascii')) stream.flush() def log(self, level, msg, *args): diff --git a/Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst b/Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst new file mode 100644 index 000000000000..cc1db086f0c4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-07-10-57-00.bpo-34421.AKJISD.rst @@ -0,0 +1 @@ +Fix distutils logging for non-ASCII strings. This caused installation issues on Windows. From solipsis at pitrou.net Sun Sep 9 05:09:51 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 09 Sep 2018 09:09:51 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=5 Message-ID: <20180909090951.1.710B374100B5B5C1@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [-7, 1, 7] memory blocks, sum=1 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [0, -2, 2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogRauWFq', '--timeout', '7200'] From webhook-mailer at python.org Sun Sep 9 07:26:58 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Sun, 09 Sep 2018 11:26:58 -0000 Subject: [Python-checkins] [3.6] bpo-34604: Fix possible mojibake in pwd.getpwnam() and grp.getgrnam() (GH-9098) (GH-9105) Message-ID: https://github.com/python/cpython/commit/1bcd891c200b8122493ddad5a203331e1a3bfcb5 commit: 1bcd891c200b8122493ddad5a203331e1a3bfcb5 branch: 3.6 author: William Grzybowski committer: Victor Stinner date: 2018-09-09T13:26:48+02:00 summary: [3.6] bpo-34604: Fix possible mojibake in pwd.getpwnam() and grp.getgrnam() (GH-9098) (GH-9105) Pass the user/group name as Unicode to the formatting function, instead of always decoding a bytes string from UTF-8.. (cherry picked from commit 28658485a54ad5f9df52ecc12d9046269f1654ec) Co-authored-by: William Grzybowski files: A Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst M Modules/grpmodule.c M Modules/pwdmodule.c diff --git a/Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst b/Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst new file mode 100644 index 000000000000..562a69124b3d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst @@ -0,0 +1,2 @@ +Fix possible mojibake in the error message of `pwd.getpwnam` and +`grp.getgrnam`. Patch by William Grzybowski. diff --git a/Modules/grpmodule.c b/Modules/grpmodule.c index f577fd3ab4ec..43e45ef7aad5 100644 --- a/Modules/grpmodule.c +++ b/Modules/grpmodule.c @@ -156,7 +156,7 @@ grp_getgrnam_impl(PyObject *module, PyObject *name) goto out; if ((p = getgrnam(name_chars)) == NULL) { - PyErr_Format(PyExc_KeyError, "getgrnam(): name not found: %s", name_chars); + PyErr_Format(PyExc_KeyError, "getgrnam(): name not found: %S", name); goto out; } retval = mkgrent(p); diff --git a/Modules/pwdmodule.c b/Modules/pwdmodule.c index bbef2de9c522..21c2b546f6dd 100644 --- a/Modules/pwdmodule.c +++ b/Modules/pwdmodule.c @@ -163,7 +163,7 @@ pwd_getpwnam_impl(PyObject *module, PyObject *arg) goto out; if ((p = getpwnam(name)) == NULL) { PyErr_Format(PyExc_KeyError, - "getpwnam(): name not found: %s", name); + "getpwnam(): name not found: %S", arg); goto out; } retval = mkpwent(p); From webhook-mailer at python.org Sun Sep 9 07:27:35 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Sun, 09 Sep 2018 11:27:35 -0000 Subject: [Python-checkins] [3.7] bpo-34604: Fix possible mojibake in pwd.getpwnam() and grp.getgrnam() (GH-9098) (GH-9104) Message-ID: https://github.com/python/cpython/commit/7a633ed79cfba2cfc0f80410ddcaeecadc2030e9 commit: 7a633ed79cfba2cfc0f80410ddcaeecadc2030e9 branch: 3.7 author: William Grzybowski committer: Victor Stinner date: 2018-09-09T13:27:31+02:00 summary: [3.7] bpo-34604: Fix possible mojibake in pwd.getpwnam() and grp.getgrnam() (GH-9098) (GH-9104) Pass the user/group name as Unicode to the formatting function, instead of always decoding a bytes string from UTF-8.. (cherry picked from commit 28658485a54ad5f9df52ecc12d9046269f1654ec) Co-authored-by: William Grzybowski files: A Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst M Modules/grpmodule.c M Modules/pwdmodule.c diff --git a/Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst b/Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst new file mode 100644 index 000000000000..562a69124b3d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-07-10-16-34.bpo-34604.xL7-kG.rst @@ -0,0 +1,2 @@ +Fix possible mojibake in the error message of `pwd.getpwnam` and +`grp.getgrnam`. Patch by William Grzybowski. diff --git a/Modules/grpmodule.c b/Modules/grpmodule.c index f577fd3ab4ec..43e45ef7aad5 100644 --- a/Modules/grpmodule.c +++ b/Modules/grpmodule.c @@ -156,7 +156,7 @@ grp_getgrnam_impl(PyObject *module, PyObject *name) goto out; if ((p = getgrnam(name_chars)) == NULL) { - PyErr_Format(PyExc_KeyError, "getgrnam(): name not found: %s", name_chars); + PyErr_Format(PyExc_KeyError, "getgrnam(): name not found: %S", name); goto out; } retval = mkgrent(p); diff --git a/Modules/pwdmodule.c b/Modules/pwdmodule.c index bbef2de9c522..21c2b546f6dd 100644 --- a/Modules/pwdmodule.c +++ b/Modules/pwdmodule.c @@ -163,7 +163,7 @@ pwd_getpwnam_impl(PyObject *module, PyObject *arg) goto out; if ((p = getpwnam(name)) == NULL) { PyErr_Format(PyExc_KeyError, - "getpwnam(): name not found: %s", name); + "getpwnam(): name not found: %S", arg); goto out; } retval = mkpwent(p); From webhook-mailer at python.org Mon Sep 10 02:27:35 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Mon, 10 Sep 2018 06:27:35 -0000 Subject: [Python-checkins] Remove unneeded PyUnicode_READY() in tokenizer.c (GH-9114) Message-ID: https://github.com/python/cpython/commit/5061a74a4c8686764a299d82df048bf858dd263b commit: 5061a74a4c8686764a299d82df048bf858dd263b branch: master author: Zackery Spytz committer: Serhiy Storchaka date: 2018-09-10T09:27:31+03:00 summary: Remove unneeded PyUnicode_READY() in tokenizer.c (GH-9114) files: M Parser/tokenizer.c diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 6566fdead380..fc75bae53766 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1324,7 +1324,7 @@ verify_identifier(struct tok_state *tok) if (tok->decoding_erred) return 0; s = PyUnicode_DecodeUTF8(tok->start, tok->cur - tok->start, NULL); - if (s == NULL || PyUnicode_READY(s) == -1) { + if (s == NULL) { if (PyErr_ExceptionMatches(PyExc_UnicodeDecodeError)) { PyErr_Clear(); tok->done = E_IDENTIFIER; From webhook-mailer at python.org Mon Sep 10 05:07:27 2018 From: webhook-mailer at python.org (Xiang Zhang) Date: Mon, 10 Sep 2018 09:07:27 -0000 Subject: [Python-checkins] Revert "Fix misindented yaml in logging how to example (GH-8604)" (GH-9081) Message-ID: https://github.com/python/cpython/commit/290a60bd8af7b1d7e7931aa4dd4eace60d355d76 commit: 290a60bd8af7b1d7e7931aa4dd4eace60d355d76 branch: master author: R?my HUBSCHER committer: Xiang Zhang date: 2018-09-10T17:07:15+08:00 summary: Revert "Fix misindented yaml in logging how to example (GH-8604)" (GH-9081) This reverts commit 10b59f1b019cd00c940dd7f4a74c4f667a20f25f. files: M Doc/howto/logging.rst diff --git a/Doc/howto/logging.rst b/Doc/howto/logging.rst index 6f47baeee2f5..47b5c680c424 100644 --- a/Doc/howto/logging.rst +++ b/Doc/howto/logging.rst @@ -750,9 +750,9 @@ the new dictionary-based approach: level: DEBUG handlers: [console] propagate: no - root: - level: DEBUG - handlers: [console] + root: + level: DEBUG + handlers: [console] For more information about logging using a dictionary, see :ref:`logging-config-api`. From solipsis at pitrou.net Mon Sep 10 05:07:33 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 10 Sep 2018 09:07:33 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=3 Message-ID: <20180910090733.1.5535083142C260FE@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_forkserver leaked [-2, 1, 0] memory blocks, sum=-1 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog8sHDRk', '--timeout', '7200'] From webhook-mailer at python.org Mon Sep 10 05:13:24 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 09:13:24 -0000 Subject: [Python-checkins] Revert "Fix misindented yaml in logging how to example (GH-8604)" (GH-9081) Message-ID: https://github.com/python/cpython/commit/9c9ac9e69735a2854c2e0d41948385110b139abe commit: 9c9ac9e69735a2854c2e0d41948385110b139abe branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T02:13:18-07:00 summary: Revert "Fix misindented yaml in logging how to example (GH-8604)" (GH-9081) This reverts commit 10b59f1b019cd00c940dd7f4a74c4f667a20f25f. (cherry picked from commit 290a60bd8af7b1d7e7931aa4dd4eace60d355d76) Co-authored-by: R?my HUBSCHER files: M Doc/howto/logging.rst diff --git a/Doc/howto/logging.rst b/Doc/howto/logging.rst index 6f47baeee2f5..47b5c680c424 100644 --- a/Doc/howto/logging.rst +++ b/Doc/howto/logging.rst @@ -750,9 +750,9 @@ the new dictionary-based approach: level: DEBUG handlers: [console] propagate: no - root: - level: DEBUG - handlers: [console] + root: + level: DEBUG + handlers: [console] For more information about logging using a dictionary, see :ref:`logging-config-api`. From webhook-mailer at python.org Mon Sep 10 05:19:25 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 09:19:25 -0000 Subject: [Python-checkins] Revert "Fix misindented yaml in logging how to example (GH-8604)" (GH-9081) Message-ID: https://github.com/python/cpython/commit/8eda821d1b8db374bf963cf18b5c7f063366b062 commit: 8eda821d1b8db374bf963cf18b5c7f063366b062 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T02:19:20-07:00 summary: Revert "Fix misindented yaml in logging how to example (GH-8604)" (GH-9081) This reverts commit 10b59f1b019cd00c940dd7f4a74c4f667a20f25f. (cherry picked from commit 290a60bd8af7b1d7e7931aa4dd4eace60d355d76) Co-authored-by: R?my HUBSCHER files: M Doc/howto/logging.rst diff --git a/Doc/howto/logging.rst b/Doc/howto/logging.rst index 78a237f5ae63..9914f8d154ab 100644 --- a/Doc/howto/logging.rst +++ b/Doc/howto/logging.rst @@ -750,9 +750,9 @@ the new dictionary-based approach: level: DEBUG handlers: [console] propagate: no - root: - level: DEBUG - handlers: [console] + root: + level: DEBUG + handlers: [console] For more information about logging using a dictionary, see :ref:`logging-config-api`. From webhook-mailer at python.org Mon Sep 10 05:42:14 2018 From: webhook-mailer at python.org (Xiang Zhang) Date: Mon, 10 Sep 2018 09:42:14 -0000 Subject: [Python-checkins] Test dict values iterator pickling with pickle.HIGHEST_PROTOCOL. (GH-9052) Message-ID: https://github.com/python/cpython/commit/1f36bf6077d93cb43fd84bea4a8a625fa772d1fa commit: 1f36bf6077d93cb43fd84bea4a8a625fa772d1fa branch: master author: Sergey Fedoseev committer: Xiang Zhang date: 2018-09-10T17:42:09+08:00 summary: Test dict values iterator pickling with pickle.HIGHEST_PROTOCOL. (GH-9052) files: M Lib/test/test_dict.py diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py index aa149d31eb0e..38521bbf6630 100644 --- a/Lib/test/test_dict.py +++ b/Lib/test/test_dict.py @@ -1056,7 +1056,7 @@ def test_itemiterator_pickling(self): self.assertEqual(dict(it), data) def test_valuesiterator_pickling(self): - for proto in range(pickle.HIGHEST_PROTOCOL): + for proto in range(pickle.HIGHEST_PROTOCOL + 1): data = {1:"a", 2:"b", 3:"c"} # data.values() isn't picklable, only its iterator it = iter(data.values()) From webhook-mailer at python.org Mon Sep 10 06:32:57 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 10:32:57 -0000 Subject: [Python-checkins] Test dict values iterator pickling with pickle.HIGHEST_PROTOCOL. (GH-9052) Message-ID: https://github.com/python/cpython/commit/9f5351edbe8610f75174013be66a767fecf78fe2 commit: 9f5351edbe8610f75174013be66a767fecf78fe2 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T03:32:53-07:00 summary: Test dict values iterator pickling with pickle.HIGHEST_PROTOCOL. (GH-9052) (cherry picked from commit 1f36bf6077d93cb43fd84bea4a8a625fa772d1fa) Co-authored-by: Sergey Fedoseev files: M Lib/test/test_dict.py diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py index 4386eda3ae48..639e05f74a8a 100644 --- a/Lib/test/test_dict.py +++ b/Lib/test/test_dict.py @@ -1010,7 +1010,7 @@ def test_itemiterator_pickling(self): self.assertEqual(dict(it), data) def test_valuesiterator_pickling(self): - for proto in range(pickle.HIGHEST_PROTOCOL): + for proto in range(pickle.HIGHEST_PROTOCOL + 1): data = {1:"a", 2:"b", 3:"c"} # data.values() isn't picklable, only its iterator it = iter(data.values()) From webhook-mailer at python.org Mon Sep 10 06:45:57 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 10:45:57 -0000 Subject: [Python-checkins] Test dict values iterator pickling with pickle.HIGHEST_PROTOCOL. (GH-9052) Message-ID: https://github.com/python/cpython/commit/e9119a5de67d5525f3be957fda239143453513bd commit: e9119a5de67d5525f3be957fda239143453513bd branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T03:45:54-07:00 summary: Test dict values iterator pickling with pickle.HIGHEST_PROTOCOL. (GH-9052) (cherry picked from commit 1f36bf6077d93cb43fd84bea4a8a625fa772d1fa) Co-authored-by: Sergey Fedoseev files: M Lib/test/test_dict.py diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py index aa149d31eb0e..38521bbf6630 100644 --- a/Lib/test/test_dict.py +++ b/Lib/test/test_dict.py @@ -1056,7 +1056,7 @@ def test_itemiterator_pickling(self): self.assertEqual(dict(it), data) def test_valuesiterator_pickling(self): - for proto in range(pickle.HIGHEST_PROTOCOL): + for proto in range(pickle.HIGHEST_PROTOCOL + 1): data = {1:"a", 2:"b", 3:"c"} # data.values() isn't picklable, only its iterator it = iter(data.values()) From webhook-mailer at python.org Mon Sep 10 09:11:08 2018 From: webhook-mailer at python.org (Tal Einat) Date: Mon, 10 Sep 2018 13:11:08 -0000 Subject: [Python-checkins] bpo-30977: rework code changes according to post-merge code review (GH-9106) Message-ID: https://github.com/python/cpython/commit/54752533b2ed1c898ffe5ec2e795c6910ee46a39 commit: 54752533b2ed1c898ffe5ec2e795c6910ee46a39 branch: master author: Tal Einat committer: GitHub date: 2018-09-10T16:11:04+03:00 summary: bpo-30977: rework code changes according to post-merge code review (GH-9106) also mention the change and its consequences in What's New files: M Doc/whatsnew/3.8.rst M Lib/test/test_uuid.py M Lib/uuid.py diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 2de7a50a471d..a9b689d53728 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -131,6 +131,10 @@ Optimizations objects (e.g. tuple, list, dict) size is reduced 4 or 8 bytes. (Contributed by Inada Naoki in :issue:`33597`) +* :class:`uuid.UUID` now uses ``__slots__`` to reduce its memory footprint. + Note that this means that instances can no longer be weak-referenced and + that arbitrary attributes can no longer be added to them. + Build and C API Changes ======================= @@ -275,6 +279,9 @@ Changes in the Python API * The function :func:`math.factorial` no longer accepts arguments that are not int-like. (Contributed by Pablo Galindo in :issue:`33083`.) +* :class:`uuid.UUID` now uses ``__slots__``, therefore instances can no longer + be weak-referenced and attributes can no longer be added. + CPython bytecode changes ------------------------ diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py index 9ec59d519573..c1b35c4a7f6c 100644 --- a/Lib/test/test_uuid.py +++ b/Lib/test/test_uuid.py @@ -2,6 +2,8 @@ from test import support import builtins import contextlib +import copy +from functools import partial import io import os import pickle @@ -313,63 +315,139 @@ def test_getnode(self): node2 = self.uuid.getnode() self.assertEqual(node1, node2, '%012x != %012x' % (node1, node2)) - def _setup_for_pickle(self): - orig_uuid = sys.modules.get('uuid') - sys.modules['uuid'] = self.uuid - - def restore_uuid_module(): - if orig_uuid is not None: - sys.modules['uuid'] = orig_uuid - else: - del sys.modules['uuid'] - self.addCleanup(restore_uuid_module) - def test_pickle_roundtrip(self): - self._setup_for_pickle() - - u = self.uuid.UUID('12345678123456781234567812345678') - self.assertEqual(u, pickle.loads(pickle.dumps(u))) + def check(actual, expected): + self.assertEqual(actual, expected) + self.assertEqual(actual.is_safe, expected.is_safe) + + with support.swap_item(sys.modules, 'uuid', self.uuid): + for is_safe in self.uuid.SafeUUID: + u = self.uuid.UUID('d82579ce6642a0de7ddf490a7aec7aa5', + is_safe=is_safe) + check(copy.copy(u), u) + check(copy.deepcopy(u), u) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(protocol=proto): + check(pickle.loads(pickle.dumps(u, proto)), u) def test_unpickle_previous_python_versions(self): - self._setup_for_pickle() - - u = self.uuid.UUID('12345678123456781234567812345678') - - # Python 2.7 protocol 0-2 pickles of u - py27_pickles = [ - b'ccopy_reg\n_reconstructor\np0\n(cuuid\nUUID\np1\nc__builtin__\nob' - b'ject\np2\nNtp3\nRp4\n(dp5\nS\'int\'\np6\nL24197857161011715162171' - b'839636988778104L\nsb.', - b'ccopy_reg\n_reconstructor\nq\x00(cuuid\nUUID\nq\x01c__builtin__\n' - b'object\nq\x02Ntq\x03Rq\x04}q\x05U\x03intq\x06L2419785716101171516' - b'2171839636988778104L\nsb.', - b'\x80\x02cuuid\nUUID\nq\x00)\x81q\x01}q\x02U\x03intq\x03\x8a\x10xV' - b'4\x12xV4\x12xV4\x12xV4\x12sb.', + def check(actual, expected): + self.assertEqual(actual, expected) + self.assertEqual(actual.is_safe, expected.is_safe) + + pickled_uuids = [ + # Python 2.7, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dS\'int\'\nL287307832597519156748809049798316161701L\nsb.', + # Python 2.7, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}U\x03intL287307832597519156748809049798316161701L\nsb.', + # Python 2.7, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}U\x03int\x8a\x11\xa5z\xecz\nI\xdf}' + b'\xde\xa0Bf\xcey%\xd8\x00sb.', + # Python 3.6, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dVint\nL287307832597519156748809049798316161701L\nsb.', + # Python 3.6, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}X\x03\x00\x00\x00intL287307832597519156748809049798316161701L' + b'\nsb.', + # Python 3.6, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}X\x03\x00\x00\x00int\x8a\x11\xa5z\xec' + b'z\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00sb.', + # Python 3.6, protocol 3 + b'\x80\x03cuuid\nUUID\n)\x81}X\x03\x00\x00\x00int\x8a\x11\xa5z\xec' + b'z\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00sb.', + # Python 3.6, protocol 4 + b'\x80\x04\x95+\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x8c\x04UUI' + b'D\x93)\x81}\x8c\x03int\x8a\x11\xa5z\xecz\nI\xdf}\xde\xa0Bf\xcey%' + b'\xd8\x00sb.', + # Python 3.7, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dVint\nL287307832597519156748809049798316161701L\nsVis_safe\n' + b'cuuid\nSafeUUID\n(NtRsb.', + # Python 3.7, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}(X\x03\x00\x00\x00intL287307832597519156748809049798316161701' + b'L\nX\x07\x00\x00\x00is_safecuuid\nSafeUUID\n(NtRub.', + # Python 3.7, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nN\x85Rub.', + # Python 3.7, protocol 3 + b'\x80\x03cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nN\x85Rub.', + # Python 3.7, protocol 4 + b'\x80\x04\x95F\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x94\x8c' + b'\x04UUID\x93)\x81}(\x8c\x03int\x8a\x11\xa5z\xecz\nI\xdf}\xde\xa0' + b'Bf\xcey%\xd8\x00\x8c\x07is_safeh\x00\x8c\x08SafeUUID\x93N\x85Rub' + b'.', + ] + pickled_uuids_safe = [ + # Python 3.7, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dVint\nL287307832597519156748809049798316161701L\nsVis_safe\n' + b'cuuid\nSafeUUID\n(I0\ntRsb.', + # Python 3.7, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}(X\x03\x00\x00\x00intL287307832597519156748809049798316161701' + b'L\nX\x07\x00\x00\x00is_safecuuid\nSafeUUID\n(K\x00tRub.', + # Python 3.7, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nK\x00\x85Rub.', + # Python 3.7, protocol 3 + b'\x80\x03cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nK\x00\x85Rub.', + # Python 3.7, protocol 4 + b'\x80\x04\x95G\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x94\x8c' + b'\x04UUID\x93)\x81}(\x8c\x03int\x8a\x11\xa5z\xecz\nI\xdf}\xde\xa0' + b'Bf\xcey%\xd8\x00\x8c\x07is_safeh\x00\x8c\x08SafeUUID\x93K\x00' + b'\x85Rub.', ] - # Python 3.6 protocol 0-4 pickles of u - py36_pickles = [ - b'ccopy_reg\n_reconstructor\np0\n(cuuid\nUUID\np1\nc__builtin__\nob' - b'ject\np2\nNtp3\nRp4\n(dp5\nVint\np6\nL241978571610117151621718396' - b'36988778104L\nsb.', - b'ccopy_reg\n_reconstructor\nq\x00(cuuid\nUUID\nq\x01c__builtin__\n' - b'object\nq\x02Ntq\x03Rq\x04}q\x05X\x03\x00\x00\x00intq\x06L2419785' - b'7161011715162171839636988778104L\nsb.', - b'\x80\x02cuuid\nUUID\nq\x00)\x81q\x01}q\x02X\x03\x00\x00\x00intq' - b'\x03\x8a\x10xV4\x12xV4\x12xV4\x12xV4\x12sb.', - b'\x80\x03cuuid\nUUID\nq\x00)\x81q\x01}q\x02X\x03\x00\x00\x00intq' - b'\x03\x8a\x10xV4\x12xV4\x12xV4\x12xV4\x12sb.', - b'\x80\x04\x950\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x94\x8c\x04' - b'UUID\x94\x93\x94)\x81\x94}\x94\x8c\x03int\x94\x8a\x10xV4\x12xV4' - b'\x12xV4\x12xV4\x12sb.', + pickled_uuids_unsafe = [ + # Python 3.7, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dVint\nL287307832597519156748809049798316161701L\nsVis_safe\n' + b'cuuid\nSafeUUID\n(I-1\ntRsb.', + # Python 3.7, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}(X\x03\x00\x00\x00intL287307832597519156748809049798316161701' + b'L\nX\x07\x00\x00\x00is_safecuuid\nSafeUUID\n(J\xff\xff\xff\xfftR' + b'ub.', + # Python 3.7, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nJ\xff\xff\xff\xff\x85Rub.', + # Python 3.7, protocol 3 + b'\x80\x03cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nJ\xff\xff\xff\xff\x85Rub.', + # Python 3.7, protocol 4 + b'\x80\x04\x95J\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x94\x8c' + b'\x04UUID\x93)\x81}(\x8c\x03int\x8a\x11\xa5z\xecz\nI\xdf}\xde\xa0' + b'Bf\xcey%\xd8\x00\x8c\x07is_safeh\x00\x8c\x08SafeUUID\x93J\xff' + b'\xff\xff\xff\x85Rub.', ] - for pickled in py27_pickles + py36_pickles: - unpickled = pickle.loads(pickled) - self.assertEqual(unpickled, u) - # is_safe was added in 3.7. When unpickling values from older - # versions, is_safe will be missing, so it should be set to - # SafeUUID.unknown. - self.assertEqual(unpickled.is_safe, self.uuid.SafeUUID.unknown) + u = self.uuid.UUID('d82579ce6642a0de7ddf490a7aec7aa5') + u_safe = self.uuid.UUID('d82579ce6642a0de7ddf490a7aec7aa5', + is_safe=self.uuid.SafeUUID.safe) + u_unsafe = self.uuid.UUID('d82579ce6642a0de7ddf490a7aec7aa5', + is_safe=self.uuid.SafeUUID.unsafe) + + with support.swap_item(sys.modules, 'uuid', self.uuid): + for pickled in pickled_uuids: + # is_safe was added in 3.7. When unpickling values from older + # versions, is_safe will be missing, so it should be set to + # SafeUUID.unknown. + check(pickle.loads(pickled), u) + for pickled in pickled_uuids_safe: + check(pickle.loads(pickled), u_safe) + for pickled in pickled_uuids_unsafe: + check(pickle.loads(pickled), u_unsafe) # bpo-32502: UUID1 requires a 48-bit identifier, but hardware identifiers # need not necessarily be 48 bits (e.g., EUI-64). diff --git a/Lib/uuid.py b/Lib/uuid.py index 515388221510..073ca711ab42 100644 --- a/Lib/uuid.py +++ b/Lib/uuid.py @@ -207,26 +207,19 @@ def __init__(self, hex=None, bytes=None, bytes_le=None, fields=None, object.__setattr__(self, 'is_safe', is_safe) def __getstate__(self): - d = {attr: getattr(self, attr) for attr in self.__slots__} - # is_safe is a SafeUUID instance. Return just its value, so that - # it can be unpickled in older Python versions without SafeUUID. - d['is_safe'] = d['is_safe'].value + d = {'int': self.int} + if self.is_safe != SafeUUID.unknown: + # is_safe is a SafeUUID instance. Return just its value, so that + # it can be un-pickled in older Python versions without SafeUUID. + d['is_safe'] = self.is_safe.value return d def __setstate__(self, state): - # is_safe was added in 3.7 - state.setdefault('is_safe', SafeUUID.unknown.value) - - for attr in self.__slots__: - value = state[attr] - - # for is_safe, restore the SafeUUID from the stored value - if attr == 'is_safe': - try: - value = SafeUUID(value) - except ValueError: - value = SafeUUID.unknown - object.__setattr__(self, attr, value) + object.__setattr__(self, 'int', state['int']) + # is_safe was added in 3.7; it is also omitted when it is "unknown" + object.__setattr__(self, 'is_safe', + SafeUUID(state['is_safe']) + if 'is_safe' in state else SafeUUID.unknown) def __eq__(self, other): if isinstance(other, UUID): From webhook-mailer at python.org Mon Sep 10 11:43:17 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Mon, 10 Sep 2018 15:43:17 -0000 Subject: [Python-checkins] bpo-34588: Fix an off-by-one error in traceback formatting. (GH-9077) Message-ID: https://github.com/python/cpython/commit/d545869d084e70d4838310e79b52a25a72a1ca56 commit: d545869d084e70d4838310e79b52a25a72a1ca56 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-10T08:43:10-07:00 summary: bpo-34588: Fix an off-by-one error in traceback formatting. (GH-9077) The recursive frame pruning code always undercounted the number of elided frames by one. That is, in the "[Previous line repeated N more times]" message, N would always be one too few. Near the recursive pruning cutoff, one frame could be silently dropped. That situation is demonstrated in the OP of the bug report. The fix is to start the identical frame counter at 1. files: A Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst M Lib/test/test_traceback.py M Lib/traceback.py M Python/traceback.c diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index bffc03e663ff..8a3aa8a8648f 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -373,7 +373,7 @@ def g(count=10): ' return g(count-1)\n' f' File "{__file__}", line {lineno_g+2}, in g\n' ' return g(count-1)\n' - ' [Previous line repeated 6 more times]\n' + ' [Previous line repeated 7 more times]\n' f' File "{__file__}", line {lineno_g+3}, in g\n' ' raise ValueError\n' 'ValueError\n' @@ -412,7 +412,7 @@ def h(count=10): ' return h(count-1)\n' f' File "{__file__}", line {lineno_h+2}, in h\n' ' return h(count-1)\n' - ' [Previous line repeated 6 more times]\n' + ' [Previous line repeated 7 more times]\n' f' File "{__file__}", line {lineno_h+3}, in h\n' ' g()\n' ) @@ -420,6 +420,63 @@ def h(count=10): actual = stderr_h.getvalue().splitlines() self.assertEqual(actual, expected) + # Check the boundary conditions. First, test just below the cutoff. + with captured_output("stderr") as stderr_g: + try: + g(traceback._RECURSIVE_CUTOFF) + except ValueError as exc: + render_exc() + else: + self.fail("no error raised") + result_g = ( + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+3}, in g\n' + ' raise ValueError\n' + 'ValueError\n' + ) + tb_line = ( + 'Traceback (most recent call last):\n' + f' File "{__file__}", line {lineno_g+71}, in _check_recursive_traceback_display\n' + ' g(traceback._RECURSIVE_CUTOFF)\n' + ) + expected = (tb_line + result_g).splitlines() + actual = stderr_g.getvalue().splitlines() + self.assertEqual(actual, expected) + + # Second, test just above the cutoff. + with captured_output("stderr") as stderr_g: + try: + g(traceback._RECURSIVE_CUTOFF + 1) + except ValueError as exc: + render_exc() + else: + self.fail("no error raised") + result_g = ( + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + ' [Previous line repeated 1 more time]\n' + f' File "{__file__}", line {lineno_g+3}, in g\n' + ' raise ValueError\n' + 'ValueError\n' + ) + tb_line = ( + 'Traceback (most recent call last):\n' + f' File "{__file__}", line {lineno_g+99}, in _check_recursive_traceback_display\n' + ' g(traceback._RECURSIVE_CUTOFF + 1)\n' + ) + expected = (tb_line + result_g).splitlines() + actual = stderr_g.getvalue().splitlines() + self.assertEqual(actual, expected) + def test_recursive_traceback_python(self): self._check_recursive_traceback_display(traceback.print_exc) diff --git a/Lib/traceback.py b/Lib/traceback.py index afab0a4b91f1..4e7605d15fa4 100644 --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -310,6 +310,8 @@ def walk_tb(tb): tb = tb.tb_next +_RECURSIVE_CUTOFF = 3 # Also hardcoded in traceback.c. + class StackSummary(list): """A stack of frames.""" @@ -398,18 +400,21 @@ def format(self): last_name = None count = 0 for frame in self: - if (last_file is not None and last_file == frame.filename and - last_line is not None and last_line == frame.lineno and - last_name is not None and last_name == frame.name): - count += 1 - else: - if count > 3: - result.append(f' [Previous line repeated {count-3} more times]\n') + if (last_file is None or last_file != frame.filename or + last_line is None or last_line != frame.lineno or + last_name is None or last_name != frame.name): + if count > _RECURSIVE_CUTOFF: + count -= _RECURSIVE_CUTOFF + result.append( + f' [Previous line repeated {count} more ' + f'time{"s" if count > 1 else ""}]\n' + ) last_file = frame.filename last_line = frame.lineno last_name = frame.name count = 0 - if count >= 3: + count += 1 + if count > _RECURSIVE_CUTOFF: continue row = [] row.append(' File "{}", line {}, in {}\n'.format( @@ -420,8 +425,12 @@ def format(self): for name, value in sorted(frame.locals.items()): row.append(' {name} = {value}\n'.format(name=name, value=value)) result.append(''.join(row)) - if count > 3: - result.append(f' [Previous line repeated {count-3} more times]\n') + if count > _RECURSIVE_CUTOFF: + count -= _RECURSIVE_CUTOFF + result.append( + f' [Previous line repeated {count} more ' + f'time{"s" if count > 1 else ""}]\n' + ) return result diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst new file mode 100644 index 000000000000..ec7a57f23ad2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst @@ -0,0 +1,2 @@ +Fix an off-by-one in the recursive call pruning feature of traceback +formatting. diff --git a/Python/traceback.c b/Python/traceback.c index 21fb03416002..5b5c715a0f44 100644 --- a/Python/traceback.c +++ b/Python/traceback.c @@ -511,16 +511,21 @@ tb_displayline(PyObject *f, PyObject *filename, int lineno, PyObject *name) return err; } +static const int TB_RECURSIVE_CUTOFF = 3; // Also hardcoded in traceback.py. + static int tb_print_line_repeated(PyObject *f, long cnt) { - int err; + cnt -= TB_RECURSIVE_CUTOFF; PyObject *line = PyUnicode_FromFormat( - " [Previous line repeated %ld more times]\n", cnt-3); + (cnt > 1) + ? " [Previous line repeated %ld more times]\n" + : " [Previous line repeated %ld more time]\n", + cnt); if (line == NULL) { return -1; } - err = PyFile_WriteObject(line, f, Py_PRINT_RAW); + int err = PyFile_WriteObject(line, f, Py_PRINT_RAW); Py_DECREF(line); return err; } @@ -544,15 +549,11 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit) tb = tb->tb_next; } while (tb != NULL && err == 0) { - if (last_file != NULL && - tb->tb_frame->f_code->co_filename == last_file && - last_line != -1 && tb->tb_lineno == last_line && - last_name != NULL && tb->tb_frame->f_code->co_name == last_name) - { - cnt++; - } - else { - if (cnt > 3) { + if (last_file == NULL || + tb->tb_frame->f_code->co_filename != last_file || + last_line == -1 || tb->tb_lineno != last_line || + last_name == NULL || tb->tb_frame->f_code->co_name != last_name) { + if (cnt > TB_RECURSIVE_CUTOFF) { err = tb_print_line_repeated(f, cnt); } last_file = tb->tb_frame->f_code->co_filename; @@ -560,7 +561,8 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit) last_name = tb->tb_frame->f_code->co_name; cnt = 0; } - if (err == 0 && cnt < 3) { + cnt++; + if (err == 0 && cnt <= TB_RECURSIVE_CUTOFF) { err = tb_displayline(f, tb->tb_frame->f_code->co_filename, tb->tb_lineno, @@ -571,7 +573,7 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit) } tb = tb->tb_next; } - if (err == 0 && cnt > 3) { + if (err == 0 && cnt > TB_RECURSIVE_CUTOFF) { err = tb_print_line_repeated(f, cnt); } return err; From webhook-mailer at python.org Mon Sep 10 11:47:36 2018 From: webhook-mailer at python.org (Tal Einat) Date: Mon, 10 Sep 2018 15:47:36 -0000 Subject: [Python-checkins] [3.7] bpo-34621: fix uuid.UUID (un)pickling compatbility with older Python versions (<3.7) (GH-9133) Message-ID: https://github.com/python/cpython/commit/d53f1cabe8837697df4acb70c9c6537461b5eeda commit: d53f1cabe8837697df4acb70c9c6537461b5eeda branch: 3.7 author: Tal Einat committer: GitHub date: 2018-09-10T18:47:29+03:00 summary: [3.7] bpo-34621: fix uuid.UUID (un)pickling compatbility with older Python versions (<3.7) (GH-9133) files: A Misc/NEWS.d/next/Library/2018-09-10-15-54-58.bpo-34621.Uqj5x3.rst M Lib/test/test_uuid.py M Lib/uuid.py diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py index 7af1d7aec797..dc502b97ca4e 100644 --- a/Lib/test/test_uuid.py +++ b/Lib/test/test_uuid.py @@ -2,10 +2,13 @@ from test import support import builtins import contextlib +import copy import io import os +import pickle import shutil import subprocess +import sys py_uuid = support.import_fresh_module('uuid', blocked=['_uuid']) c_uuid = support.import_fresh_module('uuid', fresh=['_uuid']) @@ -311,6 +314,140 @@ def test_getnode(self): node2 = self.uuid.getnode() self.assertEqual(node1, node2, '%012x != %012x' % (node1, node2)) + def test_pickle_roundtrip(self): + def check(actual, expected): + self.assertEqual(actual, expected) + self.assertEqual(actual.is_safe, expected.is_safe) + + with support.swap_item(sys.modules, 'uuid', self.uuid): + for is_safe in self.uuid.SafeUUID: + u = self.uuid.UUID('d82579ce6642a0de7ddf490a7aec7aa5', + is_safe=is_safe) + check(copy.copy(u), u) + check(copy.deepcopy(u), u) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(protocol=proto): + check(pickle.loads(pickle.dumps(u, proto)), u) + + def test_unpickle_previous_python_versions(self): + def check(actual, expected): + self.assertEqual(actual, expected) + self.assertEqual(actual.is_safe, expected.is_safe) + + pickled_uuids = [ + # Python 2.7, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dS\'int\'\nL287307832597519156748809049798316161701L\nsb.', + # Python 2.7, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}U\x03intL287307832597519156748809049798316161701L\nsb.', + # Python 2.7, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}U\x03int\x8a\x11\xa5z\xecz\nI\xdf}' + b'\xde\xa0Bf\xcey%\xd8\x00sb.', + # Python 3.6, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dVint\nL287307832597519156748809049798316161701L\nsb.', + # Python 3.6, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}X\x03\x00\x00\x00intL287307832597519156748809049798316161701L' + b'\nsb.', + # Python 3.6, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}X\x03\x00\x00\x00int\x8a\x11\xa5z\xec' + b'z\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00sb.', + # Python 3.6, protocol 3 + b'\x80\x03cuuid\nUUID\n)\x81}X\x03\x00\x00\x00int\x8a\x11\xa5z\xec' + b'z\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00sb.', + # Python 3.6, protocol 4 + b'\x80\x04\x95+\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x8c\x04UUI' + b'D\x93)\x81}\x8c\x03int\x8a\x11\xa5z\xecz\nI\xdf}\xde\xa0Bf\xcey%' + b'\xd8\x00sb.', + # Python 3.7, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dVint\nL287307832597519156748809049798316161701L\nsVis_safe\n' + b'cuuid\nSafeUUID\n(NtRsb.', + # Python 3.7, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}(X\x03\x00\x00\x00intL287307832597519156748809049798316161701' + b'L\nX\x07\x00\x00\x00is_safecuuid\nSafeUUID\n(NtRub.', + # Python 3.7, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nN\x85Rub.', + # Python 3.7, protocol 3 + b'\x80\x03cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nN\x85Rub.', + # Python 3.7, protocol 4 + b'\x80\x04\x95F\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x94\x8c' + b'\x04UUID\x93)\x81}(\x8c\x03int\x8a\x11\xa5z\xecz\nI\xdf}\xde\xa0' + b'Bf\xcey%\xd8\x00\x8c\x07is_safeh\x00\x8c\x08SafeUUID\x93N\x85Rub' + b'.', + ] + pickled_uuids_safe = [ + # Python 3.7, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dVint\nL287307832597519156748809049798316161701L\nsVis_safe\n' + b'cuuid\nSafeUUID\n(I0\ntRsb.', + # Python 3.7, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}(X\x03\x00\x00\x00intL287307832597519156748809049798316161701' + b'L\nX\x07\x00\x00\x00is_safecuuid\nSafeUUID\n(K\x00tRub.', + # Python 3.7, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nK\x00\x85Rub.', + # Python 3.7, protocol 3 + b'\x80\x03cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nK\x00\x85Rub.', + # Python 3.7, protocol 4 + b'\x80\x04\x95G\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x94\x8c' + b'\x04UUID\x93)\x81}(\x8c\x03int\x8a\x11\xa5z\xecz\nI\xdf}\xde\xa0' + b'Bf\xcey%\xd8\x00\x8c\x07is_safeh\x00\x8c\x08SafeUUID\x93K\x00' + b'\x85Rub.', + ] + pickled_uuids_unsafe = [ + # Python 3.7, protocol 0 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR(dVint\nL287307832597519156748809049798316161701L\nsVis_safe\n' + b'cuuid\nSafeUUID\n(I-1\ntRsb.', + # Python 3.7, protocol 1 + b'ccopy_reg\n_reconstructor\n(cuuid\nUUID\nc__builtin__\nobject\nN' + b'tR}(X\x03\x00\x00\x00intL287307832597519156748809049798316161701' + b'L\nX\x07\x00\x00\x00is_safecuuid\nSafeUUID\n(J\xff\xff\xff\xfftR' + b'ub.', + # Python 3.7, protocol 2 + b'\x80\x02cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nJ\xff\xff\xff\xff\x85Rub.', + # Python 3.7, protocol 3 + b'\x80\x03cuuid\nUUID\n)\x81}(X\x03\x00\x00\x00int\x8a\x11\xa5z' + b'\xecz\nI\xdf}\xde\xa0Bf\xcey%\xd8\x00X\x07\x00\x00\x00is_safecuu' + b'id\nSafeUUID\nJ\xff\xff\xff\xff\x85Rub.', + # Python 3.7, protocol 4 + b'\x80\x04\x95J\x00\x00\x00\x00\x00\x00\x00\x8c\x04uuid\x94\x8c' + b'\x04UUID\x93)\x81}(\x8c\x03int\x8a\x11\xa5z\xecz\nI\xdf}\xde\xa0' + b'Bf\xcey%\xd8\x00\x8c\x07is_safeh\x00\x8c\x08SafeUUID\x93J\xff' + b'\xff\xff\xff\x85Rub.', + ] + + u = self.uuid.UUID('d82579ce6642a0de7ddf490a7aec7aa5') + u_safe = self.uuid.UUID('d82579ce6642a0de7ddf490a7aec7aa5', + is_safe=self.uuid.SafeUUID.safe) + u_unsafe = self.uuid.UUID('d82579ce6642a0de7ddf490a7aec7aa5', + is_safe=self.uuid.SafeUUID.unsafe) + + with support.swap_item(sys.modules, 'uuid', self.uuid): + for pickled in pickled_uuids: + # is_safe was added in 3.7. When unpickling values from older + # versions, is_safe will be missing, so it should be set to + # SafeUUID.unknown. + check(pickle.loads(pickled), u) + for pickled in pickled_uuids_safe: + check(pickle.loads(pickled), u_safe) + for pickled in pickled_uuids_unsafe: + check(pickle.loads(pickled), u_unsafe) + # bpo-32502: UUID1 requires a 48-bit identifier, but hardware identifiers # need not necessarily be 48 bits (e.g., EUI-64). def test_uuid1_eui64(self): diff --git a/Lib/uuid.py b/Lib/uuid.py index 66383218e70c..26faa1accd09 100644 --- a/Lib/uuid.py +++ b/Lib/uuid.py @@ -204,6 +204,23 @@ def __init__(self, hex=None, bytes=None, bytes_le=None, fields=None, self.__dict__['int'] = int self.__dict__['is_safe'] = is_safe + def __getstate__(self): + state = self.__dict__ + if self.is_safe != SafeUUID.unknown: + # is_safe is a SafeUUID instance. Return just its value, so that + # it can be un-pickled in older Python versions without SafeUUID. + state = state.copy() + state['is_safe'] = self.is_safe.value + return state + + def __setstate__(self, state): + self.__dict__.update(state) + # is_safe was added in 3.7; it is also omitted when it is "unknown" + self.__dict__['is_safe'] = ( + SafeUUID(state['is_safe']) + if 'is_safe' in state else SafeUUID.unknown + ) + def __eq__(self, other): if isinstance(other, UUID): return self.int == other.int diff --git a/Misc/NEWS.d/next/Library/2018-09-10-15-54-58.bpo-34621.Uqj5x3.rst b/Misc/NEWS.d/next/Library/2018-09-10-15-54-58.bpo-34621.Uqj5x3.rst new file mode 100644 index 000000000000..1fd60608aea3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-15-54-58.bpo-34621.Uqj5x3.rst @@ -0,0 +1,2 @@ +Fix un/pickling compatbility of uuid.UUID objects with older versions of +Python (<3.7). From webhook-mailer at python.org Mon Sep 10 12:00:13 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 16:00:13 -0000 Subject: [Python-checkins] bpo-34588: Fix an off-by-one error in traceback formatting. (GH-9077) Message-ID: https://github.com/python/cpython/commit/49020174305ca3dc90a811b03a05f44873297c61 commit: 49020174305ca3dc90a811b03a05f44873297c61 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T09:00:08-07:00 summary: bpo-34588: Fix an off-by-one error in traceback formatting. (GH-9077) The recursive frame pruning code always undercounted the number of elided frames by one. That is, in the "[Previous line repeated N more times]" message, N would always be one too few. Near the recursive pruning cutoff, one frame could be silently dropped. That situation is demonstrated in the OP of the bug report. The fix is to start the identical frame counter at 1. (cherry picked from commit d545869d084e70d4838310e79b52a25a72a1ca56) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst M Lib/test/test_traceback.py M Lib/traceback.py M Python/traceback.c diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index bffc03e663ff..8a3aa8a8648f 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -373,7 +373,7 @@ def g(count=10): ' return g(count-1)\n' f' File "{__file__}", line {lineno_g+2}, in g\n' ' return g(count-1)\n' - ' [Previous line repeated 6 more times]\n' + ' [Previous line repeated 7 more times]\n' f' File "{__file__}", line {lineno_g+3}, in g\n' ' raise ValueError\n' 'ValueError\n' @@ -412,7 +412,7 @@ def h(count=10): ' return h(count-1)\n' f' File "{__file__}", line {lineno_h+2}, in h\n' ' return h(count-1)\n' - ' [Previous line repeated 6 more times]\n' + ' [Previous line repeated 7 more times]\n' f' File "{__file__}", line {lineno_h+3}, in h\n' ' g()\n' ) @@ -420,6 +420,63 @@ def h(count=10): actual = stderr_h.getvalue().splitlines() self.assertEqual(actual, expected) + # Check the boundary conditions. First, test just below the cutoff. + with captured_output("stderr") as stderr_g: + try: + g(traceback._RECURSIVE_CUTOFF) + except ValueError as exc: + render_exc() + else: + self.fail("no error raised") + result_g = ( + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+3}, in g\n' + ' raise ValueError\n' + 'ValueError\n' + ) + tb_line = ( + 'Traceback (most recent call last):\n' + f' File "{__file__}", line {lineno_g+71}, in _check_recursive_traceback_display\n' + ' g(traceback._RECURSIVE_CUTOFF)\n' + ) + expected = (tb_line + result_g).splitlines() + actual = stderr_g.getvalue().splitlines() + self.assertEqual(actual, expected) + + # Second, test just above the cutoff. + with captured_output("stderr") as stderr_g: + try: + g(traceback._RECURSIVE_CUTOFF + 1) + except ValueError as exc: + render_exc() + else: + self.fail("no error raised") + result_g = ( + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + ' [Previous line repeated 1 more time]\n' + f' File "{__file__}", line {lineno_g+3}, in g\n' + ' raise ValueError\n' + 'ValueError\n' + ) + tb_line = ( + 'Traceback (most recent call last):\n' + f' File "{__file__}", line {lineno_g+99}, in _check_recursive_traceback_display\n' + ' g(traceback._RECURSIVE_CUTOFF + 1)\n' + ) + expected = (tb_line + result_g).splitlines() + actual = stderr_g.getvalue().splitlines() + self.assertEqual(actual, expected) + def test_recursive_traceback_python(self): self._check_recursive_traceback_display(traceback.print_exc) diff --git a/Lib/traceback.py b/Lib/traceback.py index afab0a4b91f1..4e7605d15fa4 100644 --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -310,6 +310,8 @@ def walk_tb(tb): tb = tb.tb_next +_RECURSIVE_CUTOFF = 3 # Also hardcoded in traceback.c. + class StackSummary(list): """A stack of frames.""" @@ -398,18 +400,21 @@ def format(self): last_name = None count = 0 for frame in self: - if (last_file is not None and last_file == frame.filename and - last_line is not None and last_line == frame.lineno and - last_name is not None and last_name == frame.name): - count += 1 - else: - if count > 3: - result.append(f' [Previous line repeated {count-3} more times]\n') + if (last_file is None or last_file != frame.filename or + last_line is None or last_line != frame.lineno or + last_name is None or last_name != frame.name): + if count > _RECURSIVE_CUTOFF: + count -= _RECURSIVE_CUTOFF + result.append( + f' [Previous line repeated {count} more ' + f'time{"s" if count > 1 else ""}]\n' + ) last_file = frame.filename last_line = frame.lineno last_name = frame.name count = 0 - if count >= 3: + count += 1 + if count > _RECURSIVE_CUTOFF: continue row = [] row.append(' File "{}", line {}, in {}\n'.format( @@ -420,8 +425,12 @@ def format(self): for name, value in sorted(frame.locals.items()): row.append(' {name} = {value}\n'.format(name=name, value=value)) result.append(''.join(row)) - if count > 3: - result.append(f' [Previous line repeated {count-3} more times]\n') + if count > _RECURSIVE_CUTOFF: + count -= _RECURSIVE_CUTOFF + result.append( + f' [Previous line repeated {count} more ' + f'time{"s" if count > 1 else ""}]\n' + ) return result diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst new file mode 100644 index 000000000000..ec7a57f23ad2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst @@ -0,0 +1,2 @@ +Fix an off-by-one in the recursive call pruning feature of traceback +formatting. diff --git a/Python/traceback.c b/Python/traceback.c index b00864b06e43..95bef64e734b 100644 --- a/Python/traceback.c +++ b/Python/traceback.c @@ -511,16 +511,21 @@ tb_displayline(PyObject *f, PyObject *filename, int lineno, PyObject *name) return err; } +static const int TB_RECURSIVE_CUTOFF = 3; // Also hardcoded in traceback.py. + static int tb_print_line_repeated(PyObject *f, long cnt) { - int err; + cnt -= TB_RECURSIVE_CUTOFF; PyObject *line = PyUnicode_FromFormat( - " [Previous line repeated %ld more times]\n", cnt-3); + (cnt > 1) + ? " [Previous line repeated %ld more times]\n" + : " [Previous line repeated %ld more time]\n", + cnt); if (line == NULL) { return -1; } - err = PyFile_WriteObject(line, f, Py_PRINT_RAW); + int err = PyFile_WriteObject(line, f, Py_PRINT_RAW); Py_DECREF(line); return err; } @@ -544,15 +549,11 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit) tb = tb->tb_next; } while (tb != NULL && err == 0) { - if (last_file != NULL && - tb->tb_frame->f_code->co_filename == last_file && - last_line != -1 && tb->tb_lineno == last_line && - last_name != NULL && tb->tb_frame->f_code->co_name == last_name) - { - cnt++; - } - else { - if (cnt > 3) { + if (last_file == NULL || + tb->tb_frame->f_code->co_filename != last_file || + last_line == -1 || tb->tb_lineno != last_line || + last_name == NULL || tb->tb_frame->f_code->co_name != last_name) { + if (cnt > TB_RECURSIVE_CUTOFF) { err = tb_print_line_repeated(f, cnt); } last_file = tb->tb_frame->f_code->co_filename; @@ -560,7 +561,8 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit) last_name = tb->tb_frame->f_code->co_name; cnt = 0; } - if (err == 0 && cnt < 3) { + cnt++; + if (err == 0 && cnt <= TB_RECURSIVE_CUTOFF) { err = tb_displayline(f, tb->tb_frame->f_code->co_filename, tb->tb_lineno, @@ -571,7 +573,7 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit) } tb = tb->tb_next; } - if (err == 0 && cnt > 3) { + if (err == 0 && cnt > TB_RECURSIVE_CUTOFF) { err = tb_print_line_repeated(f, cnt); } return err; From webhook-mailer at python.org Mon Sep 10 12:10:25 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 16:10:25 -0000 Subject: [Python-checkins] bpo-34588: Fix an off-by-one error in traceback formatting. (GH-9077) Message-ID: https://github.com/python/cpython/commit/afb25bc2b5767ac3a83bc8c4d2826e8fdcb6b0e7 commit: afb25bc2b5767ac3a83bc8c4d2826e8fdcb6b0e7 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T09:10:21-07:00 summary: bpo-34588: Fix an off-by-one error in traceback formatting. (GH-9077) The recursive frame pruning code always undercounted the number of elided frames by one. That is, in the "[Previous line repeated N more times]" message, N would always be one too few. Near the recursive pruning cutoff, one frame could be silently dropped. That situation is demonstrated in the OP of the bug report. The fix is to start the identical frame counter at 1. (cherry picked from commit d545869d084e70d4838310e79b52a25a72a1ca56) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst M Lib/test/test_traceback.py M Lib/traceback.py M Python/traceback.c diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index bffc03e663ff..8a3aa8a8648f 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -373,7 +373,7 @@ def g(count=10): ' return g(count-1)\n' f' File "{__file__}", line {lineno_g+2}, in g\n' ' return g(count-1)\n' - ' [Previous line repeated 6 more times]\n' + ' [Previous line repeated 7 more times]\n' f' File "{__file__}", line {lineno_g+3}, in g\n' ' raise ValueError\n' 'ValueError\n' @@ -412,7 +412,7 @@ def h(count=10): ' return h(count-1)\n' f' File "{__file__}", line {lineno_h+2}, in h\n' ' return h(count-1)\n' - ' [Previous line repeated 6 more times]\n' + ' [Previous line repeated 7 more times]\n' f' File "{__file__}", line {lineno_h+3}, in h\n' ' g()\n' ) @@ -420,6 +420,63 @@ def h(count=10): actual = stderr_h.getvalue().splitlines() self.assertEqual(actual, expected) + # Check the boundary conditions. First, test just below the cutoff. + with captured_output("stderr") as stderr_g: + try: + g(traceback._RECURSIVE_CUTOFF) + except ValueError as exc: + render_exc() + else: + self.fail("no error raised") + result_g = ( + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+3}, in g\n' + ' raise ValueError\n' + 'ValueError\n' + ) + tb_line = ( + 'Traceback (most recent call last):\n' + f' File "{__file__}", line {lineno_g+71}, in _check_recursive_traceback_display\n' + ' g(traceback._RECURSIVE_CUTOFF)\n' + ) + expected = (tb_line + result_g).splitlines() + actual = stderr_g.getvalue().splitlines() + self.assertEqual(actual, expected) + + # Second, test just above the cutoff. + with captured_output("stderr") as stderr_g: + try: + g(traceback._RECURSIVE_CUTOFF + 1) + except ValueError as exc: + render_exc() + else: + self.fail("no error raised") + result_g = ( + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + ' [Previous line repeated 1 more time]\n' + f' File "{__file__}", line {lineno_g+3}, in g\n' + ' raise ValueError\n' + 'ValueError\n' + ) + tb_line = ( + 'Traceback (most recent call last):\n' + f' File "{__file__}", line {lineno_g+99}, in _check_recursive_traceback_display\n' + ' g(traceback._RECURSIVE_CUTOFF + 1)\n' + ) + expected = (tb_line + result_g).splitlines() + actual = stderr_g.getvalue().splitlines() + self.assertEqual(actual, expected) + def test_recursive_traceback_python(self): self._check_recursive_traceback_display(traceback.print_exc) diff --git a/Lib/traceback.py b/Lib/traceback.py index d2b102b73c40..1e5190954b95 100644 --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -311,6 +311,8 @@ def walk_tb(tb): tb = tb.tb_next +_RECURSIVE_CUTOFF = 3 # Also hardcoded in traceback.c. + class StackSummary(list): """A stack of frames.""" @@ -399,18 +401,21 @@ def format(self): last_name = None count = 0 for frame in self: - if (last_file is not None and last_file == frame.filename and - last_line is not None and last_line == frame.lineno and - last_name is not None and last_name == frame.name): - count += 1 - else: - if count > 3: - result.append(f' [Previous line repeated {count-3} more times]\n') + if (last_file is None or last_file != frame.filename or + last_line is None or last_line != frame.lineno or + last_name is None or last_name != frame.name): + if count > _RECURSIVE_CUTOFF: + count -= _RECURSIVE_CUTOFF + result.append( + f' [Previous line repeated {count} more ' + f'time{"s" if count > 1 else ""}]\n' + ) last_file = frame.filename last_line = frame.lineno last_name = frame.name count = 0 - if count >= 3: + count += 1 + if count > _RECURSIVE_CUTOFF: continue row = [] row.append(' File "{}", line {}, in {}\n'.format( @@ -421,8 +426,12 @@ def format(self): for name, value in sorted(frame.locals.items()): row.append(' {name} = {value}\n'.format(name=name, value=value)) result.append(''.join(row)) - if count > 3: - result.append(f' [Previous line repeated {count-3} more times]\n') + if count > _RECURSIVE_CUTOFF: + count -= _RECURSIVE_CUTOFF + result.append( + f' [Previous line repeated {count} more ' + f'time{"s" if count > 1 else ""}]\n' + ) return result diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst new file mode 100644 index 000000000000..ec7a57f23ad2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-05-22-56-52.bpo-34588.UIuPmL.rst @@ -0,0 +1,2 @@ +Fix an off-by-one in the recursive call pruning feature of traceback +formatting. diff --git a/Python/traceback.c b/Python/traceback.c index d9620675f747..145d028ba353 100644 --- a/Python/traceback.c +++ b/Python/traceback.c @@ -413,16 +413,21 @@ tb_displayline(PyObject *f, PyObject *filename, int lineno, PyObject *name) return err; } +static const int TB_RECURSIVE_CUTOFF = 3; // Also hardcoded in traceback.py. + static int tb_print_line_repeated(PyObject *f, long cnt) { - int err; + cnt -= TB_RECURSIVE_CUTOFF; PyObject *line = PyUnicode_FromFormat( - " [Previous line repeated %ld more times]\n", cnt-3); + (cnt > 1) + ? " [Previous line repeated %ld more times]\n" + : " [Previous line repeated %ld more time]\n", + cnt); if (line == NULL) { return -1; } - err = PyFile_WriteObject(line, f, Py_PRINT_RAW); + int err = PyFile_WriteObject(line, f, Py_PRINT_RAW); Py_DECREF(line); return err; } @@ -446,15 +451,11 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit) tb = tb->tb_next; } while (tb != NULL && err == 0) { - if (last_file != NULL && - tb->tb_frame->f_code->co_filename == last_file && - last_line != -1 && tb->tb_lineno == last_line && - last_name != NULL && tb->tb_frame->f_code->co_name == last_name) - { - cnt++; - } - else { - if (cnt > 3) { + if (last_file == NULL || + tb->tb_frame->f_code->co_filename != last_file || + last_line == -1 || tb->tb_lineno != last_line || + last_name == NULL || tb->tb_frame->f_code->co_name != last_name) { + if (cnt > TB_RECURSIVE_CUTOFF) { err = tb_print_line_repeated(f, cnt); } last_file = tb->tb_frame->f_code->co_filename; @@ -462,7 +463,8 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit) last_name = tb->tb_frame->f_code->co_name; cnt = 0; } - if (err == 0 && cnt < 3) { + cnt++; + if (err == 0 && cnt <= TB_RECURSIVE_CUTOFF) { err = tb_displayline(f, tb->tb_frame->f_code->co_filename, tb->tb_lineno, @@ -473,7 +475,7 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit) } tb = tb->tb_next; } - if (err == 0 && cnt > 3) { + if (err == 0 && cnt > TB_RECURSIVE_CUTOFF) { err = tb_print_line_repeated(f, cnt); } return err; From webhook-mailer at python.org Mon Sep 10 12:39:53 2018 From: webhook-mailer at python.org (Ethan Furman) Date: Mon, 10 Sep 2018 16:39:53 -0000 Subject: [Python-checkins] [3.7] bpo-34282: Fix Enum._convert method shadowing members named _convert (GH-9034) Message-ID: https://github.com/python/cpython/commit/c0d63bf73b35df374e6e66c08b0e297fb828d744 commit: c0d63bf73b35df374e6e66c08b0e297fb828d744 branch: 3.7 author: orlnub123 committer: Ethan Furman date: 2018-09-10T09:39:48-07:00 summary: [3.7] bpo-34282: Fix Enum._convert method shadowing members named _convert (GH-9034) * Fix Enum._convert shadowing members named _convert files: A Misc/NEWS.d/next/Library/2018-09-02-13-33-35.bpo-34282.ztyXH8.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 576de031805d..69b41fe7cbc0 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -171,9 +171,11 @@ def __new__(metacls, cls, bases, classdict): enum_class._member_map_ = OrderedDict() # name->value map enum_class._member_type_ = member_type - # save attributes from super classes so we know if we can take - # the shortcut of storing members in the class dict - base_attributes = {a for b in enum_class.mro() for a in b.__dict__} + # save DynamicClassAttribute attributes from super classes so we know + # if we can take the shortcut of storing members in the class dict + dynamic_attributes = {k for c in enum_class.mro() + for k, v in c.__dict__.items() + if isinstance(v, DynamicClassAttribute)} # Reverse value->name map for hashable values. enum_class._value2member_map_ = {} @@ -233,7 +235,7 @@ def __new__(metacls, cls, bases, classdict): enum_class._member_names_.append(member_name) # performance boost for any member that would not shadow # a DynamicClassAttribute - if member_name not in base_attributes: + if member_name not in dynamic_attributes: setattr(enum_class, member_name, enum_member) # now add to _member_map_ enum_class._member_map_[member_name] = enum_member diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index ef2d1daaf942..4b1722894601 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1516,6 +1516,23 @@ class MoreColor(Color): yellow = 6 self.assertEqual(MoreColor.magenta.hex(), '5 hexlified!') + def test_subclass_duplicate_name(self): + class Base(Enum): + def test(self): + pass + class Test(Base): + test = 1 + self.assertIs(type(Test.test), Test) + + def test_subclass_duplicate_name_dynamic(self): + from types import DynamicClassAttribute + class Base(Enum): + @DynamicClassAttribute + def test(self): + return 'dynamic' + class Test(Base): + test = 1 + self.assertEqual(Test.test.test, 'dynamic') def test_no_duplicates(self): class UniqueEnum(Enum): diff --git a/Misc/NEWS.d/next/Library/2018-09-02-13-33-35.bpo-34282.ztyXH8.rst b/Misc/NEWS.d/next/Library/2018-09-02-13-33-35.bpo-34282.ztyXH8.rst new file mode 100644 index 000000000000..c1e606ab0ce5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-02-13-33-35.bpo-34282.ztyXH8.rst @@ -0,0 +1 @@ +Fix enum members getting shadowed by parent attributes. From webhook-mailer at python.org Mon Sep 10 12:41:16 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 16:41:16 -0000 Subject: [Python-checkins] bpo-23855: Add missing NULL checks for malloc() in _msi.c (GH-9038) Message-ID: https://github.com/python/cpython/commit/f51a46631f8dcca596c08a934a766da9afe93c06 commit: f51a46631f8dcca596c08a934a766da9afe93c06 branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T09:41:12-07:00 summary: bpo-23855: Add missing NULL checks for malloc() in _msi.c (GH-9038) (cherry picked from commit 4e519377b1b84c9414a360961276993d24198825) Co-authored-by: Zackery Spytz files: M PC/_msi.c diff --git a/PC/_msi.c b/PC/_msi.c index d7700f09c749..68c4e79e2945 100644 --- a/PC/_msi.c +++ b/PC/_msi.c @@ -321,6 +321,10 @@ msierror(int status) code = MsiRecordGetInteger(err, 1); /* XXX code */ if (MsiFormatRecord(0, err, res, &size) == ERROR_MORE_DATA) { res = malloc(size+1); + if (res == NULL) { + MsiCloseHandle(err); + return PyErr_NoMemory(); + } MsiFormatRecord(0, err, res, &size); res[size]='\0'; } @@ -544,6 +548,9 @@ summary_getproperty(msiobj* si, PyObject *args) &fval, sval, &ssize); if (status == ERROR_MORE_DATA) { sval = malloc(ssize); + if (sval == NULL) { + return PyErr_NoMemory(); + } status = MsiSummaryInfoGetProperty(si->h, field, &type, &ival, &fval, sval, &ssize); } From webhook-mailer at python.org Mon Sep 10 12:41:36 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 16:41:36 -0000 Subject: [Python-checkins] bpo-23855: Add missing NULL checks for malloc() in _msi.c (GH-9038) Message-ID: https://github.com/python/cpython/commit/8a0c254fdd68cfafede168356fc5c5c3e372bc3f commit: 8a0c254fdd68cfafede168356fc5c5c3e372bc3f branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T09:41:31-07:00 summary: bpo-23855: Add missing NULL checks for malloc() in _msi.c (GH-9038) (cherry picked from commit 4e519377b1b84c9414a360961276993d24198825) Co-authored-by: Zackery Spytz files: M PC/_msi.c diff --git a/PC/_msi.c b/PC/_msi.c index a7c8fa3d092a..0487743b59aa 100644 --- a/PC/_msi.c +++ b/PC/_msi.c @@ -340,6 +340,10 @@ msierror(int status) code = MsiRecordGetInteger(err, 1); /* XXX code */ if (MsiFormatRecord(0, err, res, &size) == ERROR_MORE_DATA) { res = malloc(size+1); + if (res == NULL) { + MsiCloseHandle(err); + return PyErr_NoMemory(); + } MsiFormatRecord(0, err, res, &size); res[size]='\0'; } @@ -563,6 +567,9 @@ summary_getproperty(msiobj* si, PyObject *args) &fval, sval, &ssize); if (status == ERROR_MORE_DATA) { sval = malloc(ssize); + if (sval == NULL) { + return PyErr_NoMemory(); + } status = MsiSummaryInfoGetProperty(si->h, field, &type, &ival, &fval, sval, &ssize); } From webhook-mailer at python.org Mon Sep 10 12:46:12 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Mon, 10 Sep 2018 16:46:12 -0000 Subject: [Python-checkins] Fix misleading mentions of tp_size in comments (GH-9093) Message-ID: https://github.com/python/cpython/commit/0e0bc4e221f592f305d335faf5f8046484eb9238 commit: 0e0bc4e221f592f305d335faf5f8046484eb9238 branch: master author: Peter Eisentraut committer: Benjamin Peterson date: 2018-09-10T09:46:08-07:00 summary: Fix misleading mentions of tp_size in comments (GH-9093) Many type object initializations labeled a field "tp_size" in the comment, but the name of that field is tp_basicsize. files: M Modules/_abc.c M Modules/_blake2/blake2b_impl.c M Modules/_blake2/blake2s_impl.c M Modules/_queuemodule.c M Modules/_sha3/sha3module.c M Modules/_threadmodule.c M Modules/_xxsubinterpretersmodule.c M Modules/md5module.c M Modules/mmapmodule.c M Modules/ossaudiodev.c M Modules/sha1module.c M Modules/sha256module.c M Modules/sha512module.c M Objects/moduleobject.c M Objects/namespaceobject.c diff --git a/Modules/_abc.c b/Modules/_abc.c index ce9140fd03cc..9de199fa143f 100644 --- a/Modules/_abc.c +++ b/Modules/_abc.c @@ -68,7 +68,7 @@ PyDoc_STRVAR(abc_data_doc, static PyTypeObject _abc_data_type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "_abc_data", /*tp_name*/ - sizeof(_abc_data), /*tp_size*/ + sizeof(_abc_data), /*tp_basicsize*/ .tp_dealloc = (destructor)abc_data_dealloc, .tp_flags = Py_TPFLAGS_DEFAULT, .tp_alloc = PyType_GenericAlloc, diff --git a/Modules/_blake2/blake2b_impl.c b/Modules/_blake2/blake2b_impl.c index 7ad4b650b325..788c15c31d8c 100644 --- a/Modules/_blake2/blake2b_impl.c +++ b/Modules/_blake2/blake2b_impl.c @@ -398,7 +398,7 @@ py_blake2b_dealloc(PyObject *self) PyTypeObject PyBlake2_BLAKE2bType = { PyVarObject_HEAD_INIT(NULL, 0) "_blake2.blake2b", /* tp_name */ - sizeof(BLAKE2bObject), /* tp_size */ + sizeof(BLAKE2bObject), /* tp_basicsize */ 0, /* tp_itemsize */ py_blake2b_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Modules/_blake2/blake2s_impl.c b/Modules/_blake2/blake2s_impl.c index 8864fd829787..c8bcedeabd57 100644 --- a/Modules/_blake2/blake2s_impl.c +++ b/Modules/_blake2/blake2s_impl.c @@ -398,7 +398,7 @@ py_blake2s_dealloc(PyObject *self) PyTypeObject PyBlake2_BLAKE2sType = { PyVarObject_HEAD_INIT(NULL, 0) "_blake2.blake2s", /* tp_name */ - sizeof(BLAKE2sObject), /* tp_size */ + sizeof(BLAKE2sObject), /* tp_basicsize */ 0, /* tp_itemsize */ py_blake2s_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Modules/_queuemodule.c b/Modules/_queuemodule.c index 0eb993027459..2a9406cc4760 100644 --- a/Modules/_queuemodule.c +++ b/Modules/_queuemodule.c @@ -309,7 +309,7 @@ static PyMethodDef simplequeue_methods[] = { static PyTypeObject PySimpleQueueType = { PyVarObject_HEAD_INIT(NULL, 0) "_queue.SimpleQueue", /*tp_name*/ - sizeof(simplequeueobject), /*tp_size*/ + sizeof(simplequeueobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)simplequeue_dealloc, /*tp_dealloc*/ diff --git a/Modules/_sha3/sha3module.c b/Modules/_sha3/sha3module.c index f5032fcb72dd..46c1ff153852 100644 --- a/Modules/_sha3/sha3module.c +++ b/Modules/_sha3/sha3module.c @@ -489,7 +489,7 @@ static PyGetSetDef SHA3_getseters[] = { static PyTypeObject type_obj = { \ PyVarObject_HEAD_INIT(NULL, 0) \ type_name, /* tp_name */ \ - sizeof(SHA3object), /* tp_size */ \ + sizeof(SHA3object), /* tp_basicsize */ \ 0, /* tp_itemsize */ \ /* methods */ \ (destructor)SHA3_dealloc, /* tp_dealloc */ \ diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index f6b39defbc80..d33fa9924308 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -226,7 +226,7 @@ static PyMethodDef lock_methods[] = { static PyTypeObject Locktype = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "_thread.lock", /*tp_name*/ - sizeof(lockobject), /*tp_size*/ + sizeof(lockobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)lock_dealloc, /*tp_dealloc*/ @@ -487,7 +487,7 @@ static PyMethodDef rlock_methods[] = { static PyTypeObject RLocktype = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "_thread.RLock", /*tp_name*/ - sizeof(rlockobject), /*tp_size*/ + sizeof(rlockobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)rlock_dealloc, /*tp_dealloc*/ diff --git a/Modules/_xxsubinterpretersmodule.c b/Modules/_xxsubinterpretersmodule.c index 7b2cda218309..2eb87871694d 100644 --- a/Modules/_xxsubinterpretersmodule.c +++ b/Modules/_xxsubinterpretersmodule.c @@ -1764,7 +1764,7 @@ PyDoc_STRVAR(channelid_doc, static PyTypeObject ChannelIDtype = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "_xxsubinterpreters.ChannelID", /* tp_name */ - sizeof(channelid), /* tp_size */ + sizeof(channelid), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)channelid_dealloc, /* tp_dealloc */ 0, /* tp_print */ @@ -2173,7 +2173,7 @@ PyDoc_STRVAR(interpid_doc, static PyTypeObject InterpreterIDtype = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "interpreters.InterpreterID", /* tp_name */ - sizeof(interpid), /* tp_size */ + sizeof(interpid), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)interpid_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Modules/md5module.c b/Modules/md5module.c index b019f8287684..c66b27308f05 100644 --- a/Modules/md5module.c +++ b/Modules/md5module.c @@ -465,7 +465,7 @@ static PyGetSetDef MD5_getseters[] = { static PyTypeObject MD5type = { PyVarObject_HEAD_INIT(NULL, 0) "_md5.md5", /*tp_name*/ - sizeof(MD5object), /*tp_size*/ + sizeof(MD5object), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ MD5_dealloc, /*tp_dealloc*/ diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index af9cd7e2be8c..86632358b7fc 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -983,7 +983,7 @@ To map anonymous memory, pass -1 as the fileno (both versions)."); static PyTypeObject mmap_object_type = { PyVarObject_HEAD_INIT(NULL, 0) "mmap.mmap", /* tp_name */ - sizeof(mmap_object), /* tp_size */ + sizeof(mmap_object), /* tp_basicsize */ 0, /* tp_itemsize */ /* methods */ (destructor) mmap_object_dealloc, /* tp_dealloc */ diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c index 58ee71f90108..150a14eb388a 100644 --- a/Modules/ossaudiodev.c +++ b/Modules/ossaudiodev.c @@ -964,7 +964,7 @@ oss_getattro(oss_audio_t *self, PyObject *nameobj) static PyTypeObject OSSAudioType = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "ossaudiodev.oss_audio_device", /*tp_name*/ - sizeof(oss_audio_t), /*tp_size*/ + sizeof(oss_audio_t), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)oss_dealloc, /*tp_dealloc*/ @@ -996,7 +996,7 @@ static PyTypeObject OSSAudioType = { static PyTypeObject OSSMixerType = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "ossaudiodev.oss_mixer_device", /*tp_name*/ - sizeof(oss_mixer_t), /*tp_size*/ + sizeof(oss_mixer_t), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)oss_mixer_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha1module.c b/Modules/sha1module.c index d39190b4d5e6..358cd1e7f9ac 100644 --- a/Modules/sha1module.c +++ b/Modules/sha1module.c @@ -442,7 +442,7 @@ static PyGetSetDef SHA1_getseters[] = { static PyTypeObject SHA1type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha1.sha1", /*tp_name*/ - sizeof(SHA1object), /*tp_size*/ + sizeof(SHA1object), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA1_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha256module.c b/Modules/sha256module.c index e4cb3286ce95..3599eaf8be92 100644 --- a/Modules/sha256module.c +++ b/Modules/sha256module.c @@ -529,7 +529,7 @@ static PyMemberDef SHA_members[] = { static PyTypeObject SHA224type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha256.sha224", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA_dealloc, /*tp_dealloc*/ @@ -563,7 +563,7 @@ static PyTypeObject SHA224type = { static PyTypeObject SHA256type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha256.sha256", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha512module.c b/Modules/sha512module.c index 5ac2a2a61cf5..d8c846a450f2 100644 --- a/Modules/sha512module.c +++ b/Modules/sha512module.c @@ -594,7 +594,7 @@ static PyMemberDef SHA_members[] = { static PyTypeObject SHA384type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha512.sha384", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA512_dealloc, /*tp_dealloc*/ @@ -628,7 +628,7 @@ static PyTypeObject SHA384type = { static PyTypeObject SHA512type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha512.sha512", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA512_dealloc, /*tp_dealloc*/ diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index 2156ca0765a0..ccf5f8e6d15a 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -35,7 +35,7 @@ bad_traverse_test(PyObject *self, void *arg) { PyTypeObject PyModuleDef_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "moduledef", /* tp_name */ - sizeof(struct PyModuleDef), /* tp_size */ + sizeof(struct PyModuleDef), /* tp_basicsize */ 0, /* tp_itemsize */ }; @@ -790,7 +790,7 @@ static PyMethodDef module_methods[] = { PyTypeObject PyModule_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "module", /* tp_name */ - sizeof(PyModuleObject), /* tp_size */ + sizeof(PyModuleObject), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)module_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Objects/namespaceobject.c b/Objects/namespaceobject.c index a6c941afc0ad..0b902693849d 100644 --- a/Objects/namespaceobject.c +++ b/Objects/namespaceobject.c @@ -201,7 +201,7 @@ SimpleNamespace(**kwargs)"); PyTypeObject _PyNamespace_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "types.SimpleNamespace", /* tp_name */ - sizeof(_PyNamespaceObject), /* tp_size */ + sizeof(_PyNamespaceObject), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)namespace_dealloc, /* tp_dealloc */ 0, /* tp_print */ From webhook-mailer at python.org Mon Sep 10 13:02:36 2018 From: webhook-mailer at python.org (Petr Viktorin) Date: Mon, 10 Sep 2018 17:02:36 -0000 Subject: [Python-checkins] bpo-26502: Implement FrameSummary.__len__() (GH-8632) Message-ID: https://github.com/python/cpython/commit/9797b7ae4496627836c55333765e10201a9840e3 commit: 9797b7ae4496627836c55333765e10201a9840e3 branch: master author: Berker Peksag committer: Petr Viktorin date: 2018-09-10T10:02:33-07:00 summary: bpo-26502: Implement FrameSummary.__len__() (GH-8632) files: A Misc/NEWS.d/next/Library/2018-08-02-20-39-32.bpo-26502.eGXr_k.rst M Lib/test/test_traceback.py M Lib/traceback.py diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index 8a3aa8a8648f..3af85b81721e 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -868,6 +868,7 @@ def extract(): (__file__, lineno+2, 'test_extract_stack', 'result = extract()'), (__file__, lineno+1, 'extract', 'return traceback.extract_stack()'), ]) + self.assertEqual(len(result[0]), 4) class TestFrame(unittest.TestCase): @@ -900,6 +901,10 @@ def test_explicit_line(self): f = traceback.FrameSummary("f", 1, "dummy", line="line") self.assertEqual("line", f.line) + def test_len(self): + f = traceback.FrameSummary("f", 1, "dummy", line="line") + self.assertEqual(len(f), 4) + class TestStack(unittest.TestCase): diff --git a/Lib/traceback.py b/Lib/traceback.py index 4e7605d15fa4..ab35da94b51e 100644 --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -279,6 +279,9 @@ def __repr__(self): return "".format( filename=self.filename, lineno=self.lineno, name=self.name) + def __len__(self): + return 4 + @property def line(self): if self._line is None: diff --git a/Misc/NEWS.d/next/Library/2018-08-02-20-39-32.bpo-26502.eGXr_k.rst b/Misc/NEWS.d/next/Library/2018-08-02-20-39-32.bpo-26502.eGXr_k.rst new file mode 100644 index 000000000000..3a3fc17e22df --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-08-02-20-39-32.bpo-26502.eGXr_k.rst @@ -0,0 +1,2 @@ +Implement ``traceback.FrameSummary.__len__()`` method to preserve +compatibility with the old tuple API. From webhook-mailer at python.org Mon Sep 10 13:23:01 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Mon, 10 Sep 2018 17:23:01 -0000 Subject: [Python-checkins] [3.7] Fix misleading mentions of tp_size in comments. (GH-9136) Message-ID: https://github.com/python/cpython/commit/4d5d219a7ae1351e6f6e7c1c9845a499927d1219 commit: 4d5d219a7ae1351e6f6e7c1c9845a499927d1219 branch: 3.7 author: Benjamin Peterson committer: GitHub date: 2018-09-10T10:22:55-07:00 summary: [3.7] Fix misleading mentions of tp_size in comments. (GH-9136) Many type object initializations labeled a field "tp_size" in the comment, but the name of that field is tp_basicsize.. (cherry picked from commit 0e0bc4e221f592f305d335faf5f8046484eb9238) Co-authored-by: Peter Eisentraut files: M Modules/_abc.c M Modules/_blake2/blake2b_impl.c M Modules/_blake2/blake2s_impl.c M Modules/_queuemodule.c M Modules/_sha3/sha3module.c M Modules/_threadmodule.c M Modules/md5module.c M Modules/mmapmodule.c M Modules/ossaudiodev.c M Modules/sha1module.c M Modules/sha256module.c M Modules/sha512module.c M Objects/moduleobject.c M Objects/namespaceobject.c diff --git a/Modules/_abc.c b/Modules/_abc.c index ce9140fd03cc..9de199fa143f 100644 --- a/Modules/_abc.c +++ b/Modules/_abc.c @@ -68,7 +68,7 @@ PyDoc_STRVAR(abc_data_doc, static PyTypeObject _abc_data_type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "_abc_data", /*tp_name*/ - sizeof(_abc_data), /*tp_size*/ + sizeof(_abc_data), /*tp_basicsize*/ .tp_dealloc = (destructor)abc_data_dealloc, .tp_flags = Py_TPFLAGS_DEFAULT, .tp_alloc = PyType_GenericAlloc, diff --git a/Modules/_blake2/blake2b_impl.c b/Modules/_blake2/blake2b_impl.c index f143b5d0c2ff..31eaf3e5a799 100644 --- a/Modules/_blake2/blake2b_impl.c +++ b/Modules/_blake2/blake2b_impl.c @@ -413,7 +413,7 @@ py_blake2b_dealloc(PyObject *self) PyTypeObject PyBlake2_BLAKE2bType = { PyVarObject_HEAD_INIT(NULL, 0) "_blake2.blake2b", /* tp_name */ - sizeof(BLAKE2bObject), /* tp_size */ + sizeof(BLAKE2bObject), /* tp_basicsize */ 0, /* tp_itemsize */ py_blake2b_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Modules/_blake2/blake2s_impl.c b/Modules/_blake2/blake2s_impl.c index 7d366d394005..7c7cb4e35c7d 100644 --- a/Modules/_blake2/blake2s_impl.c +++ b/Modules/_blake2/blake2s_impl.c @@ -413,7 +413,7 @@ py_blake2s_dealloc(PyObject *self) PyTypeObject PyBlake2_BLAKE2sType = { PyVarObject_HEAD_INIT(NULL, 0) "_blake2.blake2s", /* tp_name */ - sizeof(BLAKE2sObject), /* tp_size */ + sizeof(BLAKE2sObject), /* tp_basicsize */ 0, /* tp_itemsize */ py_blake2s_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Modules/_queuemodule.c b/Modules/_queuemodule.c index 0eb993027459..2a9406cc4760 100644 --- a/Modules/_queuemodule.c +++ b/Modules/_queuemodule.c @@ -309,7 +309,7 @@ static PyMethodDef simplequeue_methods[] = { static PyTypeObject PySimpleQueueType = { PyVarObject_HEAD_INIT(NULL, 0) "_queue.SimpleQueue", /*tp_name*/ - sizeof(simplequeueobject), /*tp_size*/ + sizeof(simplequeueobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)simplequeue_dealloc, /*tp_dealloc*/ diff --git a/Modules/_sha3/sha3module.c b/Modules/_sha3/sha3module.c index f73bc99eafe9..d879e9243c6e 100644 --- a/Modules/_sha3/sha3module.c +++ b/Modules/_sha3/sha3module.c @@ -489,7 +489,7 @@ static PyGetSetDef SHA3_getseters[] = { static PyTypeObject type_obj = { \ PyVarObject_HEAD_INIT(NULL, 0) \ type_name, /* tp_name */ \ - sizeof(SHA3object), /* tp_size */ \ + sizeof(SHA3object), /* tp_basicsize */ \ 0, /* tp_itemsize */ \ /* methods */ \ (destructor)SHA3_dealloc, /* tp_dealloc */ \ diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index f594bda678a8..16d3191c621a 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -226,7 +226,7 @@ static PyMethodDef lock_methods[] = { static PyTypeObject Locktype = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "_thread.lock", /*tp_name*/ - sizeof(lockobject), /*tp_size*/ + sizeof(lockobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)lock_dealloc, /*tp_dealloc*/ @@ -487,7 +487,7 @@ static PyMethodDef rlock_methods[] = { static PyTypeObject RLocktype = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "_thread.RLock", /*tp_name*/ - sizeof(rlockobject), /*tp_size*/ + sizeof(rlockobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)rlock_dealloc, /*tp_dealloc*/ diff --git a/Modules/md5module.c b/Modules/md5module.c index b019f8287684..c66b27308f05 100644 --- a/Modules/md5module.c +++ b/Modules/md5module.c @@ -465,7 +465,7 @@ static PyGetSetDef MD5_getseters[] = { static PyTypeObject MD5type = { PyVarObject_HEAD_INIT(NULL, 0) "_md5.md5", /*tp_name*/ - sizeof(MD5object), /*tp_size*/ + sizeof(MD5object), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ MD5_dealloc, /*tp_dealloc*/ diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index 27030db49b24..ba42f7380c09 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -980,7 +980,7 @@ To map anonymous memory, pass -1 as the fileno (both versions)."); static PyTypeObject mmap_object_type = { PyVarObject_HEAD_INIT(NULL, 0) "mmap.mmap", /* tp_name */ - sizeof(mmap_object), /* tp_size */ + sizeof(mmap_object), /* tp_basicsize */ 0, /* tp_itemsize */ /* methods */ (destructor) mmap_object_dealloc, /* tp_dealloc */ diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c index 58ee71f90108..150a14eb388a 100644 --- a/Modules/ossaudiodev.c +++ b/Modules/ossaudiodev.c @@ -964,7 +964,7 @@ oss_getattro(oss_audio_t *self, PyObject *nameobj) static PyTypeObject OSSAudioType = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "ossaudiodev.oss_audio_device", /*tp_name*/ - sizeof(oss_audio_t), /*tp_size*/ + sizeof(oss_audio_t), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)oss_dealloc, /*tp_dealloc*/ @@ -996,7 +996,7 @@ static PyTypeObject OSSAudioType = { static PyTypeObject OSSMixerType = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "ossaudiodev.oss_mixer_device", /*tp_name*/ - sizeof(oss_mixer_t), /*tp_size*/ + sizeof(oss_mixer_t), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)oss_mixer_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha1module.c b/Modules/sha1module.c index d39190b4d5e6..358cd1e7f9ac 100644 --- a/Modules/sha1module.c +++ b/Modules/sha1module.c @@ -442,7 +442,7 @@ static PyGetSetDef SHA1_getseters[] = { static PyTypeObject SHA1type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha1.sha1", /*tp_name*/ - sizeof(SHA1object), /*tp_size*/ + sizeof(SHA1object), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA1_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha256module.c b/Modules/sha256module.c index e4cb3286ce95..3599eaf8be92 100644 --- a/Modules/sha256module.c +++ b/Modules/sha256module.c @@ -529,7 +529,7 @@ static PyMemberDef SHA_members[] = { static PyTypeObject SHA224type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha256.sha224", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA_dealloc, /*tp_dealloc*/ @@ -563,7 +563,7 @@ static PyTypeObject SHA224type = { static PyTypeObject SHA256type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha256.sha256", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha512module.c b/Modules/sha512module.c index 5ac2a2a61cf5..d8c846a450f2 100644 --- a/Modules/sha512module.c +++ b/Modules/sha512module.c @@ -594,7 +594,7 @@ static PyMemberDef SHA_members[] = { static PyTypeObject SHA384type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha512.sha384", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA512_dealloc, /*tp_dealloc*/ @@ -628,7 +628,7 @@ static PyTypeObject SHA384type = { static PyTypeObject SHA512type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha512.sha512", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA512_dealloc, /*tp_dealloc*/ diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index 8fb368e41474..c4bf08141268 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -35,7 +35,7 @@ bad_traverse_test(PyObject *self, void *arg) { PyTypeObject PyModuleDef_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "moduledef", /* tp_name */ - sizeof(struct PyModuleDef), /* tp_size */ + sizeof(struct PyModuleDef), /* tp_basicsize */ 0, /* tp_itemsize */ }; @@ -789,7 +789,7 @@ static PyMethodDef module_methods[] = { PyTypeObject PyModule_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "module", /* tp_name */ - sizeof(PyModuleObject), /* tp_size */ + sizeof(PyModuleObject), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)module_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Objects/namespaceobject.c b/Objects/namespaceobject.c index e5698e6378de..7de102eb06a0 100644 --- a/Objects/namespaceobject.c +++ b/Objects/namespaceobject.c @@ -201,7 +201,7 @@ SimpleNamespace(**kwargs)"); PyTypeObject _PyNamespace_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "types.SimpleNamespace", /* tp_name */ - sizeof(_PyNamespaceObject), /* tp_size */ + sizeof(_PyNamespaceObject), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)namespace_dealloc, /* tp_dealloc */ 0, /* tp_print */ From webhook-mailer at python.org Mon Sep 10 13:23:07 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Mon, 10 Sep 2018 17:23:07 -0000 Subject: [Python-checkins] [3.6] Fix misleading mentions of tp_size in comments (GH-9137) Message-ID: https://github.com/python/cpython/commit/6a939c8dae225aaf088c8c37de8cc2655c83ee61 commit: 6a939c8dae225aaf088c8c37de8cc2655c83ee61 branch: 3.6 author: Benjamin Peterson committer: GitHub date: 2018-09-10T10:23:03-07:00 summary: [3.6] Fix misleading mentions of tp_size in comments (GH-9137) Many type object initializations labeled a field "tp_size" in the comment, but the name of that field is tp_basicsize.. (cherry picked from commit 0e0bc4e221f592f305d335faf5f8046484eb9238) Co-authored-by: Peter Eisentraut files: M Modules/_blake2/blake2b_impl.c M Modules/_blake2/blake2s_impl.c M Modules/_sha3/sha3module.c M Modules/_threadmodule.c M Modules/md5module.c M Modules/mmapmodule.c M Modules/ossaudiodev.c M Modules/sha1module.c M Modules/sha256module.c M Modules/sha512module.c M Objects/moduleobject.c M Objects/namespaceobject.c diff --git a/Modules/_blake2/blake2b_impl.c b/Modules/_blake2/blake2b_impl.c index 09b200ebe172..9e6f72e67f49 100644 --- a/Modules/_blake2/blake2b_impl.c +++ b/Modules/_blake2/blake2b_impl.c @@ -423,7 +423,7 @@ py_blake2b_dealloc(PyObject *self) PyTypeObject PyBlake2_BLAKE2bType = { PyVarObject_HEAD_INIT(NULL, 0) "_blake2.blake2b", /* tp_name */ - sizeof(BLAKE2bObject), /* tp_size */ + sizeof(BLAKE2bObject), /* tp_basicsize */ 0, /* tp_itemsize */ py_blake2b_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Modules/_blake2/blake2s_impl.c b/Modules/_blake2/blake2s_impl.c index 735b6f08deb6..0743fdf2084c 100644 --- a/Modules/_blake2/blake2s_impl.c +++ b/Modules/_blake2/blake2s_impl.c @@ -423,7 +423,7 @@ py_blake2s_dealloc(PyObject *self) PyTypeObject PyBlake2_BLAKE2sType = { PyVarObject_HEAD_INIT(NULL, 0) "_blake2.blake2s", /* tp_name */ - sizeof(BLAKE2sObject), /* tp_size */ + sizeof(BLAKE2sObject), /* tp_basicsize */ 0, /* tp_itemsize */ py_blake2s_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Modules/_sha3/sha3module.c b/Modules/_sha3/sha3module.c index 11aaa1f6b0af..e8504de16154 100644 --- a/Modules/_sha3/sha3module.c +++ b/Modules/_sha3/sha3module.c @@ -505,7 +505,7 @@ static PyGetSetDef SHA3_getseters[] = { static PyTypeObject type_obj = { \ PyVarObject_HEAD_INIT(NULL, 0) \ type_name, /* tp_name */ \ - sizeof(SHA3object), /* tp_size */ \ + sizeof(SHA3object), /* tp_basicsize */ \ 0, /* tp_itemsize */ \ /* methods */ \ (destructor)SHA3_dealloc, /* tp_dealloc */ \ diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index 47e84b9e916d..c504b57b064f 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -234,7 +234,7 @@ static PyMethodDef lock_methods[] = { static PyTypeObject Locktype = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "_thread.lock", /*tp_name*/ - sizeof(lockobject), /*tp_size*/ + sizeof(lockobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)lock_dealloc, /*tp_dealloc*/ @@ -495,7 +495,7 @@ static PyMethodDef rlock_methods[] = { static PyTypeObject RLocktype = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "_thread.RLock", /*tp_name*/ - sizeof(rlockobject), /*tp_size*/ + sizeof(rlockobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)rlock_dealloc, /*tp_dealloc*/ diff --git a/Modules/md5module.c b/Modules/md5module.c index fe9da7d3af2d..01d2118339e3 100644 --- a/Modules/md5module.c +++ b/Modules/md5module.c @@ -466,7 +466,7 @@ static PyGetSetDef MD5_getseters[] = { static PyTypeObject MD5type = { PyVarObject_HEAD_INIT(NULL, 0) "_md5.md5", /*tp_name*/ - sizeof(MD5object), /*tp_size*/ + sizeof(MD5object), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ MD5_dealloc, /*tp_dealloc*/ diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index ebef501d6749..721e5ed32ea2 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -1008,7 +1008,7 @@ To map anonymous memory, pass -1 as the fileno (both versions)."); static PyTypeObject mmap_object_type = { PyVarObject_HEAD_INIT(NULL, 0) "mmap.mmap", /* tp_name */ - sizeof(mmap_object), /* tp_size */ + sizeof(mmap_object), /* tp_basicsize */ 0, /* tp_itemsize */ /* methods */ (destructor) mmap_object_dealloc, /* tp_dealloc */ diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c index 5f0505c8a7b4..da1d1631451f 100644 --- a/Modules/ossaudiodev.c +++ b/Modules/ossaudiodev.c @@ -969,7 +969,7 @@ oss_getattro(oss_audio_t *self, PyObject *nameobj) static PyTypeObject OSSAudioType = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "ossaudiodev.oss_audio_device", /*tp_name*/ - sizeof(oss_audio_t), /*tp_size*/ + sizeof(oss_audio_t), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)oss_dealloc, /*tp_dealloc*/ @@ -1001,7 +1001,7 @@ static PyTypeObject OSSAudioType = { static PyTypeObject OSSMixerType = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "ossaudiodev.oss_mixer_device", /*tp_name*/ - sizeof(oss_mixer_t), /*tp_size*/ + sizeof(oss_mixer_t), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)oss_mixer_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha1module.c b/Modules/sha1module.c index d5065ce13490..cd123586264c 100644 --- a/Modules/sha1module.c +++ b/Modules/sha1module.c @@ -443,7 +443,7 @@ static PyGetSetDef SHA1_getseters[] = { static PyTypeObject SHA1type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha1.sha1", /*tp_name*/ - sizeof(SHA1object), /*tp_size*/ + sizeof(SHA1object), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA1_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha256module.c b/Modules/sha256module.c index 8f067f182c77..70c49b5da2ee 100644 --- a/Modules/sha256module.c +++ b/Modules/sha256module.c @@ -530,7 +530,7 @@ static PyMemberDef SHA_members[] = { static PyTypeObject SHA224type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha256.sha224", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA_dealloc, /*tp_dealloc*/ @@ -564,7 +564,7 @@ static PyTypeObject SHA224type = { static PyTypeObject SHA256type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha256.sha256", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha512module.c b/Modules/sha512module.c index 17775ae57b6c..da0cb5775edb 100644 --- a/Modules/sha512module.c +++ b/Modules/sha512module.c @@ -599,7 +599,7 @@ static PyMemberDef SHA_members[] = { static PyTypeObject SHA384type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha512.sha384", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA512_dealloc, /*tp_dealloc*/ @@ -633,7 +633,7 @@ static PyTypeObject SHA384type = { static PyTypeObject SHA512type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha512.sha512", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA512_dealloc, /*tp_dealloc*/ diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index c5979329647b..0fba90c43c14 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -34,7 +34,7 @@ bad_traverse_test(PyObject *self, void *arg) { PyTypeObject PyModuleDef_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "moduledef", /* tp_name */ - sizeof(struct PyModuleDef), /* tp_size */ + sizeof(struct PyModuleDef), /* tp_basicsize */ 0, /* tp_itemsize */ }; @@ -761,7 +761,7 @@ The name must be a string; the optional doc argument can have any type."); PyTypeObject PyModule_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "module", /* tp_name */ - sizeof(PyModuleObject), /* tp_size */ + sizeof(PyModuleObject), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)module_dealloc, /* tp_dealloc */ 0, /* tp_print */ diff --git a/Objects/namespaceobject.c b/Objects/namespaceobject.c index d28c9133b4b7..6307ee0423dc 100644 --- a/Objects/namespaceobject.c +++ b/Objects/namespaceobject.c @@ -207,7 +207,7 @@ SimpleNamespace(**kwargs)"); PyTypeObject _PyNamespace_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "types.SimpleNamespace", /* tp_name */ - sizeof(_PyNamespaceObject), /* tp_size */ + sizeof(_PyNamespaceObject), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)namespace_dealloc, /* tp_dealloc */ 0, /* tp_print */ From webhook-mailer at python.org Mon Sep 10 13:29:46 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Mon, 10 Sep 2018 17:29:46 -0000 Subject: [Python-checkins] closes bpo-34525: Fix smtplib's authobject() documentation (GH-8965) Message-ID: https://github.com/python/cpython/commit/78deb7f33227972987722bc3fed5bcb45fae869e commit: 78deb7f33227972987722bc3fed5bcb45fae869e branch: master author: Sebastian Rittau committer: Benjamin Peterson date: 2018-09-10T10:29:43-07:00 summary: closes bpo-34525: Fix smtplib's authobject() documentation (GH-8965) files: M Doc/library/smtplib.rst M Lib/smtplib.py diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index 805217252ae8..6fb0934218a6 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -346,7 +346,7 @@ An :class:`SMTP` instance has the following methods: If optional keyword argument *initial_response_ok* is true, ``authobject()`` will be called first with no argument. It can return the - :rfc:`4954` "initial response" bytes which will be encoded and sent with + :rfc:`4954` "initial response" ASCII ``str`` which will be encoded and sent with the ``AUTH`` command as below. If the ``authobject()`` does not support an initial response (e.g. because it requires a challenge), it should return ``None`` when called with ``challenge=None``. If *initial_response_ok* is @@ -355,7 +355,7 @@ An :class:`SMTP` instance has the following methods: If the initial response check returns ``None``, or if *initial_response_ok* is false, ``authobject()`` will be called to process the server's challenge response; the *challenge* argument it is passed will be a ``bytes``. It - should return ``bytes`` *data* that will be base64 encoded and sent to the + should return ASCII ``str`` *data* that will be base64 encoded and sent to the server. The ``SMTP`` class provides ``authobjects`` for the ``CRAM-MD5``, ``PLAIN``, diff --git a/Lib/smtplib.py b/Lib/smtplib.py index 048c6bfb0671..5e1bc0b198ed 100755 --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -615,7 +615,7 @@ def auth(self, mechanism, authobject, *, initial_response_ok=True): It will be called to process the server's challenge response; the challenge argument it is passed will be a bytes. It should return - bytes data that will be base64 encoded and sent to the server. + an ASCII string that will be base64 encoded and sent to the server. Keyword arguments: - initial_response_ok: Allow sending the RFC 4954 initial-response From webhook-mailer at python.org Mon Sep 10 13:50:24 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Mon, 10 Sep 2018 17:50:24 -0000 Subject: [Python-checkins] [2.7] Fix misleading mentions of tp_size in comments (GH-9138) Message-ID: https://github.com/python/cpython/commit/ed62e645101fdd83aee519f6f33a4384ff79f105 commit: ed62e645101fdd83aee519f6f33a4384ff79f105 branch: 2.7 author: Benjamin Peterson committer: GitHub date: 2018-09-10T10:50:15-07:00 summary: [2.7] Fix misleading mentions of tp_size in comments (GH-9138) Many type object initializations labeled a field "tp_size" in the comment, but the name of that field is tp_basicsize.. (cherry picked from commit 0e0bc4e221f592f305d335faf5f8046484eb9238) Co-authored-by: Peter Eisentraut files: M Modules/md5module.c M Modules/mmapmodule.c M Modules/ossaudiodev.c M Modules/sha256module.c M Modules/sha512module.c M Objects/moduleobject.c diff --git a/Modules/md5module.c b/Modules/md5module.c index 103da1497dac..dd90a22c00bf 100644 --- a/Modules/md5module.c +++ b/Modules/md5module.c @@ -236,7 +236,7 @@ copy() -- return a copy of the current md5 object"); static PyTypeObject MD5type = { PyVarObject_HEAD_INIT(NULL, 0) "_md5.md5", /*tp_name*/ - sizeof(md5object), /*tp_size*/ + sizeof(md5object), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)md5_dealloc, /*tp_dealloc*/ diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index bafdce3b1f84..02b31ca438a9 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -1026,7 +1026,7 @@ To map anonymous memory, pass -1 as the fileno (both versions)."); static PyTypeObject mmap_object_type = { PyVarObject_HEAD_INIT(NULL, 0) "mmap.mmap", /* tp_name */ - sizeof(mmap_object), /* tp_size */ + sizeof(mmap_object), /* tp_basicsize */ 0, /* tp_itemsize */ /* methods */ (destructor) mmap_object_dealloc, /* tp_dealloc */ diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c index 284cc6107619..f833bb257f7a 100644 --- a/Modules/ossaudiodev.c +++ b/Modules/ossaudiodev.c @@ -842,7 +842,7 @@ oss_mixer_getattr(oss_mixer_t *self, char *name) static PyTypeObject OSSAudioType = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "ossaudiodev.oss_audio_device", /*tp_name*/ - sizeof(oss_audio_t), /*tp_size*/ + sizeof(oss_audio_t), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)oss_dealloc, /*tp_dealloc*/ @@ -856,7 +856,7 @@ static PyTypeObject OSSAudioType = { static PyTypeObject OSSMixerType = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "ossaudiodev.oss_mixer_device", /*tp_name*/ - sizeof(oss_mixer_t), /*tp_size*/ + sizeof(oss_mixer_t), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)oss_mixer_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha256module.c b/Modules/sha256module.c index 8f18faa17a59..76e52209f630 100644 --- a/Modules/sha256module.c +++ b/Modules/sha256module.c @@ -537,7 +537,7 @@ static PyMemberDef SHA_members[] = { static PyTypeObject SHA224type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha256.sha224", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA_dealloc, /*tp_dealloc*/ @@ -571,7 +571,7 @@ static PyTypeObject SHA224type = { static PyTypeObject SHA256type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha256.sha256", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA_dealloc, /*tp_dealloc*/ diff --git a/Modules/sha512module.c b/Modules/sha512module.c index abc90218053f..e3733e0443ff 100644 --- a/Modules/sha512module.c +++ b/Modules/sha512module.c @@ -603,7 +603,7 @@ static PyMemberDef SHA_members[] = { static PyTypeObject SHA384type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha512.sha384", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA512_dealloc, /*tp_dealloc*/ @@ -637,7 +637,7 @@ static PyTypeObject SHA384type = { static PyTypeObject SHA512type = { PyVarObject_HEAD_INIT(NULL, 0) "_sha512.sha512", /*tp_name*/ - sizeof(SHAobject), /*tp_size*/ + sizeof(SHAobject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ SHA512_dealloc, /*tp_dealloc*/ diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index f2fed30e90dd..fd48ff176e75 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -221,7 +221,7 @@ The name must be a string; the optional doc argument can have any type."); PyTypeObject PyModule_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "module", /* tp_name */ - sizeof(PyModuleObject), /* tp_size */ + sizeof(PyModuleObject), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)module_dealloc, /* tp_dealloc */ 0, /* tp_print */ From webhook-mailer at python.org Mon Sep 10 14:10:05 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Mon, 10 Sep 2018 18:10:05 -0000 Subject: [Python-checkins] bpo-33604: Remove deprecated HMAC default value marked for removal in 3.8 (GH-7063) Message-ID: https://github.com/python/cpython/commit/51a4743d19abd016f0772a57fb31df7af9220e18 commit: 51a4743d19abd016f0772a57fb31df7af9220e18 branch: master author: Matthias Bussonnier committer: Gregory P. Smith date: 2018-09-10T11:10:01-07:00 summary: bpo-33604: Remove deprecated HMAC default value marked for removal in 3.8 (GH-7063) HMAC's digestmod was deprecated marked for removal, this removes it as planned. files: A Misc/NEWS.d/next/Library/2018-05-22-11-55-33.bpo-33604.6V4JcO.rst M Doc/library/hmac.rst M Lib/hmac.py M Lib/test/test_hmac.py diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst index 731624ba94e3..dc994b07c35c 100644 --- a/Doc/library/hmac.rst +++ b/Doc/library/hmac.rst @@ -19,8 +19,7 @@ This module implements the HMAC algorithm as described by :rfc:`2104`. Return a new hmac object. *key* is a bytes or bytearray object giving the secret key. If *msg* is present, the method call ``update(msg)`` is made. *digestmod* is the digest name, digest constructor or module for the HMAC - object to use. It supports any name suitable to :func:`hashlib.new` and - defaults to the :data:`hashlib.md5` constructor. + object to use. It supports any name suitable to :func:`hashlib.new`. .. versionchanged:: 3.4 Parameter *key* can be a bytes or bytearray object. diff --git a/Lib/hmac.py b/Lib/hmac.py index 43b721297637..890eaba08e8f 100644 --- a/Lib/hmac.py +++ b/Lib/hmac.py @@ -35,12 +35,9 @@ def __init__(self, key, msg = None, digestmod = None): key: key for the keyed hash object. msg: Initial input for the hash, if provided. - digestmod: A module supporting PEP 247. *OR* - A hashlib constructor returning a new hash object. *OR* + digestmod: Required. A module supporting PEP 247. *OR* + A hashlib constructor returning a new hash object. *OR* A hash name suitable for hashlib.new(). - Defaults to hashlib.md5. - Implicit default to hashlib.md5 is deprecated since Python - 3.4 and will be removed in Python 3.8. Note: key and msg must be a bytes or bytearray objects. """ @@ -49,11 +46,7 @@ def __init__(self, key, msg = None, digestmod = None): raise TypeError("key: expected bytes or bytearray, but got %r" % type(key).__name__) if digestmod is None: - _warnings.warn("HMAC() without an explicit digestmod argument " - "is deprecated since Python 3.4, and will be removed " - "in 3.8", - DeprecationWarning, 2) - digestmod = _hashlib.md5 + raise ValueError('`digestmod` is required.') if callable(digestmod): self.digest_cons = digestmod diff --git a/Lib/test/test_hmac.py b/Lib/test/test_hmac.py index 7f4901307630..896bbe9ab798 100644 --- a/Lib/test/test_hmac.py +++ b/Lib/test/test_hmac.py @@ -302,45 +302,38 @@ def digest(self): hmac.HMAC(b'a', b'b', digestmod=MockCrazyHash) self.fail('Expected warning about small block_size') - def test_with_digestmod_warning(self): - with self.assertWarns(DeprecationWarning): + def test_with_digestmod_no_default(self): + with self.assertRaises(ValueError): key = b"\x0b" * 16 data = b"Hi There" - digest = "9294727A3638BB1C13F48EF8158BFC9D" - h = hmac.HMAC(key, data) - self.assertEqual(h.hexdigest().upper(), digest) - + hmac.HMAC(key, data, digestmod=None) class ConstructorTestCase(unittest.TestCase): - @ignore_warning def test_normal(self): # Standard constructor call. failed = 0 try: - h = hmac.HMAC(b"key") + h = hmac.HMAC(b"key", digestmod='md5') except Exception: self.fail("Standard constructor call raised exception.") - @ignore_warning def test_with_str_key(self): # Pass a key of type str, which is an error, because it expects a key # of type bytes with self.assertRaises(TypeError): - h = hmac.HMAC("key") + h = hmac.HMAC("key", digestmod='md5') - @ignore_warning def test_dot_new_with_str_key(self): # Pass a key of type str, which is an error, because it expects a key # of type bytes with self.assertRaises(TypeError): - h = hmac.new("key") + h = hmac.new("key", digestmod='md5') - @ignore_warning def test_withtext(self): # Constructor call with text. try: - h = hmac.HMAC(b"key", b"hash this!") + h = hmac.HMAC(b"key", b"hash this!", digestmod='md5') except Exception: self.fail("Constructor call with text argument raised exception.") self.assertEqual(h.hexdigest(), '34325b639da4cfd95735b381e28cb864') @@ -369,13 +362,6 @@ def test_withmodule(self): class SanityTestCase(unittest.TestCase): - @ignore_warning - def test_default_is_md5(self): - # Testing if HMAC defaults to MD5 algorithm. - # NOTE: this whitebox test depends on the hmac class internals - h = hmac.HMAC(b"key") - self.assertEqual(h.digest_cons, hashlib.md5) - def test_exercise_all_methods(self): # Exercising all methods once. # This must not raise any exceptions diff --git a/Misc/NEWS.d/next/Library/2018-05-22-11-55-33.bpo-33604.6V4JcO.rst b/Misc/NEWS.d/next/Library/2018-05-22-11-55-33.bpo-33604.6V4JcO.rst new file mode 100644 index 000000000000..200a9d5f661a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-05-22-11-55-33.bpo-33604.6V4JcO.rst @@ -0,0 +1,2 @@ +Remove HMAC default to md5 marked for removal in 3.8 (removal originally +planned in 3.6, bump to 3.8 in gh-7062). From webhook-mailer at python.org Mon Sep 10 14:13:17 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 18:13:17 -0000 Subject: [Python-checkins] closes bpo-34525: Fix smtplib's authobject() documentation (GH-8965) Message-ID: https://github.com/python/cpython/commit/011141f312f11b35ac163d9d52fe48e4bb61a814 commit: 011141f312f11b35ac163d9d52fe48e4bb61a814 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T11:13:13-07:00 summary: closes bpo-34525: Fix smtplib's authobject() documentation (GH-8965) (cherry picked from commit 78deb7f33227972987722bc3fed5bcb45fae869e) Co-authored-by: Sebastian Rittau files: M Doc/library/smtplib.rst M Lib/smtplib.py diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index 805217252ae8..6fb0934218a6 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -346,7 +346,7 @@ An :class:`SMTP` instance has the following methods: If optional keyword argument *initial_response_ok* is true, ``authobject()`` will be called first with no argument. It can return the - :rfc:`4954` "initial response" bytes which will be encoded and sent with + :rfc:`4954` "initial response" ASCII ``str`` which will be encoded and sent with the ``AUTH`` command as below. If the ``authobject()`` does not support an initial response (e.g. because it requires a challenge), it should return ``None`` when called with ``challenge=None``. If *initial_response_ok* is @@ -355,7 +355,7 @@ An :class:`SMTP` instance has the following methods: If the initial response check returns ``None``, or if *initial_response_ok* is false, ``authobject()`` will be called to process the server's challenge response; the *challenge* argument it is passed will be a ``bytes``. It - should return ``bytes`` *data* that will be base64 encoded and sent to the + should return ASCII ``str`` *data* that will be base64 encoded and sent to the server. The ``SMTP`` class provides ``authobjects`` for the ``CRAM-MD5``, ``PLAIN``, diff --git a/Lib/smtplib.py b/Lib/smtplib.py index 048c6bfb0671..5e1bc0b198ed 100755 --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -615,7 +615,7 @@ def auth(self, mechanism, authobject, *, initial_response_ok=True): It will be called to process the server's challenge response; the challenge argument it is passed will be a bytes. It should return - bytes data that will be base64 encoded and sent to the server. + an ASCII string that will be base64 encoded and sent to the server. Keyword arguments: - initial_response_ok: Allow sending the RFC 4954 initial-response From webhook-mailer at python.org Mon Sep 10 14:16:15 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 10 Sep 2018 18:16:15 -0000 Subject: [Python-checkins] closes bpo-34525: Fix smtplib's authobject() documentation (GH-8965) Message-ID: https://github.com/python/cpython/commit/c0db8177dc9a7f79941551576e1faea9724e027d commit: c0db8177dc9a7f79941551576e1faea9724e027d branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T11:16:11-07:00 summary: closes bpo-34525: Fix smtplib's authobject() documentation (GH-8965) (cherry picked from commit 78deb7f33227972987722bc3fed5bcb45fae869e) Co-authored-by: Sebastian Rittau files: M Doc/library/smtplib.rst M Lib/smtplib.py diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index 805217252ae8..6fb0934218a6 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -346,7 +346,7 @@ An :class:`SMTP` instance has the following methods: If optional keyword argument *initial_response_ok* is true, ``authobject()`` will be called first with no argument. It can return the - :rfc:`4954` "initial response" bytes which will be encoded and sent with + :rfc:`4954` "initial response" ASCII ``str`` which will be encoded and sent with the ``AUTH`` command as below. If the ``authobject()`` does not support an initial response (e.g. because it requires a challenge), it should return ``None`` when called with ``challenge=None``. If *initial_response_ok* is @@ -355,7 +355,7 @@ An :class:`SMTP` instance has the following methods: If the initial response check returns ``None``, or if *initial_response_ok* is false, ``authobject()`` will be called to process the server's challenge response; the *challenge* argument it is passed will be a ``bytes``. It - should return ``bytes`` *data* that will be base64 encoded and sent to the + should return ASCII ``str`` *data* that will be base64 encoded and sent to the server. The ``SMTP`` class provides ``authobjects`` for the ``CRAM-MD5``, ``PLAIN``, diff --git a/Lib/smtplib.py b/Lib/smtplib.py index 048c6bfb0671..5e1bc0b198ed 100755 --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -615,7 +615,7 @@ def auth(self, mechanism, authobject, *, initial_response_ok=True): It will be called to process the server's challenge response; the challenge argument it is passed will be a bytes. It should return - bytes data that will be base64 encoded and sent to the server. + an ASCII string that will be base64 encoded and sent to the server. Keyword arguments: - initial_response_ok: Allow sending the RFC 4954 initial-response From webhook-mailer at python.org Mon Sep 10 14:21:09 2018 From: webhook-mailer at python.org (Ethan Furman) Date: Mon, 10 Sep 2018 18:21:09 -0000 Subject: [Python-checkins] bpo-33217: Raise TypeError for non-Enum lookups in Enums (GH-6651) Message-ID: https://github.com/python/cpython/commit/9430652535f88125d8003f342a8884d34885d876 commit: 9430652535f88125d8003f342a8884d34885d876 branch: master author: Rahul Jha committer: Ethan Furman date: 2018-09-10T11:21:04-07:00 summary: bpo-33217: Raise TypeError for non-Enum lookups in Enums (GH-6651) * bpo-33217: Raise TypeError for non-Enum lookups in Enums files: A Misc/NEWS.d/next/Library/2018-04-30-13-29-47.bpo-33217.TENDzd.rst M Doc/library/enum.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index dd794b42fcbd..3f17f9947641 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -976,7 +976,7 @@ Enum Classes The :class:`EnumMeta` metaclass is responsible for providing the :meth:`__contains__`, :meth:`__dir__`, :meth:`__iter__` and other methods that allow one to do things with an :class:`Enum` class that fail on a typical -class, such as `list(Color)` or `some_var in Color`. :class:`EnumMeta` is +class, such as `list(Color)` or `some_enum_var in Color`. :class:`EnumMeta` is responsible for ensuring that various other methods on the final :class:`Enum` class are correct (such as :meth:`__new__`, :meth:`__getnewargs__`, :meth:`__str__` and :meth:`__repr__`). diff --git a/Lib/enum.py b/Lib/enum.py index 04d8ec1fa872..9d1aef372c12 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -303,6 +303,10 @@ def __call__(cls, value, names=None, *, module=None, qualname=None, type=None, s return cls._create_(value, names, module=module, qualname=qualname, type=type, start=start) def __contains__(cls, member): + if not isinstance(member, Enum): + raise TypeError( + "unsupported operand type(s) for 'in': '%s' and '%s'" % ( + type(member).__qualname__, cls.__class__.__qualname__)) return isinstance(member, cls) and member._name_ in cls._member_map_ def __delattr__(cls, attr): @@ -705,7 +709,9 @@ def _create_pseudo_member_(cls, value): def __contains__(self, other): if not isinstance(other, self.__class__): - return NotImplemented + raise TypeError( + "unsupported operand type(s) for 'in': '%s' and '%s'" % ( + type(other).__qualname__, self.__class__.__qualname__)) return other._value_ & self._value_ == other._value_ def __repr__(self): diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 97559712b1dc..68483e654254 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -325,7 +325,10 @@ class IntLogic(int, Enum): def test_contains(self): Season = self.Season self.assertIn(Season.AUTUMN, Season) - self.assertNotIn(3, Season) + with self.assertRaises(TypeError): + 3 in Season + with self.assertRaises(TypeError): + 'AUTUMN' in Season val = Season(3) self.assertIn(val, Season) @@ -1752,6 +1755,13 @@ class Open(Flag): AC = 3 CE = 1<<19 + class Color(Flag): + BLACK = 0 + RED = 1 + GREEN = 2 + BLUE = 4 + PURPLE = RED|BLUE + def test_str(self): Perm = self.Perm self.assertEqual(str(Perm.R), 'Perm.R') @@ -1954,7 +1964,21 @@ def test_pickle(self): test_pickle_dump_load(self.assertIs, FlagStooges.CURLY|FlagStooges.MOE) test_pickle_dump_load(self.assertIs, FlagStooges) - def test_containment(self): + def test_contains(self): + Open = self.Open + Color = self.Color + self.assertFalse(Color.BLACK in Open) + self.assertFalse(Open.RO in Color) + with self.assertRaises(TypeError): + 'BLACK' in Color + with self.assertRaises(TypeError): + 'RO' in Open + with self.assertRaises(TypeError): + 1 in Color + with self.assertRaises(TypeError): + 1 in Open + + def test_member_contains(self): Perm = self.Perm R, W, X = Perm RW = R | W @@ -2072,6 +2096,13 @@ class Open(IntFlag): AC = 3 CE = 1<<19 + class Color(IntFlag): + BLACK = 0 + RED = 1 + GREEN = 2 + BLUE = 4 + PURPLE = RED|BLUE + def test_type(self): Perm = self.Perm Open = self.Open @@ -2340,7 +2371,23 @@ def test_programatic_function_from_empty_tuple(self): self.assertEqual(len(lst), len(Thing)) self.assertEqual(len(Thing), 0, Thing) - def test_containment(self): + def test_contains(self): + Open = self.Open + Color = self.Color + self.assertTrue(Color.GREEN in Color) + self.assertTrue(Open.RW in Open) + self.assertFalse(Color.GREEN in Open) + self.assertFalse(Open.RW in Color) + with self.assertRaises(TypeError): + 'GREEN' in Color + with self.assertRaises(TypeError): + 'RW' in Open + with self.assertRaises(TypeError): + 2 in Color + with self.assertRaises(TypeError): + 2 in Open + + def test_member_contains(self): Perm = self.Perm R, W, X = Perm RW = R | W @@ -2359,6 +2406,8 @@ def test_containment(self): self.assertFalse(R in WX) self.assertFalse(W in RX) self.assertFalse(X in RW) + with self.assertRaises(TypeError): + self.assertFalse('test' in RW) def test_bool(self): Perm = self.Perm diff --git a/Misc/NEWS.d/next/Library/2018-04-30-13-29-47.bpo-33217.TENDzd.rst b/Misc/NEWS.d/next/Library/2018-04-30-13-29-47.bpo-33217.TENDzd.rst new file mode 100644 index 000000000000..071f5f16bfb7 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-04-30-13-29-47.bpo-33217.TENDzd.rst @@ -0,0 +1,2 @@ +Raise :exc:`TypeError` when looking up non-Enum objects in Enum classes and +Enum members. From webhook-mailer at python.org Mon Sep 10 14:33:12 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Mon, 10 Sep 2018 18:33:12 -0000 Subject: [Python-checkins] bpo-20180: itertools.groupby Argument Clinic conversion (GH-4170) Message-ID: https://github.com/python/cpython/commit/3286ce4adee85c5ce8ab3ee3089f3cd44a017fd7 commit: 3286ce4adee85c5ce8ab3ee3089f3cd44a017fd7 branch: master author: Tal Einat committer: Raymond Hettinger date: 2018-09-10T11:33:08-07:00 summary: bpo-20180: itertools.groupby Argument Clinic conversion (GH-4170) files: A Modules/clinic/itertoolsmodule.c.h M Modules/itertoolsmodule.c diff --git a/Modules/clinic/itertoolsmodule.c.h b/Modules/clinic/itertoolsmodule.c.h new file mode 100644 index 000000000000..68e67494bc93 --- /dev/null +++ b/Modules/clinic/itertoolsmodule.c.h @@ -0,0 +1,64 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +PyDoc_STRVAR(itertools_groupby__doc__, +"groupby(iterable, key=None)\n" +"--\n" +"\n" +"make an iterator that returns consecutive keys and groups from the iterable\n" +"\n" +" iterable\n" +" Elements to divide into groups according to the key function.\n" +" key\n" +" A function for computing the group category for each element.\n" +" If the key function is not specified or is None, the element itself\n" +" is used for grouping."); + +static PyObject * +itertools_groupby_impl(PyTypeObject *type, PyObject *it, PyObject *keyfunc); + +static PyObject * +itertools_groupby(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"iterable", "key", NULL}; + static _PyArg_Parser _parser = {"O|O:groupby", _keywords, 0}; + PyObject *it; + PyObject *keyfunc = Py_None; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &it, &keyfunc)) { + goto exit; + } + return_value = itertools_groupby_impl(type, it, keyfunc); + +exit: + return return_value; +} + +static PyObject * +itertools__grouper_impl(PyTypeObject *type, PyObject *parent, + PyObject *tgtkey); + +static PyObject * +itertools__grouper(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyObject *parent; + PyObject *tgtkey; + + if ((type == &_grouper_type) && + !_PyArg_NoKeywords("_grouper", kwargs)) { + goto exit; + } + if (!PyArg_ParseTuple(args, "O!O:_grouper", + &groupby_type, &parent, &tgtkey)) { + goto exit; + } + return_value = itertools__grouper_impl(type, parent, tgtkey); + +exit: + return return_value; +} +/*[clinic end generated code: output=82e10c91569d2b95 input=a9049054013a1b77]*/ diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index 8a36755bfa72..3ad7e5c80651 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -7,6 +7,17 @@ by Raymond D. Hettinger */ +/*[clinic input] +module itertools +class itertools.groupby "groupbyobject *" "&groupby_type" +class itertools._grouper "_grouperobject *" "&_grouper_type" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=9d506f5bb9177570]*/ + +static PyTypeObject groupby_type; +static PyTypeObject _grouper_type; +#include "clinic/itertoolsmodule.c.h" + /* groupby object ************************************************************/ @@ -20,19 +31,27 @@ typedef struct { const void *currgrouper; /* borrowed reference */ } groupbyobject; -static PyTypeObject groupby_type; static PyObject *_grouper_create(groupbyobject *, PyObject *); +/*[clinic input] + at classmethod +itertools.groupby.__new__ + + iterable as it: object + Elements to divide into groups according to the key function. + key as keyfunc: object = None + A function for computing the group category for each element. + If the key function is not specified or is None, the element itself + is used for grouping. + +make an iterator that returns consecutive keys and groups from the iterable +[clinic start generated code]*/ + static PyObject * -groupby_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_groupby_impl(PyTypeObject *type, PyObject *it, PyObject *keyfunc) +/*[clinic end generated code: output=cbb1ae3a90fd4141 input=6b3d123e87ff65a1]*/ { - static char *kwargs[] = {"iterable", "key", NULL}; groupbyobject *gbo; - PyObject *it, *keyfunc = Py_None; - - if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|O:groupby", kwargs, - &it, &keyfunc)) - return NULL; gbo = (groupbyobject *)type->tp_alloc(type, 0); if (gbo == NULL) @@ -186,11 +205,6 @@ static PyMethodDef groupby_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(groupby_doc, -"groupby(iterable, key=None) -> make an iterator that returns consecutive\n\ -keys and groups from the iterable. If the key function is not specified or\n\ -is None, the element itself is used for grouping.\n"); - static PyTypeObject groupby_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.groupby", /* tp_name */ @@ -214,7 +228,7 @@ static PyTypeObject groupby_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - groupby_doc, /* tp_doc */ + itertools_groupby__doc__, /* tp_doc */ (traverseproc)groupby_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -231,7 +245,7 @@ static PyTypeObject groupby_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - groupby_new, /* tp_new */ + itertools_groupby, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -244,16 +258,20 @@ typedef struct { PyObject *tgtkey; } _grouperobject; -static PyTypeObject _grouper_type; +/*[clinic input] + at classmethod +itertools._grouper.__new__ + + parent: object(subclass_of='&groupby_type') + tgtkey: object + / +[clinic start generated code]*/ static PyObject * -_grouper_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools__grouper_impl(PyTypeObject *type, PyObject *parent, + PyObject *tgtkey) +/*[clinic end generated code: output=462efb1cdebb5914 input=dc180d7771fc8c59]*/ { - PyObject *parent, *tgtkey; - - if (!PyArg_ParseTuple(args, "O!O", &groupby_type, &parent, &tgtkey)) - return NULL; - return _grouper_create((groupbyobject*) parent, tgtkey); } @@ -374,7 +392,7 @@ static PyTypeObject _grouper_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - _grouper_new, /* tp_new */ + itertools__grouper, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; From webhook-mailer at python.org Mon Sep 10 14:46:19 2018 From: webhook-mailer at python.org (Petr Viktorin) Date: Mon, 10 Sep 2018 18:46:19 -0000 Subject: [Python-checkins] bpo-25083: Python can sometimes create incorrect .pyc files (GH-8449) Message-ID: https://github.com/python/cpython/commit/f64c813de84011a84ca21d75a294861a9cc2dfdc commit: f64c813de84011a84ca21d75a294861a9cc2dfdc branch: 2.7 author: tzickel committer: Petr Viktorin date: 2018-09-10T11:46:14-07:00 summary: bpo-25083: Python can sometimes create incorrect .pyc files (GH-8449) Python 2 never checked for I/O error when reading .py files and thus could mistake an I/O error for EOF and create incorrect .pyc files. This adds an check for this and aborts on an error. files: A Misc/NEWS.d/next/Core and Builtins/2018-07-25-22-47-19.bpo-25083.HT_hXh.rst M Include/errcode.h M Parser/tokenizer.c M Python/pythonrun.c diff --git a/Include/errcode.h b/Include/errcode.h index becec80c8acf..5c5a0f7fa346 100644 --- a/Include/errcode.h +++ b/Include/errcode.h @@ -29,6 +29,7 @@ extern "C" { #define E_EOFS 23 /* EOF in triple-quoted string */ #define E_EOLS 24 /* EOL in single-quoted string */ #define E_LINECONT 25 /* Unexpected characters after a line continuation */ +#define E_IO 26 /* I/O error */ #ifdef __cplusplus } diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-07-25-22-47-19.bpo-25083.HT_hXh.rst b/Misc/NEWS.d/next/Core and Builtins/2018-07-25-22-47-19.bpo-25083.HT_hXh.rst new file mode 100644 index 000000000000..0dc44c43b387 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-07-25-22-47-19.bpo-25083.HT_hXh.rst @@ -0,0 +1,2 @@ +Adding I/O error checking when reading .py files and aborting importing on +error. diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 61bfb4e1b7af..c6e61df533e2 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1681,6 +1681,11 @@ int PyTokenizer_Get(struct tok_state *tok, char **p_start, char **p_end) { int result = tok_get(tok, p_start, p_end); + if (tok->fp && ferror(tok->fp)) { + clearerr(tok->fp); + result = ERRORTOKEN; + tok->done = E_IO; + } if (tok->decoding_erred) { result = ERRORTOKEN; tok->done = E_DECODE; diff --git a/Python/pythonrun.c b/Python/pythonrun.c index 5707c9f52452..2c9f55fbd1df 100644 --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -1654,6 +1654,9 @@ err_input(perrdetail *err) Py_XDECREF(tb); break; } + case E_IO: + msg = "I/O error while reading"; + break; case E_LINECONT: msg = "unexpected character after line continuation character"; break; From webhook-mailer at python.org Mon Sep 10 17:35:41 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Mon, 10 Sep 2018 21:35:41 -0000 Subject: [Python-checkins] switch descriptor howto to return value annotation (GH-7796) Message-ID: https://github.com/python/cpython/commit/28ea4c284724283265e95d1d1716c9f1dfc2d741 commit: 28ea4c284724283265e95d1d1716c9f1dfc2d741 branch: master author: NotAFile committer: Benjamin Peterson date: 2018-09-10T14:35:38-07:00 summary: switch descriptor howto to return value annotation (GH-7796) files: M Doc/howto/descriptor.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 6e4aa3e975f6..7b00d947f46b 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -48,11 +48,11 @@ a flexible set of new tools for everyday Python programs. Descriptor Protocol ------------------- -``descr.__get__(self, obj, type=None) --> value`` +``descr.__get__(self, obj, type=None) -> value`` -``descr.__set__(self, obj, value) --> None`` +``descr.__set__(self, obj, value) -> None`` -``descr.__delete__(self, obj) --> None`` +``descr.__delete__(self, obj) -> None`` That is all there is to it. Define any of these methods and an object is considered a descriptor and can override default behavior upon being looked up From webhook-mailer at python.org Mon Sep 10 19:16:12 2018 From: webhook-mailer at python.org (Zachary Ware) Date: Mon, 10 Sep 2018 23:16:12 -0000 Subject: [Python-checkins] bpo-8110: Refactor platform detection in subprocess (GH-9053) Message-ID: https://github.com/python/cpython/commit/880d42a3b247306f67837aa95e23f7c3471a30a3 commit: 880d42a3b247306f67837aa95e23f7c3471a30a3 branch: master author: Zachary Ware committer: GitHub date: 2018-09-10T16:16:08-07:00 summary: bpo-8110: Refactor platform detection in subprocess (GH-9053) Check for functionality via imports rather than checking sys.platform specifically for Windows files: A Misc/NEWS.d/next/Library/2018-09-03-23-54-35.bpo-8110.FExWI_.rst M Lib/subprocess.py diff --git a/Lib/subprocess.py b/Lib/subprocess.py index e070011d980e..1e04d5e57013 100644 --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -41,18 +41,56 @@ then returns a (exitcode, output) tuple """ -import sys -_mswindows = (sys.platform == "win32") - +import builtins +import errno import io import os import time import signal -import builtins +import sys +import threading import warnings -import errno from time import monotonic as _time + +__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput", + "getoutput", "check_output", "run", "CalledProcessError", "DEVNULL", + "SubprocessError", "TimeoutExpired", "CompletedProcess"] + # NOTE: We intentionally exclude list2cmdline as it is + # considered an internal implementation detail. issue10838. + +try: + import msvcrt + import _winapi + _mswindows = True +except ModuleNotFoundError: + _mswindows = False + import _posixsubprocess + import select + import selectors +else: + from _winapi import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP, + STD_INPUT_HANDLE, STD_OUTPUT_HANDLE, + STD_ERROR_HANDLE, SW_HIDE, + STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW, + ABOVE_NORMAL_PRIORITY_CLASS, BELOW_NORMAL_PRIORITY_CLASS, + HIGH_PRIORITY_CLASS, IDLE_PRIORITY_CLASS, + NORMAL_PRIORITY_CLASS, REALTIME_PRIORITY_CLASS, + CREATE_NO_WINDOW, DETACHED_PROCESS, + CREATE_DEFAULT_ERROR_MODE, CREATE_BREAKAWAY_FROM_JOB) + + __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP", + "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE", + "STD_ERROR_HANDLE", "SW_HIDE", + "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW", + "STARTUPINFO", + "ABOVE_NORMAL_PRIORITY_CLASS", "BELOW_NORMAL_PRIORITY_CLASS", + "HIGH_PRIORITY_CLASS", "IDLE_PRIORITY_CLASS", + "NORMAL_PRIORITY_CLASS", "REALTIME_PRIORITY_CLASS", + "CREATE_NO_WINDOW", "DETACHED_PROCESS", + "CREATE_DEFAULT_ERROR_MODE", "CREATE_BREAKAWAY_FROM_JOB"]) + + # Exception classes used by this module. class SubprocessError(Exception): pass @@ -123,9 +161,6 @@ def stdout(self, value): if _mswindows: - import threading - import msvcrt - import _winapi class STARTUPINFO: def __init__(self, *, dwFlags=0, hStdInput=None, hStdOutput=None, hStdError=None, wShowWindow=0, lpAttributeList=None): @@ -148,53 +183,6 @@ def copy(self): wShowWindow=self.wShowWindow, lpAttributeList=attr_list) -else: - import _posixsubprocess - import select - import selectors - import threading - - # When select or poll has indicated that the file is writable, - # we can write up to _PIPE_BUF bytes without risk of blocking. - # POSIX defines PIPE_BUF as >= 512. - _PIPE_BUF = getattr(select, 'PIPE_BUF', 512) - - # poll/select have the advantage of not requiring any extra file - # descriptor, contrarily to epoll/kqueue (also, they require a single - # syscall). - if hasattr(selectors, 'PollSelector'): - _PopenSelector = selectors.PollSelector - else: - _PopenSelector = selectors.SelectSelector - - -__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput", - "getoutput", "check_output", "run", "CalledProcessError", "DEVNULL", - "SubprocessError", "TimeoutExpired", "CompletedProcess"] - # NOTE: We intentionally exclude list2cmdline as it is - # considered an internal implementation detail. issue10838. - -if _mswindows: - from _winapi import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP, - STD_INPUT_HANDLE, STD_OUTPUT_HANDLE, - STD_ERROR_HANDLE, SW_HIDE, - STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW, - ABOVE_NORMAL_PRIORITY_CLASS, BELOW_NORMAL_PRIORITY_CLASS, - HIGH_PRIORITY_CLASS, IDLE_PRIORITY_CLASS, - NORMAL_PRIORITY_CLASS, REALTIME_PRIORITY_CLASS, - CREATE_NO_WINDOW, DETACHED_PROCESS, - CREATE_DEFAULT_ERROR_MODE, CREATE_BREAKAWAY_FROM_JOB) - - __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP", - "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE", - "STD_ERROR_HANDLE", "SW_HIDE", - "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW", - "STARTUPINFO", - "ABOVE_NORMAL_PRIORITY_CLASS", "BELOW_NORMAL_PRIORITY_CLASS", - "HIGH_PRIORITY_CLASS", "IDLE_PRIORITY_CLASS", - "NORMAL_PRIORITY_CLASS", "REALTIME_PRIORITY_CLASS", - "CREATE_NO_WINDOW", "DETACHED_PROCESS", - "CREATE_DEFAULT_ERROR_MODE", "CREATE_BREAKAWAY_FROM_JOB"]) class Handle(int): closed = False @@ -215,6 +203,19 @@ def __repr__(self): __del__ = Close __str__ = __repr__ +else: + # When select or poll has indicated that the file is writable, + # we can write up to _PIPE_BUF bytes without risk of blocking. + # POSIX defines PIPE_BUF as >= 512. + _PIPE_BUF = getattr(select, 'PIPE_BUF', 512) + + # poll/select have the advantage of not requiring any extra file + # descriptor, contrarily to epoll/kqueue (also, they require a single + # syscall). + if hasattr(selectors, 'PollSelector'): + _PopenSelector = selectors.PollSelector + else: + _PopenSelector = selectors.SelectSelector # This lists holds Popen instances for which the underlying process had not diff --git a/Misc/NEWS.d/next/Library/2018-09-03-23-54-35.bpo-8110.FExWI_.rst b/Misc/NEWS.d/next/Library/2018-09-03-23-54-35.bpo-8110.FExWI_.rst new file mode 100644 index 000000000000..c29ace1a0fc6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-03-23-54-35.bpo-8110.FExWI_.rst @@ -0,0 +1,2 @@ +Refactored :mod:`subprocess` to check for Windows-specific modules rather +than ``sys.platform == 'win32'``. From python-checkins at python.org Mon Sep 10 19:16:36 2018 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 10 Sep 2018 23:16:36 +0000 Subject: [Python-checkins] =?utf-8?q?test=3A_test_edit?= Message-ID: <20180910231636.1.082BA84C6077764B@mg.python.org> https://hg.python.org/test/rev/f2b4f6e298e4 changeset: 249:f2b4f6e298e4 user: Benjamin Peterson date: Mon Sep 10 16:16:33 2018 -0700 summary: test edit files: a | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/a b/a --- a/a +++ b/a @@ -10,4 +10,4 @@ snthdiueoa (obviously it should have gone this direction the first time) spam eggs 42 -hello, there +hello, there! -- Repository URL: https://hg.python.org/test From webhook-mailer at python.org Mon Sep 10 20:45:02 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 00:45:02 -0000 Subject: [Python-checkins] switch descriptor howto to return value annotation (GH-7796) Message-ID: https://github.com/python/cpython/commit/f3d00ae3be27ede2fc834d9d2b0028c9ad39f492 commit: f3d00ae3be27ede2fc834d9d2b0028c9ad39f492 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T17:44:58-07:00 summary: switch descriptor howto to return value annotation (GH-7796) (cherry picked from commit 28ea4c284724283265e95d1d1716c9f1dfc2d741) Co-authored-by: NotAFile files: M Doc/howto/descriptor.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 5e85a9aa6594..e5412583a674 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -48,11 +48,11 @@ a flexible set of new tools for everyday Python programs. Descriptor Protocol ------------------- -``descr.__get__(self, obj, type=None) --> value`` +``descr.__get__(self, obj, type=None) -> value`` -``descr.__set__(self, obj, value) --> None`` +``descr.__set__(self, obj, value) -> None`` -``descr.__delete__(self, obj) --> None`` +``descr.__delete__(self, obj) -> None`` That is all there is to it. Define any of these methods and an object is considered a descriptor and can override default behavior upon being looked up From webhook-mailer at python.org Mon Sep 10 20:46:26 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Tue, 11 Sep 2018 00:46:26 -0000 Subject: [Python-checkins] bpo-32270: Don't close stdin/out/err in pass_fds (GH-6242) Message-ID: https://github.com/python/cpython/commit/ce34410b8b67f49d8275c05d51b3ead50cf97f48 commit: ce34410b8b67f49d8275c05d51b3ead50cf97f48 branch: master author: Gregory P. Smith committer: GitHub date: 2018-09-10T17:46:22-07:00 summary: bpo-32270: Don't close stdin/out/err in pass_fds (GH-6242) When subprocess.Popen() stdin= stdout= or stderr= handles are specified and appear in pass_fds=, don't close the original fds after dup'ing them. This implementation and unittest primarily came from @izbyshev (see the PR) See also https://github.com/izbyshev/cpython/commit/b89b52f28490b69142d5c061604b3a3989cec66c This also removes the old manual p2cread, c2pwrite, and errwrite closing logic as inheritable flags and _close_open_fds takes care of that properly today without special treatment. This code is within child_exec() where it is the only thread so there is no race condition between the dup and _Py_set_inheritable_async_safe call. files: A Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst M Lib/test/test_subprocess.py M Modules/_posixsubprocess.c diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py index 4719773b67b7..8419061b2a90 100644 --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -2529,6 +2529,36 @@ def test_pass_fds_inheritable(self): self.assertEqual(os.get_inheritable(inheritable), True) self.assertEqual(os.get_inheritable(non_inheritable), False) + + # bpo-32270: Ensure that descriptors specified in pass_fds + # are inherited even if they are used in redirections. + # Contributed by @izbyshev. + def test_pass_fds_redirected(self): + """Regression test for https://bugs.python.org/issue32270.""" + fd_status = support.findfile("fd_status.py", subdir="subprocessdata") + pass_fds = [] + for _ in range(2): + fd = os.open(os.devnull, os.O_RDWR) + self.addCleanup(os.close, fd) + pass_fds.append(fd) + + stdout_r, stdout_w = os.pipe() + self.addCleanup(os.close, stdout_r) + self.addCleanup(os.close, stdout_w) + pass_fds.insert(1, stdout_w) + + with subprocess.Popen([sys.executable, fd_status], + stdin=pass_fds[0], + stdout=pass_fds[1], + stderr=pass_fds[2], + close_fds=True, + pass_fds=pass_fds): + output = os.read(stdout_r, 1024) + fds = {int(num) for num in output.split(b',')} + + self.assertEqual(fds, {0, 1, 2} | frozenset(pass_fds), f"output={output!a}") + + def test_stdout_stdin_are_single_inout_fd(self): with io.open(os.devnull, "r+") as inout: p = subprocess.Popen([sys.executable, "-c", "import sys; sys.exit(0)"], diff --git a/Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst b/Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst new file mode 100644 index 000000000000..83f68624c1be --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst @@ -0,0 +1,2 @@ +The subprocess module no longer mistakenly closes redirected fds even when +they were in pass_fds when outside of the default {0, 1, 2} set. diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index 0150fcb0970c..aeb10f9ecfe4 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -422,10 +422,20 @@ child_exec(char *const exec_array[], /* When duping fds, if there arises a situation where one of the fds is either 0, 1 or 2, it is possible that it is overwritten (#12607). */ - if (c2pwrite == 0) + if (c2pwrite == 0) { POSIX_CALL(c2pwrite = dup(c2pwrite)); - while (errwrite == 0 || errwrite == 1) + /* issue32270 */ + if (_Py_set_inheritable_async_safe(c2pwrite, 0, NULL) < 0) { + goto error; + } + } + while (errwrite == 0 || errwrite == 1) { POSIX_CALL(errwrite = dup(errwrite)); + /* issue32270 */ + if (_Py_set_inheritable_async_safe(errwrite, 0, NULL) < 0) { + goto error; + } + } /* Dup fds for child. dup2() removes the CLOEXEC flag but we must do it ourselves if dup2() @@ -451,14 +461,8 @@ child_exec(char *const exec_array[], else if (errwrite != -1) POSIX_CALL(dup2(errwrite, 2)); /* stderr */ - /* Close pipe fds. Make sure we don't close the same fd more than */ - /* once, or standard fds. */ - if (p2cread > 2) - POSIX_CALL(close(p2cread)); - if (c2pwrite > 2 && c2pwrite != p2cread) - POSIX_CALL(close(c2pwrite)); - if (errwrite != c2pwrite && errwrite != p2cread && errwrite > 2) - POSIX_CALL(close(errwrite)); + /* We no longer manually close p2cread, c2pwrite, and errwrite here as + * _close_open_fds takes care when it is not already non-inheritable. */ if (cwd) POSIX_CALL(chdir(cwd)); From webhook-mailer at python.org Mon Sep 10 20:54:42 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Tue, 11 Sep 2018 00:54:42 -0000 Subject: [Python-checkins] Remove obsolete comment about latin-1 in `normalize_encoding` (GH-8739) Message-ID: https://github.com/python/cpython/commit/ed2e9ab804606019f167ef914bde43bc135c725f commit: ed2e9ab804606019f167ef914bde43bc135c725f branch: master author: Anthony Sottile committer: Gregory P. Smith date: 2018-09-10T17:54:37-07:00 summary: Remove obsolete comment about latin-1 in `normalize_encoding` (GH-8739) This docstring has drifted since python2: https://github.com/python/cpython/blob/ca079a3ea30098aff3197c559a0e32d42dda6d84/Lib/encodings/__init__.py#L68 files: M Lib/encodings/__init__.py diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py index 025b7a8da3de..d737d5339dce 100644 --- a/Lib/encodings/__init__.py +++ b/Lib/encodings/__init__.py @@ -49,8 +49,7 @@ def normalize_encoding(encoding): collapsed and replaced with a single underscore, e.g. ' -;#' becomes '_'. Leading and trailing underscores are removed. - Note that encoding names should be ASCII only; if they do use - non-ASCII characters, these must be Latin-1 compatible. + Note that encoding names should be ASCII only. """ if isinstance(encoding, bytes): From webhook-mailer at python.org Mon Sep 10 21:04:38 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Tue, 11 Sep 2018 01:04:38 -0000 Subject: [Python-checkins] Lib/test/support: fix typo in docstring (GH-8506) Message-ID: https://github.com/python/cpython/commit/e578fa162eef1cef27bc7dbf7ca46ab6dc52688a commit: e578fa162eef1cef27bc7dbf7ca46ab6dc52688a branch: master author: Daniel Hahler committer: Gregory P. Smith date: 2018-09-10T18:04:33-07:00 summary: Lib/test/support: fix typo in docstring (GH-8506) files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 13b60f7e8153..de997b253a69 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1955,8 +1955,8 @@ def set_match_tests(patterns): def match_test_regex(test_id): if regex_match(test_id): - # The regex matchs the whole identifier like - # 'test.test_os.FileTests.test_access' + # The regex matches the whole identifier, for example + # 'test.test_os.FileTests.test_access'. return True else: # Try to match parts of the test identifier. From webhook-mailer at python.org Mon Sep 10 21:07:23 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Tue, 11 Sep 2018 01:07:23 -0000 Subject: [Python-checkins] Use bytes.hex instead of binascii.hexlify in pbkdf2_hmac example (GH-8420) Message-ID: https://github.com/python/cpython/commit/959625b5a56545f1908a7002fe11eb4f0411b780 commit: 959625b5a56545f1908a7002fe11eb4f0411b780 branch: master author: Ville Skytt? committer: Gregory P. Smith date: 2018-09-10T18:07:19-07:00 summary: Use bytes.hex instead of binascii.hexlify in pbkdf2_hmac example (GH-8420) files: M Doc/library/hashlib.rst diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst index baf6b0af242c..4a8d7058c68f 100644 --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -244,10 +244,10 @@ include a `salt `_. *dklen* is the length of the derived key. If *dklen* is ``None`` then the digest size of the hash algorithm *hash_name* is used, e.g. 64 for SHA-512. - >>> import hashlib, binascii + >>> import hashlib >>> dk = hashlib.pbkdf2_hmac('sha256', b'password', b'salt', 100000) - >>> binascii.hexlify(dk) - b'0394a2ede332c9a13eb82e9b24631604c31df978b4e2f0fbd2c549944f9d79a5' + >>> dk.hex() + '0394a2ede332c9a13eb82e9b24631604c31df978b4e2f0fbd2c549944f9d79a5' .. versionadded:: 3.4 From webhook-mailer at python.org Mon Sep 10 21:13:11 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Tue, 11 Sep 2018 01:13:11 -0000 Subject: [Python-checkins] bpo-33460: remove ellipsis that look like continuation prompts (GH-7851) Message-ID: https://github.com/python/cpython/commit/f019579828ed62653e2d41c95278308fa076ccaf commit: f019579828ed62653e2d41c95278308fa076ccaf branch: master author: Lew Kurtz <37632626+lew18 at users.noreply.github.com> committer: Gregory P. Smith date: 2018-09-10T18:13:08-07:00 summary: bpo-33460: remove ellipsis that look like continuation prompts (GH-7851) Remove ellipsis that look like continuation prompts, has a side benefit of putting rest of error message in proper text color. files: A Misc/NEWS.d/next/Documentation/2018-06-22-08-38-29.bpo-33460.kHt4D0.rst M Doc/tutorial/introduction.rst diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index 22a209c10333..e68c9b10d03e 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -223,10 +223,14 @@ This only works with two literals though, not with variables or expressions:: >>> prefix = 'Py' >>> prefix 'thon' # can't concatenate a variable and a string literal - ... + File "", line 1 + prefix 'thon' + ^ SyntaxError: invalid syntax >>> ('un' * 3) 'ium' - ... + File "", line 1 + ('un' * 3) 'ium' + ^ SyntaxError: invalid syntax If you want to concatenate variables or a variable and a literal, use ``+``:: @@ -320,10 +324,12 @@ Python strings cannot be changed --- they are :term:`immutable`. Therefore, assigning to an indexed position in the string results in an error:: >>> word[0] = 'J' - ... + Traceback (most recent call last): + File "", line 1, in TypeError: 'str' object does not support item assignment >>> word[2:] = 'py' - ... + Traceback (most recent call last): + File "", line 1, in TypeError: 'str' object does not support item assignment If you need a different string, you should create a new one:: diff --git a/Misc/NEWS.d/next/Documentation/2018-06-22-08-38-29.bpo-33460.kHt4D0.rst b/Misc/NEWS.d/next/Documentation/2018-06-22-08-38-29.bpo-33460.kHt4D0.rst new file mode 100644 index 000000000000..6ee63b08f682 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-06-22-08-38-29.bpo-33460.kHt4D0.rst @@ -0,0 +1 @@ +replaced ellipsis with correct error codes in tutorial chapter 3. From webhook-mailer at python.org Mon Sep 10 21:15:59 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Tue, 11 Sep 2018 01:15:59 -0000 Subject: [Python-checkins] bpo-33487: improve BZ2File Deprecation and documentation. (GH-6785) Message-ID: https://github.com/python/cpython/commit/ffa198c642f9c67b84ef192bf0f7016c4249e570 commit: ffa198c642f9c67b84ef192bf0f7016c4249e570 branch: master author: Matthias Bussonnier committer: Gregory P. Smith date: 2018-09-10T18:15:56-07:00 summary: bpo-33487: improve BZ2File Deprecation and documentation. (GH-6785) Emit warning when None passed explicitly, list Python version since deprecation in warning message and docs. files: A Misc/NEWS.d/next/Documentation/2018-05-13-14-44-30.bpo-33487.iLDzFb.rst M Doc/library/bz2.rst M Lib/bz2.py diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst index d5f622515a40..946cc67dd301 100644 --- a/Doc/library/bz2.rst +++ b/Doc/library/bz2.rst @@ -81,7 +81,7 @@ All of the classes in this module may safely be accessed from multiple threads. If *filename* is a file object (rather than an actual file name), a mode of ``'w'`` does not truncate the file, and is instead equivalent to ``'a'``. - The *buffering* argument is ignored. Its use is deprecated. + The *buffering* argument is ignored. Its use is deprecated since Python 3.0. If *mode* is ``'w'`` or ``'a'``, *compresslevel* can be a number between ``1`` and ``9`` specifying the level of compression: ``1`` produces the @@ -109,6 +109,10 @@ All of the classes in this module may safely be accessed from multiple threads. .. versionadded:: 3.3 + + .. deprecated:: 3.0 + The keyword argument *buffering* was deprecated and is now ignored. + .. versionchanged:: 3.1 Support for the :keyword:`with` statement was added. diff --git a/Lib/bz2.py b/Lib/bz2.py index 3924aaed1678..3ab099147190 100644 --- a/Lib/bz2.py +++ b/Lib/bz2.py @@ -24,6 +24,8 @@ # Value 2 no longer used _MODE_WRITE = 3 +_sentinel = object() + class BZ2File(_compression.BaseStream): @@ -36,7 +38,7 @@ class BZ2File(_compression.BaseStream): returned as bytes, and data to be written should be given as bytes. """ - def __init__(self, filename, mode="r", buffering=None, compresslevel=9): + def __init__(self, filename, mode="r", buffering=_sentinel, compresslevel=9): """Open a bzip2-compressed file. If filename is a str, bytes, or PathLike object, it gives the @@ -47,7 +49,7 @@ def __init__(self, filename, mode="r", buffering=None, compresslevel=9): 'x' for creating exclusively, or 'a' for appending. These can equivalently be given as 'rb', 'wb', 'xb', and 'ab'. - buffering is ignored. Its use is deprecated. + buffering is ignored since Python 3.0. Its use is deprecated. If mode is 'w', 'x' or 'a', compresslevel can be a number between 1 and 9 specifying the level of compression: 1 produces the least @@ -63,9 +65,11 @@ def __init__(self, filename, mode="r", buffering=None, compresslevel=9): self._closefp = False self._mode = _MODE_CLOSED - if buffering is not None: - warnings.warn("Use of 'buffering' argument is deprecated", - DeprecationWarning) + if buffering is not _sentinel: + warnings.warn("Use of 'buffering' argument is deprecated and ignored" + "since Python 3.0.", + DeprecationWarning, + stacklevel=2) if not (1 <= compresslevel <= 9): raise ValueError("compresslevel must be between 1 and 9") diff --git a/Misc/NEWS.d/next/Documentation/2018-05-13-14-44-30.bpo-33487.iLDzFb.rst b/Misc/NEWS.d/next/Documentation/2018-05-13-14-44-30.bpo-33487.iLDzFb.rst new file mode 100644 index 000000000000..0439d983d1bc --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-05-13-14-44-30.bpo-33487.iLDzFb.rst @@ -0,0 +1,3 @@ +BZ2file now emit a DeprecationWarning when buffering=None is passed, the +deprecation message and documentation also now explicitely state it is +deprecated since 3.0. From webhook-mailer at python.org Mon Sep 10 21:43:11 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Tue, 11 Sep 2018 01:43:11 -0000 Subject: [Python-checkins] Fix missing line from example shell session (GH-9143) Message-ID: https://github.com/python/cpython/commit/2064bb6d576ff7016d59318038779f428b0f0f3f commit: 2064bb6d576ff7016d59318038779f428b0f0f3f branch: master author: Raymond Hettinger committer: GitHub date: 2018-09-10T18:43:08-07:00 summary: Fix missing line from example shell session (GH-9143) files: M Doc/library/collections.rst diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index 2a83d3037277..495cfc2c234f 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -198,6 +198,7 @@ updates keys found deeper in the chain:: >>> d['lion'] = 'orange' # update an existing key two levels down >>> d['snake'] = 'red' # new keys get added to the topmost dict >>> del d['elephant'] # remove an existing key one level down + >>> d # display result DeepChainMap({'zebra': 'black', 'snake': 'red'}, {}, {'lion': 'orange'}) From webhook-mailer at python.org Mon Sep 10 21:49:42 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 01:49:42 -0000 Subject: [Python-checkins] bpo-33460: remove ellipsis that look like continuation prompts (GH-7851) Message-ID: https://github.com/python/cpython/commit/037582eb7f4d61604ef2dcc72f896117588eeb3c commit: 037582eb7f4d61604ef2dcc72f896117588eeb3c branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T18:49:39-07:00 summary: bpo-33460: remove ellipsis that look like continuation prompts (GH-7851) Remove ellipsis that look like continuation prompts, has a side benefit of putting rest of error message in proper text color. (cherry picked from commit f019579828ed62653e2d41c95278308fa076ccaf) Co-authored-by: Lew Kurtz <37632626+lew18 at users.noreply.github.com> files: A Misc/NEWS.d/next/Documentation/2018-06-22-08-38-29.bpo-33460.kHt4D0.rst M Doc/tutorial/introduction.rst diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index 22a209c10333..e68c9b10d03e 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -223,10 +223,14 @@ This only works with two literals though, not with variables or expressions:: >>> prefix = 'Py' >>> prefix 'thon' # can't concatenate a variable and a string literal - ... + File "", line 1 + prefix 'thon' + ^ SyntaxError: invalid syntax >>> ('un' * 3) 'ium' - ... + File "", line 1 + ('un' * 3) 'ium' + ^ SyntaxError: invalid syntax If you want to concatenate variables or a variable and a literal, use ``+``:: @@ -320,10 +324,12 @@ Python strings cannot be changed --- they are :term:`immutable`. Therefore, assigning to an indexed position in the string results in an error:: >>> word[0] = 'J' - ... + Traceback (most recent call last): + File "", line 1, in TypeError: 'str' object does not support item assignment >>> word[2:] = 'py' - ... + Traceback (most recent call last): + File "", line 1, in TypeError: 'str' object does not support item assignment If you need a different string, you should create a new one:: diff --git a/Misc/NEWS.d/next/Documentation/2018-06-22-08-38-29.bpo-33460.kHt4D0.rst b/Misc/NEWS.d/next/Documentation/2018-06-22-08-38-29.bpo-33460.kHt4D0.rst new file mode 100644 index 000000000000..6ee63b08f682 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-06-22-08-38-29.bpo-33460.kHt4D0.rst @@ -0,0 +1 @@ +replaced ellipsis with correct error codes in tutorial chapter 3. From webhook-mailer at python.org Mon Sep 10 22:13:05 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Tue, 11 Sep 2018 02:13:05 -0000 Subject: [Python-checkins] Fix missing line from example shell session (GH-9143) (GH-9155) Message-ID: https://github.com/python/cpython/commit/6df2005b3999e6087dc524bfdf27b0dc5896d172 commit: 6df2005b3999e6087dc524bfdf27b0dc5896d172 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2018-09-10T19:13:02-07:00 summary: Fix missing line from example shell session (GH-9143) (GH-9155) (cherry picked from commit 2064bb6d576ff7016d59318038779f428b0f0f3f) Co-authored-by: Raymond Hettinger files: M Doc/library/collections.rst diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index 2a83d3037277..495cfc2c234f 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -198,6 +198,7 @@ updates keys found deeper in the chain:: >>> d['lion'] = 'orange' # update an existing key two levels down >>> d['snake'] = 'red' # new keys get added to the topmost dict >>> del d['elephant'] # remove an existing key one level down + >>> d # display result DeepChainMap({'zebra': 'black', 'snake': 'red'}, {}, {'lion': 'orange'}) From webhook-mailer at python.org Mon Sep 10 22:13:26 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Tue, 11 Sep 2018 02:13:26 -0000 Subject: [Python-checkins] Fix missing line from example shell session (GH-9143) (GH-9156) Message-ID: https://github.com/python/cpython/commit/3cb90d1a0b32dc293de8adb52b69cdc947a18c46 commit: 3cb90d1a0b32dc293de8adb52b69cdc947a18c46 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2018-09-10T19:13:23-07:00 summary: Fix missing line from example shell session (GH-9143) (GH-9156) (cherry picked from commit 2064bb6d576ff7016d59318038779f428b0f0f3f) Co-authored-by: Raymond Hettinger files: M Doc/library/collections.rst diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index 82ba05737413..25687348d16d 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -198,6 +198,7 @@ updates keys found deeper in the chain:: >>> d['lion'] = 'orange' # update an existing key two levels down >>> d['snake'] = 'red' # new keys get added to the topmost dict >>> del d['elephant'] # remove an existing key one level down + >>> d # display result DeepChainMap({'zebra': 'black', 'snake': 'red'}, {}, {'lion': 'orange'}) From webhook-mailer at python.org Tue Sep 11 00:04:11 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 04:04:11 -0000 Subject: [Python-checkins] bpo-34625: Update vendorized expat version to 2.2.6. (GH-9150) Message-ID: https://github.com/python/cpython/commit/5033aa77aacaa5505636f150e8d54baac5bdca9c commit: 5033aa77aacaa5505636f150e8d54baac5bdca9c branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-10T21:04:00-07:00 summary: bpo-34625: Update vendorized expat version to 2.2.6. (GH-9150) files: A Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst M Modules/expat/expat.h M Modules/expat/siphash.h M Modules/expat/xmlparse.c M Modules/expat/xmltok_impl.c diff --git a/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst b/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst new file mode 100644 index 000000000000..0747ec54470f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst @@ -0,0 +1 @@ +Update vendorized expat library version to 2.2.6. diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index 1f608c02d6fd..174c3fafda3f 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -264,7 +264,7 @@ XML_ParserCreate_MM(const XML_Char *encoding, const XML_Char *namespaceSeparator); /* Prepare a parser object to be re-used. This is particularly - valuable when memory allocation overhead is disproportionatly high, + valuable when memory allocation overhead is disproportionately high, such as when a large number of small documnents need to be parsed. All handlers are cleared from the parser, except for the unknownEncodingHandler. The parser's external state is re-initialized @@ -1076,7 +1076,7 @@ XML_GetFeatureList(void); */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 2 -#define XML_MICRO_VERSION 5 +#define XML_MICRO_VERSION 6 #ifdef __cplusplus } diff --git a/Modules/expat/siphash.h b/Modules/expat/siphash.h index 581872df7b49..4d6786d7839a 100644 --- a/Modules/expat/siphash.h +++ b/Modules/expat/siphash.h @@ -11,6 +11,12 @@ * -------------------------------------------------------------------------- * HISTORY: * + * 2018-07-08 (Anton Maklakov) + * - Add "fall through" markers for GCC's -Wimplicit-fallthrough + * + * 2017-11-03 (Sebastian Pipping) + * - Hide sip_tobin and sip_binof unless SIPHASH_TOBIN macro is defined + * * 2017-07-25 (Vadim Zeitlin) * - Fix use of SIPHASH_MAIN macro * @@ -151,6 +157,8 @@ static struct sipkey *sip_tokey(struct sipkey *key, const void *src) { } /* sip_tokey() */ +#ifdef SIPHASH_TOBIN + #define sip_binof(v) sip_tobin((unsigned char[8]){ 0 }, (v)) static void *sip_tobin(void *dst, uint64_t u64) { @@ -158,6 +166,8 @@ static void *sip_tobin(void *dst, uint64_t u64) { return dst; } /* sip_tobin() */ +#endif /* SIPHASH_TOBIN */ + static void sip_round(struct siphash *H, const int rounds) { int i; @@ -231,12 +241,19 @@ static uint64_t sip24_final(struct siphash *H) { switch (left) { case 7: b |= (uint64_t)H->buf[6] << 48; + /* fall through */ case 6: b |= (uint64_t)H->buf[5] << 40; + /* fall through */ case 5: b |= (uint64_t)H->buf[4] << 32; + /* fall through */ case 4: b |= (uint64_t)H->buf[3] << 24; + /* fall through */ case 3: b |= (uint64_t)H->buf[2] << 16; + /* fall through */ case 2: b |= (uint64_t)H->buf[1] << 8; + /* fall through */ case 1: b |= (uint64_t)H->buf[0] << 0; + /* fall through */ case 0: break; } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 90a237f30eb8..c4f3ffc215c9 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 4b74aa710b4ed5ce464b0ce544852cb47bf905c85a49c7bae2749f5885cb966d (2.2.5+) +/* 19ac4776051591216f1874e34ee99b6a43a3784c8bd7d70efeb9258dd22b906a (2.2.6+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -161,6 +161,9 @@ typedef char ICHAR; /* Round up n to be a multiple of sz, where sz is a power of 2. */ #define ROUND_UP(n, sz) (((n) + ((sz) - 1)) & ~((sz) - 1)) +/* Do safe (NULL-aware) pointer arithmetic */ +#define EXPAT_SAFE_PTR_DIFF(p, q) (((p) && (q)) ? ((p) - (q)) : 0) + /* Handle the case where memmove() doesn't exist. */ #ifndef HAVE_MEMMOVE #ifdef HAVE_BCOPY @@ -1820,6 +1823,7 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) parser->m_errorCode = XML_ERROR_NO_MEMORY; return XML_STATUS_ERROR; } + /* fall through */ default: parser->m_parsingStatus.parsing = XML_PARSING; } @@ -1969,6 +1973,7 @@ XML_ParseBuffer(XML_Parser parser, int len, int isFinal) parser->m_errorCode = XML_ERROR_NO_MEMORY; return XML_STATUS_ERROR; } + /* fall through */ default: parser->m_parsingStatus.parsing = XML_PARSING; } @@ -2026,39 +2031,46 @@ XML_GetBuffer(XML_Parser parser, int len) default: ; } - if (len > parser->m_bufferLim - parser->m_bufferEnd) { + if (len > EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_bufferEnd)) { #ifdef XML_CONTEXT_BYTES int keep; #endif /* defined XML_CONTEXT_BYTES */ /* Do not invoke signed arithmetic overflow: */ - int neededSize = (int) ((unsigned)len + (unsigned)(parser->m_bufferEnd - parser->m_bufferPtr)); + int neededSize = (int) ((unsigned)len + + (unsigned)EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, + parser->m_bufferPtr)); if (neededSize < 0) { parser->m_errorCode = XML_ERROR_NO_MEMORY; return NULL; } #ifdef XML_CONTEXT_BYTES - keep = (int)(parser->m_bufferPtr - parser->m_buffer); + keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; neededSize += keep; #endif /* defined XML_CONTEXT_BYTES */ - if (neededSize <= parser->m_bufferLim - parser->m_buffer) { + if (neededSize <= EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_buffer)) { #ifdef XML_CONTEXT_BYTES - if (keep < parser->m_bufferPtr - parser->m_buffer) { - int offset = (int)(parser->m_bufferPtr - parser->m_buffer) - keep; + if (keep < EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer)) { + int offset = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer) - keep; + /* The buffer pointers cannot be NULL here; we have at least some bytes in the buffer */ memmove(parser->m_buffer, &parser->m_buffer[offset], parser->m_bufferEnd - parser->m_bufferPtr + keep); parser->m_bufferEnd -= offset; parser->m_bufferPtr -= offset; } #else - memmove(parser->m_buffer, parser->m_bufferPtr, parser->m_bufferEnd - parser->m_bufferPtr); - parser->m_bufferEnd = parser->m_buffer + (parser->m_bufferEnd - parser->m_bufferPtr); - parser->m_bufferPtr = parser->m_buffer; + if (parser->m_buffer && parser->m_bufferPtr) { + memmove(parser->m_buffer, parser->m_bufferPtr, + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr)); + parser->m_bufferEnd = parser->m_buffer + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr); + parser->m_bufferPtr = parser->m_buffer; + } #endif /* not defined XML_CONTEXT_BYTES */ } else { char *newBuf; - int bufferSize = (int)(parser->m_bufferLim - parser->m_bufferPtr); + int bufferSize = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_bufferPtr); if (bufferSize == 0) bufferSize = INIT_BUFFER_SIZE; do { @@ -2077,25 +2089,34 @@ XML_GetBuffer(XML_Parser parser, int len) parser->m_bufferLim = newBuf + bufferSize; #ifdef XML_CONTEXT_BYTES if (parser->m_bufferPtr) { - int keep = (int)(parser->m_bufferPtr - parser->m_buffer); + int keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; - memcpy(newBuf, &parser->m_bufferPtr[-keep], parser->m_bufferEnd - parser->m_bufferPtr + keep); + memcpy(newBuf, &parser->m_bufferPtr[-keep], + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr) + keep); FREE(parser, parser->m_buffer); parser->m_buffer = newBuf; - parser->m_bufferEnd = parser->m_buffer + (parser->m_bufferEnd - parser->m_bufferPtr) + keep; + parser->m_bufferEnd = parser->m_buffer + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr) + keep; parser->m_bufferPtr = parser->m_buffer + keep; } else { - parser->m_bufferEnd = newBuf + (parser->m_bufferEnd - parser->m_bufferPtr); + /* This must be a brand new buffer with no data in it yet */ + parser->m_bufferEnd = newBuf; parser->m_bufferPtr = parser->m_buffer = newBuf; } #else if (parser->m_bufferPtr) { - memcpy(newBuf, parser->m_bufferPtr, parser->m_bufferEnd - parser->m_bufferPtr); + memcpy(newBuf, parser->m_bufferPtr, + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr)); FREE(parser, parser->m_buffer); + parser->m_bufferEnd = newBuf + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr); + } + else { + /* This must be a brand new buffer with no data in it yet */ + parser->m_bufferEnd = newBuf; } - parser->m_bufferEnd = newBuf + (parser->m_bufferEnd - parser->m_bufferPtr); parser->m_bufferPtr = parser->m_buffer = newBuf; #endif /* not defined XML_CONTEXT_BYTES */ } @@ -2908,9 +2929,11 @@ doContent(XML_Parser parser, poolClear(&parser->m_tempPool); freeBindings(parser, bindings); } - if ((parser->m_tagLevel == 0) && - !((parser->m_parsingStatus.parsing == XML_FINISHED) || (parser->m_parsingStatus.parsing == XML_SUSPENDED))) { - return epilogProcessor(parser, next, end, nextPtr); + if ((parser->m_tagLevel == 0) && (parser->m_parsingStatus.parsing != XML_FINISHED)) { + if (parser->m_parsingStatus.parsing == XML_SUSPENDED) + parser->m_processor = epilogProcessor; + else + return epilogProcessor(parser, next, end, nextPtr); } break; case XML_TOK_END_TAG: @@ -4746,8 +4769,8 @@ doProlog(XML_Parser parser, return XML_ERROR_NO_MEMORY; parser->m_declEntity->publicId = NULL; } - /* fall through */ #endif /* XML_DTD */ + /* fall through */ case XML_ROLE_ENTITY_SYSTEM_ID: if (dtd->keepProcessing && parser->m_declEntity) { parser->m_declEntity->systemId = poolStoreString(&dtd->pool, enc, @@ -6643,7 +6666,6 @@ hash(XML_Parser parser, KEY s) { struct siphash state; struct sipkey key; - (void)sip_tobin; (void)sip24_valid; copy_salt_to_sipkey(parser, &key); sip24_init(&state, &key); diff --git a/Modules/expat/xmltok_impl.c b/Modules/expat/xmltok_impl.c index 0403dd3d09a0..4d9ae7dc3896 100644 --- a/Modules/expat/xmltok_impl.c +++ b/Modules/expat/xmltok_impl.c @@ -74,6 +74,7 @@ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ + /* fall through */ \ case BT_NMSTRT: \ case BT_HEX: \ case BT_DIGIT: \ @@ -102,6 +103,7 @@ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ + /* fall through */ \ case BT_NMSTRT: \ case BT_HEX: \ ptr += MINBPC(enc); \ @@ -602,7 +604,7 @@ PREFIX(scanAtts)(const ENCODING *enc, const char *ptr, const char *end, return XML_TOK_INVALID; } } - /* fall through */ + /* fall through */ case BT_EQUALS: { int open; @@ -1442,6 +1444,7 @@ PREFIX(isPublicId)(const ENCODING *enc, const char *ptr, const char *end, case BT_NMSTRT: if (!(BYTE_TO_ASCII(enc, ptr) & ~0x7f)) break; + /* fall through */ default: switch (BYTE_TO_ASCII(enc, ptr)) { case 0x24: /* $ */ @@ -1659,8 +1662,8 @@ PREFIX(nameMatchesAscii)(const ENCODING *UNUSED_P(enc), const char *ptr1, { for (; *ptr2; ptr1 += MINBPC(enc), ptr2++) { if (end1 - ptr1 < MINBPC(enc)) { - /* This line cannot be executed. THe incoming data has already - * been tokenized once, so imcomplete characters like this have + /* This line cannot be executed. The incoming data has already + * been tokenized once, so incomplete characters like this have * already been eliminated from the input. Retaining the * paranoia check is still valuable, however. */ From webhook-mailer at python.org Tue Sep 11 00:26:46 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 04:26:46 -0000 Subject: [Python-checkins] bpo-34625: Update vendorized expat version to 2.2.6. (GH-9150) Message-ID: https://github.com/python/cpython/commit/7a501def4fb98a1d6f15e4a5a006141d6027f948 commit: 7a501def4fb98a1d6f15e4a5a006141d6027f948 branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T21:26:42-07:00 summary: bpo-34625: Update vendorized expat version to 2.2.6. (GH-9150) (cherry picked from commit 5033aa77aacaa5505636f150e8d54baac5bdca9c) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst M Modules/expat/expat.h M Modules/expat/siphash.h M Modules/expat/xmlparse.c M Modules/expat/xmltok_impl.c diff --git a/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst b/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst new file mode 100644 index 000000000000..0747ec54470f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst @@ -0,0 +1 @@ +Update vendorized expat library version to 2.2.6. diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index 1f608c02d6fd..174c3fafda3f 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -264,7 +264,7 @@ XML_ParserCreate_MM(const XML_Char *encoding, const XML_Char *namespaceSeparator); /* Prepare a parser object to be re-used. This is particularly - valuable when memory allocation overhead is disproportionatly high, + valuable when memory allocation overhead is disproportionately high, such as when a large number of small documnents need to be parsed. All handlers are cleared from the parser, except for the unknownEncodingHandler. The parser's external state is re-initialized @@ -1076,7 +1076,7 @@ XML_GetFeatureList(void); */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 2 -#define XML_MICRO_VERSION 5 +#define XML_MICRO_VERSION 6 #ifdef __cplusplus } diff --git a/Modules/expat/siphash.h b/Modules/expat/siphash.h index 581872df7b49..4d6786d7839a 100644 --- a/Modules/expat/siphash.h +++ b/Modules/expat/siphash.h @@ -11,6 +11,12 @@ * -------------------------------------------------------------------------- * HISTORY: * + * 2018-07-08 (Anton Maklakov) + * - Add "fall through" markers for GCC's -Wimplicit-fallthrough + * + * 2017-11-03 (Sebastian Pipping) + * - Hide sip_tobin and sip_binof unless SIPHASH_TOBIN macro is defined + * * 2017-07-25 (Vadim Zeitlin) * - Fix use of SIPHASH_MAIN macro * @@ -151,6 +157,8 @@ static struct sipkey *sip_tokey(struct sipkey *key, const void *src) { } /* sip_tokey() */ +#ifdef SIPHASH_TOBIN + #define sip_binof(v) sip_tobin((unsigned char[8]){ 0 }, (v)) static void *sip_tobin(void *dst, uint64_t u64) { @@ -158,6 +166,8 @@ static void *sip_tobin(void *dst, uint64_t u64) { return dst; } /* sip_tobin() */ +#endif /* SIPHASH_TOBIN */ + static void sip_round(struct siphash *H, const int rounds) { int i; @@ -231,12 +241,19 @@ static uint64_t sip24_final(struct siphash *H) { switch (left) { case 7: b |= (uint64_t)H->buf[6] << 48; + /* fall through */ case 6: b |= (uint64_t)H->buf[5] << 40; + /* fall through */ case 5: b |= (uint64_t)H->buf[4] << 32; + /* fall through */ case 4: b |= (uint64_t)H->buf[3] << 24; + /* fall through */ case 3: b |= (uint64_t)H->buf[2] << 16; + /* fall through */ case 2: b |= (uint64_t)H->buf[1] << 8; + /* fall through */ case 1: b |= (uint64_t)H->buf[0] << 0; + /* fall through */ case 0: break; } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 90a237f30eb8..c4f3ffc215c9 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 4b74aa710b4ed5ce464b0ce544852cb47bf905c85a49c7bae2749f5885cb966d (2.2.5+) +/* 19ac4776051591216f1874e34ee99b6a43a3784c8bd7d70efeb9258dd22b906a (2.2.6+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -161,6 +161,9 @@ typedef char ICHAR; /* Round up n to be a multiple of sz, where sz is a power of 2. */ #define ROUND_UP(n, sz) (((n) + ((sz) - 1)) & ~((sz) - 1)) +/* Do safe (NULL-aware) pointer arithmetic */ +#define EXPAT_SAFE_PTR_DIFF(p, q) (((p) && (q)) ? ((p) - (q)) : 0) + /* Handle the case where memmove() doesn't exist. */ #ifndef HAVE_MEMMOVE #ifdef HAVE_BCOPY @@ -1820,6 +1823,7 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) parser->m_errorCode = XML_ERROR_NO_MEMORY; return XML_STATUS_ERROR; } + /* fall through */ default: parser->m_parsingStatus.parsing = XML_PARSING; } @@ -1969,6 +1973,7 @@ XML_ParseBuffer(XML_Parser parser, int len, int isFinal) parser->m_errorCode = XML_ERROR_NO_MEMORY; return XML_STATUS_ERROR; } + /* fall through */ default: parser->m_parsingStatus.parsing = XML_PARSING; } @@ -2026,39 +2031,46 @@ XML_GetBuffer(XML_Parser parser, int len) default: ; } - if (len > parser->m_bufferLim - parser->m_bufferEnd) { + if (len > EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_bufferEnd)) { #ifdef XML_CONTEXT_BYTES int keep; #endif /* defined XML_CONTEXT_BYTES */ /* Do not invoke signed arithmetic overflow: */ - int neededSize = (int) ((unsigned)len + (unsigned)(parser->m_bufferEnd - parser->m_bufferPtr)); + int neededSize = (int) ((unsigned)len + + (unsigned)EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, + parser->m_bufferPtr)); if (neededSize < 0) { parser->m_errorCode = XML_ERROR_NO_MEMORY; return NULL; } #ifdef XML_CONTEXT_BYTES - keep = (int)(parser->m_bufferPtr - parser->m_buffer); + keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; neededSize += keep; #endif /* defined XML_CONTEXT_BYTES */ - if (neededSize <= parser->m_bufferLim - parser->m_buffer) { + if (neededSize <= EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_buffer)) { #ifdef XML_CONTEXT_BYTES - if (keep < parser->m_bufferPtr - parser->m_buffer) { - int offset = (int)(parser->m_bufferPtr - parser->m_buffer) - keep; + if (keep < EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer)) { + int offset = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer) - keep; + /* The buffer pointers cannot be NULL here; we have at least some bytes in the buffer */ memmove(parser->m_buffer, &parser->m_buffer[offset], parser->m_bufferEnd - parser->m_bufferPtr + keep); parser->m_bufferEnd -= offset; parser->m_bufferPtr -= offset; } #else - memmove(parser->m_buffer, parser->m_bufferPtr, parser->m_bufferEnd - parser->m_bufferPtr); - parser->m_bufferEnd = parser->m_buffer + (parser->m_bufferEnd - parser->m_bufferPtr); - parser->m_bufferPtr = parser->m_buffer; + if (parser->m_buffer && parser->m_bufferPtr) { + memmove(parser->m_buffer, parser->m_bufferPtr, + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr)); + parser->m_bufferEnd = parser->m_buffer + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr); + parser->m_bufferPtr = parser->m_buffer; + } #endif /* not defined XML_CONTEXT_BYTES */ } else { char *newBuf; - int bufferSize = (int)(parser->m_bufferLim - parser->m_bufferPtr); + int bufferSize = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_bufferPtr); if (bufferSize == 0) bufferSize = INIT_BUFFER_SIZE; do { @@ -2077,25 +2089,34 @@ XML_GetBuffer(XML_Parser parser, int len) parser->m_bufferLim = newBuf + bufferSize; #ifdef XML_CONTEXT_BYTES if (parser->m_bufferPtr) { - int keep = (int)(parser->m_bufferPtr - parser->m_buffer); + int keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; - memcpy(newBuf, &parser->m_bufferPtr[-keep], parser->m_bufferEnd - parser->m_bufferPtr + keep); + memcpy(newBuf, &parser->m_bufferPtr[-keep], + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr) + keep); FREE(parser, parser->m_buffer); parser->m_buffer = newBuf; - parser->m_bufferEnd = parser->m_buffer + (parser->m_bufferEnd - parser->m_bufferPtr) + keep; + parser->m_bufferEnd = parser->m_buffer + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr) + keep; parser->m_bufferPtr = parser->m_buffer + keep; } else { - parser->m_bufferEnd = newBuf + (parser->m_bufferEnd - parser->m_bufferPtr); + /* This must be a brand new buffer with no data in it yet */ + parser->m_bufferEnd = newBuf; parser->m_bufferPtr = parser->m_buffer = newBuf; } #else if (parser->m_bufferPtr) { - memcpy(newBuf, parser->m_bufferPtr, parser->m_bufferEnd - parser->m_bufferPtr); + memcpy(newBuf, parser->m_bufferPtr, + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr)); FREE(parser, parser->m_buffer); + parser->m_bufferEnd = newBuf + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr); + } + else { + /* This must be a brand new buffer with no data in it yet */ + parser->m_bufferEnd = newBuf; } - parser->m_bufferEnd = newBuf + (parser->m_bufferEnd - parser->m_bufferPtr); parser->m_bufferPtr = parser->m_buffer = newBuf; #endif /* not defined XML_CONTEXT_BYTES */ } @@ -2908,9 +2929,11 @@ doContent(XML_Parser parser, poolClear(&parser->m_tempPool); freeBindings(parser, bindings); } - if ((parser->m_tagLevel == 0) && - !((parser->m_parsingStatus.parsing == XML_FINISHED) || (parser->m_parsingStatus.parsing == XML_SUSPENDED))) { - return epilogProcessor(parser, next, end, nextPtr); + if ((parser->m_tagLevel == 0) && (parser->m_parsingStatus.parsing != XML_FINISHED)) { + if (parser->m_parsingStatus.parsing == XML_SUSPENDED) + parser->m_processor = epilogProcessor; + else + return epilogProcessor(parser, next, end, nextPtr); } break; case XML_TOK_END_TAG: @@ -4746,8 +4769,8 @@ doProlog(XML_Parser parser, return XML_ERROR_NO_MEMORY; parser->m_declEntity->publicId = NULL; } - /* fall through */ #endif /* XML_DTD */ + /* fall through */ case XML_ROLE_ENTITY_SYSTEM_ID: if (dtd->keepProcessing && parser->m_declEntity) { parser->m_declEntity->systemId = poolStoreString(&dtd->pool, enc, @@ -6643,7 +6666,6 @@ hash(XML_Parser parser, KEY s) { struct siphash state; struct sipkey key; - (void)sip_tobin; (void)sip24_valid; copy_salt_to_sipkey(parser, &key); sip24_init(&state, &key); diff --git a/Modules/expat/xmltok_impl.c b/Modules/expat/xmltok_impl.c index 0403dd3d09a0..4d9ae7dc3896 100644 --- a/Modules/expat/xmltok_impl.c +++ b/Modules/expat/xmltok_impl.c @@ -74,6 +74,7 @@ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ + /* fall through */ \ case BT_NMSTRT: \ case BT_HEX: \ case BT_DIGIT: \ @@ -102,6 +103,7 @@ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ + /* fall through */ \ case BT_NMSTRT: \ case BT_HEX: \ ptr += MINBPC(enc); \ @@ -602,7 +604,7 @@ PREFIX(scanAtts)(const ENCODING *enc, const char *ptr, const char *end, return XML_TOK_INVALID; } } - /* fall through */ + /* fall through */ case BT_EQUALS: { int open; @@ -1442,6 +1444,7 @@ PREFIX(isPublicId)(const ENCODING *enc, const char *ptr, const char *end, case BT_NMSTRT: if (!(BYTE_TO_ASCII(enc, ptr) & ~0x7f)) break; + /* fall through */ default: switch (BYTE_TO_ASCII(enc, ptr)) { case 0x24: /* $ */ @@ -1659,8 +1662,8 @@ PREFIX(nameMatchesAscii)(const ENCODING *UNUSED_P(enc), const char *ptr1, { for (; *ptr2; ptr1 += MINBPC(enc), ptr2++) { if (end1 - ptr1 < MINBPC(enc)) { - /* This line cannot be executed. THe incoming data has already - * been tokenized once, so imcomplete characters like this have + /* This line cannot be executed. The incoming data has already + * been tokenized once, so incomplete characters like this have * already been eliminated from the input. Retaining the * paranoia check is still valuable, however. */ From webhook-mailer at python.org Tue Sep 11 00:36:24 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Tue, 11 Sep 2018 04:36:24 -0000 Subject: [Python-checkins] bpo-32270: Don't close stdin/out/err in pass_fds (GH-6242) (GH-9148) Message-ID: https://github.com/python/cpython/commit/9c4a63fc17efea31ad41f90d28825be37469e0e2 commit: 9c4a63fc17efea31ad41f90d28825be37469e0e2 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Gregory P. Smith date: 2018-09-10T21:36:20-07:00 summary: bpo-32270: Don't close stdin/out/err in pass_fds (GH-6242) (GH-9148) When subprocess.Popen() stdin= stdout= or stderr= handles are specified and appear in pass_fds=, don't close the original fds after dup'ing them. This implementation and unittest primarily came from @izbyshev (see the PR) See also https://github.com/izbyshev/cpython/commit/b89b52f28490b69142d5c061604b3a3989cec66c This also removes the old manual p2cread, c2pwrite, and errwrite closing logic as inheritable flags and _close_open_fds takes care of that properly today without special treatment. This code is within child_exec() where it is the only thread so there is no race condition between the dup and _Py_set_inheritable_async_safe call. (cherry picked from commit ce34410b8b67f49d8275c05d51b3ead50cf97f48) Co-authored-by: Gregory P. Smith [Google] files: A Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst M Lib/test/test_subprocess.py M Modules/_posixsubprocess.c diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py index 4719773b67b7..8419061b2a90 100644 --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -2529,6 +2529,36 @@ def test_pass_fds_inheritable(self): self.assertEqual(os.get_inheritable(inheritable), True) self.assertEqual(os.get_inheritable(non_inheritable), False) + + # bpo-32270: Ensure that descriptors specified in pass_fds + # are inherited even if they are used in redirections. + # Contributed by @izbyshev. + def test_pass_fds_redirected(self): + """Regression test for https://bugs.python.org/issue32270.""" + fd_status = support.findfile("fd_status.py", subdir="subprocessdata") + pass_fds = [] + for _ in range(2): + fd = os.open(os.devnull, os.O_RDWR) + self.addCleanup(os.close, fd) + pass_fds.append(fd) + + stdout_r, stdout_w = os.pipe() + self.addCleanup(os.close, stdout_r) + self.addCleanup(os.close, stdout_w) + pass_fds.insert(1, stdout_w) + + with subprocess.Popen([sys.executable, fd_status], + stdin=pass_fds[0], + stdout=pass_fds[1], + stderr=pass_fds[2], + close_fds=True, + pass_fds=pass_fds): + output = os.read(stdout_r, 1024) + fds = {int(num) for num in output.split(b',')} + + self.assertEqual(fds, {0, 1, 2} | frozenset(pass_fds), f"output={output!a}") + + def test_stdout_stdin_are_single_inout_fd(self): with io.open(os.devnull, "r+") as inout: p = subprocess.Popen([sys.executable, "-c", "import sys; sys.exit(0)"], diff --git a/Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst b/Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst new file mode 100644 index 000000000000..83f68624c1be --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst @@ -0,0 +1,2 @@ +The subprocess module no longer mistakenly closes redirected fds even when +they were in pass_fds when outside of the default {0, 1, 2} set. diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index 0150fcb0970c..aeb10f9ecfe4 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -422,10 +422,20 @@ child_exec(char *const exec_array[], /* When duping fds, if there arises a situation where one of the fds is either 0, 1 or 2, it is possible that it is overwritten (#12607). */ - if (c2pwrite == 0) + if (c2pwrite == 0) { POSIX_CALL(c2pwrite = dup(c2pwrite)); - while (errwrite == 0 || errwrite == 1) + /* issue32270 */ + if (_Py_set_inheritable_async_safe(c2pwrite, 0, NULL) < 0) { + goto error; + } + } + while (errwrite == 0 || errwrite == 1) { POSIX_CALL(errwrite = dup(errwrite)); + /* issue32270 */ + if (_Py_set_inheritable_async_safe(errwrite, 0, NULL) < 0) { + goto error; + } + } /* Dup fds for child. dup2() removes the CLOEXEC flag but we must do it ourselves if dup2() @@ -451,14 +461,8 @@ child_exec(char *const exec_array[], else if (errwrite != -1) POSIX_CALL(dup2(errwrite, 2)); /* stderr */ - /* Close pipe fds. Make sure we don't close the same fd more than */ - /* once, or standard fds. */ - if (p2cread > 2) - POSIX_CALL(close(p2cread)); - if (c2pwrite > 2 && c2pwrite != p2cread) - POSIX_CALL(close(c2pwrite)); - if (errwrite != c2pwrite && errwrite != p2cread && errwrite > 2) - POSIX_CALL(close(errwrite)); + /* We no longer manually close p2cread, c2pwrite, and errwrite here as + * _close_open_fds takes care when it is not already non-inheritable. */ if (cwd) POSIX_CALL(chdir(cwd)); From webhook-mailer at python.org Tue Sep 11 00:37:36 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Tue, 11 Sep 2018 04:37:36 -0000 Subject: [Python-checkins] bpo-33032: Mention the implicit cache in struct.Struct() docs (GH-7700) Message-ID: https://github.com/python/cpython/commit/3666b3c1f695a145adab1bf644c22e564e8eb0ee commit: 3666b3c1f695a145adab1bf644c22e564e8eb0ee branch: master author: Zackery Spytz committer: Gregory P. Smith date: 2018-09-10T21:37:33-07:00 summary: bpo-33032: Mention the implicit cache in struct.Struct() docs (GH-7700) Mention the implicit cache in struct.Struct() docs. Consistent with the re.compile documentation note. files: M Doc/library/struct.rst diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index d6a3cb721e83..bad33ac666ed 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -405,6 +405,12 @@ The :mod:`struct` module also defines the following type: methods is more efficient than calling the :mod:`struct` functions with the same format since the format string only needs to be compiled once. + .. note:: + + The compiled versions of the most recent format strings passed to + :class:`Struct` and the module-level functions are cached, so programs + that use only a few format strings needn't worry about reusing a single + :class:`Struct` instance. Compiled Struct objects support the following methods and attributes: From webhook-mailer at python.org Tue Sep 11 00:50:46 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 04:50:46 -0000 Subject: [Python-checkins] bpo-33032: Mention the implicit cache in struct.Struct() docs (GH-7700) Message-ID: https://github.com/python/cpython/commit/2110f78d92522499836a928d268704fdc5f7a037 commit: 2110f78d92522499836a928d268704fdc5f7a037 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T21:50:41-07:00 summary: bpo-33032: Mention the implicit cache in struct.Struct() docs (GH-7700) Mention the implicit cache in struct.Struct() docs. Consistent with the re.compile documentation note. (cherry picked from commit 3666b3c1f695a145adab1bf644c22e564e8eb0ee) Co-authored-by: Zackery Spytz files: M Doc/library/struct.rst diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index 2d0866c7e09e..f10fbe4fc0df 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -405,6 +405,12 @@ The :mod:`struct` module also defines the following type: methods is more efficient than calling the :mod:`struct` functions with the same format since the format string only needs to be compiled once. + .. note:: + + The compiled versions of the most recent format strings passed to + :class:`Struct` and the module-level functions are cached, so programs + that use only a few format strings needn't worry about reusing a single + :class:`Struct` instance. Compiled Struct objects support the following methods and attributes: From webhook-mailer at python.org Tue Sep 11 00:55:30 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 04:55:30 -0000 Subject: [Python-checkins] bpo-34625: Update vendorized expat version to 2.2.6. (GH-9150) Message-ID: https://github.com/python/cpython/commit/2bc4eea8f6675b157adcaea4a59d982d58a5e7ae commit: 2bc4eea8f6675b157adcaea4a59d982d58a5e7ae branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T21:55:27-07:00 summary: bpo-34625: Update vendorized expat version to 2.2.6. (GH-9150) (cherry picked from commit 5033aa77aacaa5505636f150e8d54baac5bdca9c) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst M Modules/expat/expat.h M Modules/expat/siphash.h M Modules/expat/xmlparse.c M Modules/expat/xmltok_impl.c diff --git a/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst b/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst new file mode 100644 index 000000000000..0747ec54470f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst @@ -0,0 +1 @@ +Update vendorized expat library version to 2.2.6. diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index 1f608c02d6fd..174c3fafda3f 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -264,7 +264,7 @@ XML_ParserCreate_MM(const XML_Char *encoding, const XML_Char *namespaceSeparator); /* Prepare a parser object to be re-used. This is particularly - valuable when memory allocation overhead is disproportionatly high, + valuable when memory allocation overhead is disproportionately high, such as when a large number of small documnents need to be parsed. All handlers are cleared from the parser, except for the unknownEncodingHandler. The parser's external state is re-initialized @@ -1076,7 +1076,7 @@ XML_GetFeatureList(void); */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 2 -#define XML_MICRO_VERSION 5 +#define XML_MICRO_VERSION 6 #ifdef __cplusplus } diff --git a/Modules/expat/siphash.h b/Modules/expat/siphash.h index 581872df7b49..4d6786d7839a 100644 --- a/Modules/expat/siphash.h +++ b/Modules/expat/siphash.h @@ -11,6 +11,12 @@ * -------------------------------------------------------------------------- * HISTORY: * + * 2018-07-08 (Anton Maklakov) + * - Add "fall through" markers for GCC's -Wimplicit-fallthrough + * + * 2017-11-03 (Sebastian Pipping) + * - Hide sip_tobin and sip_binof unless SIPHASH_TOBIN macro is defined + * * 2017-07-25 (Vadim Zeitlin) * - Fix use of SIPHASH_MAIN macro * @@ -151,6 +157,8 @@ static struct sipkey *sip_tokey(struct sipkey *key, const void *src) { } /* sip_tokey() */ +#ifdef SIPHASH_TOBIN + #define sip_binof(v) sip_tobin((unsigned char[8]){ 0 }, (v)) static void *sip_tobin(void *dst, uint64_t u64) { @@ -158,6 +166,8 @@ static void *sip_tobin(void *dst, uint64_t u64) { return dst; } /* sip_tobin() */ +#endif /* SIPHASH_TOBIN */ + static void sip_round(struct siphash *H, const int rounds) { int i; @@ -231,12 +241,19 @@ static uint64_t sip24_final(struct siphash *H) { switch (left) { case 7: b |= (uint64_t)H->buf[6] << 48; + /* fall through */ case 6: b |= (uint64_t)H->buf[5] << 40; + /* fall through */ case 5: b |= (uint64_t)H->buf[4] << 32; + /* fall through */ case 4: b |= (uint64_t)H->buf[3] << 24; + /* fall through */ case 3: b |= (uint64_t)H->buf[2] << 16; + /* fall through */ case 2: b |= (uint64_t)H->buf[1] << 8; + /* fall through */ case 1: b |= (uint64_t)H->buf[0] << 0; + /* fall through */ case 0: break; } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 90a237f30eb8..c4f3ffc215c9 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 4b74aa710b4ed5ce464b0ce544852cb47bf905c85a49c7bae2749f5885cb966d (2.2.5+) +/* 19ac4776051591216f1874e34ee99b6a43a3784c8bd7d70efeb9258dd22b906a (2.2.6+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -161,6 +161,9 @@ typedef char ICHAR; /* Round up n to be a multiple of sz, where sz is a power of 2. */ #define ROUND_UP(n, sz) (((n) + ((sz) - 1)) & ~((sz) - 1)) +/* Do safe (NULL-aware) pointer arithmetic */ +#define EXPAT_SAFE_PTR_DIFF(p, q) (((p) && (q)) ? ((p) - (q)) : 0) + /* Handle the case where memmove() doesn't exist. */ #ifndef HAVE_MEMMOVE #ifdef HAVE_BCOPY @@ -1820,6 +1823,7 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) parser->m_errorCode = XML_ERROR_NO_MEMORY; return XML_STATUS_ERROR; } + /* fall through */ default: parser->m_parsingStatus.parsing = XML_PARSING; } @@ -1969,6 +1973,7 @@ XML_ParseBuffer(XML_Parser parser, int len, int isFinal) parser->m_errorCode = XML_ERROR_NO_MEMORY; return XML_STATUS_ERROR; } + /* fall through */ default: parser->m_parsingStatus.parsing = XML_PARSING; } @@ -2026,39 +2031,46 @@ XML_GetBuffer(XML_Parser parser, int len) default: ; } - if (len > parser->m_bufferLim - parser->m_bufferEnd) { + if (len > EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_bufferEnd)) { #ifdef XML_CONTEXT_BYTES int keep; #endif /* defined XML_CONTEXT_BYTES */ /* Do not invoke signed arithmetic overflow: */ - int neededSize = (int) ((unsigned)len + (unsigned)(parser->m_bufferEnd - parser->m_bufferPtr)); + int neededSize = (int) ((unsigned)len + + (unsigned)EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, + parser->m_bufferPtr)); if (neededSize < 0) { parser->m_errorCode = XML_ERROR_NO_MEMORY; return NULL; } #ifdef XML_CONTEXT_BYTES - keep = (int)(parser->m_bufferPtr - parser->m_buffer); + keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; neededSize += keep; #endif /* defined XML_CONTEXT_BYTES */ - if (neededSize <= parser->m_bufferLim - parser->m_buffer) { + if (neededSize <= EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_buffer)) { #ifdef XML_CONTEXT_BYTES - if (keep < parser->m_bufferPtr - parser->m_buffer) { - int offset = (int)(parser->m_bufferPtr - parser->m_buffer) - keep; + if (keep < EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer)) { + int offset = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer) - keep; + /* The buffer pointers cannot be NULL here; we have at least some bytes in the buffer */ memmove(parser->m_buffer, &parser->m_buffer[offset], parser->m_bufferEnd - parser->m_bufferPtr + keep); parser->m_bufferEnd -= offset; parser->m_bufferPtr -= offset; } #else - memmove(parser->m_buffer, parser->m_bufferPtr, parser->m_bufferEnd - parser->m_bufferPtr); - parser->m_bufferEnd = parser->m_buffer + (parser->m_bufferEnd - parser->m_bufferPtr); - parser->m_bufferPtr = parser->m_buffer; + if (parser->m_buffer && parser->m_bufferPtr) { + memmove(parser->m_buffer, parser->m_bufferPtr, + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr)); + parser->m_bufferEnd = parser->m_buffer + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr); + parser->m_bufferPtr = parser->m_buffer; + } #endif /* not defined XML_CONTEXT_BYTES */ } else { char *newBuf; - int bufferSize = (int)(parser->m_bufferLim - parser->m_bufferPtr); + int bufferSize = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_bufferPtr); if (bufferSize == 0) bufferSize = INIT_BUFFER_SIZE; do { @@ -2077,25 +2089,34 @@ XML_GetBuffer(XML_Parser parser, int len) parser->m_bufferLim = newBuf + bufferSize; #ifdef XML_CONTEXT_BYTES if (parser->m_bufferPtr) { - int keep = (int)(parser->m_bufferPtr - parser->m_buffer); + int keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; - memcpy(newBuf, &parser->m_bufferPtr[-keep], parser->m_bufferEnd - parser->m_bufferPtr + keep); + memcpy(newBuf, &parser->m_bufferPtr[-keep], + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr) + keep); FREE(parser, parser->m_buffer); parser->m_buffer = newBuf; - parser->m_bufferEnd = parser->m_buffer + (parser->m_bufferEnd - parser->m_bufferPtr) + keep; + parser->m_bufferEnd = parser->m_buffer + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr) + keep; parser->m_bufferPtr = parser->m_buffer + keep; } else { - parser->m_bufferEnd = newBuf + (parser->m_bufferEnd - parser->m_bufferPtr); + /* This must be a brand new buffer with no data in it yet */ + parser->m_bufferEnd = newBuf; parser->m_bufferPtr = parser->m_buffer = newBuf; } #else if (parser->m_bufferPtr) { - memcpy(newBuf, parser->m_bufferPtr, parser->m_bufferEnd - parser->m_bufferPtr); + memcpy(newBuf, parser->m_bufferPtr, + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr)); FREE(parser, parser->m_buffer); + parser->m_bufferEnd = newBuf + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr); + } + else { + /* This must be a brand new buffer with no data in it yet */ + parser->m_bufferEnd = newBuf; } - parser->m_bufferEnd = newBuf + (parser->m_bufferEnd - parser->m_bufferPtr); parser->m_bufferPtr = parser->m_buffer = newBuf; #endif /* not defined XML_CONTEXT_BYTES */ } @@ -2908,9 +2929,11 @@ doContent(XML_Parser parser, poolClear(&parser->m_tempPool); freeBindings(parser, bindings); } - if ((parser->m_tagLevel == 0) && - !((parser->m_parsingStatus.parsing == XML_FINISHED) || (parser->m_parsingStatus.parsing == XML_SUSPENDED))) { - return epilogProcessor(parser, next, end, nextPtr); + if ((parser->m_tagLevel == 0) && (parser->m_parsingStatus.parsing != XML_FINISHED)) { + if (parser->m_parsingStatus.parsing == XML_SUSPENDED) + parser->m_processor = epilogProcessor; + else + return epilogProcessor(parser, next, end, nextPtr); } break; case XML_TOK_END_TAG: @@ -4746,8 +4769,8 @@ doProlog(XML_Parser parser, return XML_ERROR_NO_MEMORY; parser->m_declEntity->publicId = NULL; } - /* fall through */ #endif /* XML_DTD */ + /* fall through */ case XML_ROLE_ENTITY_SYSTEM_ID: if (dtd->keepProcessing && parser->m_declEntity) { parser->m_declEntity->systemId = poolStoreString(&dtd->pool, enc, @@ -6643,7 +6666,6 @@ hash(XML_Parser parser, KEY s) { struct siphash state; struct sipkey key; - (void)sip_tobin; (void)sip24_valid; copy_salt_to_sipkey(parser, &key); sip24_init(&state, &key); diff --git a/Modules/expat/xmltok_impl.c b/Modules/expat/xmltok_impl.c index 0403dd3d09a0..4d9ae7dc3896 100644 --- a/Modules/expat/xmltok_impl.c +++ b/Modules/expat/xmltok_impl.c @@ -74,6 +74,7 @@ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ + /* fall through */ \ case BT_NMSTRT: \ case BT_HEX: \ case BT_DIGIT: \ @@ -102,6 +103,7 @@ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ + /* fall through */ \ case BT_NMSTRT: \ case BT_HEX: \ ptr += MINBPC(enc); \ @@ -602,7 +604,7 @@ PREFIX(scanAtts)(const ENCODING *enc, const char *ptr, const char *end, return XML_TOK_INVALID; } } - /* fall through */ + /* fall through */ case BT_EQUALS: { int open; @@ -1442,6 +1444,7 @@ PREFIX(isPublicId)(const ENCODING *enc, const char *ptr, const char *end, case BT_NMSTRT: if (!(BYTE_TO_ASCII(enc, ptr) & ~0x7f)) break; + /* fall through */ default: switch (BYTE_TO_ASCII(enc, ptr)) { case 0x24: /* $ */ @@ -1659,8 +1662,8 @@ PREFIX(nameMatchesAscii)(const ENCODING *UNUSED_P(enc), const char *ptr1, { for (; *ptr2; ptr1 += MINBPC(enc), ptr2++) { if (end1 - ptr1 < MINBPC(enc)) { - /* This line cannot be executed. THe incoming data has already - * been tokenized once, so imcomplete characters like this have + /* This line cannot be executed. The incoming data has already + * been tokenized once, so incomplete characters like this have * already been eliminated from the input. Retaining the * paranoia check is still valuable, however. */ From webhook-mailer at python.org Tue Sep 11 00:56:53 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 04:56:53 -0000 Subject: [Python-checkins] bpo-34625: Update vendorized expat version to 2.2.6. (GH-9158) Message-ID: https://github.com/python/cpython/commit/b2260e59ff1eaf20de4738099005ddf507b7b27d commit: b2260e59ff1eaf20de4738099005ddf507b7b27d branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Benjamin Peterson date: 2018-09-10T21:56:50-07:00 summary: bpo-34625: Update vendorized expat version to 2.2.6. (GH-9158) (cherry picked from commit 5033aa77aacaa5505636f150e8d54baac5bdca9c) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst M Modules/expat/expat.h M Modules/expat/siphash.h M Modules/expat/xmlparse.c M Modules/expat/xmltok_impl.c diff --git a/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst b/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst new file mode 100644 index 000000000000..0747ec54470f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-17-46-51.bpo-34625.D2YfDz.rst @@ -0,0 +1 @@ +Update vendorized expat library version to 2.2.6. diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index 1f608c02d6fd..174c3fafda3f 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -264,7 +264,7 @@ XML_ParserCreate_MM(const XML_Char *encoding, const XML_Char *namespaceSeparator); /* Prepare a parser object to be re-used. This is particularly - valuable when memory allocation overhead is disproportionatly high, + valuable when memory allocation overhead is disproportionately high, such as when a large number of small documnents need to be parsed. All handlers are cleared from the parser, except for the unknownEncodingHandler. The parser's external state is re-initialized @@ -1076,7 +1076,7 @@ XML_GetFeatureList(void); */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 2 -#define XML_MICRO_VERSION 5 +#define XML_MICRO_VERSION 6 #ifdef __cplusplus } diff --git a/Modules/expat/siphash.h b/Modules/expat/siphash.h index 581872df7b49..4d6786d7839a 100644 --- a/Modules/expat/siphash.h +++ b/Modules/expat/siphash.h @@ -11,6 +11,12 @@ * -------------------------------------------------------------------------- * HISTORY: * + * 2018-07-08 (Anton Maklakov) + * - Add "fall through" markers for GCC's -Wimplicit-fallthrough + * + * 2017-11-03 (Sebastian Pipping) + * - Hide sip_tobin and sip_binof unless SIPHASH_TOBIN macro is defined + * * 2017-07-25 (Vadim Zeitlin) * - Fix use of SIPHASH_MAIN macro * @@ -151,6 +157,8 @@ static struct sipkey *sip_tokey(struct sipkey *key, const void *src) { } /* sip_tokey() */ +#ifdef SIPHASH_TOBIN + #define sip_binof(v) sip_tobin((unsigned char[8]){ 0 }, (v)) static void *sip_tobin(void *dst, uint64_t u64) { @@ -158,6 +166,8 @@ static void *sip_tobin(void *dst, uint64_t u64) { return dst; } /* sip_tobin() */ +#endif /* SIPHASH_TOBIN */ + static void sip_round(struct siphash *H, const int rounds) { int i; @@ -231,12 +241,19 @@ static uint64_t sip24_final(struct siphash *H) { switch (left) { case 7: b |= (uint64_t)H->buf[6] << 48; + /* fall through */ case 6: b |= (uint64_t)H->buf[5] << 40; + /* fall through */ case 5: b |= (uint64_t)H->buf[4] << 32; + /* fall through */ case 4: b |= (uint64_t)H->buf[3] << 24; + /* fall through */ case 3: b |= (uint64_t)H->buf[2] << 16; + /* fall through */ case 2: b |= (uint64_t)H->buf[1] << 8; + /* fall through */ case 1: b |= (uint64_t)H->buf[0] << 0; + /* fall through */ case 0: break; } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 90a237f30eb8..c4f3ffc215c9 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 4b74aa710b4ed5ce464b0ce544852cb47bf905c85a49c7bae2749f5885cb966d (2.2.5+) +/* 19ac4776051591216f1874e34ee99b6a43a3784c8bd7d70efeb9258dd22b906a (2.2.6+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -161,6 +161,9 @@ typedef char ICHAR; /* Round up n to be a multiple of sz, where sz is a power of 2. */ #define ROUND_UP(n, sz) (((n) + ((sz) - 1)) & ~((sz) - 1)) +/* Do safe (NULL-aware) pointer arithmetic */ +#define EXPAT_SAFE_PTR_DIFF(p, q) (((p) && (q)) ? ((p) - (q)) : 0) + /* Handle the case where memmove() doesn't exist. */ #ifndef HAVE_MEMMOVE #ifdef HAVE_BCOPY @@ -1820,6 +1823,7 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) parser->m_errorCode = XML_ERROR_NO_MEMORY; return XML_STATUS_ERROR; } + /* fall through */ default: parser->m_parsingStatus.parsing = XML_PARSING; } @@ -1969,6 +1973,7 @@ XML_ParseBuffer(XML_Parser parser, int len, int isFinal) parser->m_errorCode = XML_ERROR_NO_MEMORY; return XML_STATUS_ERROR; } + /* fall through */ default: parser->m_parsingStatus.parsing = XML_PARSING; } @@ -2026,39 +2031,46 @@ XML_GetBuffer(XML_Parser parser, int len) default: ; } - if (len > parser->m_bufferLim - parser->m_bufferEnd) { + if (len > EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_bufferEnd)) { #ifdef XML_CONTEXT_BYTES int keep; #endif /* defined XML_CONTEXT_BYTES */ /* Do not invoke signed arithmetic overflow: */ - int neededSize = (int) ((unsigned)len + (unsigned)(parser->m_bufferEnd - parser->m_bufferPtr)); + int neededSize = (int) ((unsigned)len + + (unsigned)EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, + parser->m_bufferPtr)); if (neededSize < 0) { parser->m_errorCode = XML_ERROR_NO_MEMORY; return NULL; } #ifdef XML_CONTEXT_BYTES - keep = (int)(parser->m_bufferPtr - parser->m_buffer); + keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; neededSize += keep; #endif /* defined XML_CONTEXT_BYTES */ - if (neededSize <= parser->m_bufferLim - parser->m_buffer) { + if (neededSize <= EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_buffer)) { #ifdef XML_CONTEXT_BYTES - if (keep < parser->m_bufferPtr - parser->m_buffer) { - int offset = (int)(parser->m_bufferPtr - parser->m_buffer) - keep; + if (keep < EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer)) { + int offset = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer) - keep; + /* The buffer pointers cannot be NULL here; we have at least some bytes in the buffer */ memmove(parser->m_buffer, &parser->m_buffer[offset], parser->m_bufferEnd - parser->m_bufferPtr + keep); parser->m_bufferEnd -= offset; parser->m_bufferPtr -= offset; } #else - memmove(parser->m_buffer, parser->m_bufferPtr, parser->m_bufferEnd - parser->m_bufferPtr); - parser->m_bufferEnd = parser->m_buffer + (parser->m_bufferEnd - parser->m_bufferPtr); - parser->m_bufferPtr = parser->m_buffer; + if (parser->m_buffer && parser->m_bufferPtr) { + memmove(parser->m_buffer, parser->m_bufferPtr, + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr)); + parser->m_bufferEnd = parser->m_buffer + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr); + parser->m_bufferPtr = parser->m_buffer; + } #endif /* not defined XML_CONTEXT_BYTES */ } else { char *newBuf; - int bufferSize = (int)(parser->m_bufferLim - parser->m_bufferPtr); + int bufferSize = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferLim, parser->m_bufferPtr); if (bufferSize == 0) bufferSize = INIT_BUFFER_SIZE; do { @@ -2077,25 +2089,34 @@ XML_GetBuffer(XML_Parser parser, int len) parser->m_bufferLim = newBuf + bufferSize; #ifdef XML_CONTEXT_BYTES if (parser->m_bufferPtr) { - int keep = (int)(parser->m_bufferPtr - parser->m_buffer); + int keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; - memcpy(newBuf, &parser->m_bufferPtr[-keep], parser->m_bufferEnd - parser->m_bufferPtr + keep); + memcpy(newBuf, &parser->m_bufferPtr[-keep], + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr) + keep); FREE(parser, parser->m_buffer); parser->m_buffer = newBuf; - parser->m_bufferEnd = parser->m_buffer + (parser->m_bufferEnd - parser->m_bufferPtr) + keep; + parser->m_bufferEnd = parser->m_buffer + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr) + keep; parser->m_bufferPtr = parser->m_buffer + keep; } else { - parser->m_bufferEnd = newBuf + (parser->m_bufferEnd - parser->m_bufferPtr); + /* This must be a brand new buffer with no data in it yet */ + parser->m_bufferEnd = newBuf; parser->m_bufferPtr = parser->m_buffer = newBuf; } #else if (parser->m_bufferPtr) { - memcpy(newBuf, parser->m_bufferPtr, parser->m_bufferEnd - parser->m_bufferPtr); + memcpy(newBuf, parser->m_bufferPtr, + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr)); FREE(parser, parser->m_buffer); + parser->m_bufferEnd = newBuf + + EXPAT_SAFE_PTR_DIFF(parser->m_bufferEnd, parser->m_bufferPtr); + } + else { + /* This must be a brand new buffer with no data in it yet */ + parser->m_bufferEnd = newBuf; } - parser->m_bufferEnd = newBuf + (parser->m_bufferEnd - parser->m_bufferPtr); parser->m_bufferPtr = parser->m_buffer = newBuf; #endif /* not defined XML_CONTEXT_BYTES */ } @@ -2908,9 +2929,11 @@ doContent(XML_Parser parser, poolClear(&parser->m_tempPool); freeBindings(parser, bindings); } - if ((parser->m_tagLevel == 0) && - !((parser->m_parsingStatus.parsing == XML_FINISHED) || (parser->m_parsingStatus.parsing == XML_SUSPENDED))) { - return epilogProcessor(parser, next, end, nextPtr); + if ((parser->m_tagLevel == 0) && (parser->m_parsingStatus.parsing != XML_FINISHED)) { + if (parser->m_parsingStatus.parsing == XML_SUSPENDED) + parser->m_processor = epilogProcessor; + else + return epilogProcessor(parser, next, end, nextPtr); } break; case XML_TOK_END_TAG: @@ -4746,8 +4769,8 @@ doProlog(XML_Parser parser, return XML_ERROR_NO_MEMORY; parser->m_declEntity->publicId = NULL; } - /* fall through */ #endif /* XML_DTD */ + /* fall through */ case XML_ROLE_ENTITY_SYSTEM_ID: if (dtd->keepProcessing && parser->m_declEntity) { parser->m_declEntity->systemId = poolStoreString(&dtd->pool, enc, @@ -6643,7 +6666,6 @@ hash(XML_Parser parser, KEY s) { struct siphash state; struct sipkey key; - (void)sip_tobin; (void)sip24_valid; copy_salt_to_sipkey(parser, &key); sip24_init(&state, &key); diff --git a/Modules/expat/xmltok_impl.c b/Modules/expat/xmltok_impl.c index 0403dd3d09a0..4d9ae7dc3896 100644 --- a/Modules/expat/xmltok_impl.c +++ b/Modules/expat/xmltok_impl.c @@ -74,6 +74,7 @@ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ + /* fall through */ \ case BT_NMSTRT: \ case BT_HEX: \ case BT_DIGIT: \ @@ -102,6 +103,7 @@ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ + /* fall through */ \ case BT_NMSTRT: \ case BT_HEX: \ ptr += MINBPC(enc); \ @@ -602,7 +604,7 @@ PREFIX(scanAtts)(const ENCODING *enc, const char *ptr, const char *end, return XML_TOK_INVALID; } } - /* fall through */ + /* fall through */ case BT_EQUALS: { int open; @@ -1442,6 +1444,7 @@ PREFIX(isPublicId)(const ENCODING *enc, const char *ptr, const char *end, case BT_NMSTRT: if (!(BYTE_TO_ASCII(enc, ptr) & ~0x7f)) break; + /* fall through */ default: switch (BYTE_TO_ASCII(enc, ptr)) { case 0x24: /* $ */ @@ -1659,8 +1662,8 @@ PREFIX(nameMatchesAscii)(const ENCODING *UNUSED_P(enc), const char *ptr1, { for (; *ptr2; ptr1 += MINBPC(enc), ptr2++) { if (end1 - ptr1 < MINBPC(enc)) { - /* This line cannot be executed. THe incoming data has already - * been tokenized once, so imcomplete characters like this have + /* This line cannot be executed. The incoming data has already + * been tokenized once, so incomplete characters like this have * already been eliminated from the input. Retaining the * paranoia check is still valuable, however. */ From webhook-mailer at python.org Tue Sep 11 01:12:45 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 05:12:45 -0000 Subject: [Python-checkins] closes bpo-33883: Mention type checkers in the FAQ. (GH-7760) Message-ID: https://github.com/python/cpython/commit/a37825418649873a0fa971dc7e5e6d142c124574 commit: a37825418649873a0fa971dc7e5e6d142c124574 branch: master author: Andr?s Delfino committer: Benjamin Peterson date: 2018-09-10T22:12:41-07:00 summary: closes bpo-33883: Mention type checkers in the FAQ. (GH-7760) files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 53f3b7f528c0..fd720c1a304b 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -71,6 +71,11 @@ length, whether variable names are well-formed according to your coding standard, whether declared interfaces are fully implemented, and more. https://docs.pylint.org/ provides a full list of Pylint's features. +Static type checkers such as `Mypy `_, +`Pyre `_, and +`Pytype `_ can check type hints in Python +source code. + How can I create a stand-alone binary from a Python script? ----------------------------------------------------------- From webhook-mailer at python.org Tue Sep 11 01:27:04 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 05:27:04 -0000 Subject: [Python-checkins] closes bpo-33883: Mention type checkers in the FAQ. (GH-7760) Message-ID: https://github.com/python/cpython/commit/c9ece5f4a961d4963ed42aefa934eb20f9a7222b commit: c9ece5f4a961d4963ed42aefa934eb20f9a7222b branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-10T22:27:00-07:00 summary: closes bpo-33883: Mention type checkers in the FAQ. (GH-7760) (cherry picked from commit a37825418649873a0fa971dc7e5e6d142c124574) Co-authored-by: Andr?s Delfino files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 53f3b7f528c0..fd720c1a304b 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -71,6 +71,11 @@ length, whether variable names are well-formed according to your coding standard, whether declared interfaces are fully implemented, and more. https://docs.pylint.org/ provides a full list of Pylint's features. +Static type checkers such as `Mypy `_, +`Pyre `_, and +`Pytype `_ can check type hints in Python +source code. + How can I create a stand-alone binary from a Python script? ----------------------------------------------------------- From webhook-mailer at python.org Tue Sep 11 02:00:54 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Tue, 11 Sep 2018 06:00:54 -0000 Subject: [Python-checkins] bpo-32270: Don't close stdin/out/err in pass_fds (GH-6242) (GH-9149) Message-ID: https://github.com/python/cpython/commit/2173bb818c6c726d831b106ed0d3fad7825905dc commit: 2173bb818c6c726d831b106ed0d3fad7825905dc branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Gregory P. Smith date: 2018-09-10T23:00:47-07:00 summary: bpo-32270: Don't close stdin/out/err in pass_fds (GH-6242) (GH-9149) When subprocess.Popen() stdin= stdout= or stderr= handles are specified and appear in pass_fds=, don't close the original fds after dup'ing them. This implementation and unittest primarily came from @izbyshev (see the PR) See also https://github.com/izbyshev/cpython/commit/b89b52f28490b69142d5c061604b3a3989cec66c This also removes the old manual p2cread, c2pwrite, and errwrite closing logic as inheritable flags and _close_open_fds takes care of that properly today without special treatment. This code is within child_exec() where it is the only thread so there is no race condition between the dup and _Py_set_inheritable_async_safe call. (cherry picked from commit ce34410b8b67f49d8275c05d51b3ead50cf97f48) Co-authored-by: Gregory P. Smith [Google] files: A Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst M Lib/test/test_subprocess.py M Modules/_posixsubprocess.c diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py index 9ebe3228ac03..e1b24778430e 100644 --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -2493,6 +2493,36 @@ def test_pass_fds_inheritable(self): self.assertEqual(os.get_inheritable(inheritable), True) self.assertEqual(os.get_inheritable(non_inheritable), False) + + # bpo-32270: Ensure that descriptors specified in pass_fds + # are inherited even if they are used in redirections. + # Contributed by @izbyshev. + def test_pass_fds_redirected(self): + """Regression test for https://bugs.python.org/issue32270.""" + fd_status = support.findfile("fd_status.py", subdir="subprocessdata") + pass_fds = [] + for _ in range(2): + fd = os.open(os.devnull, os.O_RDWR) + self.addCleanup(os.close, fd) + pass_fds.append(fd) + + stdout_r, stdout_w = os.pipe() + self.addCleanup(os.close, stdout_r) + self.addCleanup(os.close, stdout_w) + pass_fds.insert(1, stdout_w) + + with subprocess.Popen([sys.executable, fd_status], + stdin=pass_fds[0], + stdout=pass_fds[1], + stderr=pass_fds[2], + close_fds=True, + pass_fds=pass_fds): + output = os.read(stdout_r, 1024) + fds = {int(num) for num in output.split(b',')} + + self.assertEqual(fds, {0, 1, 2} | frozenset(pass_fds), f"output={output!a}") + + def test_stdout_stdin_are_single_inout_fd(self): with io.open(os.devnull, "r+") as inout: p = subprocess.Popen([sys.executable, "-c", "import sys; sys.exit(0)"], diff --git a/Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst b/Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst new file mode 100644 index 000000000000..83f68624c1be --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-14-15-53.bpo-32270.wSJjuD.rst @@ -0,0 +1,2 @@ +The subprocess module no longer mistakenly closes redirected fds even when +they were in pass_fds when outside of the default {0, 1, 2} set. diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index ad934dfe7d87..fe0e554618aa 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -422,10 +422,20 @@ child_exec(char *const exec_array[], /* When duping fds, if there arises a situation where one of the fds is either 0, 1 or 2, it is possible that it is overwritten (#12607). */ - if (c2pwrite == 0) + if (c2pwrite == 0) { POSIX_CALL(c2pwrite = dup(c2pwrite)); - while (errwrite == 0 || errwrite == 1) + /* issue32270 */ + if (_Py_set_inheritable_async_safe(c2pwrite, 0, NULL) < 0) { + goto error; + } + } + while (errwrite == 0 || errwrite == 1) { POSIX_CALL(errwrite = dup(errwrite)); + /* issue32270 */ + if (_Py_set_inheritable_async_safe(errwrite, 0, NULL) < 0) { + goto error; + } + } /* Dup fds for child. dup2() removes the CLOEXEC flag but we must do it ourselves if dup2() @@ -451,14 +461,8 @@ child_exec(char *const exec_array[], else if (errwrite != -1) POSIX_CALL(dup2(errwrite, 2)); /* stderr */ - /* Close pipe fds. Make sure we don't close the same fd more than */ - /* once, or standard fds. */ - if (p2cread > 2) - POSIX_CALL(close(p2cread)); - if (c2pwrite > 2 && c2pwrite != p2cread) - POSIX_CALL(close(c2pwrite)); - if (errwrite != c2pwrite && errwrite != p2cread && errwrite > 2) - POSIX_CALL(close(errwrite)); + /* We no longer manually close p2cread, c2pwrite, and errwrite here as + * _close_open_fds takes care when it is not already non-inheritable. */ if (cwd) POSIX_CALL(chdir(cwd)); From solipsis at pitrou.net Tue Sep 11 05:10:44 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 11 Sep 2018 09:10:44 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=3 Message-ID: <20180911091044.1.C1EB1FBAC2FAE48C@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_forkserver leaked [0, -2, 1] memory blocks, sum=-1 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogpFMo57', '--timeout', '7200'] From webhook-mailer at python.org Tue Sep 11 11:52:43 2018 From: webhook-mailer at python.org (Ned Deily) Date: Tue, 11 Sep 2018 15:52:43 -0000 Subject: [Python-checkins] bpo-34405: Update to OpenSSL 1.1.0i for macOS installer builds (GH-9166) Message-ID: https://github.com/python/cpython/commit/3102e24d83315eee42a94c460956fbcb92ac510f commit: 3102e24d83315eee42a94c460956fbcb92ac510f branch: master author: Ned Deily committer: GitHub date: 2018-09-11T08:52:40-07:00 summary: bpo-34405: Update to OpenSSL 1.1.0i for macOS installer builds (GH-9166) files: A Misc/NEWS.d/next/macOS/2018-09-11-08-30-55.bpo-34405.UzIi0n.rst M Mac/BuildScript/build-installer.py diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index d2b04d163ab4..15488780327e 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -215,9 +215,9 @@ def library_recipes(): result.extend([ dict( - name="OpenSSL 1.1.0h", - url="https://www.openssl.org/source/openssl-1.1.0h.tar.gz", - checksum='5271477e4d93f4ea032b665ef095ff24', + name="OpenSSL 1.1.0i", + url="https://www.openssl.org/source/openssl-1.1.0i.tar.gz", + checksum='9495126aafd2659d357ea66a969c3fe1', buildrecipe=build_universal_openssl, configure=None, install=None, diff --git a/Misc/NEWS.d/next/macOS/2018-09-11-08-30-55.bpo-34405.UzIi0n.rst b/Misc/NEWS.d/next/macOS/2018-09-11-08-30-55.bpo-34405.UzIi0n.rst new file mode 100644 index 000000000000..3bc9c4c2a037 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2018-09-11-08-30-55.bpo-34405.UzIi0n.rst @@ -0,0 +1 @@ +Update to OpenSSL 1.1.0i for macOS installer builds. From webhook-mailer at python.org Tue Sep 11 11:54:10 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 15:54:10 -0000 Subject: [Python-checkins] closes bpo-32490: Fix filename duplication in subprocess exception message. (GH-9163) Message-ID: https://github.com/python/cpython/commit/73870bfeb9cf350d84ee88bd25430c104b3c6191 commit: 73870bfeb9cf350d84ee88bd25430c104b3c6191 branch: master author: Zackery Spytz committer: Benjamin Peterson date: 2018-09-11T08:54:07-07:00 summary: closes bpo-32490: Fix filename duplication in subprocess exception message. (GH-9163) 8621bb5d93239316f97281826461b85072ff6db7 sets the filename in directly in the FileNotFoundError, so we may revert the earlier fix 5f780400572508a8179de6a6c13b58b7be417ef5. files: A Misc/NEWS.d/next/Library/2018-09-11-01-25-35.bpo-32490.ROIDO1.rst M Lib/subprocess.py M Lib/test/test_subprocess.py diff --git a/Lib/subprocess.py b/Lib/subprocess.py index 1e04d5e57013..c827113a933f 100644 --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -1512,8 +1512,6 @@ def _execute_child(self, args, executable, preexec_fn, close_fds, err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) - if errno_num == errno.ENOENT: - err_msg += ': ' + repr(err_filename) raise child_exception_type(errno_num, err_msg, err_filename) raise child_exception_type(err_msg) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py index 8419061b2a90..c56e1b632150 100644 --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -1520,7 +1520,6 @@ def _get_chdir_exception(self): # string and instead capture the exception that we want to see # below for comparison. desired_exception = e - desired_exception.strerror += ': ' + repr(self._nonexistent_dir) else: self.fail("chdir to nonexistent directory %s succeeded." % self._nonexistent_dir) @@ -1537,6 +1536,7 @@ def test_exception_cwd(self): # it up to the parent process as the correct exception. self.assertEqual(desired_exception.errno, e.errno) self.assertEqual(desired_exception.strerror, e.strerror) + self.assertEqual(desired_exception.filename, e.filename) else: self.fail("Expected OSError: %s" % desired_exception) @@ -1551,6 +1551,7 @@ def test_exception_bad_executable(self): # it up to the parent process as the correct exception. self.assertEqual(desired_exception.errno, e.errno) self.assertEqual(desired_exception.strerror, e.strerror) + self.assertEqual(desired_exception.filename, e.filename) else: self.fail("Expected OSError: %s" % desired_exception) @@ -1564,6 +1565,7 @@ def test_exception_bad_args_0(self): # it up to the parent process as the correct exception. self.assertEqual(desired_exception.errno, e.errno) self.assertEqual(desired_exception.strerror, e.strerror) + self.assertEqual(desired_exception.filename, e.filename) else: self.fail("Expected OSError: %s" % desired_exception) diff --git a/Misc/NEWS.d/next/Library/2018-09-11-01-25-35.bpo-32490.ROIDO1.rst b/Misc/NEWS.d/next/Library/2018-09-11-01-25-35.bpo-32490.ROIDO1.rst new file mode 100644 index 000000000000..16fe7b4d4c09 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-11-01-25-35.bpo-32490.ROIDO1.rst @@ -0,0 +1,2 @@ +Prevent filename duplication in :mod:`subprocess` exception messages. Patch +by Zackery Spytz. From webhook-mailer at python.org Tue Sep 11 12:07:16 2018 From: webhook-mailer at python.org (Ned Deily) Date: Tue, 11 Sep 2018 16:07:16 -0000 Subject: [Python-checkins] bpo-34405: Update to OpenSSL 1.1.0i for macOS installer builds (GH-9166) (GH-9167) Message-ID: https://github.com/python/cpython/commit/3235fac0d7d94ad6464a162261c18a424829d2e5 commit: 3235fac0d7d94ad6464a162261c18a424829d2e5 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Ned Deily date: 2018-09-11T09:07:11-07:00 summary: bpo-34405: Update to OpenSSL 1.1.0i for macOS installer builds (GH-9166) (GH-9167) (cherry picked from commit 3102e24d83315eee42a94c460956fbcb92ac510f) Co-authored-by: Ned Deily files: A Misc/NEWS.d/next/macOS/2018-09-11-08-30-55.bpo-34405.UzIi0n.rst M Mac/BuildScript/build-installer.py diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index d2b04d163ab4..15488780327e 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -215,9 +215,9 @@ def library_recipes(): result.extend([ dict( - name="OpenSSL 1.1.0h", - url="https://www.openssl.org/source/openssl-1.1.0h.tar.gz", - checksum='5271477e4d93f4ea032b665ef095ff24', + name="OpenSSL 1.1.0i", + url="https://www.openssl.org/source/openssl-1.1.0i.tar.gz", + checksum='9495126aafd2659d357ea66a969c3fe1', buildrecipe=build_universal_openssl, configure=None, install=None, diff --git a/Misc/NEWS.d/next/macOS/2018-09-11-08-30-55.bpo-34405.UzIi0n.rst b/Misc/NEWS.d/next/macOS/2018-09-11-08-30-55.bpo-34405.UzIi0n.rst new file mode 100644 index 000000000000..3bc9c4c2a037 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2018-09-11-08-30-55.bpo-34405.UzIi0n.rst @@ -0,0 +1 @@ +Update to OpenSSL 1.1.0i for macOS installer builds. From webhook-mailer at python.org Tue Sep 11 12:51:32 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 16:51:32 -0000 Subject: [Python-checkins] closes bpo-29832: Remove "getsockaddrarg" from error messages. (GH-3163) Message-ID: https://github.com/python/cpython/commit/735171e33486131d93865cf851c0c3d63fffd364 commit: 735171e33486131d93865cf851c0c3d63fffd364 branch: master author: Oren Milman committer: Benjamin Peterson date: 2018-09-11T09:51:29-07:00 summary: closes bpo-29832: Remove "getsockaddrarg" from error messages. (GH-3163) files: A Misc/NEWS.d/next/Core and Builtins/2017-09-12-08-11-01.bpo-29832.Kuf2M7.rst M Modules/socketmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2017-09-12-08-11-01.bpo-29832.Kuf2M7.rst b/Misc/NEWS.d/next/Core and Builtins/2017-09-12-08-11-01.bpo-29832.Kuf2M7.rst new file mode 100644 index 000000000000..6cf6696a71f7 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2017-09-12-08-11-01.bpo-29832.Kuf2M7.rst @@ -0,0 +1,2 @@ +Remove references to 'getsockaddrarg' from various socket error messages. +Patch by Oren Milman. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 014f3ba8b5ea..8aadc780ffeb 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -1583,7 +1583,7 @@ idna_converter(PyObject *obj, struct maybe_idna *data) static int getsockaddrarg(PySocketSockObject *s, PyObject *args, - struct sockaddr *addr_ret, int *len_ret) + struct sockaddr *addr_ret, int *len_ret, const char *caller) { switch (s->sock_family) { @@ -1649,13 +1649,17 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (!PyTuple_Check(args)) { PyErr_Format( PyExc_TypeError, - "getsockaddrarg: " - "AF_NETLINK address must be tuple, not %.500s", - Py_TYPE(args)->tp_name); + "%s(): AF_NETLINK address must be tuple, not %.500s", + caller, Py_TYPE(args)->tp_name); return 0; } - if (!PyArg_ParseTuple(args, "II:getsockaddrarg", &pid, &groups)) + if (!PyArg_ParseTuple(args, + "II;AF_NETLINK address must be a pair " + "(pid, groups)", + &pid, &groups)) + { return 0; + } addr->nl_family = AF_NETLINK; addr->nl_pid = pid; addr->nl_groups = groups; @@ -1703,14 +1707,22 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (!PyTuple_Check(args)) { PyErr_Format( PyExc_TypeError, - "getsockaddrarg: " - "AF_INET address must be tuple, not %.500s", - Py_TYPE(args)->tp_name); + "%s(): AF_INET address must be tuple, not %.500s", + caller, Py_TYPE(args)->tp_name); return 0; } - if (!PyArg_ParseTuple(args, "O&i:getsockaddrarg", + if (!PyArg_ParseTuple(args, + "O&i;AF_INET address must be a pair " + "(host, port)", idna_converter, &host, &port)) + { + assert(PyErr_Occurred()); + if (PyErr_ExceptionMatches(PyExc_OverflowError)) { + PyErr_Format(PyExc_OverflowError, + "%s(): port must be 0-65535.", caller); + } return 0; + } addr=(struct sockaddr_in*)addr_ret; result = setipaddr(host.buf, (struct sockaddr *)addr, sizeof(*addr), AF_INET); @@ -1718,9 +1730,9 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (result < 0) return 0; if (port < 0 || port > 0xffff) { - PyErr_SetString( + PyErr_Format( PyExc_OverflowError, - "getsockaddrarg: port must be 0-65535."); + "%s(): port must be 0-65535.", caller); return 0; } addr->sin_family = AF_INET; @@ -1740,14 +1752,21 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (!PyTuple_Check(args)) { PyErr_Format( PyExc_TypeError, - "getsockaddrarg: " - "AF_INET6 address must be tuple, not %.500s", - Py_TYPE(args)->tp_name); + "%s(): AF_INET6 address must be tuple, not %.500s", + caller, Py_TYPE(args)->tp_name); return 0; } - if (!PyArg_ParseTuple(args, "O&i|II", + if (!PyArg_ParseTuple(args, + "O&i|II;AF_INET6 address must be a tuple " + "(host, port[, flowinfo[, scopeid]])", idna_converter, &host, &port, &flowinfo, - &scope_id)) { + &scope_id)) + { + assert(PyErr_Occurred()); + if (PyErr_ExceptionMatches(PyExc_OverflowError)) { + PyErr_Format(PyExc_OverflowError, + "%s(): port must be 0-65535.", caller); + } return 0; } addr = (struct sockaddr_in6*)addr_ret; @@ -1757,15 +1776,15 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (result < 0) return 0; if (port < 0 || port > 0xffff) { - PyErr_SetString( + PyErr_Format( PyExc_OverflowError, - "getsockaddrarg: port must be 0-65535."); + "%s(): port must be 0-65535.", caller); return 0; } if (flowinfo > 0xfffff) { - PyErr_SetString( + PyErr_Format( PyExc_OverflowError, - "getsockaddrarg: flowinfo must be 0-1048575."); + "%s(): flowinfo must be 0-1048575.", caller); return 0; } addr->sin6_family = s->sock_family; @@ -1791,8 +1810,8 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, _BT_L2_MEMB(addr, family) = AF_BLUETOOTH; if (!PyArg_ParseTuple(args, "si", &straddr, &_BT_L2_MEMB(addr, psm))) { - PyErr_SetString(PyExc_OSError, "getsockaddrarg: " - "wrong format"); + PyErr_Format(PyExc_OSError, + "%s(): wrong format", caller); return 0; } if (setbdaddr(straddr, &_BT_L2_MEMB(addr, bdaddr)) < 0) @@ -1810,8 +1829,8 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, _BT_RC_MEMB(addr, family) = AF_BLUETOOTH; if (!PyArg_ParseTuple(args, "si", &straddr, &_BT_RC_MEMB(addr, channel))) { - PyErr_SetString(PyExc_OSError, "getsockaddrarg: " - "wrong format"); + PyErr_Format(PyExc_OSError, + "%s(): wrong format", caller); return 0; } if (setbdaddr(straddr, &_BT_RC_MEMB(addr, bdaddr)) < 0) @@ -1827,8 +1846,8 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, const char *straddr; _BT_HCI_MEMB(addr, family) = AF_BLUETOOTH; if (!PyBytes_Check(args)) { - PyErr_SetString(PyExc_OSError, "getsockaddrarg: " - "wrong format"); + PyErr_Format(PyExc_OSError, "%s: " + "wrong format", caller); return 0; } straddr = PyBytes_AS_STRING(args); @@ -1837,8 +1856,8 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, #else /* __NetBSD__ || __DragonFly__ */ _BT_HCI_MEMB(addr, family) = AF_BLUETOOTH; if (!PyArg_ParseTuple(args, "i", &_BT_HCI_MEMB(addr, dev))) { - PyErr_SetString(PyExc_OSError, "getsockaddrarg: " - "wrong format"); + PyErr_Format(PyExc_OSError, + "%s(): wrong format", caller); return 0; } #endif /* !(__NetBSD__ || __DragonFly__) */ @@ -1854,8 +1873,8 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, addr = (struct sockaddr_sco *)addr_ret; _BT_SCO_MEMB(addr, family) = AF_BLUETOOTH; if (!PyBytes_Check(args)) { - PyErr_SetString(PyExc_OSError, "getsockaddrarg: " - "wrong format"); + PyErr_Format(PyExc_OSError, + "%s(): wrong format", caller); return 0; } straddr = PyBytes_AS_STRING(args); @@ -1867,7 +1886,8 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, } #endif /* !__FreeBSD__ */ default: - PyErr_SetString(PyExc_OSError, "getsockaddrarg: unknown Bluetooth protocol"); + PyErr_Format(PyExc_OSError, + "%s(): unknown Bluetooth protocol", caller); return 0; } } @@ -1887,15 +1907,26 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (!PyTuple_Check(args)) { PyErr_Format( PyExc_TypeError, - "getsockaddrarg: " - "AF_PACKET address must be tuple, not %.500s", - Py_TYPE(args)->tp_name); + "%s(): AF_PACKET address must be tuple, not %.500s", + caller, Py_TYPE(args)->tp_name); return 0; } - if (!PyArg_ParseTuple(args, "si|iiy*", &interfaceName, - &protoNumber, &pkttype, &hatype, + /* XXX: improve the default error message according to the + documentation of AF_PACKET, which would be added as part + of bpo-25041. */ + if (!PyArg_ParseTuple(args, + "si|iiy*;AF_PACKET address must be a tuple of " + "two to five elements", + &interfaceName, &protoNumber, &pkttype, &hatype, &haddr)) + { + assert(PyErr_Occurred()); + if (PyErr_ExceptionMatches(PyExc_OverflowError)) { + PyErr_Format(PyExc_OverflowError, + "%s(): address argument out of range", caller); + } return 0; + } strncpy(ifr.ifr_name, interfaceName, sizeof(ifr.ifr_name)); ifr.ifr_name[(sizeof(ifr.ifr_name))-1] = '\0'; if (ioctl(s->sock_fd, SIOCGIFINDEX, &ifr) < 0) { @@ -1910,9 +1941,9 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, return 0; } if (protoNumber < 0 || protoNumber > 0xffff) { - PyErr_SetString( + PyErr_Format( PyExc_OverflowError, - "getsockaddrarg: protoNumber must be 0-65535."); + "%s(): protoNumber must be 0-65535.", caller); PyBuffer_Release(&haddr); return 0; } @@ -1944,16 +1975,18 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (!PyTuple_Check(args)) { PyErr_Format( PyExc_TypeError, - "getsockaddrarg: " - "AF_TIPC address must be tuple, not %.500s", - Py_TYPE(args)->tp_name); + "%s(): AF_TIPC address must be tuple, not %.500s", + caller, Py_TYPE(args)->tp_name); return 0; } if (!PyArg_ParseTuple(args, - "IIII|I;Invalid TIPC address format", - &atype, &v1, &v2, &v3, &scope)) + "IIII|I;AF_TIPC address must be a tuple " + "(addr_type, v1, v2, v3[, scope])", + &atype, &v1, &v2, &v3, &scope)) + { return 0; + } addr = (struct sockaddr_tipc *) addr_ret; memset(addr, 0, sizeof(struct sockaddr_tipc)); @@ -2002,9 +2035,19 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, Py_ssize_t len; addr = (struct sockaddr_can *)addr_ret; - if (!PyArg_ParseTuple(args, "O&", PyUnicode_FSConverter, - &interfaceName)) + if (!PyTuple_Check(args)) { + PyErr_Format(PyExc_TypeError, + "%s(): AF_CAN address must be tuple, not %.500s", + caller, Py_TYPE(args)->tp_name); + return 0; + } + if (!PyArg_ParseTuple(args, + "O&;AF_CAN address must be a tuple " + "(interface, )", + PyUnicode_FSConverter, &interfaceName)) + { return 0; + } len = PyBytes_GET_SIZE(interfaceName); @@ -2081,8 +2124,8 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, } #endif /* CAN_ISOTP */ default: - PyErr_SetString(PyExc_OSError, - "getsockaddrarg: unsupported CAN protocol"); + PyErr_Format(PyExc_OSError, + "%s(): unsupported CAN protocol", caller); return 0; } #endif /* AF_CAN && SIOCGIFINDEX */ @@ -2128,9 +2171,9 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, addr->sc_unit = 0; } else if (!PyArg_ParseTuple(args, "II", &(addr->sc_id), &(addr->sc_unit))) { - PyErr_SetString(PyExc_TypeError, "getsockaddrarg: " - "expected str or tuple of two ints"); - + PyErr_Format(PyExc_TypeError, + "%s(): PF_SYSTEM address must be a str or " + "a pair (id, unit)", caller); return 0; } @@ -2139,8 +2182,8 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, } #endif /* SYSPROTO_CONTROL */ default: - PyErr_SetString(PyExc_OSError, - "getsockaddrarg: unsupported PF_SYSTEM protocol"); + PyErr_Format(PyExc_OSError, + "%s(): unsupported PF_SYSTEM protocol", caller); return 0; } #endif /* PF_SYSTEM */ @@ -2155,9 +2198,19 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, memset(sa, 0, sizeof(*sa)); sa->salg_family = AF_ALG; - if (!PyArg_ParseTuple(args, "ss|HH:getsockaddrarg", - &type, &name, &sa->salg_feat, &sa->salg_mask)) + if (!PyTuple_Check(args)) { + PyErr_Format(PyExc_TypeError, + "%s(): AF_ALG address must be tuple, not %.500s", + caller, Py_TYPE(args)->tp_name); + return 0; + } + if (!PyArg_ParseTuple(args, + "ss|HH;AF_ALG address must be a tuple " + "(type, name[, feat[, mask]])", + &type, &name, &sa->salg_feat, &sa->salg_mask)) + { return 0; + } /* sockaddr_alg has fixed-sized char arrays for type and name */ if (strlen(type) > sizeof(sa->salg_type)) { PyErr_SetString(PyExc_ValueError, "AF_ALG type too long."); @@ -2178,7 +2231,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, /* More cases here... */ default: - PyErr_SetString(PyExc_OSError, "getsockaddrarg: bad family"); + PyErr_Format(PyExc_OSError, "%s(): bad family", caller); return 0; } @@ -2910,8 +2963,9 @@ sock_bind(PySocketSockObject *s, PyObject *addro) int addrlen; int res; - if (!getsockaddrarg(s, addro, SAS2SA(&addrbuf), &addrlen)) + if (!getsockaddrarg(s, addro, SAS2SA(&addrbuf), &addrlen, "bind")) { return NULL; + } Py_BEGIN_ALLOW_THREADS res = bind(s->sock_fd, SAS2SA(&addrbuf), addrlen); Py_END_ALLOW_THREADS @@ -3074,8 +3128,9 @@ sock_connect(PySocketSockObject *s, PyObject *addro) int addrlen; int res; - if (!getsockaddrarg(s, addro, SAS2SA(&addrbuf), &addrlen)) + if (!getsockaddrarg(s, addro, SAS2SA(&addrbuf), &addrlen, "connect")) { return NULL; + } res = internal_connect(s, SAS2SA(&addrbuf), addrlen, 1); if (res < 0) @@ -3100,8 +3155,9 @@ sock_connect_ex(PySocketSockObject *s, PyObject *addro) int addrlen; int res; - if (!getsockaddrarg(s, addro, SAS2SA(&addrbuf), &addrlen)) + if (!getsockaddrarg(s, addro, SAS2SA(&addrbuf), &addrlen, "connect_ex")) { return NULL; + } res = internal_connect(s, SAS2SA(&addrbuf), addrlen, 0); if (res < 0) @@ -4100,7 +4156,7 @@ sock_sendto(PySocketSockObject *s, PyObject *args) return select_error(); } - if (!getsockaddrarg(s, addro, SAS2SA(&addrbuf), &addrlen)) { + if (!getsockaddrarg(s, addro, SAS2SA(&addrbuf), &addrlen, "sendto")) { PyBuffer_Release(&pbuf); return NULL; } @@ -4231,8 +4287,11 @@ sock_sendmsg(PySocketSockObject *s, PyObject *args) /* Parse destination address. */ if (addr_arg != NULL && addr_arg != Py_None) { - if (!getsockaddrarg(s, addr_arg, SAS2SA(&addrbuf), &addrlen)) + if (!getsockaddrarg(s, addr_arg, SAS2SA(&addrbuf), &addrlen, + "sendmsg")) + { goto finally; + } msg.msg_name = &addrbuf; msg.msg_namelen = addrlen; } From webhook-mailer at python.org Tue Sep 11 12:54:45 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 11 Sep 2018 16:54:45 -0000 Subject: [Python-checkins] bpo-33649: First asyncio docs improvement pass (GH-9142) Message-ID: https://github.com/python/cpython/commit/7c7605ff1133cf757cac428c483827f666c7c827 commit: 7c7605ff1133cf757cac428c483827f666c7c827 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-11T09:54:40-07:00 summary: bpo-33649: First asyncio docs improvement pass (GH-9142) Rewritten/updated sections: * Event Loop APIs * Transports & Protocols * Streams * Exceptions * Policies * Queues * Subprocesses * Platforms files: A Doc/library/asyncio-exceptions.rst A Doc/library/asyncio-platforms.rst A Doc/library/asyncio-policy.rst D Doc/library/asyncio-eventloops.rst M Doc/library/asyncio-dev.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-protocol.rst M Doc/library/asyncio-queue.rst M Doc/library/asyncio-stream.rst M Doc/library/asyncio-subprocess.rst M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst M Doc/library/ipc.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.7.rst M Doc/whatsnew/3.8.rst diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index 100fff561c5b..cb574c308ad1 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -23,7 +23,7 @@ To enable all debug checks for an application: * Enable the asyncio debug mode globally by setting the environment variable :envvar:`PYTHONASYNCIODEBUG` to ``1``, using ``-X dev`` command line option (see the :option:`-X` option), or by calling - :meth:`AbstractEventLoop.set_debug`. + :meth:`loop.set_debug`. * Set the log level of the :ref:`asyncio logger ` to :py:data:`logging.DEBUG`. For example, call ``logging.basicConfig(level=logging.DEBUG)`` at startup. @@ -35,11 +35,11 @@ Examples debug checks: * Log :ref:`coroutines defined but never "yielded from" ` -* :meth:`~AbstractEventLoop.call_soon` and :meth:`~AbstractEventLoop.call_at` methods +* :meth:`loop.call_soon` and :meth:`loop.call_at` methods raise an exception if they are called from the wrong thread. * Log the execution time of the selector * Log callbacks taking more than 100 ms to be executed. The - :attr:`AbstractEventLoop.slow_callback_duration` attribute is the minimum + :attr:`loop.slow_callback_duration` attribute is the minimum duration in seconds of "slow" callbacks. * :exc:`ResourceWarning` warnings are emitted when transports and event loops are :ref:`not closed explicitly `. @@ -51,7 +51,7 @@ Examples debug checks: .. seealso:: - The :meth:`AbstractEventLoop.set_debug` method and the :ref:`asyncio logger + The :meth:`loop.set_debug` method and the :ref:`asyncio logger `. @@ -75,7 +75,7 @@ For example, write:: Don't schedule directly a call to the :meth:`~Future.set_result` or the :meth:`~Future.set_exception` method of a future with -:meth:`AbstractEventLoop.call_soon`: the future can be cancelled before its method +:meth:`loop.call_soon`: the future can be cancelled before its method is called. If you wait for a future, you should check early if the future was cancelled to @@ -96,13 +96,14 @@ The :func:`shield` function can also be used to ignore cancellation. Concurrency and multithreading ------------------------------ -An event loop runs in a thread and executes all callbacks and tasks in the same -thread. While a task is running in the event loop, no other task is running in -the same thread. But when the task uses ``await``, the task is suspended -and the event loop executes the next task. +An event loop runs in a thread (typically the main thread) and executes +all callbacks and tasks in its thread. While a task is running in the +event loop, no other task is running in the same thread. When a task +executes an ``await`` expression, the task gets suspended and the +event loop executes the next task. To schedule a callback from a different thread, the -:meth:`AbstractEventLoop.call_soon_threadsafe` method should be used. Example:: +:meth:`loop.call_soon_threadsafe` method should be used. Example:: loop.call_soon_threadsafe(callback, *args) @@ -122,7 +123,7 @@ To schedule a coroutine object from a different thread, the future = asyncio.run_coroutine_threadsafe(coro_func(), loop) result = future.result(timeout) # Wait for the result with a timeout -The :meth:`AbstractEventLoop.run_in_executor` method can be used with a thread pool +The :meth:`loop.run_in_executor` method can be used with a thread pool executor to execute a callback in different thread to not block the thread of the event loop. @@ -151,7 +152,7 @@ APIs like :ref:`protocols `. An executor can be used to run a task in a different thread or even in a different process, to not block the thread of the event loop. See the -:meth:`AbstractEventLoop.run_in_executor` method. +:meth:`loop.run_in_executor` method. .. seealso:: @@ -182,7 +183,7 @@ Detect coroutine objects never scheduled ---------------------------------------- When a coroutine function is called and its result is not passed to -:func:`ensure_future` or to the :meth:`AbstractEventLoop.create_task` method, +:func:`ensure_future` or to the :meth:`loop.create_task` method, the execution of the coroutine object will never be scheduled which is probably a bug. :ref:`Enable the debug mode of asyncio ` to :ref:`log a warning ` to detect it. @@ -204,7 +205,7 @@ Output in debug mode:: test() The fix is to call the :func:`ensure_future` function or the -:meth:`AbstractEventLoop.create_task` method with the coroutine object. +:meth:`loop.create_task` method with the coroutine object. .. seealso:: @@ -279,7 +280,7 @@ coroutine in another coroutine and use classic try/except:: loop.run_forever() loop.close() -Another option is to use the :meth:`AbstractEventLoop.run_until_complete` +Another option is to use the :meth:`loop.run_until_complete` function:: task = asyncio.ensure_future(bug()) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 10068538c7a4..c200844385c1 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1,103 +1,161 @@ .. currentmodule:: asyncio -.. _asyncio-event-loop: -Base Event Loop -=============== +========== +Event Loop +========== -**Source code:** :source:`Lib/asyncio/events.py` -The event loop is the central execution device provided by :mod:`asyncio`. -It provides multiple facilities, including: +.. rubric:: Preface -* Registering, executing and cancelling delayed calls (timeouts). +An event loop is the central component of every asyncio application. +Event loops run asynchronous tasks and callbacks, perform network +IO operations, run subprocesses, etc. -* Creating client and server :ref:`transports ` for various - kinds of communication. +In general, it is *not recommended* to use event loops directly at +the application-level asyncio code. They should only be accessed +in low-level code in libraries and frameworks. -* Launching subprocesses and the associated :ref:`transports - ` for communication with an external program. +High-level asyncio applications should not need to work with event +loops and should use the :func:`asyncio.run` function to initialize +and run asynchronous code. -* Delegating costly function calls to a pool of threads. -.. class:: BaseEventLoop +.. rubric:: Accessing Event Loop - This class is an implementation detail. It is a subclass of - :class:`AbstractEventLoop` and may be a base class of concrete - event loop implementations found in :mod:`asyncio`. It should not - be used directly; use :class:`AbstractEventLoop` instead. - ``BaseEventLoop`` should not be subclassed by third-party code; the - internal interface is not stable. +The following low-level functions can be used to get, set, or create +an event loop: -.. class:: AbstractEventLoop +.. function:: get_running_loop() - Abstract base class of event loops. + Return the running event loop in the current OS thread. - This class is :ref:`not thread safe `. + If there is no running event loop a :exc:`RuntimeError` is raised. + This function can only be called from a coroutine or a callback. -Run an event loop ------------------ + .. versionadded:: 3.7 -.. method:: AbstractEventLoop.run_forever() +.. function:: get_event_loop() - Run until :meth:`stop` is called. If :meth:`stop` is called before - :meth:`run_forever()` is called, this polls the I/O selector once - with a timeout of zero, runs all callbacks scheduled in response to - I/O events (and those that were already scheduled), and then exits. - If :meth:`stop` is called while :meth:`run_forever` is running, - this will run the current batch of callbacks and then exit. Note - that callbacks scheduled by callbacks will not run in that case; - they will run the next time :meth:`run_forever` is called. + Get the current event loop. If there is no current event loop set + in the current OS thread and :func:`set_event_loop` has not yet + been called, asyncio will create a new event loop and set it as the + current one. - .. versionchanged:: 3.5.1 + Because this function has a rather complex behavior (especially + when custom event loop policies are in use), it is recommended + to use the :func:`get_running_loop` function in coroutines and + callbacks instead. -.. method:: AbstractEventLoop.run_until_complete(future) + Consider also using the :func:`asyncio.run` function instead of + manually creating and closing an event loop. - Run until the :class:`Future` is done. +.. function:: set_event_loop(loop) - If the argument is a :ref:`coroutine object `, it is wrapped by - :func:`ensure_future`. + Set *loop* as a current event loop for the current OS thread. - Return the Future's result, or raise its exception. +.. function:: new_event_loop() -.. method:: AbstractEventLoop.is_running() + Create a new event loop object. - Returns running status of event loop. +Note that the behaviour of :func:`get_event_loop`, :func:`set_event_loop`, +and :func:`new_event_loop` functions can be altered by +:ref:`setting a custom event loop policy `. -.. method:: AbstractEventLoop.stop() - Stop running the event loop. +.. rubric:: Contents - This causes :meth:`run_forever` to exit at the next suitable - opportunity (see there for more details). +This documentation page contains the following sections: - .. versionchanged:: 3.5.1 +* The `Event Loop Methods`_ section is a reference documentation of + event loop APIs; + +* The `Callback Handles`_ section documents the :class:`Handle` and + :class:`TimerHandle`, instances of which are returned from functions + :meth:`loop.call_soon`, :meth:`loop.call_later`, etc; + +* The `Server Objects`_ sections documents types returned from + event loop methods like :meth:`loop.create_server`; + +* The `Event Loops Implementations`_ section documents the + :class:`SelectorEventLoop` and :class:`ProactorEventLoop` classes; + +* The `Examples`_ section showcases how to work with some event + loop APIs. + + +.. _asyncio-event-loop: + +Event Loop Methods +================== + +Event loops provide the following **low-level** APIs: + +.. contents:: + :depth: 1 + :local: + + +Running and stopping the loop +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. method:: loop.run_until_complete(future) + + Run until the *future* (an instance of :class:`Future`) is + completed. + + If the argument is a :ref:`coroutine object ` it + is implicitly wrapped into an :class:`asyncio.Task`. + + Return the Future's result or raise its exception. + +.. method:: loop.run_forever() + + Run the event loop until :meth:`stop` is called. + + If :meth:`stop` is called before :meth:`run_forever()` is called, + the loop will poll the I/O selector once with a timeout of zero, + run all callbacks scheduled in response to I/O events (and + those that were already scheduled), and then exit. + + If :meth:`stop` is called while :meth:`run_forever` is running, + the loop will run the current batch of callbacks and then exit. + Note that callbacks scheduled by callbacks will not run in that + case; they will run the next time :meth:`run_forever` or + :meth:`run_until_complete` is called. -.. method:: AbstractEventLoop.is_closed() +.. method:: loop.stop() - Returns ``True`` if the event loop was closed. + Stop the event loop. - .. versionadded:: 3.4.2 +.. method:: loop.is_running() -.. method:: AbstractEventLoop.close() + Return ``True`` if the event loop is currently running. - Close the event loop. The loop must not be running. Pending - callbacks will be lost. +.. method:: loop.is_closed() - This clears the queues and shuts down the executor, but does not wait for - the executor to finish. + Return ``True`` if the event loop was closed. - This is idempotent and irreversible. No other methods should be called after - this one. +.. method:: loop.close() + Close the event loop. -.. coroutinemethod:: AbstractEventLoop.shutdown_asyncgens() + The loop cannot not be running when this function is called. + Any pending callbacks will be discarded. + + This method clears all queues and shuts down the executor, but does + not wait for the executor to finish. + + This method is idempotent and irreversible. No other methods + should be called after the event loop is closed. + +.. coroutinemethod:: loop.shutdown_asyncgens() Schedule all currently open :term:`asynchronous generator` objects to close with an :meth:`~agen.aclose()` call. After calling this method, - the event loop will issue a warning whenever a new asynchronous generator - is iterated. Should be used to finalize all scheduled asynchronous - generators reliably. Example:: + the event loop will issue a warning if a new asynchronous generator + is iterated. Should be used to reliably finalize all scheduled + asynchronous generators, e.g.: try: loop.run_forever() @@ -110,163 +168,147 @@ Run an event loop .. _asyncio-pass-keywords: -Calls ------ - -Most :mod:`asyncio` functions don't accept keywords. If you want to pass -keywords to your callback, use :func:`functools.partial`. For example, -``loop.call_soon(functools.partial(print, "Hello", flush=True))`` will call -``print("Hello", flush=True)``. - -.. note:: - :func:`functools.partial` is better than ``lambda`` functions, because - :mod:`asyncio` can inspect :func:`functools.partial` object to display - parameters in debug mode, whereas ``lambda`` functions have a poor - representation. - -.. method:: AbstractEventLoop.call_soon(callback, *args, context=None) +Scheduling callbacks +^^^^^^^^^^^^^^^^^^^^ - Arrange for a callback to be called as soon as possible. The callback is - called after :meth:`call_soon` returns, when control returns to the event - loop. +.. method:: loop.call_soon(callback, *args, context=None) - This operates as a :abbr:`FIFO (first-in, first-out)` queue, callbacks - are called in the order in which they are registered. Each callback - will be called exactly once. + Schedule *callback* to be called with *args* arguments at + the next iteration of the event loop. - Any positional arguments after the callback will be passed to the - callback when it is called. + Callbacks are called in the order in which they are registered. + Each callback will be called exactly once. - An optional keyword-only *context* argument allows specifying a custom - :class:`contextvars.Context` for the *callback* to run in. The current - context is used when no *context* is provided. + An optional keyword-only *context* argument allows specifying a + custom :class:`contextvars.Context` for the *callback* to run in. + The current context is used when no *context* is provided. An instance of :class:`asyncio.Handle` is returned, which can be used to cancel the callback. - :ref:`Use functools.partial to pass keywords to the callback - `. +.. method:: loop.call_soon_threadsafe(callback, *args, context=None) - .. versionchanged:: 3.7 - The *context* keyword-only parameter was added. See :pep:`567` - for more details. - -.. method:: AbstractEventLoop.call_soon_threadsafe(callback, *args, context=None) - - Like :meth:`call_soon`, but thread safe. + A thread-safe variant of :meth:`call_soon`. Must be used to + schedule callbacks *from another thread*. See the :ref:`concurrency and multithreading ` section of the documentation. - .. versionchanged:: 3.7 - The *context* keyword-only parameter was added. See :pep:`567` - for more details. +.. versionchanged:: 3.7 + The *context* keyword-only parameter was added. See :pep:`567` + for more details. +.. note:: -.. _asyncio-delayed-calls: + Most :mod:`asyncio` scheduling functions don't allow to pass + keyword arguments. To do that, use :func:`functools.partial`, + e.g.:: -Delayed calls -------------- + # will schedule "print("Hello", flush=True)": + loop.call_soon( + functools.partial(print, "Hello", flush=True)) -The event loop has its own internal clock for computing timeouts. -Which clock is used depends on the (platform-specific) event loop -implementation; ideally it is a monotonic clock. This will generally be -a different clock than :func:`time.time`. + Using partial objects is usually more convenient than using lambdas, + as asyncio can better render partial objects in debug and error + messages. -.. method:: AbstractEventLoop.call_later(delay, callback, *args, context=None) +.. _asyncio-delayed-calls: - Arrange for the *callback* to be called after the given *delay* - seconds (either an int or float). +Scheduling delayed callbacks +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - An instance of :class:`asyncio.TimerHandle` is returned, which can be - used to cancel the callback. +Event loop provides mechanisms to schedule callback functions +to be called at some point in the future. Event loop uses monotonic +clocks to track time. + + +.. method:: loop.call_later(delay, callback, *args, context=None) - *callback* will be called exactly once per call to :meth:`call_later`. - If two callbacks are scheduled for exactly the same time, it is - undefined which will be called first. + Schedule *callback* to be called after the given *delay* + number of seconds (can be either an int or a float). - The optional positional *args* will be passed to the callback when it - is called. If you want the callback to be called with some named - arguments, use a closure or :func:`functools.partial`. + An instance of :class:`asyncio.TimerHandle` is returned which can + be used to cancel the callback. - An optional keyword-only *context* argument allows specifying a custom - :class:`contextvars.Context` for the *callback* to run in. The current - context is used when no *context* is provided. + *callback* will be called exactly once. If two callbacks are + scheduled for exactly the same time, it is undefined which will + be called first. - :ref:`Use functools.partial to pass keywords to the callback - `. + The optional positional *args* will be passed to the callback when + it is called. If you want the callback to be called with keyword + arguments use :func:`functools.partial`. + + An optional keyword-only *context* argument allows specifying a + custom :class:`contextvars.Context` for the *callback* to run in. + The current context is used when no *context* is provided. .. versionchanged:: 3.7 The *context* keyword-only parameter was added. See :pep:`567` for more details. -.. method:: AbstractEventLoop.call_at(when, callback, *args, context=None) +.. method:: loop.call_at(when, callback, *args, context=None) - Arrange for the *callback* to be called at the given absolute timestamp - *when* (an int or float), using the same time reference as - :meth:`AbstractEventLoop.time`. + Schedule *callback* to be called at the given absolute timestamp + *when* (an int or a float), using the same time reference as + :meth:`loop.time`. This method's behavior is the same as :meth:`call_later`. - An instance of :class:`asyncio.TimerHandle` is returned, which can be - used to cancel the callback. - - :ref:`Use functools.partial to pass keywords to the callback - `. + An instance of :class:`asyncio.TimerHandle` is returned which can + be used to cancel the callback. .. versionchanged:: 3.7 The *context* keyword-only parameter was added. See :pep:`567` for more details. -.. method:: AbstractEventLoop.time() +.. method:: loop.time() - Return the current time, as a :class:`float` value, according to the - event loop's internal clock. + Return the current time, as a :class:`float` value, according to + the event loop's internal monotonic clock. + +.. note:: + + Timeouts (relative *delay* or absolute *when*) should not + exceed one day. .. seealso:: The :func:`asyncio.sleep` function. -Futures -------- +Creating Futures and Tasks +^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. method:: AbstractEventLoop.create_future() +.. method:: loop.create_future() - Create an :class:`asyncio.Future` object attached to the loop. + Create an :class:`asyncio.Future` object attached to the event loop. - This is a preferred way to create futures in asyncio, as event - loop implementations can provide alternative implementations - of the Future class (with better performance or instrumentation). + This is the preferred way to create Futures in asyncio, that lets + third-party event loops to provide alternative implementations of + the Future object (with better performance or instrumentation). .. versionadded:: 3.5.2 +.. method:: loop.create_task(coro, \*, name=None) -Tasks ------ - -.. method:: AbstractEventLoop.create_task(coro, \*, name=None) + Schedule the execution of a :ref:`coroutine`. + Return a :class:`Task` object. - Schedule the execution of a :ref:`coroutine object `: wrap it in - a future. Return a :class:`Task` object. + Third-party event loops can use their own subclass of :class:`Task` + for interoperability. In this case, the result type is a subclass + of :class:`Task`. - Third-party event loops can use their own subclass of :class:`Task` for - interoperability. In this case, the result type is a subclass of - :class:`Task`. - - If the *name* argument is provided and not ``None``, it is set as the name - of the task using :meth:`Task.set_name`. - - .. versionadded:: 3.4.2 + If the *name* argument is provided and not ``None``, it is set as + the name of the task using :meth:`Task.set_name`. .. versionchanged:: 3.8 Added the ``name`` parameter. -.. method:: AbstractEventLoop.set_task_factory(factory) +.. method:: loop.set_task_factory(factory) Set a task factory that will be used by - :meth:`AbstractEventLoop.create_task`. + :meth:`loop.create_task`. If *factory* is ``None`` the default task factory will be set. @@ -275,53 +317,59 @@ Tasks event loop, *coro* will be a coroutine object. The callable must return an :class:`asyncio.Future` compatible object. - .. versionadded:: 3.4.4 +.. method:: loop.get_task_factory() + + Return a task factory or ``None`` if the default one is in use. -.. method:: AbstractEventLoop.get_task_factory() - Return a task factory, or ``None`` if the default one is in use. +Opening network connections +^^^^^^^^^^^^^^^^^^^^^^^^^^^ - .. versionadded:: 3.4.4 +.. coroutinemethod:: loop.create_connection(protocol_factory, \ + host=None, port=None, \*, ssl=None, \ + family=0, proto=0, flags=0, sock=None, \ + local_addr=None, server_hostname=None, \ + ssl_handshake_timeout=None) + Open a streaming transport connection to a given + address specified by *host* and *port*. -Creating connections --------------------- + The socket family can be either :py:data:`~socket.AF_INET` or + :py:data:`~socket.AF_INET6` depending on *host* (or the *family* + argument, if provided). -.. coroutinemethod:: AbstractEventLoop.create_connection(protocol_factory, host=None, port=None, \*, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None) + The socket type will be :py:data:`~socket.SOCK_STREAM`. - Create a streaming transport connection to a given Internet *host* and - *port*: socket family :py:data:`~socket.AF_INET` or - :py:data:`~socket.AF_INET6` depending on *host* (or *family* if specified), - socket type :py:data:`~socket.SOCK_STREAM`. *protocol_factory* must be a - callable returning a :ref:`protocol ` instance. + *protocol_factory* must be a callable returning an + :ref:`asyncio protocol ` implementation. This method will try to establish the connection in the background. When successful, it returns a ``(transport, protocol)`` pair. The chronological synopsis of the underlying operation is as follows: - #. The connection is established, and a :ref:`transport ` - is created to represent it. + #. The connection is established and a :ref:`transport ` + is created for it. - #. *protocol_factory* is called without arguments and must return a - :ref:`protocol ` instance. + #. *protocol_factory* is called without arguments and is expected to + return a :ref:`protocol ` instance. - #. The protocol instance is tied to the transport, and its - :meth:`connection_made` method is called. + #. The protocol instance is coupled with the transport by calling its + :meth:`~BaseProtocol.connection_made` method. - #. The coroutine returns successfully with the ``(transport, protocol)`` - pair. + #. A ``(transport, protocol)`` tuple is returned on success. - The created transport is an implementation-dependent bidirectional stream. + The created transport is an implementation-dependent bidirectional + stream. .. note:: *protocol_factory* can be any kind of callable, not necessarily a class. For example, if you want to use a pre-created protocol instance, you can pass ``lambda: my_protocol``. - Options that change how the connection is created: + Other arguments: - * *ssl*: if given and not false, a SSL/TLS transport is created + * *ssl*: if given and not false, an SSL/TLS transport is created (by default a plain TCP transport is created). If *ssl* is a :class:`ssl.SSLContext` object, this context is used to create the transport; if *ssl* is :const:`True`, a context with some @@ -359,28 +407,41 @@ Creating connections The *ssl_handshake_timeout* parameter. + .. versionchanged:: 3.6 + + The socket option :py:data:`~socket.TCP_NODELAY` is set by default + for all TCP connections. + .. versionchanged:: 3.5 - On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. + Added support for SSL/TLS for :class:`ProactorEventLoop`. .. seealso:: - The :func:`open_connection` function can be used to get a pair of - (:class:`StreamReader`, :class:`StreamWriter`) instead of a protocol. + The :func:`open_connection` function is a high-level alternative + API. It returns a pair of (:class:`StreamReader`, :class:`StreamWriter`) + that can be used directly in async/await code. +.. coroutinemethod:: loop.create_datagram_endpoint(protocol_factory, \ + local_addr=None, remote_addr=None, \*, \ + family=0, proto=0, flags=0, \ + reuse_address=None, reuse_port=None, \ + allow_broadcast=None, sock=None) -.. coroutinemethod:: AbstractEventLoop.create_datagram_endpoint(protocol_factory, local_addr=None, remote_addr=None, \*, family=0, proto=0, flags=0, reuse_address=None, reuse_port=None, allow_broadcast=None, sock=None) + Create a datagram connection. - Create datagram connection: socket family :py:data:`~socket.AF_INET`, - :py:data:`~socket.AF_INET6` or :py:data:`~socket.AF_UNIX` depending on - *host* (or *family* if specified), socket type - :py:data:`~socket.SOCK_DGRAM`. *protocol_factory* must be a - callable returning a :ref:`protocol ` instance. + The socket family can be either :py:data:`~socket.AF_INET`, + :py:data:`~socket.AF_INET6`, or :py:data:`~socket.AF_UNIX`, + depending on *host* (or the *family* argument, if provided). - This method will try to establish the connection in the background. - When successful, it returns a ``(transport, protocol)`` pair. + The socket type will be :py:data:`~socket.SOCK_DGRAM`. + + *protocol_factory* must be a callable returning a + :ref:`protocol ` implementation. + + A tuple of ``(transport, protocol)`` is returned on success. - Options changing how the connection is created: + Other arguments: * *local_addr*, if given, is a ``(local_host, local_port)`` tuple used to bind the socket to locally. The *local_host* and *local_port* @@ -396,7 +457,7 @@ Creating connections corresponding :mod:`socket` module constants. * *reuse_address* tells the kernel to reuse a local socket in - TIME_WAIT state, without waiting for its natural timeout to + ``TIME_WAIT`` state, without waiting for its natural timeout to expire. If not specified will automatically be set to ``True`` on UNIX. @@ -423,21 +484,24 @@ Creating connections The *family*, *proto*, *flags*, *reuse_address*, *reuse_port, *allow_broadcast*, and *sock* parameters were added. -.. coroutinemethod:: AbstractEventLoop.create_unix_connection(protocol_factory, path=None, \*, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None) +.. coroutinemethod:: loop.create_unix_connection(protocol_factory, \ + path=None, \*, ssl=None, sock=None, \ + server_hostname=None, ssl_handshake_timeout=None) - Create UNIX connection: socket family :py:data:`~socket.AF_UNIX`, socket - type :py:data:`~socket.SOCK_STREAM`. The :py:data:`~socket.AF_UNIX` socket - family is used to communicate between processes on the same machine - efficiently. + Create UNIX connection. - This method will try to establish the connection in the background. - When successful, it returns a ``(transport, protocol)`` pair. + The socket family will be :py:data:`~socket.AF_UNIX`; socket + type will be :py:data:`~socket.SOCK_STREAM`. - *path* is the name of a UNIX domain socket, and is required unless a *sock* - parameter is specified. Abstract UNIX sockets, :class:`str`, - :class:`bytes`, and :class:`~pathlib.Path` paths are supported. + A tuple of ``(transport, protocol)`` is returned on success. - See the :meth:`AbstractEventLoop.create_connection` method for parameters. + *path* is the name of a UNIX domain socket and is required, + unless a *sock* parameter is specified. Abstract UNIX sockets, + :class:`str`, :class:`bytes`, and :class:`~pathlib.Path` paths are + supported. + + See the documentation of the :meth:`loop.create_connection` method + for information about arguments to this method. Availability: UNIX. @@ -450,19 +514,23 @@ Creating connections The *path* parameter can now be a :term:`path-like object`. -Creating listening connections ------------------------------- +Creating network servers +^^^^^^^^^^^^^^^^^^^^^^^^ -.. coroutinemethod:: AbstractEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, ssl_handshake_timeout=None, start_serving=True) +.. coroutinemethod:: loop.create_server(protocol_factory, \ + host=None, port=None, \*, \ + family=socket.AF_UNSPEC, \ + flags=socket.AI_PASSIVE, \ + sock=None, backlog=100, ssl=None, \ + reuse_address=None, reuse_port=None, \ + ssl_handshake_timeout=None, start_serving=True) - Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) bound to - *host* and *port*. + Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) listening + on the *host* and *port* address. - Return a :class:`Server` object, its :attr:`~Server.sockets` attribute - contains created sockets. Use the :meth:`Server.close` method to stop the - server: close listening sockets. + Returns a :class:`Server` object. - Parameters: + Arguments: * The *host* parameter can be a string, in that case the TCP server is bound to *host* and *port*. The *host* parameter can also be a sequence @@ -472,8 +540,9 @@ Creating listening connections for IPv4 and another one for IPv6). * *family* can be set to either :data:`socket.AF_INET` or - :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set - it will be determined from host (defaults to :data:`socket.AF_UNSPEC`). + :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. + If not set it will be determined from host name + (defaults to :data:`~socket.AF_UNSPEC`). * *flags* is a bitmask for :meth:`getaddrinfo`. @@ -488,7 +557,7 @@ Creating listening connections accepted connections. * *reuse_address* tells the kernel to reuse a local socket in - TIME_WAIT state, without waiting for its natural timeout to + ``TIME_WAIT`` state, without waiting for its natural timeout to expire. If not specified will automatically be set to ``True`` on UNIX. @@ -509,30 +578,40 @@ Creating listening connections .. versionadded:: 3.7 - *ssl_handshake_timeout* and *start_serving* parameters. + Added *ssl_handshake_timeout* and *start_serving* parameters. - .. versionchanged:: 3.5 + .. versionchanged:: 3.6 - On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. + The socket option :py:data:`~socket.TCP_NODELAY` is set by default + for all TCP connections. - .. seealso:: + .. versionchanged:: 3.5 - The function :func:`start_server` creates a (:class:`StreamReader`, - :class:`StreamWriter`) pair and calls back a function with this pair. + Added support for SSL/TLS on Windows with + :class:`ProactorEventLoop`. .. versionchanged:: 3.5.1 - The *host* parameter can now be a sequence of strings. + The *host* parameter can be a sequence of strings. + + .. seealso:: + + The :func:`start_server` function is a higher-level alternative API + that returns a pair of :class:`StreamReader` and :class:`StreamWriter` + that can be used in an async/await code. -.. coroutinemethod:: AbstractEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, start_serving=True) +.. coroutinemethod:: loop.create_unix_server(protocol_factory, path=None, \ + \*, sock=None, backlog=100, ssl=None, \ + ssl_handshake_timeout=None, start_serving=True) - Similar to :meth:`AbstractEventLoop.create_server`, but specific to the - socket family :py:data:`~socket.AF_UNIX`. + Similar to :meth:`loop.create_server` but works with the + :py:data:`~socket.AF_UNIX` socket family. - *path* is the name of a UNIX domain socket, and is required unless a *sock* - parameter is specified. Abstract UNIX sockets, :class:`str`, - :class:`bytes`, and :class:`~pathlib.Path` paths are supported. + *path* is the name of a UNIX domain socket, and is required, + unless a *sock* argument is provided. Abstract UNIX sockets, + :class:`str`, :class:`bytes`, and :class:`~pathlib.Path` paths + are supported. Availability: UNIX. @@ -544,26 +623,27 @@ Creating listening connections The *path* parameter can now be a :class:`~pathlib.Path` object. -.. coroutinemethod:: BaseEventLoop.connect_accepted_socket(protocol_factory, sock, \*, ssl=None, ssl_handshake_timeout=None) +.. coroutinemethod:: loop.connect_accepted_socket(protocol_factory, \ + sock, \*, ssl=None, ssl_handshake_timeout=None) - Handle an accepted connection. + Wrap an already accepted connection into a transport/protocol pair. - This is used by servers that accept connections outside of - asyncio but that use asyncio to handle them. + This method can be used by servers that accept connections outside + of asyncio but that use asyncio to handle them. Parameters: - * *sock* is a preexisting socket object returned from an ``accept`` - call. + * *sock* is a preexisting socket object returned from + :meth:`socket.accept `. - * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the - accepted connections. + * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over + the accepted connections. * *ssl_handshake_timeout* is (for an SSL connection) the time in seconds to wait for the SSL handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). - When completed it returns a ``(transport, protocol)`` pair. + Returns a ``(transport, protocol)`` pair. .. versionadded:: 3.7 @@ -572,15 +652,14 @@ Creating listening connections .. versionadded:: 3.5.3 -File Transferring ------------------ +Transferring files +^^^^^^^^^^^^^^^^^^ -.. coroutinemethod:: AbstractEventLoop.sendfile(transport, file, \ - offset=0, count=None, \ - *, fallback=True) +.. coroutinemethod:: loop.sendfile(transport, file, \ + offset=0, count=None, *, fallback=True) - Send a *file* to *transport*, return the total number of bytes - which were sent. + Send a *file* over a *transport*. Return the total number of bytes + sent. The method uses high-performance :meth:`os.sendfile` if available. @@ -594,7 +673,7 @@ File Transferring which were sent. *fallback* set to ``True`` makes asyncio to manually read and send - the file when the platform does not support the sendfile syscall + the file when the platform does not support the sendfile system call (e.g. Windows or SSL socket on Unix). Raise :exc:`SendfileNotAvailableError` if the system does not support @@ -604,26 +683,28 @@ File Transferring TLS Upgrade ------------ +^^^^^^^^^^^ -.. coroutinemethod:: AbstractEventLoop.start_tls(transport, protocol, sslcontext, \*, server_side=False, server_hostname=None, ssl_handshake_timeout=None) +.. coroutinemethod:: loop.start_tls(transport, protocol, \ + sslcontext, \*, server_side=False, \ + server_hostname=None, ssl_handshake_timeout=None) - Upgrades an existing connection to TLS. + Upgrade an existing transport-based connection to TLS. - Returns a new transport instance, that the *protocol* must start using + Return a new transport instance, that the *protocol* must start using immediately after the *await*. The *transport* instance passed to the *start_tls* method should never be used again. Parameters: * *transport* and *protocol* instances that methods like - :meth:`~AbstractEventLoop.create_server` and - :meth:`~AbstractEventLoop.create_connection` return. + :meth:`~loop.create_server` and + :meth:`~loop.create_connection` return. * *sslcontext*: a configured instance of :class:`~ssl.SSLContext`. * *server_side* pass ``True`` when a server-side connection is being - upgraded (like the one created by :meth:`~AbstractEventLoop.create_server`). + upgraded (like the one created by :meth:`~loop.create_server`). * *server_hostname*: sets or overrides the host name that the target server's certificate will be matched against. @@ -635,120 +716,112 @@ TLS Upgrade .. versionadded:: 3.7 -Watch file descriptors ----------------------- - -On Windows with :class:`SelectorEventLoop`, only socket handles are supported -(ex: pipe file descriptors are not supported). - -On Windows with :class:`ProactorEventLoop`, these methods are not supported. - -.. method:: AbstractEventLoop.add_reader(fd, callback, \*args) +Watching file descriptors +^^^^^^^^^^^^^^^^^^^^^^^^^ - Start watching the file descriptor for read availability and then call the - *callback* with specified arguments. +.. method:: loop.add_reader(fd, callback, \*args) - :ref:`Use functools.partial to pass keywords to the callback - `. + Start watching the file descriptor for read availability and + call the *callback* with specified arguments. -.. method:: AbstractEventLoop.remove_reader(fd) +.. method:: loop.remove_reader(fd) Stop watching the file descriptor for read availability. -.. method:: AbstractEventLoop.add_writer(fd, callback, \*args) +.. method:: loop.add_writer(fd, callback, \*args) - Start watching the file descriptor for write availability and then call the - *callback* with specified arguments. + Start watching the file descriptor for write availability and then + call the *callback* with specified arguments. - :ref:`Use functools.partial to pass keywords to the callback - `. + Use :func:`functools.partial` :ref:`to pass keywords + ` to *func*. -.. method:: AbstractEventLoop.remove_writer(fd) +.. method:: loop.remove_writer(fd) Stop watching the file descriptor for write availability. -The :ref:`watch a file descriptor for read events ` -example uses the low-level :meth:`AbstractEventLoop.add_reader` method to register -the file descriptor of a socket. +See also :ref:`Platform Support ` section +for some limitations of these methods. -Low-level socket operations ---------------------------- +Working with socket objects directly +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. coroutinemethod:: AbstractEventLoop.sock_recv(sock, nbytes) +In general, protocols implementations that use transport-based APIs +such as :meth:`loop.create_connection` and :meth:`loop.create_server` +are faster than implementations that work with sockets directly. +However, there are use cases when performance is not critical and +working with :class:`~socket.socket` objects directly is more +convenient. - Receive data from the socket. Modeled after blocking - :meth:`socket.socket.recv` method. +.. coroutinemethod:: loop.sock_recv(sock, nbytes) - The return value is a bytes object - representing the data received. The maximum amount of data to be received - at once is specified by *nbytes*. + Receive data. Asynchronous version of + :meth:`socket.recv() `. - With :class:`SelectorEventLoop` event loop, the socket *sock* must be - non-blocking. + The received data is returned as a bytes object. The maximum amount + of data to be received is specified by the *nbytes* argument. + + The socket *sock* must be non-blocking. .. versionchanged:: 3.7 Even though the method was always documented as a coroutine method, before Python 3.7 it returned a :class:`Future`. - Since Python 3.7, this is an ``async def`` method. + Since Python 3.7 this is an ``async def`` method. -.. coroutinemethod:: AbstractEventLoop.sock_recv_into(sock, buf) +.. coroutinemethod:: loop.sock_recv_into(sock, buf) - Receive data from the socket. Modeled after blocking - :meth:`socket.socket.recv_into` method. + Receive data into a buffer. Modeled after the blocking + :meth:`socket.recv_into() ` method. - The received data is written into *buf* (a writable buffer). - The return value is the number of bytes written. + Return the number of bytes written to the buffer. - With :class:`SelectorEventLoop` event loop, the socket *sock* must be - non-blocking. + The socket *sock* must be non-blocking. .. versionadded:: 3.7 -.. coroutinemethod:: AbstractEventLoop.sock_sendall(sock, data) +.. coroutinemethod:: loop.sock_sendall(sock, data) - Send data to the socket. Modeled after blocking - :meth:`socket.socket.sendall` method. + Send data to the socket. Asynchronous version of + :meth:`socket.sendall() `. - The socket must be connected to a remote socket. This method continues to send data from *data* until either all data has been sent or an error occurs. ``None`` is returned on success. On error, an exception is raised, and there is no way to determine how much data, if any, was successfully processed by the receiving end of the connection. - With :class:`SelectorEventLoop` event loop, the socket *sock* must be - non-blocking. + The socket *sock* must be non-blocking. .. versionchanged:: 3.7 Even though the method was always documented as a coroutine method, before Python 3.7 it returned an :class:`Future`. Since Python 3.7, this is an ``async def`` method. -.. coroutinemethod:: AbstractEventLoop.sock_connect(sock, address) +.. coroutinemethod:: loop.sock_connect(sock, address) + + Connect to a remote socket at *address*. - Connect to a remote socket at *address*. Modeled after - blocking :meth:`socket.socket.connect` method. + Asynchronous version of :meth:`socket.connect() `. - With :class:`SelectorEventLoop` event loop, the socket *sock* must be - non-blocking. + The socket *sock* must be non-blocking. .. versionchanged:: 3.5.2 ``address`` no longer needs to be resolved. ``sock_connect`` will try to check if the *address* is already resolved by calling :func:`socket.inet_pton`. If not, - :meth:`AbstractEventLoop.getaddrinfo` will be used to resolve the + :meth:`loop.getaddrinfo` will be used to resolve the *address*. .. seealso:: - :meth:`AbstractEventLoop.create_connection` + :meth:`loop.create_connection` and :func:`asyncio.open_connection() `. -.. coroutinemethod:: AbstractEventLoop.sock_accept(sock) +.. coroutinemethod:: loop.sock_accept(sock) - Accept a connection. Modeled after blocking - :meth:`socket.socket.accept`. + Accept a connection. Modeled after the blocking + :meth:`socket.accept() ` method. The socket must be bound to an address and listening for connections. The return value is a pair ``(conn, address)`` where *conn* @@ -765,16 +838,15 @@ Low-level socket operations .. seealso:: - :meth:`AbstractEventLoop.create_server` and :func:`start_server`. + :meth:`loop.create_server` and :func:`start_server`. -.. coroutinemethod:: AbstractEventLoop.sock_sendfile(sock, file, \ - offset=0, count=None, \ - *, fallback=True) +.. coroutinemethod:: loop.sock_sendfile(sock, file, offset=0, count=None, \ + \*, fallback=True) - Send a file using high-performance :mod:`os.sendfile` if possible - and return the total number of bytes which were sent. + Send a file using high-performance :mod:`os.sendfile` if possible. + Return the total number of bytes which were sent. - Asynchronous version of :meth:`socket.socket.sendfile`. + Asynchronous version of :meth:`socket.sendfile() `. *sock* must be non-blocking :class:`~socket.socket` of :const:`socket.SOCK_STREAM` type. @@ -795,21 +867,22 @@ Low-level socket operations Raise :exc:`SendfileNotAvailableError` if the system does not support *sendfile* syscall and *fallback* is ``False``. + The socket *sock* must be non-blocking. + .. versionadded:: 3.7 -Resolve host name ------------------ +DNS +^^^ -.. coroutinemethod:: AbstractEventLoop.getaddrinfo(host, port, \*, family=0, type=0, proto=0, flags=0) +.. coroutinemethod:: loop.getaddrinfo(host, port, \*, family=0, \ + type=0, proto=0, flags=0) - This method is a :ref:`coroutine `, similar to - :meth:`socket.getaddrinfo` function but non-blocking. + Asynchronous version of :meth:`socket.getaddrinfo`. -.. coroutinemethod:: AbstractEventLoop.getnameinfo(sockaddr, flags=0) +.. coroutinemethod:: loop.getnameinfo(sockaddr, flags=0) - This method is a :ref:`coroutine `, similar to - :meth:`socket.getnameinfo` function but non-blocking. + Asynchronous version of :meth:`socket.getnameinfo`. .. versionchanged:: 3.7 Both *getaddrinfo* and *getnameinfo* methods were always documented @@ -818,95 +891,99 @@ Resolve host name both methods are coroutines. -Connect pipes -------------- +Working with pipes +^^^^^^^^^^^^^^^^^^ -On Windows with :class:`SelectorEventLoop`, these methods are not supported. -Use :class:`ProactorEventLoop` to support pipes on Windows. +.. coroutinemethod:: loop.connect_read_pipe(protocol_factory, pipe) -.. coroutinemethod:: AbstractEventLoop.connect_read_pipe(protocol_factory, pipe) + Register a read-pipe in the event loop. - Register read pipe in eventloop. + *protocol_factory* must be a callable returning an + :ref:`asyncio protocol ` implementation. - *protocol_factory* should instantiate object with :class:`Protocol` - interface. *pipe* is a :term:`file-like object `. - Return pair ``(transport, protocol)``, where *transport* supports the - :class:`ReadTransport` interface. + *pipe* is a :term:`file-like object `. + + Return pair ``(transport, protocol)``, where *transport* supports + the :class:`ReadTransport` interface. With :class:`SelectorEventLoop` event loop, the *pipe* is set to non-blocking mode. -.. coroutinemethod:: AbstractEventLoop.connect_write_pipe(protocol_factory, pipe) +.. coroutinemethod:: loop.connect_write_pipe(protocol_factory, pipe) + + Register a write-pipe in the event loop. + + *protocol_factory* must be a callable returning an + :ref:`asyncio protocol ` implementation. - Register write pipe in eventloop. + *pipe* is :term:`file-like object `. - *protocol_factory* should instantiate object with :class:`BaseProtocol` - interface. *pipe* is :term:`file-like object `. Return pair ``(transport, protocol)``, where *transport* supports :class:`WriteTransport` interface. With :class:`SelectorEventLoop` event loop, the *pipe* is set to non-blocking mode. +.. note:: + + :class:`SelectorEventLoop` does not support the above methods on + Windows. Use :class:`ProactorEventLoop` instead. + .. seealso:: - The :meth:`AbstractEventLoop.subprocess_exec` and - :meth:`AbstractEventLoop.subprocess_shell` methods. + The :meth:`loop.subprocess_exec` and + :meth:`loop.subprocess_shell` methods. UNIX signals ------------- +^^^^^^^^^^^^ -Availability: UNIX only. - -.. method:: AbstractEventLoop.add_signal_handler(signum, callback, \*args) +.. method:: loop.add_signal_handler(signum, callback, \*args) Add a handler for a signal. Raise :exc:`ValueError` if the signal number is invalid or uncatchable. Raise :exc:`RuntimeError` if there is a problem setting up the handler. - :ref:`Use functools.partial to pass keywords to the callback - `. + Use :func:`functools.partial` :ref:`to pass keywords + ` to *func*. -.. method:: AbstractEventLoop.remove_signal_handler(sig) +.. method:: loop.remove_signal_handler(sig) Remove a handler for a signal. Return ``True`` if a signal handler was removed, ``False`` if not. +Availability: UNIX. + .. seealso:: The :mod:`signal` module. -Executor --------- - -Call a function in an :class:`~concurrent.futures.Executor` (pool of threads or -pool of processes). By default, an event loop uses a thread pool executor -(:class:`~concurrent.futures.ThreadPoolExecutor`). +Executing code in thread or process pools +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. method:: AbstractEventLoop.run_in_executor(executor, func, \*args) +.. method:: loop.run_in_executor(executor, func, \*args) Arrange for a *func* to be called in the specified executor. - The *executor* argument should be an :class:`~concurrent.futures.Executor` + The *executor* argument should be an :class:`concurrent.futures.Executor` instance. The default executor is used if *executor* is ``None``. - :ref:`Use functools.partial to pass keywords to the *func* - `. + Use :func:`functools.partial` :ref:`to pass keywords + ` to *func*. This method returns a :class:`asyncio.Future` object. .. versionchanged:: 3.5.3 - :meth:`BaseEventLoop.run_in_executor` no longer configures the + :meth:`loop.run_in_executor` no longer configures the ``max_workers`` of the thread pool executor it creates, instead leaving it up to the thread pool executor (:class:`~concurrent.futures.ThreadPoolExecutor`) to set the default. -.. method:: AbstractEventLoop.set_default_executor(executor) +.. method:: loop.set_default_executor(executor) Set *executor* as the default executor used by :meth:`run_in_executor`. *executor* should be an instance of @@ -917,13 +994,16 @@ pool of processes). By default, an event loop uses a thread pool executor :class:`~concurrent.futures.ThreadPoolExecutor` is deprecated and will trigger an error in Python 3.9. + *executor* must be an instance of + :class:`concurrent.futures.ThreadPoolExecutor`. + Error Handling API ------------------- +^^^^^^^^^^^^^^^^^^ Allows customizing how exceptions are handled in the event loop. -.. method:: AbstractEventLoop.set_exception_handler(handler) +.. method:: loop.set_exception_handler(handler) Set *handler* as the new event loop exception handler. @@ -936,14 +1016,14 @@ Allows customizing how exceptions are handled in the event loop. will be a ``dict`` object (see :meth:`call_exception_handler` documentation for details about context). -.. method:: AbstractEventLoop.get_exception_handler() +.. method:: loop.get_exception_handler() Return the exception handler, or ``None`` if the default one is in use. .. versionadded:: 3.5.2 -.. method:: AbstractEventLoop.default_exception_handler(context) +.. method:: loop.default_exception_handler(context) Default exception handler. @@ -954,7 +1034,7 @@ Allows customizing how exceptions are handled in the event loop. *context* parameter has the same meaning as in :meth:`call_exception_handler`. -.. method:: AbstractEventLoop.call_exception_handler(context) +.. method:: loop.call_exception_handler(context) Call the current event loop exception handler. @@ -975,10 +1055,10 @@ Allows customizing how exceptions are handled in the event loop. event loops. For any custom exception handling, use :meth:`set_exception_handler()` method. -Debug mode ----------- +Enabling debug mode +^^^^^^^^^^^^^^^^^^^ -.. method:: AbstractEventLoop.get_debug() +.. method:: loop.get_debug() Get the debug mode (:class:`bool`) of the event loop. @@ -986,29 +1066,167 @@ Debug mode :envvar:`PYTHONASYNCIODEBUG` is set to a non-empty string, ``False`` otherwise. - .. versionadded:: 3.4.2 - -.. method:: AbstractEventLoop.set_debug(enabled: bool) +.. method:: loop.set_debug(enabled: bool) Set the debug mode of the event loop. - .. versionadded:: 3.4.2 - .. seealso:: The :ref:`debug mode of asyncio `. -Server ------- -.. class:: Server +Running Subprocesses +^^^^^^^^^^^^^^^^^^^^ + +Methods described in this subsections are low-level. In an +async/await code consider using high-level convenient +:func:`asyncio.create_subprocess_shell` and +:func:`asyncio.create_subprocess_exec` functions instead. + +.. note:: + + The default event loop that asyncio is pre-configured + to use on **Windows** does not support subprocesses. + See :ref:`Subprocess Support on Windows ` + for details. + +.. coroutinemethod:: loop.subprocess_exec(protocol_factory, \*args, \ + stdin=subprocess.PIPE, stdout=subprocess.PIPE, \ + stderr=subprocess.PIPE, \*\*kwargs) + + Create a subprocess from one or more string arguments specified by + *args*. + + *args* must be a list of strings represented by: + + * :class:`str`; + * or :class:`bytes`, encoded to the + :ref:`filesystem encoding `. + + The first string specifies the program to execute, + and the remaining strings specify the arguments. Together string + arguments form the ``argv`` of the program. + + This is similar to the standard library :class:`subprocess.Popen` + class called with ``shell=False`` and the list of strings passed as + the first argument; however, where :class:`~subprocess.Popen` takes + a single argument which is list of strings, *subprocess_exec* + takes multiple string arguments. + + The *protocol_factory* must instantiate a subclass of the + :class:`asyncio.SubprocessProtocol` class. + + Other parameters: + + * *stdin*: either a file-like object representing a pipe to be + connected to the subprocess's standard input stream using + :meth:`~loop.connect_write_pipe`, or the + :const:`subprocess.PIPE` constant (default). By default a new + pipe will be created and connected. - Server listening on sockets. + * *stdout*: either a file-like object representing the pipe to be + connected to the subprocess's standard output stream using + :meth:`~loop.connect_read_pipe`, or the + :const:`subprocess.PIPE` constant (default). By default a new pipe + will be created and connected. + + * *stderr*: either a file-like object representing the pipe to be + connected to the subprocess's standard error stream using + :meth:`~loop.connect_read_pipe`, or one of + :const:`subprocess.PIPE` (default) or :const:`subprocess.STDOUT` + constants. + + By default a new pipe will be created and connected. When + :const:`subprocess.STDOUT` is specified, the subprocess' standard + error stream will be connected to the same pipe as the standard + output stream. + + * All other keyword arguments are passed to :class:`subprocess.Popen` + without interpretation, except for *bufsize*, *universal_newlines* + and *shell*, which should not be specified at all. + + See the constructor of the :class:`subprocess.Popen` class + for documentation on other arguments. + + Returns a pair of ``(transport, protocol)``, where *transport* + conforms to the :class:`asyncio.SubprocessTransport` base class. + +.. coroutinemethod:: loop.subprocess_shell(protocol_factory, cmd, \*, \ + stdin=subprocess.PIPE, stdout=subprocess.PIPE, \ + stderr=subprocess.PIPE, \*\*kwargs) + + Create a subprocess from *cmd*, which can be a :class:`str` or a + :class:`bytes` string encoded to the + :ref:`filesystem encoding `, + using the platform's "shell" syntax. + + This is similar to the standard library :class:`subprocess.Popen` + class called with ``shell=True``. + + The *protocol_factory* must instantiate a subclass of the + :class:`SubprocessProtocol` class. + + See :meth:`~loop.subprocess_exec` for more details about + the remaining arguments. + + Returns a pair of ``(transport, protocol)``, where *transport* + conforms to the :class:`SubprocessTransport` base class. + +.. note:: + It is the application's responsibility to ensure that all whitespace + and metacharacters are quoted appropriately to avoid `shell injection + `_ + vulnerabilities. The :func:`shlex.quote` function can be used to + properly escape whitespace and shell metacharacters in strings that + are going to be used to construct shell commands. - Object created by :meth:`AbstractEventLoop.create_server`, - :meth:`AbstractEventLoop.create_unix_server`, :func:`start_server`, - and :func:`start_unix_server` functions. Don't instantiate the class - directly. + +Callback Handles +================ + +.. class:: Handle + + A callback wrapper object returned by :meth:`loop.call_soon`, + :meth:`loop.call_soon_threadsafe`. + + .. method:: cancel() + + Cancel the call. If the callback is already canceled or executed, + this method has no effect. + + .. method:: cancelled() + + Return ``True`` if the call was cancelled. + + .. versionadded:: 3.7 + +.. class:: TimerHandle + + A callback wrapper object returned by :meth:`loop.call_later`, + and :meth:`loop.call_at`. + + The class is inherited from :class:`Handle`. + + .. method:: when() + + Return a scheduled callback time as :class:`float` seconds. + + The time is an absolute timestamp, using the same time + reference as :meth:`loop.time`. + + .. versionadded:: 3.7 + + +Server Objects +============== + +Server objects are created by :meth:`loop.create_server`, +:meth:`loop.create_unix_server`, :func:`start_server`, +and :func:`start_unix_server` functions. + +Do not instantiate the class directly. + +.. class:: Server *Server* objects are asynchronous context managers. When used in an ``async with`` statement, it's guaranteed that the Server object is @@ -1020,7 +1238,8 @@ Server async with srv: # some code - # At this point, srv is closed and no longer accepts new connections. + # At this point, srv is closed and no longer accepts new + connections. .. versionchanged:: 3.7 @@ -1031,8 +1250,8 @@ Server Stop serving: close listening sockets and set the :attr:`sockets` attribute to ``None``. - The sockets that represent existing incoming client connections are left - open. + The sockets that represent existing incoming client connections + are left open. The server is closed asynchronously, use the :meth:`wait_closed` coroutine to wait until the server is closed. @@ -1051,7 +1270,7 @@ Server the server is already being serving. The new *start_serving* keyword-only parameter to - :meth:`AbstractEventLoop.create_server` and + :meth:`loop.create_server` and :meth:`asyncio.start_server` allows to create a Server object that is not accepting connections right away. In which case this method, or :meth:`Server.serve_forever` can be used @@ -1097,8 +1316,8 @@ Server .. attribute:: sockets - List of :class:`socket.socket` objects the server is listening to, or - ``None`` if the server is closed. + List of :class:`socket.socket` objects the server is listening to, + or ``None`` if the server is closed. .. versionchanged:: 3.7 Prior to Python 3.7 ``Server.sockets`` used to return the @@ -1106,65 +1325,85 @@ Server of that list is returned. -Handle ------- +.. _asyncio-event-loops: -.. class:: Handle +Event Loops Implementations +=========================== - A callback wrapper object returned by :func:`AbstractEventLoop.call_soon`, - :func:`AbstractEventLoop.call_soon_threadsafe`. +asyncio ships with two different event loop implementations: +:class:`SelectorEventLoop` and :class:`ProactorEventLoop`. - .. method:: cancel() +By default asyncio is configured to use :class:`SelectorEventLoop` +on all platforms. - Cancel the call. If the callback is already canceled or executed, - this method has no effect. - .. method:: cancelled() +.. class:: SelectorEventLoop - Return ``True`` if the call was cancelled. + An event loop based on the :mod:`selectors` module. - .. versionadded:: 3.7 + Uses the most efficient *selector* available for the given + platform. It is also possible to manually configure what + exact selector implementation should be used:: -.. class:: TimerHandle + import asyncio + import selectors - A callback wrapper object returned by :func:`AbstractEventLoop.call_later`, - and :func:`AbstractEventLoop.call_at`. + selector = selectors.SelectSelector() + loop = asyncio.SelectorEventLoop(selector) + asyncio.set_event_loop(loop) - The class is inherited from :class:`Handle`. - .. method:: when() + Availability: UNIX, Windows. - Return a scheduled callback time as :class:`float` seconds. - The time is an absolute timestamp, using the same time - reference as :meth:`AbstractEventLoop.time`. +.. class:: ProactorEventLoop - .. versionadded:: 3.7 + An event loop for Windows that uses "I/O Completion Ports" (IOCP). + + Availability: Windows. + + An example how to use :class:`ProactorEventLoop` on Windows:: + + import asyncio + import sys + + if sys.platform == 'win32': + loop = asyncio.ProactorEventLoop() + asyncio.set_event_loop(loop) + + .. seealso:: + + `MSDN documentation on I/O Completion Ports + `_. -SendfileNotAvailableError -------------------------- +.. class:: AbstractEventLoop + Abstract base class for asyncio-compliant event loops. -.. exception:: SendfileNotAvailableError + The :ref:`Event Loop Methods ` section lists all + methods that an alternative implementation of ``AbstractEventLoop`` + should have defined. - Sendfile syscall is not available, subclass of :exc:`RuntimeError`. - Raised if the OS does not support sendfile syscall for - given socket or file type. +Examples +======== +Note that all examples in this section **purposefully** show how +to use low-level event loop APIs such as :meth:`loop.run_forever` +and :meth:`loop.call_soon`. Modern asyncio applications rarely +need to be written this way; consider using high-level functions +like :func:`asyncio.run`. -Event loop examples -------------------- .. _asyncio-hello-world-callback: Hello World with call_soon() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Example using the :meth:`AbstractEventLoop.call_soon` method to schedule a -callback. The callback displays ``"Hello World"`` and then stops the event -loop:: +An example using the :meth:`loop.call_soon` method to schedule a +callback. The callback displays ``"Hello World"`` and then stops the +event loop:: import asyncio @@ -1178,13 +1417,15 @@ loop:: loop.call_soon(hello_world, loop) # Blocking call interrupted by loop.stop() - loop.run_forever() - loop.close() + try: + loop.run_forever() + finally: + loop.close() .. seealso:: - The :ref:`Hello World coroutine ` example - uses a :ref:`coroutine `. + A similar :ref:`Hello World ` + example created with a coroutine and the :func:`run` function. .. _asyncio-date-callback: @@ -1192,9 +1433,9 @@ loop:: Display the current date with call_later() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Example of callback displaying the current date every second. The callback uses -the :meth:`AbstractEventLoop.call_later` method to reschedule itself during 5 -seconds, and then stops the event loop:: +An example of callback displaying the current date every second. The +callback uses the :meth:`loop.call_later` method to reschedule itself +during 5 seconds, and then stops the event loop:: import asyncio import datetime @@ -1213,14 +1454,15 @@ seconds, and then stops the event loop:: loop.call_soon(display_date, end_time, loop) # Blocking call interrupted by loop.stop() - loop.run_forever() - loop.close() + try: + loop.run_forever() + finally: + loop.close() .. seealso:: - The :ref:`coroutine displaying the current date - ` example uses a :ref:`coroutine - `. + A similar :ref:`current date ` example + created with a coroutine and the :func:`run` function. .. _asyncio-watch-read-event: @@ -1229,7 +1471,7 @@ Watch a file descriptor for read events ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Wait until a file descriptor received some data using the -:meth:`AbstractEventLoop.add_reader` method and then close the event loop:: +:meth:`loop.add_reader` method and then close the event loop:: import asyncio from socket import socketpair @@ -1241,8 +1483,10 @@ Wait until a file descriptor received some data using the def reader(): data = rsock.recv(100) print("Received:", data.decode()) + # We are done: unregister the file descriptor loop.remove_reader(rsock) + # Stop the event loop loop.stop() @@ -1252,30 +1496,33 @@ Wait until a file descriptor received some data using the # Simulate the reception of data from the network loop.call_soon(wsock.send, 'abc'.encode()) - # Run the event loop - loop.run_forever() - - # We are done, close sockets and the event loop - rsock.close() - wsock.close() - loop.close() + try: + # Run the event loop + loop.run_forever() + finally: + # We are done, close sockets and the event loop + rsock.close() + wsock.close() + loop.close() .. seealso:: - The :ref:`register an open socket to wait for data using a protocol - ` example uses a low-level protocol created by the - :meth:`AbstractEventLoop.create_connection` method. + * A similar :ref:`example ` + using transports, protocols, and the + :meth:`loop.create_connection` method. - The :ref:`register an open socket to wait for data using streams - ` example uses high-level streams - created by the :func:`open_connection` function in a coroutine. + * Another similar :ref:`example ` + using the high-level :func:`asyncio.open_connection` function + and streams. Set signal handlers for SIGINT and SIGTERM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM` using -the :meth:`AbstractEventLoop.add_signal_handler` method:: +(This example only works on UNIX.) + +Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM` +using the :meth:`loop.add_signal_handler` method:: import asyncio import functools @@ -1286,16 +1533,17 @@ the :meth:`AbstractEventLoop.add_signal_handler` method:: print("got signal %s: exit" % signame) loop.stop() - loop = asyncio.get_event_loop() - for signame in ('SIGINT', 'SIGTERM'): - loop.add_signal_handler(getattr(signal, signame), - functools.partial(ask_exit, signame)) + async def main(): + loop = asyncio.get_running_loop() - print("Event loop running forever, press Ctrl+C to interrupt.") - print("pid %s: send SIGINT or SIGTERM to exit." % os.getpid()) - try: - loop.run_forever() - finally: - loop.close() + for signame in {'SIGINT', 'SIGTERM'}: + loop.add_signal_handler( + getattr(signal, signame), + functools.partial(ask_exit, signame)) + + await asyncio.sleep(3600) + + print("Event loop running for 1 hour, press Ctrl+C to interrupt.") + print(f"pid {os.getpid()}: send SIGINT or SIGTERM to exit.") -This example only works on UNIX. + asyncio.run(main()) diff --git a/Doc/library/asyncio-eventloops.rst b/Doc/library/asyncio-eventloops.rst deleted file mode 100644 index 7f6e9535a8a5..000000000000 --- a/Doc/library/asyncio-eventloops.rst +++ /dev/null @@ -1,244 +0,0 @@ -.. currentmodule:: asyncio - -Event loops -=========== - -**Source code:** :source:`Lib/asyncio/events.py` - -Event loop functions --------------------- - -The following functions are convenient shortcuts to accessing the methods of the -global policy. Note that this provides access to the default policy, unless an -alternative policy was set by calling :func:`set_event_loop_policy` earlier in -the execution of the process. - -.. function:: get_event_loop() - - Equivalent to calling ``get_event_loop_policy().get_event_loop()``. - -.. function:: set_event_loop(loop) - - Equivalent to calling ``get_event_loop_policy().set_event_loop(loop)``. - -.. function:: new_event_loop() - - Equivalent to calling ``get_event_loop_policy().new_event_loop()``. - -.. function:: get_running_loop() - - Return the running event loop in the current OS thread. If there - is no running event loop a :exc:`RuntimeError` is raised. - - .. versionadded:: 3.7 - - -.. _asyncio-event-loops: - -Available event loops ---------------------- - -asyncio currently provides two implementations of event loops: -:class:`SelectorEventLoop` and :class:`ProactorEventLoop`. - -.. class:: SelectorEventLoop - - Event loop based on the :mod:`selectors` module. Subclass of - :class:`AbstractEventLoop`. - - Use the most efficient selector available on the platform. - - On Windows, only sockets are supported (ex: pipes are not supported): - see the `MSDN documentation of select - `_. - -.. class:: ProactorEventLoop - - Proactor event loop for Windows using "I/O Completion Ports" aka IOCP. - Subclass of :class:`AbstractEventLoop`. - - Availability: Windows. - - .. seealso:: - - `MSDN documentation on I/O Completion Ports - `_. - -Example to use a :class:`ProactorEventLoop` on Windows:: - - import asyncio, sys - - if sys.platform == 'win32': - loop = asyncio.ProactorEventLoop() - asyncio.set_event_loop(loop) - -.. _asyncio-platform-support: - -Platform support ----------------- - -The :mod:`asyncio` module has been designed to be portable, but each platform -still has subtle differences and may not support all :mod:`asyncio` features. - -Windows -^^^^^^^ - -Common limits of Windows event loops: - -- :meth:`~AbstractEventLoop.create_unix_connection` and - :meth:`~AbstractEventLoop.create_unix_server` are not supported: the socket - family :data:`socket.AF_UNIX` is specific to UNIX -- :meth:`~AbstractEventLoop.add_signal_handler` and - :meth:`~AbstractEventLoop.remove_signal_handler` are not supported -- :meth:`EventLoopPolicy.set_child_watcher` is not supported. - :class:`ProactorEventLoop` supports subprocesses. It has only one - implementation to watch child processes, there is no need to configure it. - -:class:`SelectorEventLoop` specific limits: - -- :class:`~selectors.SelectSelector` is used which only supports sockets - and is limited to 512 sockets. -- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` only - accept file descriptors of sockets -- Pipes are not supported - (ex: :meth:`~AbstractEventLoop.connect_read_pipe`, - :meth:`~AbstractEventLoop.connect_write_pipe`) -- :ref:`Subprocesses ` are not supported - (ex: :meth:`~AbstractEventLoop.subprocess_exec`, - :meth:`~AbstractEventLoop.subprocess_shell`) - -:class:`ProactorEventLoop` specific limits: - -- :meth:`~AbstractEventLoop.create_datagram_endpoint` (UDP) is not supported -- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` are - not supported - -The resolution of the monotonic clock on Windows is usually around 15.6 msec. -The best resolution is 0.5 msec. The resolution depends on the hardware -(availability of `HPET -`_) and on the Windows -configuration. See :ref:`asyncio delayed calls `. - -.. versionchanged:: 3.5 - - :class:`ProactorEventLoop` now supports SSL. - - -Mac OS X -^^^^^^^^ - -Character devices like PTY are only well supported since Mavericks (Mac OS -10.9). They are not supported at all on Mac OS 10.5 and older. - -On Mac OS 10.6, 10.7 and 10.8, the default event loop is -:class:`SelectorEventLoop` which uses :class:`selectors.KqueueSelector`. -:class:`selectors.KqueueSelector` does not support character devices on these -versions. The :class:`SelectorEventLoop` can be used with -:class:`~selectors.SelectSelector` or :class:`~selectors.PollSelector` to -support character devices on these versions of Mac OS X. Example:: - - import asyncio - import selectors - - selector = selectors.SelectSelector() - loop = asyncio.SelectorEventLoop(selector) - asyncio.set_event_loop(loop) - - -Event loop policies and the default policy ------------------------------------------- - -Event loop management is abstracted with a *policy* pattern, to provide maximal -flexibility for custom platforms and frameworks. Throughout the execution of a -process, a single global policy object manages the event loops available to the -process based on the calling context. A policy is an object implementing the -:class:`AbstractEventLoopPolicy` interface. - -For most users of :mod:`asyncio`, policies never have to be dealt with -explicitly, since the default global policy is sufficient (see below). - -The module-level functions -:func:`get_event_loop` and :func:`set_event_loop` provide convenient access to -event loops managed by the default policy. - - -Event loop policy interface ---------------------------- - -An event loop policy must implement the following interface: - -.. class:: AbstractEventLoopPolicy - - Event loop policy. - - .. method:: get_event_loop() - - Get the event loop for the current context. - - Returns an event loop object implementing the :class:`AbstractEventLoop` - interface. In case called from coroutine, it returns the currently - running event loop. - - Raises an exception in case no event loop has been set for the current - context and the current policy does not specify to create one. It must - never return ``None``. - - .. versionchanged:: 3.6 - - .. method:: set_event_loop(loop) - - Set the event loop for the current context to *loop*. - - .. method:: new_event_loop() - - Create and return a new event loop object according to this policy's - rules. - - If there's need to set this loop as the event loop for the current - context, :meth:`set_event_loop` must be called explicitly. - - -The default policy defines context as the current thread, and manages an event -loop per thread that interacts with :mod:`asyncio`. An exception to this rule -happens when :meth:`~AbstractEventLoopPolicy.get_event_loop` is called from a -running future/coroutine, in which case it will return the current loop -running that future/coroutine. - -If the current thread doesn't already have an event loop associated with it, -the default policy's :meth:`~AbstractEventLoopPolicy.get_event_loop` method -creates one when called from the main thread, but raises :exc:`RuntimeError` -otherwise. - - -Access to the global loop policy --------------------------------- - -.. function:: get_event_loop_policy() - - Get the current event loop policy. - -.. function:: set_event_loop_policy(policy) - - Set the current event loop policy. If *policy* is ``None``, the default - policy is restored. - - -Customizing the event loop policy ---------------------------------- - -To implement a new event loop policy, it is recommended you subclass the -concrete default event loop policy :class:`DefaultEventLoopPolicy` -and override the methods for which you want to change behavior, for example:: - - class MyEventLoopPolicy(asyncio.DefaultEventLoopPolicy): - - def get_event_loop(self): - """Get the event loop. - - This may be None or an instance of EventLoop. - """ - loop = super().get_event_loop() - # Do something with loop ... - return loop - - asyncio.set_event_loop_policy(MyEventLoopPolicy()) diff --git a/Doc/library/asyncio-exceptions.rst b/Doc/library/asyncio-exceptions.rst new file mode 100644 index 000000000000..bcd3599477f7 --- /dev/null +++ b/Doc/library/asyncio-exceptions.rst @@ -0,0 +1,88 @@ +.. currentmodule:: asyncio + + +========== +Exceptions +========== + + +.. exception:: TimeoutError + + The operation has exceeded the given deadline. + + .. note:: + This exception is different from the builtin :exc:`TimeoutError` + exception. + + +.. exception:: CancelledError + + The operation has been cancelled. + + This exception can be caught to perform custom operations on + when asyncio Tasks are cancelled. In almost all situations the + exception must always be re-raised. + + .. note:: + This exception is a subclass of :exc:`Exception`, so it can be + accidentally suppressed by ``try..except`` block:: + + try: + await operation + except Exception: + # The cancellation is broken because the *except* block + # suppresses the CancelledError exception. + log.log('an error has occurred') + + Instead, the following pattern should be used:: + + try: + await operation + except asyncio.CancelledError: + raise + except Exception: + log.log('an error has occurred') + + +.. exception:: InvalidStateError + + Invalid internal state of :class:`Task` or :class:`Future`. + + Can be raised in situations like setting a result value for a + *Future* object that already has a result value set. + + +.. exception:: SendfileNotAvailableError + + The "sendfile" syscall for is not available for the given + socket or file type. + + A subclass of :exc:`RuntimeError`. + + +.. exception:: IncompleteReadError + + Incomplete read error. + + Raised by :ref:`asyncio streams ` APIs. + + This exception is a subclass of :exc:`EOFError`. + + .. attribute:: expected + + Total number (:class:`int`) of expected bytes. + + .. attribute:: partial + + Read :class:`bytes` string before the end of stream was reached. + + +.. exception:: LimitOverrunError + + Reached the buffer limit while looking for a separator. + + Raised by :ref:`asyncio streams ` APIs. + + .. attribute:: consumed + + Total number of to be consumed bytes. diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst new file mode 100644 index 000000000000..afdbce67cdd4 --- /dev/null +++ b/Doc/library/asyncio-platforms.rst @@ -0,0 +1,105 @@ +.. currentmodule:: asyncio + + +.. _asyncio-platform-support: + + +================= +Platforms Support +================= + +The :mod:`asyncio` module has been designed to be portable, +but some platforms have subtle differences and limitations. + + +All Platforms +============= + +* :meth:`loop.add_reader` and :meth:`loop.add_writer` + cannot be used to monitor file IO. + + +Windows +======= + +All event loops on Windows do not support the following methods: + +* :meth:`loop.create_unix_connection` and + :meth:`loop.create_unix_server` are not supported. + The :data:`socket.AF_UNIX` socket family is specific to UNIX/ + +* :meth:`loop.add_signal_handler` and + :meth:`loop.remove_signal_handler` are not supported. + +:class:`SelectorEventLoop` has the following limitations: + +* :class:`~selectors.SelectSelector` is used to wait on socket events: + it supports sockets and is limited to 512 sockets. + +* :meth:`loop.add_reader` and :meth:`loop.add_writer` only accept + socket handles (e.g. pipe file descriptors are not supported). + +* Pipes are not supported, so the :meth:`loop.connect_read_pipe` + and :meth:`loop.connect_write_pipe` methods are not implemented. + +* :ref:`Subprocesses ` are not supported, i.e. + :meth:`loop.subprocess_exec` and :meth:`loop.subprocess_shell` + methods are not implemented. + +:class:`ProactorEventLoop` has the following limitations: + +* The :meth:`loop.create_datagram_endpoint` method + is not supported. + +* The :meth:`loop.add_reader` and :meth:`loop.add_writer` + methods are not supported. + +The resolution of the monotonic clock on Windows is usually around 15.6 +msec. The best resolution is 0.5 msec. The resolution depends on the +hardware (availability of `HPET +`_) and on the +Windows configuration. + + +.. _asyncio-windows-subprocess: + +Subprocess Support on Windows +----------------------------- + +:class:`SelectorEventLoop` on Windows does not support subproceses, +so :class:`ProactorEventLoop` should be used instead:: + + import asyncio + + asyncio.set_event_loop_policy( + asyncio.WindowsProactorEventLoopPolicy()) + + asyncio.run(your_code()) + + +The :meth:`policy.set_child_watcher() +` function is also +not supported, as :class:`ProactorEventLoop` has a different mechanism +to watch child processes. + + +macOS +===== + +Modern macOS versions are fully supported. + +.. rubric:: macOS <= 10.8 + +On macOS 10.6, 10.7 and 10.8, the default event loop +uses :class:`selectors.KqueueSelector`, which does not support +character devices on these versions. The :class:`SelectorEventLoop` +can be manually configured to use :class:`~selectors.SelectSelector` +or :class:`~selectors.PollSelector` to support character devices on +these older versions of macOS. Example:: + + import asyncio + import selectors + + selector = selectors.SelectSelector() + loop = asyncio.SelectorEventLoop(selector) + asyncio.set_event_loop(loop) diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst new file mode 100644 index 000000000000..7bb1c1587b7a --- /dev/null +++ b/Doc/library/asyncio-policy.rst @@ -0,0 +1,222 @@ +.. currentmodule:: asyncio + + +.. _asyncio-policies: + +======== +Policies +======== + +An event loop policy, a global per-process object, controls +management of the event loop. Each event loop has a default +policy, which can be changed and customized using the API. + +A policy defines the notion of context and manages a +separate event loop per context. The default policy +defines context to be the current thread. + +By using a custom event loop policy, the behavior of +:func:`get_event_loop`, :func:`set_event_loop`, and +:func:`new_event_loop` functions can be customized. + +Policy objects should implement the APIs defined +in the abstract base class :class:`AbstractEventLoopPolicy`. + + +Access the Policy +================= + +The following functions can be used to get and set the policy +for the current process: + +.. function:: get_event_loop_policy() + + Return the current process-wide policy. + +.. function:: set_event_loop_policy(policy) + + Set the current process-wide policy to *policy*. + + If *policy* is set to ``None``, the default policy is restored. + + +Policy Objects +============== + +The abstract event loop policy base class is defined as follows: + +.. class:: AbstractEventLoopPolicy + + An abstract base class for asyncio policies. + + .. method:: get_event_loop() + + Get the event loop for the current context. + + Return an event loop object implementing the + :class:`AbstractEventLoop` interface. + + This method should never return ``None``. + + .. versionchanged:: 3.6 + + .. method:: set_event_loop(loop) + + Set the event loop for the current context to *loop*. + + .. method:: new_event_loop() + + Create and return a new event loop object. + + This method should never return ``None``. + + .. method:: get_child_watcher() + + Get a child process watcher object. + + Return a watcher object implementing the + :class:`AbstractChildWatcher` interface. + + This function is Unix specific. + + .. method:: set_child_watcher(watcher) + + Get the current child process watcher to *watcher*. + + This function is Unix specific. + + +asyncio ships with the following built-in policies: + + +.. class:: DefaultEventLoopPolicy + + The default asyncio policy. Uses :class:`SelectorEventLoop` + on both Unix and Windows platforms. + + There is no need to install the default policy manually; asyncio + is configured to use it automatically. + + +.. class:: WindowsProactorEventLoopPolicy + + An alternative event loop policy that uses the + :class:`ProactorEventLoop` event loop implementation. + + Availability: Windows. + + +Process Watchers +================ + +A process watcher allows customization of how an event loop monitors +child processes on Unix. Specifically, the event loop needs to know +when a child process has finished its execution. + +In asyncio, child processes are created with +:func:`create_subprocess_exec` and :meth:`loop.subprocess_exec` +functions. + +asyncio defines an abstract base class :class:`AbstractChildWatcher` +that child watchers should implement, and has two different +implementations: :class:`SafeChildWatcher` (configured to be used +by default) and :class:`FastChildWatcher`. + +See also the :ref:`Subprocess and Threads ` +section. + +The following two functions can be used to customize the watcher +implementation used by the asyncio event loop: + +.. function:: get_child_watcher() + + Return the current child watcher for the current policy. + +.. function:: set_child_watcher(watcher) + + Set the current child watcher to *watcher* for the current + policy. *watcher* must implement methods defined in the + :class:`AbstractChildWatcher` base class. + +.. note:: + Third-party event loops implementations might not support + custom child watchers. For such event loops, using + :func:`set_child_watcher` might have no effect or even can + be prohibited. + +.. class:: AbstractChildWatcher + + .. method:: add_child_handler(pid, callback, \*args) + + Register a new child handler. + + Arrange for ``callback(pid, returncode, *args)`` to be called + when a process with PID equal to *pid* terminates. Specifying + another callback for the same process replaces the previous + handler. + + *callback* callable must be thread-safe. + + .. method:: remove_child_handler(pid) + + Removes the handler for process with PID equal to *pid*. + + The function returns ``True`` if the handler was successfully + removed, ``False`` if there was nothing to remove. + + .. method:: attach_loop(loop) + + Attach the watcher to an event loop. + + If the watcher was previously attached to an event loop, then + it is first detached before attaching to the new loop. + + Note: loop may be ``None``. + + .. method:: close() + + Close the watcher. + + This method has to be called to ensure that underlying + resources are cleaned-up. + +.. class:: SafeChildWatcher + + This implementation avoids disrupting other code spawning processes + by polling every process explicitly on a :py:data:`SIGCHLD` signal. + + This is a safe solution but it has a significant overhead when + handling a big number of processes (*O(n)* each time a + :py:data:`SIGCHLD` is received). + + asyncio uses this implementation by default. + +.. class:: FastChildWatcher + + This implementation reaps every terminated processes by calling + ``os.waitpid(-1)`` directly, possibly breaking other code spawning + processes and waiting for their termination. + + There is no noticeable overhead when handling a big number of + children (*O(1)* each time a child terminates). + + +Custom Policies +=============== + +To implement a new event loop policy, it is recommended to subclass +:class:`DefaultEventLoopPolicy` and override the methods for which +custom behavior is wanted, e.g.:: + + class MyEventLoopPolicy(asyncio.DefaultEventLoopPolicy): + + def get_event_loop(self): + """Get the event loop. + + This may be None or an instance of EventLoop. + """ + loop = super().get_event_loop() + # Do something with loop ... + return loop + + asyncio.set_event_loop_policy(MyEventLoopPolicy()) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst index 9a08a4a49021..348ced57772f 100644 --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -1,12 +1,65 @@ .. currentmodule:: asyncio -+++++++++++++++++++++++++++++++++++++++++++++ -Transports and protocols (callback based API) -+++++++++++++++++++++++++++++++++++++++++++++ -**Source code:** :source:`Lib/asyncio/transports.py` +======================== +Transports and Protocols +======================== + +.. rubric:: Preface + +Transports and Protocols are used by **low-level** event loop +APIs such as :meth:`loop.create_connection`. They require using +callback-based programming style and enable high-performance +implementations of network or IPC protocols (e.g. HTTP). + +Essentially, transports and protocols should only be used in +libraries and frameworks and never in high-level asyncio +applications. + +This documentation page covers both `Transports`_ and `Protocols`_. + +.. rubric:: Introduction + +At the highest level, the transport is concerned with *how* bytes +are transmitted, while the protocol determines *which* bytes to +transmit (and to some extent when). + +A different way of saying the same thing: a transport is an +abstraction for a socket (or similar I/O endpoint) while a protocol +is an abstraction for an application, from the transport's point +of view. + +Yet another view is simply that the transport and protocol interfaces +together define an abstract interface for using network I/O and +interprocess I/O. + +There is always a 1:1 relationship between transport and protocol +objects: the protocol calls transport methods to send data, +while the transport calls protocol methods to pass it data that +has been received. + +Most of connection oriented event loop methods +(such as :meth:`loop.create_connection`) usually accept a +*protocol_factory* argument used to create a *Protocol* object +for an accepted connection, represented by a *Transport* object. +Such methods usually return a tuple of ``(transport, protocol)``. + +.. rubric:: Contents + +This documentation page contains the following sections: + +* The `Transports`_ section documents asyncio :class:`BaseTransport`, + :class:`ReadTransport`, :class:`WriteTransport`, :class:`Transport`, + :class:`DatagramTransport`, and :class:`SubprocessTransport` + classes. + +* The `Protocols`_ section documents asyncio :class:`BaseProtocol`, + :class:`Protocol`, :class:`BufferedProtocol`, + :class:`DatagramProtocol`, and :class:`SubprocessProtocol` classes. + +* The `Examples`_ section showcases how to work with transports, + protocols, and low-level event loop APIs. -**Source code:** :source:`Lib/asyncio/protocols.py` .. _asyncio-transport: @@ -14,293 +67,356 @@ Transports ========== Transports are classes provided by :mod:`asyncio` in order to abstract -various kinds of communication channels. You generally won't instantiate -a transport yourself; instead, you will call an :class:`AbstractEventLoop` method -which will create the transport and try to initiate the underlying -communication channel, calling you back when it succeeds. +various kinds of communication channels. -Once the communication channel is established, a transport is always -paired with a :ref:`protocol ` instance. The protocol can -then call the transport's methods for various purposes. +Transport objects are always instantiated by an +ref:`asyncio event loop `. -:mod:`asyncio` currently implements transports for TCP, UDP, SSL, and -subprocess pipes. The methods available on a transport depend on -the transport's kind. +asyncio implements transports for TCP, UDP, SSL, and subprocess pipes. +The methods available on a transport depend on the transport's kind. The transport classes are :ref:`not thread safe `. -.. versionchanged:: 3.6 - The socket option ``TCP_NODELAY`` is now set by default. - -BaseTransport -------------- +Transports Hierarchy +-------------------- .. class:: BaseTransport - Base class for transports. + Base class for all transports. Contains methods that all + asyncio transports share. - .. method:: close() +.. class:: WriteTransport(BaseTransport) - Close the transport. If the transport has a buffer for outgoing - data, buffered data will be flushed asynchronously. No more data - will be received. After all buffered data is flushed, the - protocol's :meth:`connection_lost` method will be called with - :const:`None` as its argument. + A base transport for write-only connections. - .. method:: is_closing() + Instances of the *WriteTransport* class are returned from + the :meth:`loop.connect_write_pipe` event loop method and + are also used by subprocess-related methods like + :meth:`loop.subprocess_exec`. - Return ``True`` if the transport is closing or is closed. +.. class:: ReadTransport(BaseTransport) - .. versionadded:: 3.5.1 + A base transport for read-only connections. - .. method:: get_extra_info(name, default=None) + Instances of the *ReadTransport* class are returned from + the :meth:`loop.connect_read_pipe` event loop method and + are also used by subprocess-related methods like + :meth:`loop.subprocess_exec`. - Return optional transport information. *name* is a string representing - the piece of transport-specific information to get, *default* is the - value to return if the information doesn't exist. +.. class:: Transport(WriteTransport, ReadTransport) - This method allows transport implementations to easily expose - channel-specific information. + Interface representing a bidirectional transport, such as a + TCP connection. - * socket: + The user never instantiates a transport directly; they call a + utility function, passing it a protocol factory and other + information necessary to create the transport and protocol. - - ``'peername'``: the remote address to which the socket is connected, - result of :meth:`socket.socket.getpeername` (``None`` on error) - - ``'socket'``: :class:`socket.socket` instance - - ``'sockname'``: the socket's own address, - result of :meth:`socket.socket.getsockname` + Instances of the *Transport* class are returned from or used by + event loop methods like :meth:`loop.create_connection`, + :meth:`loop.create_unix_connection`, + :meth:`loop.create_server`, :meth:`loop.sendfile`, etc. - * SSL socket: - - ``'compression'``: the compression algorithm being used as a string, - or ``None`` if the connection isn't compressed; result of - :meth:`ssl.SSLSocket.compression` - - ``'cipher'``: a three-value tuple containing the name of the cipher - being used, the version of the SSL protocol that defines its use, and - the number of secret bits being used; result of - :meth:`ssl.SSLSocket.cipher` - - ``'peercert'``: peer certificate; result of - :meth:`ssl.SSLSocket.getpeercert` - - ``'sslcontext'``: :class:`ssl.SSLContext` instance - - ``'ssl_object'``: :class:`ssl.SSLObject` or :class:`ssl.SSLSocket` - instance +.. class:: DatagramTransport(BaseTransport) - * pipe: + A transport for datagram (UDP) connections. - - ``'pipe'``: pipe object + Instances of the *DatagramTransport* class are returned from + the :meth:`loop.create_datagram_endpoint` event loop method. - * subprocess: - - ``'subprocess'``: :class:`subprocess.Popen` instance +.. class:: SubprocessTransport(BaseTransport) - .. method:: set_protocol(protocol) + An abstraction to represent a connection between a parent and its + child OS process. - Set a new protocol. Switching protocol should only be done when both - protocols are documented to support the switch. + Instances of the *SubprocessTransport* class are returned from + event loop methods :meth:`loop.subprocess_shell` and + :meth:`loop.subprocess_exec`. - .. versionadded:: 3.5.3 - .. method:: get_protocol +Base Transport +-------------- - Return the current protocol. +.. method:: BaseTransport.close() - .. versionadded:: 3.5.3 + Close the transport. - .. versionchanged:: 3.5.1 - ``'ssl_object'`` info was added to SSL sockets. + If the transport has a buffer for outgoing + data, buffered data will be flushed asynchronously. No more data + will be received. After all buffered data is flushed, the + protocol's :meth:`protocol.connection_lost() + ` method will be called with + :const:`None` as its argument. +.. method:: BaseTransport.is_closing() -ReadTransport -------------- + Return ``True`` if the transport is closing or is closed. -.. class:: ReadTransport +.. method:: BaseTransport.get_extra_info(name, default=None) - Interface for read-only transports. + Return information about the transport or underlying resources + it uses. - .. method:: is_reading() + *name* is a string representing the piece of transport-specific + information to get. - Return ``True`` if the transport is receiving new data. + *default* is the value to return if the information is not + available, or if the transport does not support querying it + with the given third-party event loop implementation or on the + current platform. - .. versionadded:: 3.7 + For example, the following code attempts to get the underlying + socket object of the transport:: - .. method:: pause_reading() + sock = transport.get_extra_info('socket') + if sock is not None: + print(sock.getsockopt(...)) - Pause the receiving end of the transport. No data will be passed to - the protocol's :meth:`data_received` method until :meth:`resume_reading` - is called. + Categories of information that can be queried on some transports: - .. versionchanged:: 3.7 - The method is idempotent, i.e. it can be called when the - transport is already paused or closed. + * socket: - .. method:: resume_reading() + - ``'peername'``: the remote address to which the socket is + connected, result of :meth:`socket.socket.getpeername` + (``None`` on error) - Resume the receiving end. The protocol's :meth:`data_received` method - will be called once again if some data is available for reading. + - ``'socket'``: :class:`socket.socket` instance - .. versionchanged:: 3.7 - The method is idempotent, i.e. it can be called when the - transport is already reading. + - ``'sockname'``: the socket's own address, + result of :meth:`socket.socket.getsockname` + * SSL socket: -WriteTransport --------------- + - ``'compression'``: the compression algorithm being used as a + string, or ``None`` if the connection isn't compressed; result + of :meth:`ssl.SSLSocket.compression` -.. class:: WriteTransport + - ``'cipher'``: a three-value tuple containing the name of the + cipher being used, the version of the SSL protocol that defines + its use, and the number of secret bits being used; result of + :meth:`ssl.SSLSocket.cipher` - Interface for write-only transports. + - ``'peercert'``: peer certificate; result of + :meth:`ssl.SSLSocket.getpeercert` - .. method:: abort() + - ``'sslcontext'``: :class:`ssl.SSLContext` instance - Close the transport immediately, without waiting for pending operations - to complete. Buffered data will be lost. No more data will be received. - The protocol's :meth:`connection_lost` method will eventually be - called with :const:`None` as its argument. + - ``'ssl_object'``: :class:`ssl.SSLObject` or + :class:`ssl.SSLSocket` instance - .. method:: can_write_eof() + * pipe: - Return :const:`True` if the transport supports :meth:`write_eof`, - :const:`False` if not. + - ``'pipe'``: pipe object - .. method:: get_write_buffer_size() + * subprocess: - Return the current size of the output buffer used by the transport. + - ``'subprocess'``: :class:`subprocess.Popen` instance - .. method:: get_write_buffer_limits() +.. method:: BaseTransport.set_protocol(protocol) - Get the *high*- and *low*-water limits for write flow control. Return a - tuple ``(low, high)`` where *low* and *high* are positive number of - bytes. + Set a new protocol. - Use :meth:`set_write_buffer_limits` to set the limits. + Switching protocol should only be done when both + protocols are documented to support the switch. - .. versionadded:: 3.4.2 +.. method:: BaseTransport.get_protocol() - .. method:: set_write_buffer_limits(high=None, low=None) + Return the current protocol. - Set the *high*- and *low*-water limits for write flow control. - These two values (measured in number of - bytes) control when the protocol's - :meth:`pause_writing` and :meth:`resume_writing` methods are called. - If specified, the low-water limit must be less than or equal to the - high-water limit. Neither *high* nor *low* can be negative. +Read-only Transports +-------------------- - :meth:`pause_writing` is called when the buffer size becomes greater - than or equal to the *high* value. If writing has been paused, - :meth:`resume_writing` is called when the buffer size becomes less - than or equal to the *low* value. +.. method:: ReadTransport.is_reading() - The defaults are implementation-specific. If only the - high-water limit is given, the low-water limit defaults to an - implementation-specific value less than or equal to the - high-water limit. Setting *high* to zero forces *low* to zero as - well, and causes :meth:`pause_writing` to be called whenever the - buffer becomes non-empty. Setting *low* to zero causes - :meth:`resume_writing` to be called only once the buffer is empty. - Use of zero for either limit is generally sub-optimal as it - reduces opportunities for doing I/O and computation - concurrently. + Return ``True`` if the transport is receiving new data. - Use :meth:`get_write_buffer_limits` to get the limits. + .. versionadded:: 3.7 - .. method:: write(data) +.. method:: ReadTransport.pause_reading() - Write some *data* bytes to the transport. + Pause the receiving end of the transport. No data will be passed to + the protocol's :meth:`protocol.data_received() ` + method until :meth:`resume_reading` is called. - This method does not block; it buffers the data and arranges for it - to be sent out asynchronously. + .. versionchanged:: 3.7 + The method is idempotent, i.e. it can be called when the + transport is already paused or closed. - .. method:: writelines(list_of_data) +.. method:: ReadTransport.resume_reading() - Write a list (or any iterable) of data bytes to the transport. - This is functionally equivalent to calling :meth:`write` on each - element yielded by the iterable, but may be implemented more efficiently. + Resume the receiving end. The protocol's + :meth:`protocol.data_received() ` method + will be called once again if some data is available for reading. - .. method:: write_eof() + .. versionchanged:: 3.7 + The method is idempotent, i.e. it can be called when the + transport is already reading. - Close the write end of the transport after flushing buffered data. - Data may still be received. - This method can raise :exc:`NotImplementedError` if the transport - (e.g. SSL) doesn't support half-closes. +Write-only Transports +--------------------- +.. method:: WriteTransport.abort() -DatagramTransport ------------------ + Close the transport immediately, without waiting for pending operations + to complete. Buffered data will be lost. No more data will be received. + The protocol's :meth:`protocol.connection_lost() + ` method will eventually be + called with :const:`None` as its argument. -.. method:: DatagramTransport.sendto(data, addr=None) +.. method:: WriteTransport.can_write_eof() - Send the *data* bytes to the remote peer given by *addr* (a - transport-dependent target address). If *addr* is :const:`None`, the - data is sent to the target address given on transport creation. + Return :const:`True` if the transport supports + :meth:`~WriteTransport.write_eof`, :const:`False` if not. + +.. method:: WriteTransport.get_write_buffer_size() + + Return the current size of the output buffer used by the transport. + +.. method:: WriteTransport.get_write_buffer_limits() + + Get the *high*- and *low*-water limits for write flow control. Return a + tuple ``(low, high)`` where *low* and *high* are positive number of + bytes. + + Use :meth:`set_write_buffer_limits` to set the limits. + + .. versionadded:: 3.4.2 + +.. method:: WriteTransport.set_write_buffer_limits(high=None, low=None) + + Set the *high*- and *low*-water limits for write flow control. + + These two values (measured in number of + bytes) control when the protocol's + :meth:`protocol.pause_writing() ` + and :meth:`protocol.resume_writing() ` + methods are called. If specified, the low-water limit must be less + than or equal to the high-water limit. Neither *high* nor *low* + can be negative. + + :meth:`~BaseProtocol.pause_writing` is called when the buffer size + becomes greater than or equal to the *high* value. If writing has + been paused, :meth:`~BaseProtocol.resume_writing` is called when + the buffer size becomes less than or equal to the *low* value. + + The defaults are implementation-specific. If only the + high-water limit is given, the low-water limit defaults to an + implementation-specific value less than or equal to the + high-water limit. Setting *high* to zero forces *low* to zero as + well, and causes :meth:`~BaseProtocol.pause_writing` to be called + whenever the buffer becomes non-empty. Setting *low* to zero causes + :meth:`~BaseProtocol.resume_writing` to be called only once the + buffer is empty. Use of zero for either limit is generally + sub-optimal as it reduces opportunities for doing I/O and + computation concurrently. + + Use :meth:`~WriteTransport.get_write_buffer_limits` + to get the limits. + +.. method:: WriteTransport.write(data) + + Write some *data* bytes to the transport. This method does not block; it buffers the data and arranges for it to be sent out asynchronously. +.. method:: WriteTransport.writelines(list_of_data) + + Write a list (or any iterable) of data bytes to the transport. + This is functionally equivalent to calling :meth:`write` on each + element yielded by the iterable, but may be implemented more + efficiently. + +.. method:: WriteTransport.write_eof() + + Close the write end of the transport after flushing buffered data. + Data may still be received. + + This method can raise :exc:`NotImplementedError` if the transport + (e.g. SSL) doesn't support half-closes. + + +Datagram Transports +------------------- + +.. method:: DatagramTransport.sendto(data, addr=None) + + Send the *data* bytes to the remote peer given by *addr* (a + transport-dependent target address). If *addr* is :const:`None`, + the data is sent to the target address given on transport + creation. + + This method does not block; it buffers the data and arranges + for it to be sent out asynchronously. + .. method:: DatagramTransport.abort() - Close the transport immediately, without waiting for pending operations - to complete. Buffered data will be lost. No more data will be received. - The protocol's :meth:`connection_lost` method will eventually be - called with :const:`None` as its argument. + Close the transport immediately, without waiting for pending + operations to complete. Buffered data will be lost. + No more data will be received. The protocol's + :meth:`protocol.connection_lost() ` + method will eventually be called with :const:`None` as its argument. -BaseSubprocessTransport ------------------------ +.. _asyncio-subprocess-transports: -.. class:: BaseSubprocessTransport +Subprocess Transports +--------------------- - .. method:: get_pid() +.. method:: SubprocessTransport.get_pid() - Return the subprocess process id as an integer. + Return the subprocess process id as an integer. - .. method:: get_pipe_transport(fd) +.. method:: SubprocessTransport.get_pipe_transport(fd) - Return the transport for the communication pipe corresponding to the - integer file descriptor *fd*: + Return the transport for the communication pipe corresponding to the + integer file descriptor *fd*: - * ``0``: readable streaming transport of the standard input (*stdin*), - or :const:`None` if the subprocess was not created with ``stdin=PIPE`` - * ``1``: writable streaming transport of the standard output (*stdout*), - or :const:`None` if the subprocess was not created with ``stdout=PIPE`` - * ``2``: writable streaming transport of the standard error (*stderr*), - or :const:`None` if the subprocess was not created with ``stderr=PIPE`` - * other *fd*: :const:`None` + * ``0``: readable streaming transport of the standard input (*stdin*), + or :const:`None` if the subprocess was not created with ``stdin=PIPE`` + * ``1``: writable streaming transport of the standard output (*stdout*), + or :const:`None` if the subprocess was not created with ``stdout=PIPE`` + * ``2``: writable streaming transport of the standard error (*stderr*), + or :const:`None` if the subprocess was not created with ``stderr=PIPE`` + * other *fd*: :const:`None` - .. method:: get_returncode() +.. method:: SubprocessTransport.get_returncode() - Return the subprocess returncode as an integer or :const:`None` - if it hasn't returned, similarly to the - :attr:`subprocess.Popen.returncode` attribute. + Return the subprocess return code as an integer or :const:`None` + if it hasn't returned, similarly to the + :attr:`subprocess.Popen.returncode` attribute. - .. method:: kill() +.. method:: SubprocessTransport.kill() - Kill the subprocess, as in :meth:`subprocess.Popen.kill`. + Kill the subprocess, as in :meth:`subprocess.Popen.kill`. - On POSIX systems, the function sends SIGKILL to the subprocess. - On Windows, this method is an alias for :meth:`terminate`. + On POSIX systems, the function sends SIGKILL to the subprocess. + On Windows, this method is an alias for :meth:`terminate`. - .. method:: send_signal(signal) +.. method:: SubprocessTransport.send_signal(signal) - Send the *signal* number to the subprocess, as in - :meth:`subprocess.Popen.send_signal`. + Send the *signal* number to the subprocess, as in + :meth:`subprocess.Popen.send_signal`. - .. method:: terminate() +.. method:: SubprocessTransport.terminate() - Ask the subprocess to stop, as in :meth:`subprocess.Popen.terminate`. - This method is an alias for the :meth:`close` method. + Ask the subprocess to stop, as in :meth:`subprocess.Popen.terminate`. + This method is an alias for the :meth:`close` method. - On POSIX systems, this method sends SIGTERM to the subprocess. - On Windows, the Windows API function TerminateProcess() is called to - stop the subprocess. + On POSIX systems, this method sends SIGTERM to the subprocess. + On Windows, the Windows API function TerminateProcess() is called to + stop the subprocess. - .. method:: close() +.. method:: SubprocessTransport.close() - Ask the subprocess to stop by calling the :meth:`terminate` method if the - subprocess hasn't returned yet, and close transports of all pipes - (*stdin*, *stdout* and *stderr*). + Ask the subprocess to stop by calling the :meth:`terminate` method + if the subprocess hasn't returned yet, and close transports of all + pipes (*stdin*, *stdout* and *stderr*). .. _asyncio-protocol: @@ -308,65 +424,62 @@ BaseSubprocessTransport Protocols ========= -:mod:`asyncio` provides base classes that you can subclass to implement -your network protocols. Those classes are used in conjunction with -:ref:`transports ` (see below): the protocol parses incoming -data and asks for the writing of outgoing data, while the transport is -responsible for the actual I/O and buffering. +asyncio provides a set of abstract base classes that should be used +to implement network protocols. Those classes are meant to be used +together with :ref:`transports `. -When subclassing a protocol class, it is recommended you override certain -methods. Those methods are callbacks: they will be called by the transport -on certain events (for example when some data is received); you shouldn't -call them yourself, unless you are implementing a transport. +Subclasses of abstract base protocol classes can implement some or +all methods. All those methods are callbacks: they are called by +transports on certain events, for example when some data is received. +Base protocol methods are not supposed to be called by anything but +the corresponding transport. -.. note:: - All callbacks have default implementations, which are empty. Therefore, - you only need to implement the callbacks for the events in which you - are interested. +Base Protocols +-------------- + +.. class:: BaseProtocol -Protocol classes ----------------- + Base protocol with methods that all protocols share. -.. class:: Protocol +.. class:: Protocol(BaseProtocol) - The base class for implementing streaming protocols (for use with - e.g. TCP and SSL transports). + The base class for implementing streaming protocols + (TCP, Unix sockets, etc). -.. class:: BufferedProtocol +.. class:: BufferedProtocol(BaseProtocol) A base class for implementing streaming protocols with manual control of the receive buffer. - .. versionadded:: 3.7 - **Important:** this has been added to asyncio in Python 3.7 - *on a provisional basis*! Treat it as an experimental API that - might be changed or removed in Python 3.8. - -.. class:: DatagramProtocol +.. class:: DatagramProtocol(BaseProtocol) - The base class for implementing datagram protocols (for use with - e.g. UDP transports). + The base class for implementing datagram (UDP) protocols. -.. class:: SubprocessProtocol +.. class:: SubprocessProtocol(BaseProtocol) The base class for implementing protocols communicating with child - processes (through a set of unidirectional pipes). + processes (unidirectional pipes). -Connection callbacks --------------------- +Base Protocol +------------- + +All asyncio protocols can implement Base Protocol callbacks. -These callbacks may be called on :class:`Protocol`, :class:`DatagramProtocol` -and :class:`SubprocessProtocol` instances: +.. rubric:: Connection Callbacks + +Connection callbacks are called on all protocols, exactly once per +a successful connection. All other protocol callbacks can only be +called between those two methods. .. method:: BaseProtocol.connection_made(transport) Called when a connection is made. The *transport* argument is the transport representing the - connection. You are responsible for storing it somewhere - (e.g. as an attribute) if you need to. + connection. The protocol is responsible for storing the reference + to its transport. .. method:: BaseProtocol.connection_lost(exc) @@ -376,65 +489,77 @@ and :class:`SubprocessProtocol` instances: The latter means a regular EOF is received, or the connection was aborted or closed by this side of the connection. -:meth:`~BaseProtocol.connection_made` and :meth:`~BaseProtocol.connection_lost` -are called exactly once per successful connection. All other callbacks will be -called between those two methods, which allows for easier resource management -in your protocol implementation. -The following callbacks may be called only on :class:`SubprocessProtocol` -instances: +.. rubric:: Flow Control Callbacks -.. method:: SubprocessProtocol.pipe_data_received(fd, data) +Flow control callbacks can be called by transports to pause or +resume writing performed by the protocol. - Called when the child process writes data into its stdout or stderr pipe. - *fd* is the integer file descriptor of the pipe. *data* is a non-empty - bytes object containing the data. +See the documentation of the :meth:`~WriteTransport.set_write_buffer_limits` +method for more details. -.. method:: SubprocessProtocol.pipe_connection_lost(fd, exc) +.. method:: BaseProtocol.pause_writing() - Called when one of the pipes communicating with the child process - is closed. *fd* is the integer file descriptor that was closed. + Called when the transport's buffer goes over the high-water mark. -.. method:: SubprocessProtocol.process_exited() +.. method:: BaseProtocol.resume_writing() - Called when the child process has exited. + Called when the transport's buffer drains below the low-water mark. +If the buffer size equals the high-water mark, +:meth:`~BaseProtocol.pause_writing` is not called: the buffer size must +go strictly over. -Streaming protocols +Conversely, :meth:`~BaseProtocol.resume_writing` is called when the +buffer size is equal or lower than the low-water mark. These end +conditions are important to ensure that things go as expected when +either mark is zero. + + +Streaming Protocols ------------------- -The following callbacks are called on :class:`Protocol` instances: +Event methods, such as :meth:`loop.create_server`, +:meth:`loop.create_unix_server`, :meth:`loop.create_connection`, +:meth:`loop.create_unix_connection`, :meth:`loop.connect_accepted_socket`, +:meth:`loop.connect_read_pipe`, and :meth:`loop.connect_write_pipe` +accept factories that return streaming protocols. .. method:: Protocol.data_received(data) - Called when some data is received. *data* is a non-empty bytes object - containing the incoming data. + Called when some data is received. *data* is a non-empty bytes + object containing the incoming data. + + Whether the data is buffered, chunked or reassembled depends on + the transport. In general, you shouldn't rely on specific semantics + and instead make your parsing generic and flexible enough. However, + data is always received in the correct order. + + The method can be called an arbitrary number of times during + a connection. - .. note:: - Whether the data is buffered, chunked or reassembled depends on - the transport. In general, you shouldn't rely on specific semantics - and instead make your parsing generic and flexible enough. However, - data is always received in the correct order. + However, :meth:`protocol.eof_received() ` + is called at most once and, if called, + :meth:`protocol.data_received() ` + won't be called after it. .. method:: Protocol.eof_received() Called when the other end signals it won't send any more data - (for example by calling :meth:`write_eof`, if the other end also uses + (for example by calling :meth:`transport.write_eof() + `, if the other end also uses asyncio). This method may return a false value (including ``None``), in which case the transport will close itself. Conversely, if this method returns a true value, closing the transport is up to the protocol. Since the - default implementation returns ``None``, it implicitly closes the connection. + default implementation returns ``None``, it implicitly closes the + connection. - .. note:: - Some transports such as SSL don't support half-closed connections, - in which case returning true from this method will not prevent closing - the connection. + Some transports such as SSL don't support half-closed connections, + in which case returning true from this method will not prevent closing + the connection. -:meth:`data_received` can be called an arbitrary number of times during -a connection. However, :meth:`eof_received` is called at most once -and, if called, :meth:`data_received` won't be called after it. State machine: @@ -446,20 +571,18 @@ State machine: -> connection_lost -> end -Streaming protocols with manual receive buffer control ------------------------------------------------------- +Buffered Streaming Protocols +---------------------------- .. versionadded:: 3.7 - **Important:** :class:`BufferedProtocol` has been added to - asyncio in Python 3.7 *on a provisional basis*! Consider it as an - experimental API that might be changed or removed in Python 3.8. + **Important:** this has been added to asyncio in Python 3.7 + *on a provisional basis*! This is as an experimental API that + might be changed or removed completely in Python 3.8. +Buffered Protocols can be used with any event loop method +that supports `Streaming Protocols`_. -Event methods, such as :meth:`AbstractEventLoop.create_server` and -:meth:`AbstractEventLoop.create_connection`, accept factories that -return protocols that implement this interface. - -The idea of BufferedProtocol is that it allows to manually allocate +The idea of ``BufferedProtocol`` is that it allows to manually allocate and control the receive buffer. Event loops can then use the buffer provided by the protocol to avoid unnecessary data copies. This can result in noticeable performance improvement for protocols that @@ -489,13 +612,15 @@ instances: .. method:: BufferedProtocol.eof_received() - See the documentation of the :meth:`Protocol.eof_received` method. + See the documentation of the :meth:`protocol.eof_received() + ` method. -:meth:`get_buffer` can be called an arbitrary number of times during -a connection. However, :meth:`eof_received` is called at most once -and, if called, :meth:`get_buffer` and :meth:`buffer_updated` -won't be called after it. +:meth:`~BufferedProtocol.get_buffer` can be called an arbitrary number +of times during a connection. However, :meth:`protocol.eof_received() +` is called at most once +and, if called, :meth:`~BufferedProtocol.get_buffer` and +:meth:`~BufferedProtocol.buffer_updated` won't be called after it. State machine: @@ -509,10 +634,11 @@ State machine: -> connection_lost -> end -Datagram protocols +Datagram Protocols ------------------ -The following callbacks are called on :class:`DatagramProtocol` instances. +Datagram Protocol instances should be constructed by protocol +factories passed to the :meth:`loop.create_datagram_endpoint` method. .. method:: DatagramProtocol.datagram_received(data, addr) @@ -530,76 +656,116 @@ The following callbacks are called on :class:`DatagramProtocol` instances. In many conditions though, undeliverable datagrams will be silently dropped. +.. note:: -Flow control callbacks ----------------------- + On BSD systems (macOS, FreeBSD, etc.) flow control is not supported + for datagram protocols, because send failures caused by + writing too many packets cannot be detected easily. -These callbacks may be called on :class:`Protocol`, -:class:`DatagramProtocol` and :class:`SubprocessProtocol` instances: + The socket always appears 'ready' and excess packets are dropped; an + :class:`OSError` with ``errno`` set to :const:`errno.ENOBUFS` may + or may not be raised; if it is raised, it will be reported to + :meth:`DatagramProtocol.error_received` but otherwise ignored. -.. method:: BaseProtocol.pause_writing() - Called when the transport's buffer goes over the high-water mark. +.. _asyncio-subprocess-protocols: -.. method:: BaseProtocol.resume_writing() +Subprocess Protocols +-------------------- - Called when the transport's buffer drains below the low-water mark. +Datagram Protocol instances should be constructed by protocol +factories passed to the :meth:`loop.subprocess_exec` and +:meth:`loop.subprocess_shell` methods. +.. method:: SubprocessProtocol.pipe_data_received(fd, data) -:meth:`pause_writing` and :meth:`resume_writing` calls are paired -- -:meth:`pause_writing` is called once when the buffer goes strictly over -the high-water mark (even if subsequent writes increases the buffer size -even more), and eventually :meth:`resume_writing` is called once when the -buffer size reaches the low-water mark. + Called when the child process writes data into its stdout or stderr + pipe. -.. note:: - If the buffer size equals the high-water mark, - :meth:`pause_writing` is not called -- it must go strictly over. - Conversely, :meth:`resume_writing` is called when the buffer size is - equal or lower than the low-water mark. These end conditions - are important to ensure that things go as expected when either - mark is zero. + *fd* is the integer file descriptor of the pipe. -.. note:: - On BSD systems (OS X, FreeBSD, etc.) flow control is not supported - for :class:`DatagramProtocol`, because send failures caused by - writing too many packets cannot be detected easily. The socket - always appears 'ready' and excess packets are dropped; an - :class:`OSError` with errno set to :const:`errno.ENOBUFS` may or - may not be raised; if it is raised, it will be reported to - :meth:`DatagramProtocol.error_received` but otherwise ignored. + *data* is a non-empty bytes object containing the received data. + +.. method:: SubprocessProtocol.pipe_connection_lost(fd, exc) + + Called when one of the pipes communicating with the child process + is closed. + + *fd* is the integer file descriptor that was closed. + +.. method:: SubprocessProtocol.process_exited() + + Called when the child process has exited. -Coroutines and protocols ------------------------- +Examples +======== -Coroutines can be scheduled in a protocol method using :func:`ensure_future`, -but there is no guarantee made about the execution order. Protocols are not -aware of coroutines created in protocol methods and so will not wait for them. +.. _asyncio-tcp-echo-server-protocol: + +TCP Echo Server +--------------- + +TCP echo server using the :meth:`loop.create_server` method, send back +received data and close the connection:: + + import asyncio + + + class EchoServerClientProtocol(asyncio.Protocol): + def connection_made(self, transport): + peername = transport.get_extra_info('peername') + print('Connection from {}'.format(peername)) + self.transport = transport + + def data_received(self, data): + message = data.decode() + print('Data received: {!r}'.format(message)) + + print('Send: {!r}'.format(message)) + self.transport.write(data) + + print('Close the client socket') + self.transport.close() + + + async def main(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() + + server = await loop.create_server( + lambda: EchoServerClientProtocol(), + '127.0.0.1', 8888) -To have a reliable execution order, -use :ref:`stream objects ` in a -coroutine with ``await``. For example, the :meth:`StreamWriter.drain` -coroutine can be used to wait until the write buffer is flushed. + async with server: + await server.serve_forever() -Protocol examples -================= + asyncio.run(main()) + + +.. seealso:: + + The :ref:`TCP echo server using streams ` + example uses the high-level :func:`asyncio.start_server` function. .. _asyncio-tcp-echo-client-protocol: -TCP echo client protocol ------------------------- +TCP Echo Client +--------------- -TCP echo client using the :meth:`AbstractEventLoop.create_connection` method, send +TCP echo client using the :meth:`loop.create_connection` method, send data and wait until the connection is closed:: import asyncio + class EchoClientProtocol(asyncio.Protocol): - def __init__(self, message, loop): + def __init__(self, message, on_con_lost, loop): self.message = message self.loop = loop + self.on_con_lost = on_con_lost def connection_made(self, transport): transport.write(self.message.encode()) @@ -610,99 +776,99 @@ data and wait until the connection is closed:: def connection_lost(self, exc): print('The server closed the connection') - print('Stop the event loop') - self.loop.stop() - - loop = asyncio.get_event_loop() - message = 'Hello World!' - coro = loop.create_connection(lambda: EchoClientProtocol(message, loop), - '127.0.0.1', 8888) - loop.run_until_complete(coro) - loop.run_forever() - loop.close() - -The event loop is running twice. The -:meth:`~AbstractEventLoop.run_until_complete` method is preferred in this short -example to raise an exception if the server is not listening, instead of -having to write a short coroutine to handle the exception and stop the -running loop. At :meth:`~AbstractEventLoop.run_until_complete` exit, the loop is -no longer running, so there is no need to stop the loop in case of an error. + self.on_con_lost.set_result(True) + + + async def main(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() + + on_con_lost = loop.create_future() + message = 'Hello World!' + + transport, protocol = await loop.create_connection( + lambda: EchoClientProtocol(message, on_con_lost, loop), + '127.0.0.1', 8888) + + # Wait until the protocol signals that the connection + # is lost and close the transport. + try: + await on_con_lost + finally: + transport.close() + + + asyncio.run(main()) + .. seealso:: The :ref:`TCP echo client using streams ` - example uses the :func:`asyncio.open_connection` function. + example uses the high-level :func:`asyncio.open_connection` function. -.. _asyncio-tcp-echo-server-protocol: +.. _asyncio-udp-echo-server-protocol: -TCP echo server protocol ------------------------- +UDP Echo Server +--------------- -TCP echo server using the :meth:`AbstractEventLoop.create_server` method, send back -received data and close the connection:: +UDP echo server using the :meth:`loop.create_datagram_endpoint` +method, send back received data:: import asyncio - class EchoServerClientProtocol(asyncio.Protocol): + + class EchoServerProtocol: def connection_made(self, transport): - peername = transport.get_extra_info('peername') - print('Connection from {}'.format(peername)) self.transport = transport - def data_received(self, data): + def datagram_received(self, data, addr): message = data.decode() - print('Data received: {!r}'.format(message)) - - print('Send: {!r}'.format(message)) - self.transport.write(data) + print('Received %r from %s' % (message, addr)) + print('Send %r to %s' % (message, addr)) + self.transport.sendto(data, addr) - print('Close the client socket') - self.transport.close() - loop = asyncio.get_event_loop() - # Each client connection will create a new protocol instance - coro = loop.create_server(EchoServerClientProtocol, '127.0.0.1', 8888) - server = loop.run_until_complete(coro) + async def main(): + print("Starting UDP server") - # Serve requests until Ctrl+C is pressed - print('Serving on {}'.format(server.sockets[0].getsockname())) - try: - loop.run_forever() - except KeyboardInterrupt: - pass + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() - # Close the server - server.close() - loop.run_until_complete(server.wait_closed()) - loop.close() + # One protocol instance will be created to serve all + # client requests. + transport, protocol = await loop.create_datagram_endpoint( + lambda: EchoServerProtocol(), + local_addr=('127.0.0.1', 9999)) -:meth:`Transport.close` can be called immediately after -:meth:`WriteTransport.write` even if data are not sent yet on the socket: both -methods are asynchronous. ``await`` is not needed because these transport -methods are not coroutines. + try: + await asyncio.sleep(3600) # Serve for 1 hour. + finally: + transport.close() -.. seealso:: - The :ref:`TCP echo server using streams ` - example uses the :func:`asyncio.start_server` function. + asyncio.run(main()) .. _asyncio-udp-echo-client-protocol: -UDP echo client protocol ------------------------- +UDP Echo Client +--------------- -UDP echo client using the :meth:`AbstractEventLoop.create_datagram_endpoint` +UDP echo client using the :meth:`loop.create_datagram_endpoint` method, send data and close the transport when we received the answer:: import asyncio + class EchoClientProtocol: def __init__(self, message, loop): self.message = message self.loop = loop self.transport = None + self.on_con_lost = loop.create_future() def connection_made(self, transport): self.transport = transport @@ -719,75 +885,46 @@ method, send data and close the transport when we received the answer:: print('Error received:', exc) def connection_lost(self, exc): - print("Socket closed, stop the event loop") - loop = asyncio.get_event_loop() - loop.stop() + print("Connection closed") + self.on_con_lost.set_result(True) - loop = asyncio.get_event_loop() - message = "Hello World!" - connect = loop.create_datagram_endpoint( - lambda: EchoClientProtocol(message, loop), - remote_addr=('127.0.0.1', 9999)) - transport, protocol = loop.run_until_complete(connect) - loop.run_forever() - transport.close() - loop.close() + async def main(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() -.. _asyncio-udp-echo-server-protocol: - -UDP echo server protocol ------------------------- + message = "Hello World!" + transport, protocol = await loop.create_datagram_endpoint( + lambda: EchoClientProtocol(message, loop), + remote_addr=('127.0.0.1', 9999)) -UDP echo server using the :meth:`AbstractEventLoop.create_datagram_endpoint` -method, send back received data:: + try: + await protocol.on_con_lost + finally: + transport.close() - import asyncio - class EchoServerProtocol: - def connection_made(self, transport): - self.transport = transport - - def datagram_received(self, data, addr): - message = data.decode() - print('Received %r from %s' % (message, addr)) - print('Send %r to %s' % (message, addr)) - self.transport.sendto(data, addr) - - loop = asyncio.get_event_loop() - print("Starting UDP server") - # One protocol instance will be created to serve all client requests - listen = loop.create_datagram_endpoint( - EchoServerProtocol, local_addr=('127.0.0.1', 9999)) - transport, protocol = loop.run_until_complete(listen) - - try: - loop.run_forever() - except KeyboardInterrupt: - pass - - transport.close() - loop.close() + asyncio.run(main()) .. _asyncio-register-socket: -Register an open socket to wait for data using a protocol ---------------------------------------------------------- +Connecting Existing Sockets +--------------------------- Wait until a socket receives data using the -:meth:`AbstractEventLoop.create_connection` method with a protocol, and then close -the event loop :: +:meth:`loop.create_connection` method with a protocol:: import asyncio - from socket import socketpair + import socket - # Create a pair of connected sockets - rsock, wsock = socketpair() - loop = asyncio.get_event_loop() class MyProtocol(asyncio.Protocol): - transport = None + + def __init__(self, loop): + self.transport = None + self.on_con_lost = loop.create_future() def connection_made(self, transport): self.transport = transport @@ -795,35 +932,102 @@ the event loop :: def data_received(self, data): print("Received:", data.decode()) - # We are done: close the transport (it will call connection_lost()) + # We are done: close the transport; + # connection_lost() will be called automatically. self.transport.close() def connection_lost(self, exc): - # The socket has been closed, stop the event loop - loop.stop() + # The socket has been closed + self.on_con_lost.set_result(True) + + + async def main(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() - # Register the socket to wait for data - connect_coro = loop.create_connection(MyProtocol, sock=rsock) - transport, protocol = loop.run_until_complete(connect_coro) + # Create a pair of connected sockets + rsock, wsock = socket.socketpair() - # Simulate the reception of data from the network - loop.call_soon(wsock.send, 'abc'.encode()) + # Register the socket to wait for data. + transport, protocol = await loop.create_connection( + lambda: MyProtocol(loop), sock=rsock) - # Run the event loop - loop.run_forever() + # Simulate the reception of data from the network. + loop.call_soon(wsock.send, 'abc'.encode()) - # We are done, close sockets and the event loop - rsock.close() - wsock.close() - loop.close() + try: + await protocol.on_con_lost + finally: + transport.close() + wsock.close() + + asyncio.run(main()) .. seealso:: The :ref:`watch a file descriptor for read events ` example uses the low-level - :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a - socket. + :meth:`loop.add_reader` method to register an FD. The :ref:`register an open socket to wait for data using streams ` example uses high-level streams created by the :func:`open_connection` function in a coroutine. + +.. _asyncio-subprocess-proto-example: + +loop.subprocess_exec() and SubprocessProtocol +--------------------------------------------- + +An example of a subprocess protocol using to get the output of a +subprocess and to wait for the subprocess exit. + +The subprocess is created by th :meth:`loop.subprocess_exec` method:: + + import asyncio + import sys + + class DateProtocol(asyncio.SubprocessProtocol): + def __init__(self, exit_future): + self.exit_future = exit_future + self.output = bytearray() + + def pipe_data_received(self, fd, data): + self.output.extend(data) + + def process_exited(self): + self.exit_future.set_result(True) + + async def get_date(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() + + code = 'import datetime; print(datetime.datetime.now())' + exit_future = asyncio.Future(loop=loop) + + # Create the subprocess controlled by DateProtocol; + # redirect the standard output into a pipe. + transport, protocol = await loop.subprocess_exec( + lambda: DateProtocol(exit_future), + sys.executable, '-c', code, + stdin=None, stderr=None) + + # Wait for the subprocess exit using the process_exited() + # method of the protocol. + await exit_future + + # Close the stdout pipe. + transport.close() + + # Read the output which was collected by the + # pipe_data_received() method of the protocol. + data = bytes(protocol.output) + return data.decode('ascii').rstrip() + + if sys.platform == "win32": + asyncio.set_event_loop_policy( + asyncio.WindowsProactorEventLoopPolicy()) + + date = asyncio.run(get_date()) + print(f"Current date: {date}") diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index 65497f29d895..1e4470adee98 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -1,41 +1,41 @@ .. currentmodule:: asyncio + +====== Queues ====== -**Source code:** :source:`Lib/asyncio/queues.py` - -Queues: +asyncio queues are designed to be similar to classes of the +:mod:`queue` module. Although asyncio queues are not thread-safe, +they are designed to be used specifically in async/await code. -* :class:`Queue` -* :class:`PriorityQueue` -* :class:`LifoQueue` +Note that methods on asyncio queues don't have a *timeout* parameter; +use :func:`asyncio.wait_for` function to do queue operations with a +timeout. -asyncio queue API was designed to be close to classes of the :mod:`queue` -module (:class:`~queue.Queue`, :class:`~queue.PriorityQueue`, -:class:`~queue.LifoQueue`), but it has no *timeout* parameter. The -:func:`asyncio.wait_for` function can be used to cancel a task after a timeout. +See also the `Examples`_ section below. Queue ------ +===== .. class:: Queue(maxsize=0, \*, loop=None) - A queue, useful for coordinating producer and consumer coroutines. + A first in, first out (FIFO) queue. - If *maxsize* is less than or equal to zero, the queue size is infinite. If - it is an integer greater than ``0``, then ``await put()`` will block - when the queue reaches *maxsize*, until an item is removed by :meth:`get`. + If *maxsize* is less than or equal to zero, the queue size is + infinite. If it is an integer greater than ``0``, then + ``await put()`` blocks when the queue reaches *maxsize* + until an item is removed by :meth:`get`. - Unlike the standard library :mod:`queue`, you can reliably know this Queue's - size with :meth:`qsize`, since your single-threaded asyncio application won't - be interrupted between calling :meth:`qsize` and doing an operation on the - Queue. + Unlike the standard library threading :mod:`queue`, the size of + the queue is always known and can be returned by calling the + :meth:`qsize` method. This class is :ref:`not thread safe `. - .. versionchanged:: 3.4.4 - New :meth:`join` and :meth:`task_done` methods. + .. attribute:: maxsize + + Number of items allowed in the queue. .. method:: empty() @@ -45,26 +45,16 @@ Queue Return ``True`` if there are :attr:`maxsize` items in the queue. - .. note:: - - If the Queue was initialized with ``maxsize=0`` (the default), then - :meth:`full()` is never ``True``. + If the queue was initialized with ``maxsize=0`` (the default), + then :meth:`full()` never returns ``True``. .. coroutinemethod:: get() - Remove and return an item from the queue. If queue is empty, wait until - an item is available. - - This method is a :ref:`coroutine `. - - .. seealso:: - - The :meth:`empty` method. + Remove and return an item from the queue. If queue is empty, + wait until an item is available. .. method:: get_nowait() - Remove and return an item from the queue. - Return an item if one is immediately available, else raise :exc:`QueueEmpty`. @@ -72,26 +62,16 @@ Queue Block until all items in the queue have been gotten and processed. - The count of unfinished tasks goes up whenever an item is added to the - queue. The count goes down whenever a consumer thread calls - :meth:`task_done` to indicate that the item was retrieved and all work on - it is complete. When the count of unfinished tasks drops to zero, - :meth:`join` unblocks. - - This method is a :ref:`coroutine `. - - .. versionadded:: 3.4.4 + The count of unfinished tasks goes up whenever an item is added + to the queue. The count goes down whenever a consumer thread calls + :meth:`task_done` to indicate that the item was retrieved and all + work on it is complete. When the count of unfinished tasks drops + to zero, :meth:`join` unblocks. .. coroutinemethod:: put(item) - Put an item into the queue. If the queue is full, wait until a free slot - is available before adding item. - - This method is a :ref:`coroutine `. - - .. seealso:: - - The :meth:`full` method. + Put an item into the queue. If the queue is full, wait until a + free slot is available before adding item. .. method:: put_nowait(item) @@ -107,54 +87,111 @@ Queue Indicate that a formerly enqueued task is complete. - Used by queue consumers. For each :meth:`~Queue.get` used to fetch a task, a - subsequent call to :meth:`task_done` tells the queue that the processing - on the task is complete. - - If a :meth:`join` is currently blocking, it will resume when all items - have been processed (meaning that a :meth:`task_done` call was received - for every item that had been :meth:`~Queue.put` into the queue). + Used by queue consumers. For each :meth:`~Queue.get` used to + fetch a task, a subsequent call to :meth:`task_done` tells the + queue that the processing on the task is complete. - Raises :exc:`ValueError` if called more times than there were items - placed in the queue. + If a :meth:`join` is currently blocking, it will resume when all + items have been processed (meaning that a :meth:`task_done` + call was received for every item that had been :meth:`~Queue.put` + into the queue). - .. versionadded:: 3.4.4 + Raises :exc:`ValueError` if called more times than there were + items placed in the queue. - .. attribute:: maxsize - - Number of items allowed in the queue. - -PriorityQueue -------------- +Priority Queue +============== .. class:: PriorityQueue - A subclass of :class:`Queue`; retrieves entries in priority order (lowest - first). + A variant of :class:`Queue`; retrieves entries in priority order + (lowest first). - Entries are typically tuples of the form: (priority number, data). + Entries are typically tuples of the form + ``(priority_number, data)``. -LifoQueue ---------- +LIFO Queue +========== .. class:: LifoQueue - A subclass of :class:`Queue` that retrieves most recently added entries - first. + A variant of :class:`Queue` that retrieves most recently added + entries first (last in, first out). Exceptions -^^^^^^^^^^ +========== .. exception:: QueueEmpty - Exception raised when the :meth:`~Queue.get_nowait` method is called on a - :class:`Queue` object which is empty. + This exception is raised when the :meth:`~Queue.get_nowait` method + is called on an empty queue. .. exception:: QueueFull - Exception raised when the :meth:`~Queue.put_nowait` method is called on a - :class:`Queue` object which is full. + Exception raised when the :meth:`~Queue.put_nowait` method is called + on a queue that has reached its *maxsize*. + + +Examples +======== + +Queues can be used to distribute workload between several +concurrent tasks:: + + import asyncio + import random + import time + + + async def worker(name, queue): + while True: + # Get a "work item" out of the queue. + sleep_for = await queue.get() + + # Sleep for the "sleep_for" seconds. + await asyncio.sleep(sleep_for) + + # Notify the queue that the "work item" has been processed. + queue.task_done() + + print(f'{name} has slept for {sleep_for:.2f} seconds') + + + async def main(): + # Create a queue that we will use to store our "workload". + queue = asyncio.Queue() + + # Generate random timings and put them into the queue. + total_sleep_time = 0 + for _ in range(20): + sleep_for = random.uniform(0.05, 1.0) + total_sleep_time += sleep_for + queue.put_nowait(sleep_for) + + # Create three worker tasks to process the queue concurrently. + tasks = [] + for i in range(3): + task = asyncio.create_task(worker(f'worker-{i}', queue)) + tasks.append(task) + + # Wait until the queue is fully processed. + started_at = time.monotonic() + await queue.join() + total_slept_for = time.monotonic() - started_at + + # Cancel our worker tasks. + for task in tasks: + task.cancel() + # Wait until all worker tasks are cancelled. + await asyncio.gather(*tasks, return_exceptions=True) + + print('====') + print(f'3 workers slept in parallel for {total_slept_for:.2f} seconds') + print(f'total expected sleep time: {total_sleep_time:.2f} seconds') + + + asyncio.run(main()) diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index f662e7223337..3fe7ac7b4abc 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -2,83 +2,107 @@ .. _asyncio-streams: -+++++++++++++++++++++++++++++ -Streams (coroutine based API) -+++++++++++++++++++++++++++++ +======= +Streams +======= -**Source code:** :source:`Lib/asyncio/streams.py` +Streams are high-level async/await-ready primitives to work with +network connections. Streams allow send and receive data without +using callbacks or low-level protocols and transports. -Stream functions -================ +Here's an example of a TCP echo client written using asyncio +streams:: -.. note:: + import asyncio + + async def tcp_echo_client(message): + reader, writer = await asyncio.open_connection( + '127.0.0.1', 8888) + + print(f'Send: {message!r}') + writer.write(message.encode()) + + data = await reader.read(100) + print(f'Received: {data.decode()!r}') + + print('Close the connection') + writer.close() - The top-level functions in this module are meant as convenience wrappers - only; there's really nothing special there, and if they don't do - exactly what you want, feel free to copy their code. + asyncio.run(tcp_echo_client('Hello World!')) -.. coroutinefunction:: open_connection(host=None, port=None, \*, loop=None, limit=None, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None) +.. rubric:: Stream Functions - A wrapper for :meth:`~AbstractEventLoop.create_connection()` returning a (reader, - writer) pair. +The following top-level asyncio functions can be used to create +and work with streams: - The reader returned is a :class:`StreamReader` instance; the writer is - a :class:`StreamWriter` instance. - When specified, the *loop* argument determines which event loop to use, - and the *limit* argument determines the buffer size limit used by the +.. coroutinefunction:: open_connection(host=None, port=None, \*, \ + loop=None, limit=None, ssl=None, family=0, \ + proto=0, flags=0, sock=None, local_addr=None, \ + server_hostname=None, ssl_handshake_timeout=None) + + Establish a network connection and return a pair of + ``(reader, writer)``. + + The returned *reader* and *writer* objects are instances of + :class:`StreamReader` and :class:`StreamWriter` classes. + + The *loop* argument is optional and can always be determined + automatically when this method is awaited from a coroutine. + + *limit* determines the buffer size limit used by the returned :class:`StreamReader` instance. The rest of the arguments are passed directly to - :meth:`AbstractEventLoop.create_connection`. - - This function is a :ref:`coroutine `. + :meth:`loop.create_connection`. .. versionadded:: 3.7 The *ssl_handshake_timeout* parameter. -.. coroutinefunction:: start_server(client_connected_cb, host=None, port=None, \*, loop=None, limit=None, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, ssl_handshake_timeout=None, start_serving=True) +.. coroutinefunction:: start_server(client_connected_cb, host=None, \ + port=None, \*, loop=None, limit=None, \ + family=socket.AF_UNSPEC, \ + flags=socket.AI_PASSIVE, sock=None, \ + backlog=100, ssl=None, reuse_address=None, \ + reuse_port=None, ssl_handshake_timeout=None, \ + start_serving=True) - Start a socket server, with a callback for each client connected. The return - value is the same as :meth:`~AbstractEventLoop.create_server()`. + Start a socket server. The *client_connected_cb* callback is called whenever a new client - connection is established. It receives a reader/writer pair as two - arguments, the first is a :class:`StreamReader` instance, - and the second is a :class:`StreamWriter` instance. + connection is established. It receives a ``(reader, writer)`` pair + as two arguments, instances of the :class:`StreamReader` and + :class:`StreamWriter` classes. - *client_connected_cb* accepts a plain callable or a + *client_connected_cb* can be a plain callable or a :ref:`coroutine function `; if it is a coroutine function, - it will be automatically converted into a :class:`Task`. + it will be automatically wrapped into a :class:`Task`. - When specified, the *loop* argument determines which event loop to use, - and the *limit* argument determines the buffer size limit used by the - :class:`StreamReader` instance passed to *client_connected_cb*. + The *loop* argument is optional and can always be determined + automatically when this method is awaited from a coroutine. - The rest of the arguments are passed directly to - :meth:`~AbstractEventLoop.create_server()`. + *limit* determines the buffer size limit used by the + returned :class:`StreamReader` instance. - This function is a :ref:`coroutine `. + The rest of the arguments are passed directly to + :meth:`loop.create_server`. .. versionadded:: 3.7 The *ssl_handshake_timeout* and *start_serving* parameters. -.. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, limit=None, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None) - - A wrapper for :meth:`~AbstractEventLoop.create_unix_connection()` returning - a (reader, writer) pair. +.. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, \ + limit=None, ssl=None, sock=None, \ + server_hostname=None, ssl_handshake_timeout=None) - When specified, the *loop* argument determines which event loop to use, - and the *limit* argument determines the buffer size limit used by the - returned :class:`StreamReader` instance. + Establish a UNIX socket connection and return a pair of + ``(reader, writer)``. - The rest of the arguments are passed directly to - :meth:`~AbstractEventLoop.create_unix_connection()`. + Similar to :func:`open_connection` but operates on UNIX sockets. - This function is a :ref:`coroutine `. + See also the documentation of :meth:`loop.create_unix_connection`. Availability: UNIX. @@ -90,27 +114,16 @@ Stream functions The *path* parameter can now be a :term:`path-like object` -.. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \*, loop=None, limit=None, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, start_serving=True) - - Start a UNIX Domain Socket server, with a callback for each client connected. - - The *client_connected_cb* callback is called whenever a new client - connection is established. It receives a reader/writer pair as two - arguments, the first is a :class:`StreamReader` instance, - and the second is a :class:`StreamWriter` instance. - - *client_connected_cb* accepts a plain callable or a - :ref:`coroutine function `; if it is a coroutine function, - it will be automatically converted into a :class:`Task`. +.. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \ + \*, loop=None, limit=None, sock=None, \ + backlog=100, ssl=None, ssl_handshake_timeout=None, \ + start_serving=True) - When specified, the *loop* argument determines which event loop to use, - and the *limit* argument determines the buffer size limit used by the - :class:`StreamReader` instance passed to *client_connected_cb*. + Start a UNIX socket server. - The rest of the arguments are passed directly to - :meth:`~AbstractEventLoop.create_unix_server()`. + Similar to :func:`start_server` but operates on UNIX sockets. - This function is a :ref:`coroutine `. + See also the documentation of :meth:`loop.create_unix_server`. Availability: UNIX. @@ -123,6 +136,13 @@ Stream functions The *path* parameter can now be a :term:`path-like object`. +.. rubric:: Contents + +* `StreamReader`_ and `StreamWriter`_ +* `StreamReaderProtocol`_ +* `Examples`_ + + StreamReader ============ @@ -159,8 +179,6 @@ StreamReader If the EOF was received and the internal buffer is empty, return an empty ``bytes`` object. - This method is a :ref:`coroutine `. - .. coroutinemethod:: readline() Read one line, where "line" is a sequence of bytes ending with ``\n``. @@ -171,8 +189,6 @@ StreamReader If the EOF was received and the internal buffer is empty, return an empty ``bytes`` object. - This method is a :ref:`coroutine `. - .. coroutinemethod:: readexactly(n) Read exactly *n* bytes. Raise an :exc:`IncompleteReadError` if the end of @@ -180,8 +196,6 @@ StreamReader :attr:`IncompleteReadError.partial` attribute of the exception contains the partial read bytes. - This method is a :ref:`coroutine `. - .. coroutinemethod:: readuntil(separator=b'\\n') Read data from the stream until ``separator`` is found. @@ -208,7 +222,8 @@ StreamReader .. method:: at_eof() - Return ``True`` if the buffer is empty and :meth:`feed_eof` was called. + Return ``True`` if the buffer is empty and :meth:`feed_eof` + was called. StreamWriter @@ -299,7 +314,8 @@ StreamWriter StreamReaderProtocol ==================== -.. class:: StreamReaderProtocol(stream_reader, client_connected_cb=None, loop=None) +.. class:: StreamReaderProtocol(stream_reader, client_connected_cb=None, \ + loop=None) Trivial helper class to adapt between :class:`Protocol` and :class:`StreamReader`. Subclass of :class:`Protocol`. @@ -314,36 +330,8 @@ StreamReaderProtocol accidentally calling inappropriate methods of the protocol.) -IncompleteReadError -=================== - -.. exception:: IncompleteReadError - - Incomplete read error, subclass of :exc:`EOFError`. - - .. attribute:: expected - - Total number of expected bytes (:class:`int`). - - .. attribute:: partial - - Read bytes string before the end of stream was reached (:class:`bytes`). - - -LimitOverrunError -================= - -.. exception:: LimitOverrunError - - Reached the buffer limit while looking for a separator. - - .. attribute:: consumed - - Total number of to be consumed bytes. - - -Stream examples -=============== +Examples +======== .. _asyncio-tcp-echo-client-streams: @@ -354,28 +342,26 @@ TCP echo client using the :func:`asyncio.open_connection` function:: import asyncio - async def tcp_echo_client(message, loop): - reader, writer = await asyncio.open_connection('127.0.0.1', 8888, - loop=loop) + async def tcp_echo_client(message): + reader, writer = await asyncio.open_connection( + '127.0.0.1', 8888) - print('Send: %r' % message) + print(f'Send: {message!r}') writer.write(message.encode()) data = await reader.read(100) - print('Received: %r' % data.decode()) + print(f'Received: {data.decode()!r}') - print('Close the socket') + print('Close the connection') writer.close() - message = 'Hello World!' - loop = asyncio.get_event_loop() - loop.run_until_complete(tcp_echo_client(message, loop)) - loop.close() + asyncio.run(tcp_echo_client('Hello World!')) + .. seealso:: The :ref:`TCP echo client protocol ` - example uses the :meth:`AbstractEventLoop.create_connection` method. + example uses the low-level :meth:`loop.create_connection` method. .. _asyncio-tcp-echo-server-streams: @@ -391,35 +377,33 @@ TCP echo server using the :func:`asyncio.start_server` function:: data = await reader.read(100) message = data.decode() addr = writer.get_extra_info('peername') - print("Received %r from %r" % (message, addr)) - print("Send: %r" % message) + print(f"Received {message!r} from {addr!r}") + + print(f"Send: {message!r}") writer.write(data) await writer.drain() - print("Close the client socket") + print("Close the connection") writer.close() - loop = asyncio.get_event_loop() - coro = asyncio.start_server(handle_echo, '127.0.0.1', 8888, loop=loop) - server = loop.run_until_complete(coro) + async def main(): + server = await asyncio.start_server( + handle_echo, '127.0.0.1', 8888) + + addr = server.sockets[0].getsockname() + print(f'Serving on {addr}') - # Serve requests until Ctrl+C is pressed - print('Serving on {}'.format(server.sockets[0].getsockname())) - try: - loop.run_forever() - except KeyboardInterrupt: - pass + async with server: + await server.serve_forever() + + asyncio.run(main()) - # Close the server - server.close() - loop.run_until_complete(server.wait_closed()) - loop.close() .. seealso:: The :ref:`TCP echo server protocol ` - example uses the :meth:`AbstractEventLoop.create_server` method. + example uses the :meth:`loop.create_server` method. Get HTTP headers @@ -434,30 +418,34 @@ Simple example querying HTTP headers of the URL passed on the command line:: async def print_http_headers(url): url = urllib.parse.urlsplit(url) if url.scheme == 'https': - connect = asyncio.open_connection(url.hostname, 443, ssl=True) + reader, writer = await asyncio.open_connection( + url.hostname, 443, ssl=True) else: - connect = asyncio.open_connection(url.hostname, 80) - reader, writer = await connect - query = ('HEAD {path} HTTP/1.0\r\n' - 'Host: {hostname}\r\n' - '\r\n').format(path=url.path or '/', hostname=url.hostname) + reader, writer = await asyncio.open_connection( + url.hostname, 80) + + query = ( + f"HEAD {url.path or '/'} HTTP/1.0\r\n" + f"Host: {url.hostname}\r\n" + f"\r\n" + ) + writer.write(query.encode('latin-1')) while True: line = await reader.readline() if not line: break + line = line.decode('latin1').rstrip() if line: - print('HTTP header> %s' % line) + print(f'HTTP header> {line}') # Ignore the body, close the socket writer.close() url = sys.argv[1] - loop = asyncio.get_event_loop() - task = asyncio.ensure_future(print_http_headers(url)) - loop.run_until_complete(task) - loop.close() + asyncio.run(print_http_headers(url)) + Usage:: @@ -467,6 +455,7 @@ or with HTTPS:: python example.py https://example.com/path/page.html + .. _asyncio-register-socket-streams: Register an open socket to wait for data using streams @@ -476,14 +465,18 @@ Coroutine waiting until a socket receives data using the :func:`open_connection` function:: import asyncio - from socket import socketpair + import socket + + async def wait_for_data(): + # Get a reference to the current event loop because + # we want to access low-level APIs. + loop = asyncio.get_running_loop() - async def wait_for_data(loop): - # Create a pair of connected sockets - rsock, wsock = socketpair() + # Create a pair of connected sockets. + rsock, wsock = socket.socketpair() - # Register the open socket to wait for data - reader, writer = await asyncio.open_connection(sock=rsock, loop=loop) + # Register the open socket to wait for data. + reader, writer = await asyncio.open_connection(sock=rsock) # Simulate the reception of data from the network loop.call_soon(wsock.send, 'abc'.encode()) @@ -498,17 +491,14 @@ Coroutine waiting until a socket receives data using the # Close the second socket wsock.close() - loop = asyncio.get_event_loop() - loop.run_until_complete(wait_for_data(loop)) - loop.close() + asyncio.run(wait_for_data()) .. seealso:: The :ref:`register an open socket to wait for data using a protocol - ` example uses a low-level protocol created by the - :meth:`AbstractEventLoop.create_connection` method. + ` example uses a low-level protocol and + the :meth:`loop.create_connection` method. The :ref:`watch a file descriptor for read events ` example uses the low-level - :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a - socket. + :meth:`loop.add_reader` method to watch a file descriptor. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 60e174574b04..b05c236bb0c8 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -2,137 +2,91 @@ .. _asyncio-subprocess: -Subprocess -========== +============ +Subprocesses +============ -**Source code:** :source:`Lib/asyncio/subprocess.py` +This section describes high-level async/await asyncio APIs to +create and manage subprocesses. -Windows event loop ------------------- +Here's an example of how asyncio can run a shell command and +communicate its result back:: -On Windows, the default event loop is :class:`SelectorEventLoop` which does not -support subprocesses. :class:`ProactorEventLoop` should be used instead. -Example to use it on Windows:: - - import asyncio, sys - - if sys.platform == 'win32': - loop = asyncio.ProactorEventLoop() - asyncio.set_event_loop(loop) - -.. seealso:: - - :ref:`Available event loops ` and :ref:`Platform - support `. - - -Create a subprocess: high-level API using Process -------------------------------------------------- - -.. coroutinefunction:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) - - Create a subprocess. - - The *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_exec` for other - parameters. - - Return a :class:`~asyncio.subprocess.Process` instance. - - This function is a :ref:`coroutine `. - -.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) - - Run the shell command *cmd*. - - The *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_shell` for other - parameters. + import asyncio - Return a :class:`~asyncio.subprocess.Process` instance. + async def run(cmd): + proc = await asyncio.create_subprocess_shell( + cmd, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE) - It is the application's responsibility to ensure that all whitespace and - metacharacters are quoted appropriately to avoid `shell injection - `_ - vulnerabilities. The :func:`shlex.quote` function can be used to properly - escape whitespace and shell metacharacters in strings that are going to be - used to construct shell commands. + stdout, stderr = await proc.communicate() - This function is a :ref:`coroutine `. + print(f'[{cmd!r} exited with {proc.returncode}]') + if stdout: + print(f'[stdout]\n{stdout.decode()}') + if stderr: + print(f'[stderr]\n{stderr.decode()}') -Use the :meth:`AbstractEventLoop.connect_read_pipe` and -:meth:`AbstractEventLoop.connect_write_pipe` methods to connect pipes. + asyncio.run(run('ls /zzz')) +will print:: -Create a subprocess: low-level API using subprocess.Popen ---------------------------------------------------------- + ['ls /zzz' exited with 1] + [stderr] + ls: /zzz: No such file or directory -Run subprocesses asynchronously using the :mod:`subprocess` module. +Because all asyncio subprocess functions are asynchronous and asyncio +provides many tools to work with such functions, it is easy to execute +and monitor multiple subprocesses in parallel. It is indeed trivial +to modify the above example to run a few commands at once:: -.. coroutinemethod:: AbstractEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) + async def main(): + await asyncio.gather( + run('ls /zzz'), + run('sleep 1; echo "hello"')) - Create a subprocess from one or more string arguments (character strings or - bytes strings encoded to the :ref:`filesystem encoding - `), where the first string - specifies the program to execute, and the remaining strings specify the - program's arguments. (Thus, together the string arguments form the - ``sys.argv`` value of the program, assuming it is a Python script.) This is - similar to the standard library :class:`subprocess.Popen` class called with - shell=False and the list of strings passed as the first argument; - however, where :class:`~subprocess.Popen` takes a single argument which is - list of strings, :func:`subprocess_exec` takes multiple string arguments. + asyncio.run(main()) - The *protocol_factory* must instantiate a subclass of the - :class:`asyncio.SubprocessProtocol` class. +See also the `Examples`_ subsection. - Other parameters: - * *stdin*: Either a file-like object representing the pipe to be connected - to the subprocess's standard input stream using - :meth:`~AbstractEventLoop.connect_write_pipe`, or the constant - :const:`subprocess.PIPE` (the default). By default a new pipe will be - created and connected. +Creating Subprocesses +===================== - * *stdout*: Either a file-like object representing the pipe to be connected - to the subprocess's standard output stream using - :meth:`~AbstractEventLoop.connect_read_pipe`, or the constant - :const:`subprocess.PIPE` (the default). By default a new pipe will be - created and connected. +.. coroutinefunction:: create_subprocess_exec(\*args, stdin=None, \ + stdout=None, stderr=None, loop=None, \ + limit=None, \*\*kwds) - * *stderr*: Either a file-like object representing the pipe to be connected - to the subprocess's standard error stream using - :meth:`~AbstractEventLoop.connect_read_pipe`, or one of the constants - :const:`subprocess.PIPE` (the default) or :const:`subprocess.STDOUT`. - By default a new pipe will be created and connected. When - :const:`subprocess.STDOUT` is specified, the subprocess's standard error - stream will be connected to the same pipe as the standard output stream. + Create a subprocess. - * All other keyword arguments are passed to :class:`subprocess.Popen` - without interpretation, except for *bufsize*, *universal_newlines* and - *shell*, which should not be specified at all. + The *limit* argument sets the buffer limit for :class:`StreamReader` + wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` + (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* + arguments). - Returns a pair of ``(transport, protocol)``, where *transport* is an - instance of :class:`BaseSubprocessTransport`. + Return a :class:`~asyncio.subprocess.Process` instance. - This method is a :ref:`coroutine `. + See the documentation of :meth:`loop.subprocess_exec` for other + parameters. - See the constructor of the :class:`subprocess.Popen` class for parameters. +.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \ + stdout=None, stderr=None, loop=None, \ + limit=None, \*\*kwds) -.. coroutinemethod:: AbstractEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) + Run the shell command *cmd*. - Create a subprocess from *cmd*, which is a character string or a bytes - string encoded to the :ref:`filesystem encoding `, - using the platform's "shell" syntax. This is similar to the standard library - :class:`subprocess.Popen` class called with ``shell=True``. + The *limit* argument sets the buffer limit for :class:`StreamReader` + wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` + (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* + arguments). - The *protocol_factory* must instantiate a subclass of the - :class:`asyncio.SubprocessProtocol` class. + Return a :class:`~asyncio.subprocess.Process` instance. - See :meth:`~AbstractEventLoop.subprocess_exec` for more details about - the remaining arguments. + See the documentation of :meth:`loop.subprocess_shell` for other + parameters. - Returns a pair of ``(transport, protocol)``, where *transport* is an - instance of :class:`BaseSubprocessTransport`. +.. note:: It is the application's responsibility to ensure that all whitespace and metacharacters are quoted appropriately to avoid `shell injection @@ -141,106 +95,125 @@ Run subprocesses asynchronously using the :mod:`subprocess` module. escape whitespace and shell metacharacters in strings that are going to be used to construct shell commands. - This method is a :ref:`coroutine `. +.. note:: + + The default event loop that asyncio is pre-configured + to use on **Windows** does not support subprocesses. + See :ref:`Subprocess Support on Windows ` + for details. .. seealso:: - The :meth:`AbstractEventLoop.connect_read_pipe` and - :meth:`AbstractEventLoop.connect_write_pipe` methods. + asyncio has also *low-level* APIs to work with subprocesses: + :meth:`loop.subprocess_exec`, :meth:`loop.subprocess_shell`, + :meth:`loop.connect_read_pipe`, :meth:`loop.connect_write_pipe`, + as well as the :ref:`Subprocess Transports ` + and :ref:`Subprocess Protocols `. Constants ---------- +========= .. data:: asyncio.subprocess.PIPE - Special value that can be used as the *stdin*, *stdout* or *stderr* argument - to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and - indicates that a pipe to the standard stream should be opened. + Can be passed to the *stdin*, *stdout* or *stderr* parameters. + + If *PIPE* is passed to *stdin* argument, the + :attr:`Process.stdin ` attribute + will point to a :class:`StreamWriter` instance. + + If *PIPE* is passed to *stdout* or *stderr* arguments, the + :attr:`Process.stdout ` and + :attr:`Process.stderr ` + attributes will point to :class:`StreamReader` instances. .. data:: asyncio.subprocess.STDOUT - Special value that can be used as the *stderr* argument to - :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and - indicates that standard error should go into the same handle as standard - output. + Can be passed to the *stderr* parameter to redirect process' + *stderr* to *stdout*. .. data:: asyncio.subprocess.DEVNULL - Special value that can be used as the *stdin*, *stdout* or *stderr* argument - to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and - indicates that the special file :data:`os.devnull` will be used. + Can be passed as the *stdin*, *stdout* or *stderr* parameters + to redirect the corresponding subprocess' IO to :data:`os.devnull`. -Process -------- +Interacting with Subprocesses +============================= + +Both :func:`create_subprocess_exec` and :func:`create_subprocess_shell` +functions return instances of the *Process* class. It is a high-level +wrapper that allows to watch for subprocesses completion and +communicate with them. .. class:: asyncio.subprocess.Process - A subprocess created by the :func:`create_subprocess_exec` or the - :func:`create_subprocess_shell` function. + An object that wraps OS processes created by the + :func:`create_subprocess_exec` and :func:`create_subprocess_shell` + functions. + + This class is designed to have a similar API to the + :class:`subprocess.Popen` class, but there are some + notable differences: + + * unlike Popen, Process instances do not have an equivalent to + the :meth:`~subprocess.Popen.poll` method; + + * the :meth:`~asyncio.subprocess.Process.communicate` and + :meth:`~asyncio.subprocess.Process.wait` methods don't take a + *timeout* parameter: use the :func:`wait_for` function; + + * the :meth:`Process.wait() ` method + is asynchronous, whereas :meth:`subprocess.Popen.wait` method + is implemented as a blocking busy loop; - The API of the :class:`~asyncio.subprocess.Process` class was designed to be - close to the API of the :class:`subprocess.Popen` class, but there are some - differences: + * the *universal_newlines* parameter is not supported. - * There is no explicit :meth:`~subprocess.Popen.poll` method - * The :meth:`~subprocess.Popen.communicate` and - :meth:`~subprocess.Popen.wait` methods don't take a *timeout* parameter: - use the :func:`wait_for` function - * The *universal_newlines* parameter is not supported (only bytes strings - are supported) - * The :meth:`~asyncio.subprocess.Process.wait` method of - the :class:`~asyncio.subprocess.Process` class is asynchronous whereas the - :meth:`~subprocess.Popen.wait` method of the :class:`~subprocess.Popen` - class is implemented as a busy loop. + This class is :ref:`not thread safe `. - This class is :ref:`not thread safe `. See also the - :ref:`Subprocess and threads ` section. + See also the :ref:`Subprocess and Threads ` + section. .. coroutinemethod:: wait() - Wait for child process to terminate. Set and return :attr:`returncode` - attribute. + Wait for child process to terminate. - This method is a :ref:`coroutine `. + Set and return the :attr:`returncode` attribute. .. note:: - This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE`` and - the child process generates enough output to a pipe such that it - blocks waiting for the OS pipe buffer to accept more data. Use the - :meth:`communicate` method when using pipes to avoid that. + This method can deadlock when using ``stdout=PIPE`` or + ``stderr=PIPE`` and the child process generates so much output + that it blocks waiting for the OS pipe buffer to accept + more data. Use the :meth:`communicate` method when using pipes + to avoid this condition. .. coroutinemethod:: communicate(input=None) - Interact with process: Send data to stdin. Read data from stdout and - stderr, until end-of-file is reached. Wait for process to terminate. - The optional *input* argument should be data to be sent to the child - process, or ``None``, if no data should be sent to the child. The type - of *input* must be bytes. + Interact with process: - :meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``. + 1. send data to *stdin* (if *input* is not ``None``); + 2. read data from *stdout* and *stderr*, until EOF is reached; + 3. wait for process to terminate. - If a :exc:`BrokenPipeError` or :exc:`ConnectionResetError` exception is - raised when writing *input* into stdin, the exception is ignored. It - occurs when the process exits before all data are written into stdin. + The optional *input* argument is the data (:class:`bytes` object) + that will be sent to the child process. - Note that if you want to send data to the process's stdin, you need to - create the Process object with ``stdin=PIPE``. Similarly, to get anything - other than ``None`` in the result tuple, you need to give ``stdout=PIPE`` - and/or ``stderr=PIPE`` too. + Return a tuple ``(stdout_data, stderr_data)``. - This method is a :ref:`coroutine `. + If either :exc:`BrokenPipeError` or :exc:`ConnectionResetError` + exception is raised when writing *input* into *stdin*, the + exception is ignored. This condition occurs when the process + exits before all data are written into *stdin*. - .. note:: - - The data read is buffered in memory, so do not use this method if the - data size is large or unlimited. + If its desired to send data to the process' *stdin*, + the process needs to be created with ``stdin=PIPE``. Similarly, + to get anything other than ``None`` in the result tuple, the + process has to be created with ``stdout=PIPE`` and/or + ``stderr=PIPE`` arguments. - .. versionchanged:: 3.4.2 - The method now ignores :exc:`BrokenPipeError` and - :exc:`ConnectionResetError`. + Note, that the data read is buffered in memory, so do not use + this method if the data size is large or unlimited. .. method:: send_signal(signal) @@ -255,67 +228,81 @@ Process .. method:: terminate() - Stop the child. On Posix OSs the method sends :py:data:`signal.SIGTERM` - to the child. On Windows the Win32 API function - :c:func:`TerminateProcess` is called to stop the child. + Stop the child. + + On Posix OSs the method sends :py:data:`signal.SIGTERM` to the + child process. + + On Windows the Win32 API function :c:func:`TerminateProcess` is + called to stop the child process. .. method:: kill() - Kills the child. On Posix OSs the function sends :py:data:`SIGKILL` to - the child. On Windows :meth:`kill` is an alias for :meth:`terminate`. + Kill the child. + + On Posix OSs the function sends :py:data:`SIGKILL` to the child + process. + + On Windows this method is an alias for :meth:`terminate`. .. attribute:: stdin - Standard input stream (:class:`StreamWriter`), ``None`` if the process - was created with ``stdin=None``. + Standard input stream (:class:`StreamWriter`) or ``None`` + if the process was created with ``stdin=None``. .. attribute:: stdout - Standard output stream (:class:`StreamReader`), ``None`` if the process - was created with ``stdout=None``. + Standard output stream (:class:`StreamReader`) or ``None`` + if the process was created with ``stdout=None``. .. attribute:: stderr - Standard error stream (:class:`StreamReader`), ``None`` if the process - was created with ``stderr=None``. + Standard error stream (:class:`StreamReader`) or ``None`` + if the process was created with ``stderr=None``. .. warning:: - Use the :meth:`communicate` method rather than :attr:`.stdin.write - `, :attr:`.stdout.read ` or :attr:`.stderr.read ` - to avoid deadlocks due to streams pausing reading or writing and blocking - the child process. + Use the :meth:`communicate` method rather than + :attr:`process.stdin.write() `, + :attr:`await process.stdout.read() ` or + :attr:`await process.stderr.read ` + to avoid deadlocks due to streams pausing reading or writing + and blocking the child process. .. attribute:: pid - The identifier of the process. + Process identification number (PID). Note that for processes created by the :func:`create_subprocess_shell` - function, this attribute is the process identifier of the spawned shell. + function, this attribute is the PID of the spawned shell. .. attribute:: returncode - Return code of the process when it exited. A ``None`` value indicates - that the process has not terminated yet. + Return code of the process when it exits. + + A ``None`` value indicates that the process has not terminated yet. - A negative value ``-N`` indicates that the child was terminated by signal - ``N`` (Unix only). + A negative value ``-N`` indicates that the child was terminated + by signal ``N`` (Unix only). .. _asyncio-subprocess-threads: -Subprocess and threads +Subprocess and Threads ---------------------- -asyncio supports running subprocesses from different threads, but there -are limits: +asyncio built-in event loops support running subprocesses from +different threads, but there are the following limitations: -* An event loop must run in the main thread -* The child watcher must be instantiated in the main thread, before executing - subprocesses from other threads. Call the :func:`get_child_watcher` - function in the main thread to instantiate the child watcher. +* An event loop must run in the main thread. -The :class:`asyncio.subprocess.Process` class is not thread safe. +* The child watcher must be instantiated in the main thread, + before executing subprocesses from other threads. Call the + :func:`get_child_watcher` function in the main thread to instantiate + the child watcher. + +Note, that alternative event loop implementations might not share +the above limitations; please refer to their documentation. .. seealso:: @@ -323,97 +310,43 @@ The :class:`asyncio.subprocess.Process` class is not thread safe. ` section. -Subprocess examples -------------------- - -Subprocess using transport and protocol -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Example of a subprocess protocol using to get the output of a subprocess and to -wait for the subprocess exit. The subprocess is created by the -:meth:`AbstractEventLoop.subprocess_exec` method:: - - import asyncio - import sys - - class DateProtocol(asyncio.SubprocessProtocol): - def __init__(self, exit_future): - self.exit_future = exit_future - self.output = bytearray() - - def pipe_data_received(self, fd, data): - self.output.extend(data) - - def process_exited(self): - self.exit_future.set_result(True) - - async def get_date(loop): - code = 'import datetime; print(datetime.datetime.now())' - exit_future = asyncio.Future(loop=loop) - - # Create the subprocess controlled by the protocol DateProtocol, - # redirect the standard output into a pipe - transport, protocol = await loop.subprocess_exec( - lambda: DateProtocol(exit_future), - sys.executable, '-c', code, - stdin=None, stderr=None) - - # Wait for the subprocess exit using the process_exited() method - # of the protocol - await exit_future - - # Close the stdout pipe - transport.close() +Examples +-------- - # Read the output which was collected by the pipe_data_received() - # method of the protocol - data = bytes(protocol.output) - return data.decode('ascii').rstrip() +An example using the :class:`~asyncio.subprocess.Process` class to +control a subprocess and the :class:`StreamReader` class to read from +the *stdout*. - if sys.platform == "win32": - loop = asyncio.ProactorEventLoop() - asyncio.set_event_loop(loop) - else: - loop = asyncio.get_event_loop() - - date = loop.run_until_complete(get_date(loop)) - print("Current date: %s" % date) - loop.close() - - -Subprocess using streams -^^^^^^^^^^^^^^^^^^^^^^^^ - -Example using the :class:`~asyncio.subprocess.Process` class to control the -subprocess and the :class:`StreamReader` class to read from the standard -output. The subprocess is created by the :func:`create_subprocess_exec` +The subprocess is created by the :func:`create_subprocess_exec` function:: - import asyncio.subprocess + import asyncio import sys async def get_date(): code = 'import datetime; print(datetime.datetime.now())' - # Create the subprocess, redirect the standard output into a pipe + # Create the subprocess; redirect the standard output + # into a pipe. proc = await asyncio.create_subprocess_exec( sys.executable, '-c', code, stdout=asyncio.subprocess.PIPE) - # Read one line of output + # Read one line of output. data = await proc.stdout.readline() line = data.decode('ascii').rstrip() - # Wait for the subprocess exit + # Wait for the subprocess exit. await proc.wait() return line if sys.platform == "win32": - loop = asyncio.ProactorEventLoop() - asyncio.set_event_loop(loop) - else: - loop = asyncio.get_event_loop() - - date = loop.run_until_complete(get_date()) - print("Current date: %s" % date) - loop.close() + asyncio.set_event_loop_policy( + asyncio.WindowsProactorEventLoopPolicy()) + + date = asyncio.run(get_date()) + print(f"Current date: {date}") + + +See also the :ref:`same example ` +written using low-level APIs. diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index c73f36b1adb9..9f4433ddc339 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -63,7 +63,7 @@ the coroutine object returned by the call doesn't do anything until you schedule its execution. There are two basic ways to start it running: call ``await coroutine`` or ``yield from coroutine`` from another coroutine (assuming the other coroutine is already running!), or schedule its execution -using the :func:`ensure_future` function or the :meth:`AbstractEventLoop.create_task` +using the :func:`ensure_future` function or the :meth:`loop.create_task` method. @@ -129,7 +129,7 @@ Example of coroutine displaying ``"Hello World"``:: .. seealso:: The :ref:`Hello World with call_soon() ` - example uses the :meth:`AbstractEventLoop.call_soon` method to schedule a + example uses the :meth:`loop.call_soon` method to schedule a callback. @@ -159,7 +159,7 @@ using the :meth:`sleep` function:: The :ref:`display the current date with call_later() ` example uses a callback with the - :meth:`AbstractEventLoop.call_later` method. + :meth:`loop.call_later` method. Example: Chain coroutines @@ -190,32 +190,12 @@ Sequence diagram of the example: .. image:: tulip_coro.png :align: center -The "Task" is created by the :meth:`AbstractEventLoop.run_until_complete` method +The "Task" is created by the :meth:`loop.run_until_complete` method when it gets a coroutine object instead of a task. The diagram shows the control flow, it does not describe exactly how things work internally. For example, the sleep coroutine creates an internal future -which uses :meth:`AbstractEventLoop.call_later` to wake up the task in 1 second. - - -InvalidStateError ------------------ - -.. exception:: InvalidStateError - - The operation is not allowed in this state. - - -TimeoutError ------------- - -.. exception:: TimeoutError - - The operation exceeded the given deadline. - -.. note:: - - This exception is different from the builtin :exc:`TimeoutError` exception! +which uses :meth:`loop.call_later` to wake up the task in 1 second. Future @@ -231,7 +211,7 @@ Future raise an exception when the future isn't done yet. - Callbacks registered with :meth:`add_done_callback` are always called - via the event loop's :meth:`~AbstractEventLoop.call_soon`. + via the event loop's :meth:`loop.call_soon`. - This class is not compatible with the :func:`~concurrent.futures.wait` and :func:`~concurrent.futures.as_completed` functions in the @@ -281,7 +261,7 @@ Future The *callback* is called with a single argument - the future object. If the future is already done when this is called, the callback is scheduled - with :meth:`~AbstractEventLoop.call_soon`. + with :meth:`loop.call_soon`. An optional keyword-only *context* argument allows specifying a custom :class:`contextvars.Context` for the *callback* to run in. The current @@ -344,11 +324,11 @@ Example combining a :class:`Future` and a :ref:`coroutine function The coroutine function is responsible for the computation (which takes 1 second) and it stores the result into the future. The -:meth:`~AbstractEventLoop.run_until_complete` method waits for the completion of +:meth:`loop.run_until_complete` method waits for the completion of the future. .. note:: - The :meth:`~AbstractEventLoop.run_until_complete` method uses internally the + The :meth:`loop.run_until_complete` method uses internally the :meth:`~Future.add_done_callback` method to be notified when the future is done. @@ -433,7 +413,7 @@ Task logged: see :ref:`Pending task destroyed `. Don't directly create :class:`Task` instances: use the :func:`create_task` - function or the :meth:`AbstractEventLoop.create_task` method. + function or the :meth:`loop.create_task` method. Tasks support the :mod:`contextvars` module. When a Task is created it copies the current context and later runs its coroutine @@ -644,7 +624,7 @@ Task functions .. seealso:: The :func:`create_task` function and - :meth:`AbstractEventLoop.create_task` method. + :meth:`loop.create_task` method. .. function:: wrap_future(future, \*, loop=None) diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index b076b7d0093b..0d58a94c8b9b 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -50,7 +50,8 @@ Table of contents: :maxdepth: 3 asyncio-eventloop.rst - asyncio-eventloops.rst + asyncio-policy.rst + asyncio-platforms.rst asyncio-task.rst asyncio-protocol.rst asyncio-stream.rst @@ -58,6 +59,7 @@ Table of contents: asyncio-sync.rst asyncio-queue.rst asyncio-dev.rst + asyncio-exceptions.rst .. seealso:: diff --git a/Doc/library/ipc.rst b/Doc/library/ipc.rst index 6b1756331ec0..8f5b3b216c78 100644 --- a/Doc/library/ipc.rst +++ b/Doc/library/ipc.rst @@ -16,11 +16,11 @@ The list of modules described in this chapter is: .. toctree:: + asyncio.rst socket.rst ssl.rst select.rst selectors.rst - asyncio.rst asyncore.rst asynchat.rst signal.rst diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 96f86e6c6142..d3aed84d250e 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -777,18 +777,18 @@ all changes introduced in Python 3.5 have also been backported to Python 3.4.x. Notable changes in the :mod:`asyncio` module since Python 3.4.0: -* New debugging APIs: :meth:`loop.set_debug() ` - and :meth:`loop.get_debug() ` methods. +* New debugging APIs: :meth:`loop.set_debug() ` + and :meth:`loop.get_debug() ` methods. (Contributed by Victor Stinner.) * The proactor event loop now supports SSL. (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) -* A new :meth:`loop.is_closed() ` method to +* A new :meth:`loop.is_closed() ` method to check if the event loop is closed. (Contributed by Victor Stinner in :issue:`21326`.) -* A new :meth:`loop.create_task() ` +* A new :meth:`loop.create_task() ` to conveniently create and schedule a new :class:`~asyncio.Task` for a coroutine. The ``create_task`` method is also used by all asyncio functions that wrap coroutines into tasks, such as @@ -805,10 +805,10 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0: (Contributed by Yury Selivanov.) * New :meth:`loop.set_task_factory() - ` and - :meth:`loop.get_task_factory() ` + ` and + :meth:`loop.get_task_factory() ` methods to customize the task factory that :meth:`loop.create_task() - ` method uses. (Contributed by Yury + ` method uses. (Contributed by Yury Selivanov.) * New :meth:`Queue.join() ` and @@ -822,7 +822,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0: Updates in 3.5.1: * The :func:`~asyncio.ensure_future` function and all functions that - use it, such as :meth:`loop.run_until_complete() `, + use it, such as :meth:`loop.run_until_complete() `, now accept all kinds of :term:`awaitable objects `. (Contributed by Yury Selivanov.) @@ -834,20 +834,20 @@ Updates in 3.5.1: method to check if the transport is closing or closed. (Contributed by Yury Selivanov.) -* The :meth:`loop.create_server() ` +* The :meth:`loop.create_server() ` method can now accept a list of hosts. (Contributed by Yann Sionneau.) Updates in 3.5.2: -* New :meth:`loop.create_future() ` +* New :meth:`loop.create_future() ` method to create Future objects. This allows alternative event loop implementations, such as `uvloop `_, to provide a faster :class:`asyncio.Future` implementation. (Contributed by Yury Selivanov.) -* New :meth:`loop.get_exception_handler() ` +* New :meth:`loop.get_exception_handler() ` method to get the current exception handler. (Contributed by Yury Selivanov.) @@ -856,13 +856,13 @@ Updates in 3.5.2: sequence appears. (Contributed by Mark Korenberg.) -* The :meth:`loop.create_connection() ` - and :meth:`loop.create_server() ` +* The :meth:`loop.create_connection() ` + and :meth:`loop.create_server() ` methods are optimized to avoid calling the system ``getaddrinfo`` function if the address is already resolved. (Contributed by A. Jesse Jiryu Davis.) -* The :meth:`loop.sock_connect(sock, address) ` +* The :meth:`loop.sock_connect(sock, address) ` no longer requires the *address* to be resolved prior to the call. (Contributed by A. Jesse Jiryu Davis.) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 1360af79d073..b413b5a65d52 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -824,7 +824,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 (Contributed by Yury Selivanov in :issue:`28613`.) * The :func:`~asyncio.ensure_future` function and all functions that - use it, such as :meth:`loop.run_until_complete() `, + use it, such as :meth:`loop.run_until_complete() `, now accept all kinds of :term:`awaitable objects `. (Contributed by Yury Selivanov.) @@ -836,18 +836,18 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 method to check if the transport is closing or closed. (Contributed by Yury Selivanov.) -* The :meth:`loop.create_server() ` +* The :meth:`loop.create_server() ` method can now accept a list of hosts. (Contributed by Yann Sionneau.) -* New :meth:`loop.create_future() ` +* New :meth:`loop.create_future() ` method to create Future objects. This allows alternative event loop implementations, such as `uvloop `_, to provide a faster :class:`asyncio.Future` implementation. (Contributed by Yury Selivanov in :issue:`27041`.) -* New :meth:`loop.get_exception_handler() ` +* New :meth:`loop.get_exception_handler() ` method to get the current exception handler. (Contributed by Yury Selivanov in :issue:`27040`.) @@ -860,12 +860,12 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 has been improved. (Contributed by Mark Korenberg in :issue:`28370`.) -* The :meth:`loop.getaddrinfo() ` +* The :meth:`loop.getaddrinfo() ` method is optimized to avoid calling the system ``getaddrinfo`` function if the address is already resolved. (Contributed by A. Jesse Jiryu Davis.) -* The :meth:`loop.stop() ` +* The :meth:`loop.stop() ` method has been changed to stop the loop immediately after the current iteration. Any new callbacks scheduled as a result of the last iteration will be discarded. @@ -876,7 +876,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 the :exc:`StopIteration` exception. (Contributed by Chris Angelico in :issue:`26221`.) -* New :meth:`loop.connect_accepted_socket() ` +* New :meth:`loop.connect_accepted_socket() ` method to be used by servers that accept connections outside of asyncio, but that use asyncio to handle them. (Contributed by Jim Fulton in :issue:`27392`.) @@ -884,7 +884,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 * ``TCP_NODELAY`` flag is now set for all TCP transports by default. (Contributed by Yury Selivanov in :issue:`27456`.) -* New :meth:`loop.shutdown_asyncgens() ` +* New :meth:`loop.shutdown_asyncgens() ` to properly close pending asynchronous generators before closing the loop. (Contributed by Yury Selivanov in :issue:`28003`.) diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index cb4865e48403..fbaa2cf15f94 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -637,10 +637,10 @@ include: (Contributed by Yury Selivanov in :issue:`32314`.) * asyncio gained support for :mod:`contextvars`. - :meth:`loop.call_soon() `, - :meth:`loop.call_soon_threadsafe() `, - :meth:`loop.call_later() `, - :meth:`loop.call_at() `, and + :meth:`loop.call_soon() `, + :meth:`loop.call_soon_threadsafe() `, + :meth:`loop.call_later() `, + :meth:`loop.call_at() `, and :meth:`Future.add_done_callback() ` have a new optional keyword-only *context* parameter. :class:`Tasks ` now track their context automatically. @@ -651,11 +651,11 @@ include: to ``asyncio.get_event_loop().create_task()``. (Contributed by Andrew Svetlov in :issue:`32311`.) -* The new :meth:`loop.start_tls() ` +* The new :meth:`loop.start_tls() ` method can be used to upgrade an existing connection to TLS. (Contributed by Yury Selivanov in :issue:`23749`.) -* The new :meth:`loop.sock_recv_into() ` +* The new :meth:`loop.sock_recv_into() ` method allows reading data from a socket directly into a provided buffer making it possible to reduce data copies. (Contributed by Antoine Pitrou in :issue:`31819`.) @@ -683,13 +683,13 @@ include: can be used to determine if the writer is closing. (Contributed by Andrew Svetlov in :issue:`32391`.) -* The new :meth:`loop.sock_sendfile() ` +* The new :meth:`loop.sock_sendfile() ` coroutine method allows sending files using :mod:`os.sendfile` when possible. (Contributed by Andrew Svetlov in :issue:`32410`.) -* The new :meth:`Task.get_loop() ` and - :meth:`Future.get_loop() ` methods - return the instance of the loop on which a task or a future were created. +* The new :meth:`Future.get_loop() ` and + ``Task.get_loop()`` methods return the instance of the loop on which a task or + a future were created. :meth:`Server.get_loop() ` allows doing the same for :class:`asyncio.Server` objects. (Contributed by Yury Selivanov in :issue:`32415` and @@ -698,8 +698,8 @@ include: * It is now possible to control how instances of :class:`asyncio.Server` begin serving. Previously, the server would start serving immediately when created. The new *start_serving* keyword argument to - :meth:`loop.create_server() ` and - :meth:`loop.create_unix_server() `, + :meth:`loop.create_server() ` and + :meth:`loop.create_unix_server() `, as well as :meth:`Server.start_serving() `, and :meth:`Server.serve_forever() ` can be used to decouple server instantiation and serving. The new @@ -717,20 +717,20 @@ include: (Contributed by Yury Selivanov in :issue:`32662`.) * Callback objects returned by - :func:`loop.call_later() ` + :func:`loop.call_later() ` gained the new :meth:`when() ` method which returns an absolute scheduled callback timestamp. (Contributed by Andrew Svetlov in :issue:`32741`.) * The :meth:`loop.create_datagram_endpoint() \ - ` method + ` method gained support for Unix sockets. (Contributed by Quentin Dawans in :issue:`31245`.) * The :func:`asyncio.open_connection`, :func:`asyncio.start_server` functions, - :meth:`loop.create_connection() `, - :meth:`loop.create_server() `, - :meth:`loop.create_accepted_socket() ` + :meth:`loop.create_connection() `, + :meth:`loop.create_server() `, + :meth:`loop.create_accepted_socket() ` methods and their corresponding UNIX socket variants now accept the *ssl_handshake_timeout* keyword argument. (Contributed by Neil Aspinall in :issue:`29970`.) @@ -2360,11 +2360,11 @@ Changes in the Python API (Contributed by Brett Cannon in :issue:`33169`.) * In :mod:`asyncio`, - :meth:`loop.sock_recv() `, - :meth:`loop.sock_sendall() `, - :meth:`loop.sock_accept() `, - :meth:`loop.getaddrinfo() `, - :meth:`loop.getnameinfo() ` + :meth:`loop.sock_recv() `, + :meth:`loop.sock_sendall() `, + :meth:`loop.sock_accept() `, + :meth:`loop.getaddrinfo() `, + :meth:`loop.getnameinfo() ` have been changed to be proper coroutine methods to match their documentation. Previously, these methods returned :class:`asyncio.Future` instances. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index a9b689d53728..259dcf65f2e0 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -170,7 +170,7 @@ Deprecated * Passing an object that is not an instance of :class:`concurrent.futures.ThreadPoolExecutor` to - :meth:`asyncio.AbstractEventLoop.set_default_executor()` is + :meth:`asyncio.loop.set_default_executor()` is deprecated and will be prohibited in Python 3.9. (Contributed by Elvis Pranskevichus in :issue:`34075`.) @@ -264,7 +264,7 @@ Changes in the Python API * Asyncio tasks can now be named, either by passing the ``name`` keyword argument to :func:`asyncio.create_task` or - the :meth:`~asyncio.AbstractEventLoop.create_task` event loop method, or by + the :meth:`~asyncio.loop.create_task` event loop method, or by calling the :meth:`~asyncio.Task.set_name` method on the task object. The task name is visible in the ``repr()`` output of :class:`asyncio.Task` and can also be retrieved using the :meth:`~asyncio.Task.get_name` method. From webhook-mailer at python.org Tue Sep 11 13:13:08 2018 From: webhook-mailer at python.org (Andrew Svetlov) Date: Tue, 11 Sep 2018 17:13:08 -0000 Subject: [Python-checkins] bpo-34622: Extract asyncio exceptions into a separate module (GH-9141) Message-ID: https://github.com/python/cpython/commit/0baa72f4b2e7185298d09cf64c7b591efcd22af0 commit: 0baa72f4b2e7185298d09cf64c7b591efcd22af0 branch: master author: Andrew Svetlov committer: GitHub date: 2018-09-11T10:13:04-07:00 summary: bpo-34622: Extract asyncio exceptions into a separate module (GH-9141) files: A Lib/asyncio/exceptions.py A Misc/NEWS.d/next/Library/2018-09-10-13-04-40.bpo-34622.tpv_rN.rst M Lib/asyncio/__init__.py M Lib/asyncio/base_events.py M Lib/asyncio/base_futures.py M Lib/asyncio/events.py M Lib/asyncio/futures.py M Lib/asyncio/locks.py M Lib/asyncio/proactor_events.py M Lib/asyncio/streams.py M Lib/asyncio/tasks.py M Lib/asyncio/unix_events.py M Lib/asyncio/windows_events.py M Lib/test/test_asyncio/test_base_events.py M Lib/test/test_asyncio/test_events.py M Lib/test/test_asyncio/test_proactor_events.py M Lib/test/test_asyncio/test_unix_events.py M Modules/_asynciomodule.c diff --git a/Lib/asyncio/__init__.py b/Lib/asyncio/__init__.py index 26859024300e..28c2e2c429f3 100644 --- a/Lib/asyncio/__init__.py +++ b/Lib/asyncio/__init__.py @@ -8,6 +8,7 @@ from .base_events import * from .coroutines import * from .events import * +from .exceptions import * from .futures import * from .locks import * from .protocols import * @@ -25,6 +26,7 @@ __all__ = (base_events.__all__ + coroutines.__all__ + events.__all__ + + exceptions.__all__ + futures.__all__ + locks.__all__ + protocols.__all__ + diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index ee13d1a78028..046743864fdd 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -37,6 +37,7 @@ from . import constants from . import coroutines from . import events +from . import exceptions from . import futures from . import protocols from . import sslproto @@ -327,7 +328,7 @@ def close(self): try: await self._serving_forever_fut - except futures.CancelledError: + except exceptions.CancelledError: try: self.close() await self.wait_closed() @@ -800,7 +801,7 @@ def _getaddrinfo_debug(self, host, port, family, type, proto, flags): try: return await self._sock_sendfile_native(sock, file, offset, count) - except events.SendfileNotAvailableError as exc: + except exceptions.SendfileNotAvailableError as exc: if not fallback: raise return await self._sock_sendfile_fallback(sock, file, @@ -809,7 +810,7 @@ def _getaddrinfo_debug(self, host, port, family, type, proto, flags): async def _sock_sendfile_native(self, sock, file, offset, count): # NB: sendfile syscall is not supported for SSL sockets and # non-mmap files even if sendfile is supported by OS - raise events.SendfileNotAvailableError( + raise exceptions.SendfileNotAvailableError( f"syscall sendfile is not available for socket {sock!r} " "and file {file!r} combination") @@ -1053,7 +1054,7 @@ def _check_sendfile_params(self, sock, file, offset, count): try: return await self._sendfile_native(transport, file, offset, count) - except events.SendfileNotAvailableError as exc: + except exceptions.SendfileNotAvailableError as exc: if not fallback: raise @@ -1066,7 +1067,7 @@ def _check_sendfile_params(self, sock, file, offset, count): offset, count) async def _sendfile_native(self, transp, file, offset, count): - raise events.SendfileNotAvailableError( + raise exceptions.SendfileNotAvailableError( "sendfile syscall is not supported") async def _sendfile_fallback(self, transp, file, offset, count): diff --git a/Lib/asyncio/base_futures.py b/Lib/asyncio/base_futures.py index bd65beec553c..22f298069c50 100644 --- a/Lib/asyncio/base_futures.py +++ b/Lib/asyncio/base_futures.py @@ -1,15 +1,9 @@ __all__ = () -import concurrent.futures import reprlib from . import format_helpers -CancelledError = concurrent.futures.CancelledError -TimeoutError = concurrent.futures.TimeoutError -InvalidStateError = concurrent.futures.InvalidStateError - - # States for Future. _PENDING = 'PENDING' _CANCELLED = 'CANCELLED' diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py index 58a60a0b16a0..163b868afeee 100644 --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -3,7 +3,7 @@ __all__ = ( 'AbstractEventLoopPolicy', 'AbstractEventLoop', 'AbstractServer', - 'Handle', 'TimerHandle', 'SendfileNotAvailableError', + 'Handle', 'TimerHandle', 'get_event_loop_policy', 'set_event_loop_policy', 'get_event_loop', 'set_event_loop', 'new_event_loop', 'get_child_watcher', 'set_child_watcher', @@ -19,14 +19,7 @@ import threading from . import format_helpers - - -class SendfileNotAvailableError(RuntimeError): - """Sendfile syscall is not available. - - Raised if OS does not support sendfile syscall for given socket or - file type. - """ +from . import exceptions class Handle: diff --git a/Lib/asyncio/exceptions.py b/Lib/asyncio/exceptions.py new file mode 100644 index 000000000000..cac31a54d253 --- /dev/null +++ b/Lib/asyncio/exceptions.py @@ -0,0 +1,60 @@ +"""asyncio exceptions.""" + + +__all__ = ('CancelledError', 'InvalidStateError', 'TimeoutError', + 'IncompleteReadError', 'LimitOverrunError', + 'SendfileNotAvailableError') + +import concurrent.futures +from . import base_futures + + +class CancelledError(concurrent.futures.CancelledError): + """The Future or Task was cancelled.""" + + +class TimeoutError(concurrent.futures.TimeoutError): + """The operation exceeded the given deadline.""" + + +class InvalidStateError(concurrent.futures.InvalidStateError): + """The operation is not allowed in this state.""" + + +class SendfileNotAvailableError(RuntimeError): + """Sendfile syscall is not available. + + Raised if OS does not support sendfile syscall for given socket or + file type. + """ + + +class IncompleteReadError(EOFError): + """ + Incomplete read error. Attributes: + + - partial: read bytes string before the end of stream was reached + - expected: total number of expected bytes (or None if unknown) + """ + def __init__(self, partial, expected): + super().__init__(f'{len(partial)} bytes read on a total of ' + f'{expected!r} expected bytes') + self.partial = partial + self.expected = expected + + def __reduce__(self): + return type(self), (self.partial, self.expected) + + +class LimitOverrunError(Exception): + """Reached the buffer limit while looking for a separator. + + Attributes: + - consumed: total number of to be consumed bytes. + """ + def __init__(self, message, consumed): + super().__init__(message) + self.consumed = consumed + + def __reduce__(self): + return type(self), (self.args[0], self.consumed) diff --git a/Lib/asyncio/futures.py b/Lib/asyncio/futures.py index 0e0e696a2535..98a5308ed061 100644 --- a/Lib/asyncio/futures.py +++ b/Lib/asyncio/futures.py @@ -1,7 +1,6 @@ """A Future class similar to the one in PEP 3148.""" __all__ = ( - 'CancelledError', 'TimeoutError', 'InvalidStateError', 'Future', 'wrap_future', 'isfuture', ) @@ -12,12 +11,10 @@ from . import base_futures from . import events +from . import exceptions from . import format_helpers -CancelledError = base_futures.CancelledError -InvalidStateError = base_futures.InvalidStateError -TimeoutError = base_futures.TimeoutError isfuture = base_futures.isfuture @@ -170,9 +167,9 @@ def result(self): the future is done and has an exception set, this exception is raised. """ if self._state == _CANCELLED: - raise CancelledError + raise exceptions.CancelledError if self._state != _FINISHED: - raise InvalidStateError('Result is not ready.') + raise exceptions.InvalidStateError('Result is not ready.') self.__log_traceback = False if self._exception is not None: raise self._exception @@ -187,9 +184,9 @@ def exception(self): InvalidStateError. """ if self._state == _CANCELLED: - raise CancelledError + raise exceptions.CancelledError if self._state != _FINISHED: - raise InvalidStateError('Exception is not set.') + raise exceptions.InvalidStateError('Exception is not set.') self.__log_traceback = False return self._exception @@ -231,7 +228,7 @@ def set_result(self, result): InvalidStateError. """ if self._state != _PENDING: - raise InvalidStateError('{}: {!r}'.format(self._state, self)) + raise exceptions.InvalidStateError(f'{self._state}: {self!r}') self._result = result self._state = _FINISHED self.__schedule_callbacks() @@ -243,7 +240,7 @@ def set_exception(self, exception): InvalidStateError. """ if self._state != _PENDING: - raise InvalidStateError('{}: {!r}'.format(self._state, self)) + raise exceptions.InvalidStateError(f'{self._state}: {self!r}') if isinstance(exception, type): exception = exception() if type(exception) is StopIteration: @@ -288,6 +285,18 @@ def _set_result_unless_cancelled(fut, result): fut.set_result(result) +def _convert_future_exc(exc): + exc_class = type(exc) + if exc_class is concurrent.futures.CancelledError: + return exceptions.CancelledError(*exc.args) + elif exc_class is concurrent.futures.TimeoutError: + return exceptions.TimeoutError(*exc.args) + elif exc_class is concurrent.futures.InvalidStateError: + return exceptions.InvalidStateError(*exc.args) + else: + return exc + + def _set_concurrent_future_state(concurrent, source): """Copy state from a future to a concurrent.futures.Future.""" assert source.done() @@ -297,7 +306,7 @@ def _set_concurrent_future_state(concurrent, source): return exception = source.exception() if exception is not None: - concurrent.set_exception(exception) + concurrent.set_exception(_convert_future_exc(exception)) else: result = source.result() concurrent.set_result(result) @@ -317,7 +326,7 @@ def _copy_future_state(source, dest): else: exception = source.exception() if exception is not None: - dest.set_exception(exception) + dest.set_exception(_convert_future_exc(exception)) else: result = source.result() dest.set_result(result) diff --git a/Lib/asyncio/locks.py b/Lib/asyncio/locks.py index 91f7a01de8ad..639bd11bd068 100644 --- a/Lib/asyncio/locks.py +++ b/Lib/asyncio/locks.py @@ -7,6 +7,7 @@ from . import events from . import futures +from . import exceptions from .coroutines import coroutine @@ -192,7 +193,7 @@ def locked(self): await fut finally: self._waiters.remove(fut) - except futures.CancelledError: + except exceptions.CancelledError: if not self._locked: self._wake_up_first() raise @@ -363,11 +364,11 @@ def __repr__(self): try: await self.acquire() break - except futures.CancelledError: + except exceptions.CancelledError: cancelled = True if cancelled: - raise futures.CancelledError + raise exceptions.CancelledError async def wait_for(self, predicate): """Wait until a predicate becomes true. diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py index 66bfb0ab11ee..ad23918802fa 100644 --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -15,6 +15,7 @@ from . import constants from . import events from . import futures +from . import exceptions from . import protocols from . import sslproto from . import transports @@ -282,7 +283,7 @@ def _loop_reading(self, fut=None): self._force_close(exc) except OSError as exc: self._fatal_error(exc, 'Fatal read error on pipe transport') - except futures.CancelledError: + except exceptions.CancelledError: if not self._closing: raise else: @@ -555,11 +556,11 @@ def close(self): try: fileno = file.fileno() except (AttributeError, io.UnsupportedOperation) as err: - raise events.SendfileNotAvailableError("not a regular file") + raise exceptions.SendfileNotAvailableError("not a regular file") try: fsize = os.fstat(fileno).st_size except OSError as err: - raise events.SendfileNotAvailableError("not a regular file") + raise exceptions.SendfileNotAvailableError("not a regular file") blocksize = count if count else fsize if not blocksize: return 0 # empty file @@ -615,7 +616,7 @@ def _loop_self_reading(self, f=None): if f is not None: f.result() # may raise f = self._proactor.recv(self._ssock, 4096) - except futures.CancelledError: + except exceptions.CancelledError: # _close_self_pipe() has been called, stop waiting for data return except Exception as exc: @@ -666,7 +667,7 @@ def loop(f=None): elif self._debug: logger.debug("Accept failed on socket %r", sock, exc_info=True) - except futures.CancelledError: + except exceptions.CancelledError: sock.close() else: self._accept_futures[sock.fileno()] = f diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py index d6531f88a74d..9dab49b35e46 100644 --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -1,8 +1,6 @@ __all__ = ( 'StreamReader', 'StreamWriter', 'StreamReaderProtocol', - 'open_connection', 'start_server', - 'IncompleteReadError', 'LimitOverrunError', -) + 'open_connection', 'start_server') import socket @@ -11,6 +9,7 @@ from . import coroutines from . import events +from . import exceptions from . import protocols from .log import logger from .tasks import sleep @@ -19,37 +18,6 @@ _DEFAULT_LIMIT = 2 ** 16 # 64 KiB -class IncompleteReadError(EOFError): - """ - Incomplete read error. Attributes: - - - partial: read bytes string before the end of stream was reached - - expected: total number of expected bytes (or None if unknown) - """ - def __init__(self, partial, expected): - super().__init__(f'{len(partial)} bytes read on a total of ' - f'{expected!r} expected bytes') - self.partial = partial - self.expected = expected - - def __reduce__(self): - return type(self), (self.partial, self.expected) - - -class LimitOverrunError(Exception): - """Reached the buffer limit while looking for a separator. - - Attributes: - - consumed: total number of to be consumed bytes. - """ - def __init__(self, message, consumed): - super().__init__(message) - self.consumed = consumed - - def __reduce__(self): - return type(self), (self.args[0], self.consumed) - - async def open_connection(host=None, port=None, *, loop=None, limit=_DEFAULT_LIMIT, **kwds): """A wrapper for create_connection() returning a (reader, writer) pair. @@ -494,9 +462,9 @@ def feed_data(self, data): seplen = len(sep) try: line = await self.readuntil(sep) - except IncompleteReadError as e: + except exceptions.IncompleteReadError as e: return e.partial - except LimitOverrunError as e: + except exceptions.LimitOverrunError as e: if self._buffer.startswith(sep, e.consumed): del self._buffer[:e.consumed + seplen] else: @@ -571,7 +539,7 @@ def feed_data(self, data): # see upper comment for explanation. offset = buflen + 1 - seplen if offset > self._limit: - raise LimitOverrunError( + raise exceptions.LimitOverrunError( 'Separator is not found, and chunk exceed the limit', offset) @@ -582,13 +550,13 @@ def feed_data(self, data): if self._eof: chunk = bytes(self._buffer) self._buffer.clear() - raise IncompleteReadError(chunk, None) + raise exceptions.IncompleteReadError(chunk, None) # _wait_for_data() will resume reading if stream was paused. await self._wait_for_data('readuntil') if isep > self._limit: - raise LimitOverrunError( + raise exceptions.LimitOverrunError( 'Separator is found, but chunk is longer than limit', isep) chunk = self._buffer[:isep + seplen] @@ -674,7 +642,7 @@ def feed_data(self, data): if self._eof: incomplete = bytes(self._buffer) self._buffer.clear() - raise IncompleteReadError(incomplete, n) + raise exceptions.IncompleteReadError(incomplete, n) await self._wait_for_data('readexactly') diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py index 03d71d37f01a..7121aa65da0e 100644 --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -21,6 +21,7 @@ from . import base_tasks from . import coroutines from . import events +from . import exceptions from . import futures from .coroutines import coroutine @@ -228,11 +229,11 @@ def cancel(self): def __step(self, exc=None): if self.done(): - raise futures.InvalidStateError( + raise exceptions.InvalidStateError( f'_step(): already done: {self!r}, {exc!r}') if self._must_cancel: - if not isinstance(exc, futures.CancelledError): - exc = futures.CancelledError() + if not isinstance(exc, exceptions.CancelledError): + exc = exceptions.CancelledError() self._must_cancel = False coro = self._coro self._fut_waiter = None @@ -250,10 +251,10 @@ def __step(self, exc=None): if self._must_cancel: # Task is cancelled right before coro stops. self._must_cancel = False - super().set_exception(futures.CancelledError()) + super().set_exception(exceptions.CancelledError()) else: super().set_result(exc.value) - except futures.CancelledError: + except exceptions.CancelledError: super().cancel() # I.e., Future.cancel(self). except Exception as exc: super().set_exception(exc) @@ -419,7 +420,7 @@ def _release_waiter(waiter, *args): return fut.result() fut.cancel() - raise futures.TimeoutError() + raise exceptions.TimeoutError() waiter = loop.create_future() timeout_handle = loop.call_later(timeout, _release_waiter, waiter) @@ -432,7 +433,7 @@ def _release_waiter(waiter, *args): # wait until the future completes or the timeout try: await waiter - except futures.CancelledError: + except exceptions.CancelledError: fut.remove_done_callback(cb) fut.cancel() raise @@ -445,7 +446,7 @@ def _release_waiter(waiter, *args): # after wait_for() returns. # See https://bugs.python.org/issue32751 await _cancel_and_wait(fut, loop=loop) - raise futures.TimeoutError() + raise exceptions.TimeoutError() finally: timeout_handle.cancel() @@ -554,7 +555,7 @@ def _on_completion(f): f = await done.get() if f is None: # Dummy value from _on_timeout(). - raise futures.TimeoutError + raise exceptions.TimeoutError return f.result() # May raise f.exception(). for f in todo: @@ -701,7 +702,7 @@ def _done_callback(fut): # Check if 'fut' is cancelled first, as # 'fut.exception()' will *raise* a CancelledError # instead of returning it. - exc = futures.CancelledError() + exc = exceptions.CancelledError() outer.set_exception(exc) return else: @@ -720,7 +721,7 @@ def _done_callback(fut): # Check if 'fut' is cancelled first, as # 'fut.exception()' will *raise* a CancelledError # instead of returning it. - res = futures.CancelledError() + res = exceptions.CancelledError() else: res = fut.exception() if res is None: @@ -731,7 +732,7 @@ def _done_callback(fut): # If gather is being cancelled we must propagate the # cancellation regardless of *return_exceptions* argument. # See issue 32684. - outer.set_exception(futures.CancelledError()) + outer.set_exception(exceptions.CancelledError()) else: outer.set_result(results) diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py index 7cad7e3637a1..1a62db4f59bc 100644 --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -18,6 +18,7 @@ from . import constants from . import coroutines from . import events +from . import exceptions from . import futures from . import selector_events from . import tasks @@ -319,16 +320,16 @@ def _child_watcher_callback(self, pid, returncode, transp): try: os.sendfile except AttributeError as exc: - raise events.SendfileNotAvailableError( + raise exceptions.SendfileNotAvailableError( "os.sendfile() is not available") try: fileno = file.fileno() except (AttributeError, io.UnsupportedOperation) as err: - raise events.SendfileNotAvailableError("not a regular file") + raise exceptions.SendfileNotAvailableError("not a regular file") try: fsize = os.fstat(fileno).st_size except OSError as err: - raise events.SendfileNotAvailableError("not a regular file") + raise exceptions.SendfileNotAvailableError("not a regular file") blocksize = count if count else fsize if not blocksize: return 0 # empty file @@ -382,7 +383,7 @@ def _sock_sendfile_native_impl(self, fut, registered_fd, sock, fileno, # one being 'file' is not a regular mmap(2)-like # file, in which case we'll fall back on using # plain send(). - err = events.SendfileNotAvailableError( + err = exceptions.SendfileNotAvailableError( "os.sendfile call failed") self._sock_sendfile_update_filepos(fileno, offset, total_sent) fut.set_exception(err) diff --git a/Lib/asyncio/windows_events.py b/Lib/asyncio/windows_events.py index 2ec542764375..fdde8e9e0bf4 100644 --- a/Lib/asyncio/windows_events.py +++ b/Lib/asyncio/windows_events.py @@ -12,6 +12,7 @@ from . import events from . import base_subprocess from . import futures +from . import exceptions from . import proactor_events from . import selector_events from . import tasks @@ -351,7 +352,7 @@ def loop_accept_pipe(f=None): elif self._debug: logger.warning("Accept pipe failed on pipe %r", pipe, exc_info=True) - except futures.CancelledError: + except exceptions.CancelledError: if pipe: pipe.close() else: @@ -497,7 +498,7 @@ def finish_accept(trans, key, ov): # Coroutine closing the accept socket if the future is cancelled try: await future - except futures.CancelledError: + except exceptions.CancelledError: conn.close() raise diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py index e10863795335..fe3c38371d0c 100644 --- a/Lib/test/test_asyncio/test_base_events.py +++ b/Lib/test/test_asyncio/test_base_events.py @@ -1946,7 +1946,7 @@ def cleanup(): def test__sock_sendfile_native_failure(self): sock, proto = self.prepare() - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "sendfile is not available"): self.run_loop(self.loop._sock_sendfile_native(sock, self.file, 0, None)) @@ -1957,7 +1957,7 @@ def test__sock_sendfile_native_failure(self): def test_sock_sendfile_no_fallback(self): sock, proto = self.prepare() - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "sendfile is not available"): self.run_loop(self.loop.sock_sendfile(sock, self.file, fallback=False)) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py index 11cd950df1ce..01f616ab5b19 100644 --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -2393,7 +2393,7 @@ def sendfile_native(transp, file, offset, count): self.loop._sendfile_native = sendfile_native - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "not supported"): self.run_loop( self.loop.sendfile(cli_proto.transport, self.file, diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py index dddbe3b2a107..afc4c19a96b0 100644 --- a/Lib/test/test_asyncio/test_proactor_events.py +++ b/Lib/test/test_asyncio/test_proactor_events.py @@ -951,7 +951,7 @@ def cleanup(): def test_sock_sendfile_not_a_file(self): sock, proto = self.prepare() f = object() - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "not a regular file"): self.run_loop(self.loop._sock_sendfile_native(sock, f, 0, None)) @@ -960,7 +960,7 @@ def test_sock_sendfile_not_a_file(self): def test_sock_sendfile_iobuffer(self): sock, proto = self.prepare() f = io.BytesIO() - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "not a regular file"): self.run_loop(self.loop._sock_sendfile_native(sock, f, 0, None)) @@ -970,7 +970,7 @@ def test_sock_sendfile_not_regular_file(self): sock, proto = self.prepare() f = mock.Mock() f.fileno.return_value = -1 - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "not a regular file"): self.run_loop(self.loop._sock_sendfile_native(sock, f, 0, None)) diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py index 29b345b142e8..62545c0f98c3 100644 --- a/Lib/test/test_asyncio/test_unix_events.py +++ b/Lib/test/test_asyncio/test_unix_events.py @@ -521,7 +521,7 @@ def cleanup(): def test_sock_sendfile_not_available(self): sock, proto = self.prepare() with mock.patch('asyncio.unix_events.os', spec=[]): - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "os[.]sendfile[(][)] is not available"): self.run_loop(self.loop._sock_sendfile_native(sock, self.file, 0, None)) @@ -530,7 +530,7 @@ def test_sock_sendfile_not_available(self): def test_sock_sendfile_not_a_file(self): sock, proto = self.prepare() f = object() - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "not a regular file"): self.run_loop(self.loop._sock_sendfile_native(sock, f, 0, None)) @@ -539,7 +539,7 @@ def test_sock_sendfile_not_a_file(self): def test_sock_sendfile_iobuffer(self): sock, proto = self.prepare() f = io.BytesIO() - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "not a regular file"): self.run_loop(self.loop._sock_sendfile_native(sock, f, 0, None)) @@ -549,7 +549,7 @@ def test_sock_sendfile_not_regular_file(self): sock, proto = self.prepare() f = mock.Mock() f.fileno.return_value = -1 - with self.assertRaisesRegex(events.SendfileNotAvailableError, + with self.assertRaisesRegex(asyncio.SendfileNotAvailableError, "not a regular file"): self.run_loop(self.loop._sock_sendfile_native(sock, f, 0, None)) @@ -605,7 +605,7 @@ def test_sock_sendfile_os_error_first_call(self): with self.assertRaises(KeyError): self.loop._selector.get_key(sock) exc = fut.exception() - self.assertIsInstance(exc, events.SendfileNotAvailableError) + self.assertIsInstance(exc, asyncio.SendfileNotAvailableError) self.assertEqual(0, self.file.tell()) def test_sock_sendfile_os_error_next_call(self): @@ -630,7 +630,7 @@ def test_sock_sendfile_exception(self): fileno = self.file.fileno() fut = self.loop.create_future() - err = events.SendfileNotAvailableError() + err = asyncio.SendfileNotAvailableError() with mock.patch('os.sendfile', side_effect=err): self.loop._sock_sendfile_native_impl(fut, sock.fileno(), sock, fileno, diff --git a/Misc/NEWS.d/next/Library/2018-09-10-13-04-40.bpo-34622.tpv_rN.rst b/Misc/NEWS.d/next/Library/2018-09-10-13-04-40.bpo-34622.tpv_rN.rst new file mode 100644 index 000000000000..493d6abd910b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-13-04-40.bpo-34622.tpv_rN.rst @@ -0,0 +1,4 @@ +Create a dedicated ``asyncio.CancelledError``, ``asyncio.InvalidStateError`` +and ``asyncio.TimeoutError`` exception classes. Inherit them from +corresponding exceptions from ``concurrent.futures`` package. Extract +``asyncio`` exceptions into a separate file. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 3d7ce01a680c..2ed033cacd85 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -3306,6 +3306,8 @@ module_init(void) WITH_MOD("asyncio.base_futures") GET_MOD_ATTR(asyncio_future_repr_info_func, "_future_repr_info") + + WITH_MOD("asyncio.exceptions") GET_MOD_ATTR(asyncio_InvalidStateError, "InvalidStateError") GET_MOD_ATTR(asyncio_CancelledError, "CancelledError") From webhook-mailer at python.org Tue Sep 11 13:29:52 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Tue, 11 Sep 2018 17:29:52 -0000 Subject: [Python-checkins] bpo-29386: Pass -1 to epoll_wait() when timeout is < -1 (GH-9040) Message-ID: https://github.com/python/cpython/commit/b690b9b04729ba3d91c59bff1bb11c3dcc1b50fc commit: b690b9b04729ba3d91c59bff1bb11c3dcc1b50fc branch: master author: Berker Peksag committer: GitHub date: 2018-09-11T20:29:48+03:00 summary: bpo-29386: Pass -1 to epoll_wait() when timeout is < -1 (GH-9040) Although the kernel accepts any negative value for timeout, the documented value to block indefinitely is -1. This commit also makes the code similar to select.poll.poll(). files: M Modules/selectmodule.c diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c index 836af5429af5..d86727a8978d 100644 --- a/Modules/selectmodule.c +++ b/Modules/selectmodule.c @@ -1498,17 +1498,12 @@ select_epoll_poll_impl(pyEpoll_Object *self, PyObject *timeout_obj, int nfds, i; PyObject *elist = NULL, *etuple = NULL; struct epoll_event *evs = NULL; - _PyTime_t timeout, ms, deadline; + _PyTime_t timeout = -1, ms = -1, deadline = 0; if (self->epfd < 0) return pyepoll_err_closed(); - if (timeout_obj == Py_None) { - timeout = -1; - ms = -1; - deadline = 0; /* initialize to prevent gcc warning */ - } - else { + if (timeout_obj != Py_None) { /* epoll_wait() has a resolution of 1 millisecond, round towards infinity to wait at least timeout seconds. */ if (_PyTime_FromSecondsObject(&timeout, timeout_obj, @@ -1525,8 +1520,20 @@ select_epoll_poll_impl(pyEpoll_Object *self, PyObject *timeout_obj, PyErr_SetString(PyExc_OverflowError, "timeout is too large"); return NULL; } + /* epoll_wait(2) treats all arbitrary negative numbers the same + for the timeout argument, but -1 is the documented way to block + indefinitely in the epoll_wait(2) documentation, so we set ms + to -1 if the value of ms is a negative number. + + Note that we didn't use INFTIM here since it's non-standard and + isn't available under Linux. */ + if (ms < 0) { + ms = -1; + } - deadline = _PyTime_GetMonotonicClock() + timeout; + if (timeout >= 0) { + deadline = _PyTime_GetMonotonicClock() + timeout; + } } if (maxevents == -1) { From webhook-mailer at python.org Tue Sep 11 13:35:12 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 17:35:12 -0000 Subject: [Python-checkins] Fix Tools/gdb/libpython.py Message-ID: https://github.com/python/cpython/commit/ed74a258aab2d5ca56a311acffdc7a46ca8ce2a1 commit: ed74a258aab2d5ca56a311acffdc7a46ca8ce2a1 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T10:35:08-07:00 summary: Fix Tools/gdb/libpython.py Backport https://github.com/python/cpython/commit/11659d00b9185c8f02ea6b642fa475a80e21f1a9 into this change instead of leaving it a separate followup change. files: A Misc/NEWS.d/next/Core and Builtins/2018-04-19-08-30-07.bpo-33312.mDe2iL.rst M Objects/dict-common.h M Objects/dictobject.c M Tools/gdb/libpython.py diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-04-19-08-30-07.bpo-33312.mDe2iL.rst b/Misc/NEWS.d/next/Core and Builtins/2018-04-19-08-30-07.bpo-33312.mDe2iL.rst new file mode 100644 index 000000000000..40b51b902c9f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-04-19-08-30-07.bpo-33312.mDe2iL.rst @@ -0,0 +1,3 @@ +Fixed clang ubsan (undefined behavior sanitizer) warnings in dictobject.c by +adjusting how the internal struct _dictkeysobject shared keys structure is +declared. diff --git a/Objects/dict-common.h b/Objects/dict-common.h index ce9edabd8959..d3ae07bfd3f4 100644 --- a/Objects/dict-common.h +++ b/Objects/dict-common.h @@ -59,15 +59,8 @@ struct _dictkeysobject { - 4 bytes if dk_size <= 0xffffffff (int32_t*) - 8 bytes otherwise (int64_t*) - Dynamically sized, 8 is minimum. */ - union { - int8_t as_1[8]; - int16_t as_2[4]; - int32_t as_4[2]; -#if SIZEOF_VOID_P > 4 - int64_t as_8[1]; -#endif - } dk_indices; + Dynamically sized, SIZEOF_VOID_P is minimum. */ + char dk_indices[]; /* char is required to avoid strict aliasing. */ /* "PyDictKeyEntry dk_entries[dk_usable];" array follows: see the DK_ENTRIES() macro */ diff --git a/Objects/dictobject.c b/Objects/dictobject.c index ddd05826ea5f..0768d1154542 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -300,7 +300,7 @@ PyDict_Fini(void) 2 : sizeof(int32_t)) #endif #define DK_ENTRIES(dk) \ - ((PyDictKeyEntry*)(&(dk)->dk_indices.as_1[DK_SIZE(dk) * DK_IXSIZE(dk)])) + ((PyDictKeyEntry*)(&((int8_t*)((dk)->dk_indices))[DK_SIZE(dk) * DK_IXSIZE(dk)])) #define DK_DEBUG_INCREF _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA #define DK_DEBUG_DECREF _Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA @@ -318,21 +318,21 @@ dk_get_index(PyDictKeysObject *keys, Py_ssize_t i) Py_ssize_t ix; if (s <= 0xff) { - int8_t *indices = keys->dk_indices.as_1; + int8_t *indices = (int8_t*)(keys->dk_indices); ix = indices[i]; } else if (s <= 0xffff) { - int16_t *indices = keys->dk_indices.as_2; + int16_t *indices = (int16_t*)(keys->dk_indices); ix = indices[i]; } #if SIZEOF_VOID_P > 4 else if (s > 0xffffffff) { - int64_t *indices = keys->dk_indices.as_8; + int64_t *indices = (int64_t*)(keys->dk_indices); ix = indices[i]; } #endif else { - int32_t *indices = keys->dk_indices.as_4; + int32_t *indices = (int32_t*)(keys->dk_indices); ix = indices[i]; } assert(ix >= DKIX_DUMMY); @@ -348,23 +348,23 @@ dk_set_index(PyDictKeysObject *keys, Py_ssize_t i, Py_ssize_t ix) assert(ix >= DKIX_DUMMY); if (s <= 0xff) { - int8_t *indices = keys->dk_indices.as_1; + int8_t *indices = (int8_t*)(keys->dk_indices); assert(ix <= 0x7f); indices[i] = (char)ix; } else if (s <= 0xffff) { - int16_t *indices = keys->dk_indices.as_2; + int16_t *indices = (int16_t*)(keys->dk_indices); assert(ix <= 0x7fff); indices[i] = (int16_t)ix; } #if SIZEOF_VOID_P > 4 else if (s > 0xffffffff) { - int64_t *indices = keys->dk_indices.as_8; + int64_t *indices = (int64_t*)(keys->dk_indices); indices[i] = ix; } #endif else { - int32_t *indices = keys->dk_indices.as_4; + int32_t *indices = (int32_t*)(keys->dk_indices); assert(ix <= 0x7fffffff); indices[i] = (int32_t)ix; } @@ -424,8 +424,8 @@ static PyDictKeysObject empty_keys_struct = { lookdict_split, /* dk_lookup */ 0, /* dk_usable (immutable) */ 0, /* dk_nentries */ - .dk_indices = { .as_1 = {DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, - DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY}}, + {DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, + DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY}, /* dk_indices */ }; static PyObject *empty_values[1] = { NULL }; @@ -533,7 +533,6 @@ static PyDictKeysObject *new_keys_object(Py_ssize_t size) } else { dk = PyObject_MALLOC(sizeof(PyDictKeysObject) - - Py_MEMBER_SIZE(PyDictKeysObject, dk_indices) + es * size + sizeof(PyDictKeyEntry) * usable); if (dk == NULL) { @@ -546,7 +545,7 @@ static PyDictKeysObject *new_keys_object(Py_ssize_t size) dk->dk_usable = usable; dk->dk_lookup = lookdict_unicode_nodummy; dk->dk_nentries = 0; - memset(&dk->dk_indices.as_1[0], 0xff, es * size); + memset(&dk->dk_indices[0], 0xff, es * size); memset(DK_ENTRIES(dk), 0, sizeof(PyDictKeyEntry) * usable); return dk; } @@ -3086,7 +3085,6 @@ _PyDict_SizeOf(PyDictObject *mp) in the type object. */ if (mp->ma_keys->dk_refcnt == 1) res += (sizeof(PyDictKeysObject) - - Py_MEMBER_SIZE(PyDictKeysObject, dk_indices) + DK_IXSIZE(mp->ma_keys) * size + sizeof(PyDictKeyEntry) * usable); return res; @@ -3096,7 +3094,6 @@ Py_ssize_t _PyDict_KeysSize(PyDictKeysObject *keys) { return (sizeof(PyDictKeysObject) - - Py_MEMBER_SIZE(PyDictKeysObject, dk_indices) + DK_IXSIZE(keys) * DK_SIZE(keys) + USABLE_FRACTION(DK_SIZE(keys)) * sizeof(PyDictKeyEntry)); } diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py index 6d7a94247bc3..6d3d52f397d4 100755 --- a/Tools/gdb/libpython.py +++ b/Tools/gdb/libpython.py @@ -736,7 +736,7 @@ def _get_entries(self, keys): else: offset = 8 * dk_size - ent_addr = keys['dk_indices']['as_1'].address + ent_addr = keys['dk_indices'].address ent_addr = ent_addr.cast(_type_unsigned_char_ptr()) + offset ent_ptr_t = gdb.lookup_type('PyDictKeyEntry').pointer() ent_addr = ent_addr.cast(ent_ptr_t) From webhook-mailer at python.org Tue Sep 11 13:44:56 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 17:44:56 -0000 Subject: [Python-checkins] bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086) Message-ID: https://github.com/python/cpython/commit/08bcf647d8a92e4bd47531588b284c6820b7a7ef commit: 08bcf647d8a92e4bd47531588b284c6820b7a7ef branch: master author: wim glenn committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T10:44:52-07:00 summary: bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086) https://bugs.python.org/issue28617 files: A Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 87c64214c67d..f7f59cd3a6bb 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -197,8 +197,8 @@ exception. operator: not in Two more operations with the same syntactic priority, :keyword:`in` and -:keyword:`not in`, are supported only by sequence types (below). - +:keyword:`not in`, are supported by types that are :term:`iterable` or +implement the :meth:`__contains__` method. .. _typesnumeric: diff --git a/Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst b/Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst new file mode 100644 index 000000000000..281afad71d27 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst @@ -0,0 +1,2 @@ +Fixed info in the stdtypes docs concerning the types that support membership +tests. From webhook-mailer at python.org Tue Sep 11 13:47:49 2018 From: webhook-mailer at python.org (Steve Dower) Date: Tue, 11 Sep 2018 17:47:49 -0000 Subject: [Python-checkins] Update VSTS to Azure DevOps and simplify dependencies (GH-9168) Message-ID: https://github.com/python/cpython/commit/b7d613fc5636d5d8c5ec32d8004c095e20cc4d7d commit: b7d613fc5636d5d8c5ec32d8004c095e20cc4d7d branch: master author: Steve Dower committer: GitHub date: 2018-09-11T10:47:46-07:00 summary: Update VSTS to Azure DevOps and simplify dependencies (GH-9168) files: A .vsts/install_deps.sh D .vsts/linux-deps.yml M .vsts/docs.yml M .vsts/linux-buildbot.yml M .vsts/linux-coverage.yml M .vsts/linux-pr.yml M .vsts/macos-buildbot.yml M .vsts/macos-pr.yml M README.rst diff --git a/.vsts/docs.yml b/.vsts/docs.yml index 93a7282f770a..0be07b31dfcc 100644 --- a/.vsts/docs.yml +++ b/.vsts/docs.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: diff --git a/.vsts/install_deps.sh b/.vsts/install_deps.sh new file mode 100755 index 000000000000..7b98cfddb6c4 --- /dev/null +++ b/.vsts/install_deps.sh @@ -0,0 +1,19 @@ +sudo apt-get update + +sudo apt-get -yq install \ + build-essential \ + zlib1g-dev \ + libbz2-dev \ + liblzma-dev \ + libncurses5-dev \ + libreadline6-dev \ + libsqlite3-dev \ + libssl-dev \ + libgdbm-dev \ + tk-dev \ + lzma \ + lzma-dev \ + liblzma-dev \ + libffi-dev \ + uuid-dev \ + xvfb diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml index fc2c8ca2486e..517040048937 100644 --- a/.vsts/linux-buildbot.yml +++ b/.vsts/linux-buildbot.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: @@ -30,31 +30,9 @@ steps: #- template: linux-deps.yml -# See https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted-templates.md -# For now, we copy/paste the steps -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - - script: echo ##vso[task.prependpath]$(OPENSSL_DIR) displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb +- script: ./.vsts/install_deps.sh displayName: 'Install dependencies' - script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux displayName: 'python multissltests.py' diff --git a/.vsts/linux-coverage.yml b/.vsts/linux-coverage.yml index 1112555ab93f..cc03e4258ab4 100644 --- a/.vsts/linux-coverage.yml +++ b/.vsts/linux-coverage.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: @@ -40,32 +40,9 @@ steps: #- template: linux-deps.yml -# See https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted-templates.md -# For now, we copy/paste the steps -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - - script: echo ##vso[task.prependpath]$(OPENSSL_DIR) displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb +- script: ./.vsts/install_deps.sh displayName: 'Install dependencies' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux diff --git a/.vsts/linux-deps.yml b/.vsts/linux-deps.yml deleted file mode 100644 index 83b0b5961721..000000000000 --- a/.vsts/linux-deps.yml +++ /dev/null @@ -1,37 +0,0 @@ -# Note: this file is not currently used, but when template support comes to VSTS it -# will be referenced from the other scripts.. - -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -parameters: - OPENSSL: 1.1.0g - OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" - -steps: -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - -- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) - displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb - displayName: 'Install dependencies' -- script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux - displayName: 'python multissltests.py' diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml index 145ebb38016a..6e4ac7c65c4d 100644 --- a/.vsts/linux-pr.yml +++ b/.vsts/linux-pr.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: @@ -40,34 +40,11 @@ steps: #- template: linux-deps.yml -# See https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted-templates.md -# For now, we copy/paste the steps -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - - script: echo ##vso[task.prependpath]$(OPENSSL_DIR) displayName: 'Add $(OPENSSL_DIR) to PATH' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb +- script: ./.vsts/install_deps.sh displayName: 'Install dependencies' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/macos-buildbot.yml b/.vsts/macos-buildbot.yml index d9b2297283b0..f58ea1626144 100644 --- a/.vsts/macos-buildbot.yml +++ b/.vsts/macos-buildbot.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted macOS Preview + name: Hosted macOS trigger: branches: diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml index a3fd4487ed3a..c56e66b5090b 100644 --- a/.vsts/macos-pr.yml +++ b/.vsts/macos-pr.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted macOS Preview + name: Hosted macOS trigger: branches: diff --git a/README.rst b/README.rst index 6047e7aef2cb..c65f6e29d413 100644 --- a/README.rst +++ b/README.rst @@ -9,17 +9,17 @@ This is Python version 3.8.0 alpha 0 :alt: CPython build status on Appveyor :target: https://ci.appveyor.com/project/python/cpython/branch/master -.. image:: https://python.visualstudio.com/cpython/_apis/build/status/Linux-Buildbot?branchName=master&label=Linux - :alt: CPython build status on VSTS (Linux) - :target: https://python.visualstudio.com/cpython/_build/latest?definitionId=6&branchName=master +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Linux-Buildbot?branchName=master&label=Linux + :alt: CPython build status on Azure DevOps (Linux) + :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=6&branchName=master -.. image:: https://python.visualstudio.com/cpython/_apis/build/status/macOS-Buildbot?branchName=master&label=macOS - :alt: CPython build status on VSTS (macOS) - :target: https://python.visualstudio.com/cpython/_build/latest?definitionId=5&branchName=master +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/macOS-Buildbot?branchName=master&label=macOS + :alt: CPython build status on Azure DevOps (macOS) + :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=5&branchName=master -.. image:: https://python.visualstudio.com/cpython/_apis/build/status/Windows-Buildbot?branchName=master&label=Windows - :alt: CPython build status on VSTS (Windows) - :target: https://python.visualstudio.com/cpython/_build/latest?definitionId=4&branchName=master +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Windows-Buildbot?branchName=master&label=Windows + :alt: CPython build status on Azure DevOps (Windows) + :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=4&branchName=master .. image:: https://codecov.io/gh/python/cpython/branch/master/graph/badge.svg :alt: CPython code coverage on Codecov From webhook-mailer at python.org Tue Sep 11 14:13:38 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 18:13:38 -0000 Subject: [Python-checkins] bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086) Message-ID: https://github.com/python/cpython/commit/3e648f8371e342ccfa663ad77e82a538fcc8c9fb commit: 3e648f8371e342ccfa663ad77e82a538fcc8c9fb branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T11:13:33-07:00 summary: bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086) https://bugs.python.org/issue28617 (cherry picked from commit 08bcf647d8a92e4bd47531588b284c6820b7a7ef) Co-authored-by: wim glenn files: A Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index e71ce75b8329..efa0a89490e1 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -197,8 +197,8 @@ exception. operator: not in Two more operations with the same syntactic priority, :keyword:`in` and -:keyword:`not in`, are supported only by sequence types (below). - +:keyword:`not in`, are supported by types that are :term:`iterable` or +implement the :meth:`__contains__` method. .. _typesnumeric: diff --git a/Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst b/Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst new file mode 100644 index 000000000000..281afad71d27 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst @@ -0,0 +1,2 @@ +Fixed info in the stdtypes docs concerning the types that support membership +tests. From webhook-mailer at python.org Tue Sep 11 14:18:19 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 18:18:19 -0000 Subject: [Python-checkins] bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086) Message-ID: https://github.com/python/cpython/commit/889f080a4d5cdb1cfb901b953f4b89f3ea806bbe commit: 889f080a4d5cdb1cfb901b953f4b89f3ea806bbe branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T11:18:15-07:00 summary: bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086) https://bugs.python.org/issue28617 (cherry picked from commit 08bcf647d8a92e4bd47531588b284c6820b7a7ef) Co-authored-by: wim glenn files: A Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 0172d34ade6c..6de7e1a2df5a 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -197,8 +197,8 @@ exception. operator: not in Two more operations with the same syntactic priority, :keyword:`in` and -:keyword:`not in`, are supported only by sequence types (below). - +:keyword:`not in`, are supported by types that are :term:`iterable` or +implement the :meth:`__contains__` method. .. _typesnumeric: diff --git a/Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst b/Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst new file mode 100644 index 000000000000..281afad71d27 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-09-06-22-39-47.bpo-28617.MjnJLz.rst @@ -0,0 +1,2 @@ +Fixed info in the stdtypes docs concerning the types that support membership +tests. From webhook-mailer at python.org Tue Sep 11 14:45:31 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 18:45:31 -0000 Subject: [Python-checkins] bpo-34613: document the correct value of limit argument of asyncio.StreamReader (GH-9121) Message-ID: https://github.com/python/cpython/commit/b4ec36200a959da70eba94c19826446a8efdffdd commit: b4ec36200a959da70eba94c19826446a8efdffdd branch: master author: Bram committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T11:45:26-07:00 summary: bpo-34613: document the correct value of limit argument of asyncio.StreamReader (GH-9121) The default value of asyncio.StreamReader *limit* is `_DEFAULT_LIMIT` instead of `None`. https://bugs.python.org/issue34613 files: M Doc/library/asyncio-stream.rst diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 3fe7ac7b4abc..27b5205f1c5b 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -146,10 +146,12 @@ and work with streams: StreamReader ============ -.. class:: StreamReader(limit=None, loop=None) +.. class:: StreamReader(limit=_DEFAULT_LIMIT, loop=None) This class is :ref:`not thread safe `. + The *limit* argument's default value is set to _DEFAULT_LIMIT which is 2**16 (64 KiB) + .. method:: exception() Get the exception. From webhook-mailer at python.org Tue Sep 11 14:46:58 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 18:46:58 -0000 Subject: [Python-checkins] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788) Message-ID: https://github.com/python/cpython/commit/24bd50bdcc97d65130c07d6cd26085fd06c3e972 commit: 24bd50bdcc97d65130c07d6cd26085fd06c3e972 branch: master author: Oren Milman committer: Benjamin Peterson date: 2018-09-11T11:46:55-07:00 summary: closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788) files: A Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst M Lib/test/test_deque.py M Modules/_collectionsmodule.c diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py index e895c3c9f0df..921136069d77 100644 --- a/Lib/test/test_deque.py +++ b/Lib/test/test_deque.py @@ -892,6 +892,21 @@ def __iter__(self): d1 == d2 # not clear if this is supposed to be True or False, # but it used to give a SystemError + @support.cpython_only + def test_bug_31608(self): + # The interpreter used to crash in specific cases where a deque + # subclass returned a non-deque. + class X(deque): + pass + d = X() + def bad___new__(cls, *args, **kwargs): + return [42] + X.__new__ = bad___new__ + with self.assertRaises(TypeError): + d * 42 # shouldn't crash + with self.assertRaises(TypeError): + d + deque([1, 2, 3]) # shouldn't crash + class SubclassWithKwargs(deque): def __init__(self, newarg=1): diff --git a/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst b/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst new file mode 100644 index 000000000000..d657a8697361 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst @@ -0,0 +1,2 @@ +Raise a ``TypeError`` instead of crashing if a ``collections.deque`` subclass +returns a non-deque from ``__new__``. Patch by Oren Milman. diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c index c04485771402..935b4348a8ff 100644 --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -478,6 +478,7 @@ deque_inplace_concat(dequeobject *deque, PyObject *other) static PyObject * deque_copy(PyObject *deque, PyObject *Py_UNUSED(ignored)) { + PyObject *result; dequeobject *old_deque = (dequeobject *)deque; if (Py_TYPE(deque) == &deque_type) { dequeobject *new_deque; @@ -502,11 +503,19 @@ deque_copy(PyObject *deque, PyObject *Py_UNUSED(ignored)) return NULL; } if (old_deque->maxlen < 0) - return PyObject_CallFunctionObjArgs((PyObject *)(Py_TYPE(deque)), - deque, NULL); + result = PyObject_CallFunctionObjArgs((PyObject *)(Py_TYPE(deque)), + deque, NULL); else - return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", - deque, old_deque->maxlen, NULL); + result = PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", + deque, old_deque->maxlen, NULL); + if (result != NULL && !PyObject_TypeCheck(result, &deque_type)) { + PyErr_Format(PyExc_TypeError, + "%.200s() must return a deque, not %.200s", + Py_TYPE(deque)->tp_name, Py_TYPE(result)->tp_name); + Py_DECREF(result); + return NULL; + } + return result; } PyDoc_STRVAR(copy_doc, "Return a shallow copy of a deque."); From webhook-mailer at python.org Tue Sep 11 14:57:51 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 18:57:51 -0000 Subject: [Python-checkins] bpo-34613: document the correct value of limit argument of asyncio.StreamReader (GH-9121) Message-ID: https://github.com/python/cpython/commit/cb51dd7cac6a6e2a7ba67fa4cd328a68f630095b commit: cb51dd7cac6a6e2a7ba67fa4cd328a68f630095b branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T11:57:45-07:00 summary: bpo-34613: document the correct value of limit argument of asyncio.StreamReader (GH-9121) The default value of asyncio.StreamReader *limit* is `_DEFAULT_LIMIT` instead of `None`. https://bugs.python.org/issue34613 (cherry picked from commit b4ec36200a959da70eba94c19826446a8efdffdd) Co-authored-by: Bram files: M Doc/library/asyncio-stream.rst diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 491afdd610ca..a510e1e638a7 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -88,10 +88,12 @@ Stream functions StreamReader ============ -.. class:: StreamReader(limit=None, loop=None) +.. class:: StreamReader(limit=_DEFAULT_LIMIT, loop=None) This class is :ref:`not thread safe `. + The *limit* argument's default value is set to _DEFAULT_LIMIT which is 2**16 (64 KiB) + .. method:: exception() Get the exception. From webhook-mailer at python.org Tue Sep 11 14:59:33 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 18:59:33 -0000 Subject: [Python-checkins] bpo-34613: document the correct value of limit argument of asyncio.StreamReader (GH-9121) Message-ID: https://github.com/python/cpython/commit/e02ca4270ef258162215e345c23025bec27f9eb0 commit: e02ca4270ef258162215e345c23025bec27f9eb0 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T11:59:29-07:00 summary: bpo-34613: document the correct value of limit argument of asyncio.StreamReader (GH-9121) The default value of asyncio.StreamReader *limit* is `_DEFAULT_LIMIT` instead of `None`. https://bugs.python.org/issue34613 (cherry picked from commit b4ec36200a959da70eba94c19826446a8efdffdd) Co-authored-by: Bram files: M Doc/library/asyncio-stream.rst diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index f662e7223337..ca7daabdadd5 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -126,10 +126,12 @@ Stream functions StreamReader ============ -.. class:: StreamReader(limit=None, loop=None) +.. class:: StreamReader(limit=_DEFAULT_LIMIT, loop=None) This class is :ref:`not thread safe `. + The *limit* argument's default value is set to _DEFAULT_LIMIT which is 2**16 (64 KiB) + .. method:: exception() Get the exception. From webhook-mailer at python.org Tue Sep 11 15:08:15 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 19:08:15 -0000 Subject: [Python-checkins] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788) Message-ID: https://github.com/python/cpython/commit/536e45accf8f05355dd943a6966b9968cdb15f5a commit: 536e45accf8f05355dd943a6966b9968cdb15f5a branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T12:08:10-07:00 summary: closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788) (cherry picked from commit 24bd50bdcc97d65130c07d6cd26085fd06c3e972) Co-authored-by: Oren Milman files: A Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst M Lib/test/test_deque.py M Modules/_collectionsmodule.c diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py index e895c3c9f0df..921136069d77 100644 --- a/Lib/test/test_deque.py +++ b/Lib/test/test_deque.py @@ -892,6 +892,21 @@ def __iter__(self): d1 == d2 # not clear if this is supposed to be True or False, # but it used to give a SystemError + @support.cpython_only + def test_bug_31608(self): + # The interpreter used to crash in specific cases where a deque + # subclass returned a non-deque. + class X(deque): + pass + d = X() + def bad___new__(cls, *args, **kwargs): + return [42] + X.__new__ = bad___new__ + with self.assertRaises(TypeError): + d * 42 # shouldn't crash + with self.assertRaises(TypeError): + d + deque([1, 2, 3]) # shouldn't crash + class SubclassWithKwargs(deque): def __init__(self, newarg=1): diff --git a/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst b/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst new file mode 100644 index 000000000000..d657a8697361 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst @@ -0,0 +1,2 @@ +Raise a ``TypeError`` instead of crashing if a ``collections.deque`` subclass +returns a non-deque from ``__new__``. Patch by Oren Milman. diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c index 4116d4053547..cfd8905edeb3 100644 --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -514,6 +514,7 @@ deque_inplace_concat(dequeobject *deque, PyObject *other) static PyObject * deque_copy(PyObject *deque) { + PyObject *result; dequeobject *old_deque = (dequeobject *)deque; if (Py_TYPE(deque) == &deque_type) { dequeobject *new_deque; @@ -538,11 +539,19 @@ deque_copy(PyObject *deque) return NULL; } if (old_deque->maxlen < 0) - return PyObject_CallFunctionObjArgs((PyObject *)(Py_TYPE(deque)), - deque, NULL); + result = PyObject_CallFunctionObjArgs((PyObject *)(Py_TYPE(deque)), + deque, NULL); else - return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", - deque, old_deque->maxlen, NULL); + result = PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", + deque, old_deque->maxlen, NULL); + if (result != NULL && !PyObject_TypeCheck(result, &deque_type)) { + PyErr_Format(PyExc_TypeError, + "%.200s() must return a deque, not %.200s", + Py_TYPE(deque)->tp_name, Py_TYPE(result)->tp_name); + Py_DECREF(result); + return NULL; + } + return result; } PyDoc_STRVAR(copy_doc, "Return a shallow copy of a deque."); From webhook-mailer at python.org Tue Sep 11 15:12:46 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 19:12:46 -0000 Subject: [Python-checkins] [3.6] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-9178) Message-ID: https://github.com/python/cpython/commit/ccbbdd0a1e00ecad6f0005438dd6ff6d84fd9ceb commit: ccbbdd0a1e00ecad6f0005438dd6ff6d84fd9ceb branch: 3.6 author: Benjamin Peterson committer: GitHub date: 2018-09-11T12:12:42-07:00 summary: [3.6] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-9178) (cherry picked from commit 24bd50bdcc97d65130c07d6cd26085fd06c3e972) Co-authored-by: Oren Milman files: A Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst M Lib/test/test_deque.py M Modules/_collectionsmodule.c diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py index ce517b51d58e..ed9ea0251a74 100644 --- a/Lib/test/test_deque.py +++ b/Lib/test/test_deque.py @@ -893,6 +893,21 @@ def __iter__(self): d1 == d2 # not clear if this is supposed to be True or False, # but it used to give a SystemError + @support.cpython_only + def test_bug_31608(self): + # The interpreter used to crash in specific cases where a deque + # subclass returned a non-deque. + class X(deque): + pass + d = X() + def bad___new__(cls, *args, **kwargs): + return [42] + X.__new__ = bad___new__ + with self.assertRaises(TypeError): + d * 42 # shouldn't crash + with self.assertRaises(TypeError): + d + deque([1, 2, 3]) # shouldn't crash + class SubclassWithKwargs(deque): def __init__(self, newarg=1): diff --git a/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst b/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst new file mode 100644 index 000000000000..d657a8697361 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst @@ -0,0 +1,2 @@ +Raise a ``TypeError`` instead of crashing if a ``collections.deque`` subclass +returns a non-deque from ``__new__``. Patch by Oren Milman. diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c index d7b344be692c..85037d002703 100644 --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -514,6 +514,7 @@ deque_inplace_concat(dequeobject *deque, PyObject *other) static PyObject * deque_copy(PyObject *deque) { + PyObject *result; dequeobject *old_deque = (dequeobject *)deque; if (Py_TYPE(deque) == &deque_type) { dequeobject *new_deque; @@ -538,10 +539,19 @@ deque_copy(PyObject *deque) return NULL; } if (old_deque->maxlen < 0) - return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "O", deque, NULL); + result = PyObject_CallFunctionObjArgs((PyObject *)(Py_TYPE(deque)), + deque, NULL); else - return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", - deque, old_deque->maxlen, NULL); + result = PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", + deque, old_deque->maxlen, NULL); + if (result != NULL && !PyObject_TypeCheck(result, &deque_type)) { + PyErr_Format(PyExc_TypeError, + "%.200s() must return a deque, not %.200s", + Py_TYPE(deque)->tp_name, Py_TYPE(result)->tp_name); + Py_DECREF(result); + return NULL; + } + return result; } PyDoc_STRVAR(copy_doc, "Return a shallow copy of a deque."); From webhook-mailer at python.org Tue Sep 11 16:42:01 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 20:42:01 -0000 Subject: [Python-checkins] [2.7] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-9179) Message-ID: https://github.com/python/cpython/commit/253279c616d4f34287c5749df15e20eb2eb988d6 commit: 253279c616d4f34287c5749df15e20eb2eb988d6 branch: 2.7 author: Benjamin Peterson committer: GitHub date: 2018-09-11T13:41:57-07:00 summary: [2.7] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-9179) files: A Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst M Lib/test/test_deque.py M Modules/_collectionsmodule.c diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py index c81064d9f2a6..e6307ab5c32e 100644 --- a/Lib/test/test_deque.py +++ b/Lib/test/test_deque.py @@ -659,6 +659,21 @@ def __iter__(self): d1 == d2 # not clear if this is supposed to be True or False, # but it used to give a SystemError + @test_support.cpython_only + def test_bug_31608(self): + # The interpreter used to crash in specific cases where a deque + # subclass returned a non-deque. + class X(deque): + pass + d = X() + def bad___new__(cls, *args, **kwargs): + return [42] + X.__new__ = bad___new__ + with self.assertRaises(TypeError): + d * 42 # shouldn't crash + with self.assertRaises(TypeError): + d + deque([1, 2, 3]) # shouldn't crash + class SubclassWithKwargs(deque): def __init__(self, newarg=1): diff --git a/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst b/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst new file mode 100644 index 000000000000..d657a8697361 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2017-10-29-10-37-55.bpo-31608.wkp8Nw.rst @@ -0,0 +1,2 @@ +Raise a ``TypeError`` instead of crashing if a ``collections.deque`` subclass +returns a non-deque from ``__new__``. Patch by Oren Milman. diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c index 9364aba549b1..3ca393082f1d 100644 --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -859,11 +859,20 @@ deque_traverse(dequeobject *deque, visitproc visit, void *arg) static PyObject * deque_copy(PyObject *deque) { + PyObject *result; if (((dequeobject *)deque)->maxlen == -1) - return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "O", deque, NULL); + result = PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "O", deque, NULL); else - return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", + result = PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", deque, ((dequeobject *)deque)->maxlen, NULL); + if (result != NULL && !PyObject_TypeCheck(result, &deque_type)) { + PyErr_Format(PyExc_TypeError, + "%.200s() must return a deque, not %.200s", + Py_TYPE(deque)->tp_name, Py_TYPE(result)->tp_name); + Py_DECREF(result); + return NULL; + } + return result; } PyDoc_STRVAR(copy_doc, "Return a shallow copy of a deque."); From webhook-mailer at python.org Tue Sep 11 16:55:01 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 20:55:01 -0000 Subject: [Python-checkins] [2.7] bpo-32502: Discard 64-bit (and other invalid) hardware addresses (GH-9125) Message-ID: https://github.com/python/cpython/commit/d919c60e6936f853ad15040017f2c0bce0f027f8 commit: d919c60e6936f853ad15040017f2c0bce0f027f8 branch: 2.7 author: Chih-Hsuan Yen committer: Benjamin Peterson date: 2018-09-11T13:54:57-07:00 summary: [2.7] bpo-32502: Discard 64-bit (and other invalid) hardware addresses (GH-9125) (cherry picked from commit 6b273f7f4056f8276f61a97c789d6bb4425e653c) Co-authored-by: Bo Bayles files: A Misc/NEWS.d/next/Library/2018-01-20-17-15-34.bpo-32502.OXJfn7.rst M Lib/test/test_uuid.py M Lib/uuid.py diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py index 3749564c930e..6cd2c392932e 100644 --- a/Lib/test/test_uuid.py +++ b/Lib/test/test_uuid.py @@ -287,6 +287,39 @@ def test_getnode(self): node2 = uuid.getnode() self.assertEqual(node1, node2, '%012x != %012x' % (node1, node2)) + # bpo-32502: UUID1 requires a 48-bit identifier, but hardware identifiers + # need not necessarily be 48 bits (e.g., EUI-64). + def test_uuid1_eui64(self): + # Confirm that uuid.getnode ignores hardware addresses larger than 48 + # bits. Mock out each platform's *_getnode helper functions to return + # something just larger than 48 bits to test. This will cause + # uuid.getnode to fall back on uuid._random_getnode, which will + # generate a valid value. + too_large_getter = lambda: 1 << 48 + + uuid_real__node = uuid._node + uuid_real__NODE_GETTERS_WIN32 = uuid._NODE_GETTERS_WIN32 + uuid_real__NODE_GETTERS_UNIX = uuid._NODE_GETTERS_UNIX + uuid._node = None + uuid._NODE_GETTERS_WIN32 = [too_large_getter] + uuid._NODE_GETTERS_UNIX = [too_large_getter] + try: + node = uuid.getnode() + finally: + uuid._node = uuid_real__node + uuid._NODE_GETTERS_WIN32 = uuid_real__NODE_GETTERS_WIN32 + uuid._NODE_GETTERS_UNIX = uuid_real__NODE_GETTERS_UNIX + + self.assertTrue(0 < node < (1 << 48), '%012x' % node) + + # Confirm that uuid1 can use the generated node, i.e., the that + # uuid.getnode fell back on uuid._random_getnode() rather than using + # the value from too_large_getter above. + try: + uuid.uuid1(node=node) + except ValueError as e: + self.fail('uuid1 was given an invalid node ID') + @unittest.skipUnless(importable('ctypes'), 'requires ctypes') def test_uuid1(self): equal = self.assertEqual diff --git a/Lib/uuid.py b/Lib/uuid.py index 973013c00636..80d33c0bd83f 100644 --- a/Lib/uuid.py +++ b/Lib/uuid.py @@ -522,6 +522,11 @@ def _random_getnode(): _node = None +_NODE_GETTERS_WIN32 = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] + +_NODE_GETTERS_UNIX = [_unixdll_getnode, _ifconfig_getnode, _arp_getnode, + _lanscan_getnode, _netstat_getnode] + def getnode(): """Get the hardware address as a 48-bit positive integer. @@ -537,18 +542,19 @@ def getnode(): import sys if sys.platform == 'win32': - getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] + getters = _NODE_GETTERS_WIN32 else: - getters = [_unixdll_getnode, _ifconfig_getnode, _arp_getnode, - _lanscan_getnode, _netstat_getnode] + getters = _NODE_GETTERS_UNIX for getter in getters + [_random_getnode]: try: _node = getter() except: continue - if _node is not None: + if (_node is not None) and (0 <= _node < (1 << 48)): return _node + assert False, '_random_getnode() returned invalid value: {}'.format(_node) + _last_timestamp = None diff --git a/Misc/NEWS.d/next/Library/2018-01-20-17-15-34.bpo-32502.OXJfn7.rst b/Misc/NEWS.d/next/Library/2018-01-20-17-15-34.bpo-32502.OXJfn7.rst new file mode 100644 index 000000000000..8338632aa2c5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-01-20-17-15-34.bpo-32502.OXJfn7.rst @@ -0,0 +1,2 @@ +uuid.uuid1 no longer raises an exception if a 64-bit hardware address is +encountered. From webhook-mailer at python.org Tue Sep 11 16:59:28 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 20:59:28 -0000 Subject: [Python-checkins] bpo-34365: Update date object documentation (GH-8814) Message-ID: https://github.com/python/cpython/commit/9c223794c754408644c16349b85dd27fdba8a926 commit: 9c223794c754408644c16349b85dd27fdba8a926 branch: master author: Danish Prakash committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T13:59:23-07:00 summary: bpo-34365: Update date object documentation (GH-8814) Python 3.x does not fall back to comparing object addresses when comparing two `dt` objects. https://bugs.python.org/issue34365 files: M Doc/library/datetime.rst diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst index 53c181c4014a..22582476cf95 100644 --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -526,10 +526,9 @@ Notes: (4) In other words, ``date1 < date2`` if and only if ``date1.toordinal() < - date2.toordinal()``. In order to stop comparison from falling back to the - default scheme of comparing object addresses, date comparison normally raises - :exc:`TypeError` if the other comparand isn't also a :class:`date` object. - However, ``NotImplemented`` is returned instead if the other comparand has a + date2.toordinal()``. Date comparison raises :exc:`TypeError` if + the other comparand isn't also a :class:`date` object. However, + ``NotImplemented`` is returned instead if the other comparand has a :meth:`timetuple` attribute. This hook gives other kinds of date objects a chance at implementing mixed-type comparison. If not, when a :class:`date` object is compared to an object of a different type, :exc:`TypeError` is raised From webhook-mailer at python.org Tue Sep 11 17:01:17 2018 From: webhook-mailer at python.org (Steve Dower) Date: Tue, 11 Sep 2018 21:01:17 -0000 Subject: [Python-checkins] Update VSTS to Azure DevOps and simplify dependencies (GH-9168) Message-ID: https://github.com/python/cpython/commit/d289df13d376d69764884982c236dbc3879ad43c commit: d289df13d376d69764884982c236dbc3879ad43c branch: 3.7 author: Steve Dower committer: GitHub date: 2018-09-11T14:01:13-07:00 summary: Update VSTS to Azure DevOps and simplify dependencies (GH-9168) files: A .vsts/install_deps.sh D .vsts/linux-deps.yml M .vsts/docs.yml M .vsts/linux-buildbot.yml M .vsts/linux-coverage.yml M .vsts/linux-pr.yml M .vsts/macos-buildbot.yml M .vsts/macos-pr.yml M README.rst diff --git a/.vsts/docs.yml b/.vsts/docs.yml index 93a7282f770a..0be07b31dfcc 100644 --- a/.vsts/docs.yml +++ b/.vsts/docs.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: diff --git a/.vsts/install_deps.sh b/.vsts/install_deps.sh new file mode 100755 index 000000000000..7b98cfddb6c4 --- /dev/null +++ b/.vsts/install_deps.sh @@ -0,0 +1,19 @@ +sudo apt-get update + +sudo apt-get -yq install \ + build-essential \ + zlib1g-dev \ + libbz2-dev \ + liblzma-dev \ + libncurses5-dev \ + libreadline6-dev \ + libsqlite3-dev \ + libssl-dev \ + libgdbm-dev \ + tk-dev \ + lzma \ + lzma-dev \ + liblzma-dev \ + libffi-dev \ + uuid-dev \ + xvfb diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml index fc2c8ca2486e..517040048937 100644 --- a/.vsts/linux-buildbot.yml +++ b/.vsts/linux-buildbot.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: @@ -30,31 +30,9 @@ steps: #- template: linux-deps.yml -# See https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted-templates.md -# For now, we copy/paste the steps -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - - script: echo ##vso[task.prependpath]$(OPENSSL_DIR) displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb +- script: ./.vsts/install_deps.sh displayName: 'Install dependencies' - script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux displayName: 'python multissltests.py' diff --git a/.vsts/linux-coverage.yml b/.vsts/linux-coverage.yml index 1112555ab93f..cc03e4258ab4 100644 --- a/.vsts/linux-coverage.yml +++ b/.vsts/linux-coverage.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: @@ -40,32 +40,9 @@ steps: #- template: linux-deps.yml -# See https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted-templates.md -# For now, we copy/paste the steps -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - - script: echo ##vso[task.prependpath]$(OPENSSL_DIR) displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb +- script: ./.vsts/install_deps.sh displayName: 'Install dependencies' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux diff --git a/.vsts/linux-deps.yml b/.vsts/linux-deps.yml deleted file mode 100644 index 83b0b5961721..000000000000 --- a/.vsts/linux-deps.yml +++ /dev/null @@ -1,37 +0,0 @@ -# Note: this file is not currently used, but when template support comes to VSTS it -# will be referenced from the other scripts.. - -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -parameters: - OPENSSL: 1.1.0g - OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" - -steps: -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - -- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) - displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb - displayName: 'Install dependencies' -- script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux - displayName: 'python multissltests.py' diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml index 145ebb38016a..6e4ac7c65c4d 100644 --- a/.vsts/linux-pr.yml +++ b/.vsts/linux-pr.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: @@ -40,34 +40,11 @@ steps: #- template: linux-deps.yml -# See https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted-templates.md -# For now, we copy/paste the steps -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - - script: echo ##vso[task.prependpath]$(OPENSSL_DIR) displayName: 'Add $(OPENSSL_DIR) to PATH' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb +- script: ./.vsts/install_deps.sh displayName: 'Install dependencies' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/macos-buildbot.yml b/.vsts/macos-buildbot.yml index d9b2297283b0..f58ea1626144 100644 --- a/.vsts/macos-buildbot.yml +++ b/.vsts/macos-buildbot.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted macOS Preview + name: Hosted macOS trigger: branches: diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml index a3fd4487ed3a..c56e66b5090b 100644 --- a/.vsts/macos-pr.yml +++ b/.vsts/macos-pr.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted macOS Preview + name: Hosted macOS trigger: branches: diff --git a/README.rst b/README.rst index b330b41abdb3..377aa3dfce6f 100644 --- a/README.rst +++ b/README.rst @@ -9,17 +9,17 @@ This is Python version 3.7.0+ :alt: CPython build status on Appveyor :target: https://ci.appveyor.com/project/python/cpython/branch/master -.. image:: https://python.visualstudio.com/cpython/_apis/build/status/Linux-Buildbot?branchName=3.7&label=Linux +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Linux-Buildbot?branchName=3.7&label=Linux :alt: CPython build status on VSTS (Linux) - :target: https://python.visualstudio.com/cpython/_build/latest?definitionId=6&branchName=3.7 + :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=6&branchName=3.7 -.. image:: https://python.visualstudio.com/cpython/_apis/build/status/macOS-Buildbot?branchName=3.7&label=macOS +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/macOS-Buildbot?branchName=3.7&label=macOS :alt: CPython build status on VSTS (macOS) - :target: https://python.visualstudio.com/cpython/_build/latest?definitionId=5&branchName=3.7 + :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=5&branchName=3.7 -.. image:: https://python.visualstudio.com/cpython/_apis/build/status/Windows-Buildbot?branchName=3.7&label=Windows +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Windows-Buildbot?branchName=3.7&label=Windows :alt: CPython build status on VSTS (Windows) - :target: https://python.visualstudio.com/cpython/_build/latest?definitionId=4&branchName=3.7 + :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=4&branchName=3.7 .. image:: https://codecov.io/gh/python/cpython/branch/master/graph/badge.svg :alt: CPython code coverage on Codecov From webhook-mailer at python.org Tue Sep 11 17:01:22 2018 From: webhook-mailer at python.org (Steve Dower) Date: Tue, 11 Sep 2018 21:01:22 -0000 Subject: [Python-checkins] Update VSTS to Azure DevOps and simplify dependencies (GH-9168) Message-ID: https://github.com/python/cpython/commit/7f76eebf407c5caf8564ec3244ed7409b828974d commit: 7f76eebf407c5caf8564ec3244ed7409b828974d branch: 3.6 author: Steve Dower committer: GitHub date: 2018-09-11T14:01:19-07:00 summary: Update VSTS to Azure DevOps and simplify dependencies (GH-9168) files: A .vsts/install_deps.sh D .vsts/linux-deps.yml M .vsts/docs.yml M .vsts/linux-buildbot.yml M .vsts/linux-coverage.yml M .vsts/linux-pr.yml M .vsts/macos-buildbot.yml M .vsts/macos-pr.yml M README.rst diff --git a/.vsts/docs.yml b/.vsts/docs.yml index 93a7282f770a..0be07b31dfcc 100644 --- a/.vsts/docs.yml +++ b/.vsts/docs.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: diff --git a/.vsts/install_deps.sh b/.vsts/install_deps.sh new file mode 100755 index 000000000000..7b98cfddb6c4 --- /dev/null +++ b/.vsts/install_deps.sh @@ -0,0 +1,19 @@ +sudo apt-get update + +sudo apt-get -yq install \ + build-essential \ + zlib1g-dev \ + libbz2-dev \ + liblzma-dev \ + libncurses5-dev \ + libreadline6-dev \ + libsqlite3-dev \ + libssl-dev \ + libgdbm-dev \ + tk-dev \ + lzma \ + lzma-dev \ + liblzma-dev \ + libffi-dev \ + uuid-dev \ + xvfb diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml index 4c01bdd165d1..09084e07df77 100644 --- a/.vsts/linux-buildbot.yml +++ b/.vsts/linux-buildbot.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: @@ -25,31 +25,7 @@ steps: clean: true fetchDepth: 5 -#- template: linux-deps.yml - -# See https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted-templates.md -# For now, we copy/paste the steps -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb +- script: ./.vsts/install_deps.sh displayName: 'Install dependencies' - script: ./configure --with-pydebug diff --git a/.vsts/linux-coverage.yml b/.vsts/linux-coverage.yml index 62fafe347688..4eb75efc8040 100644 --- a/.vsts/linux-coverage.yml +++ b/.vsts/linux-coverage.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: @@ -35,34 +35,9 @@ steps: displayName: Detect doc-only changes condition: and(succeeded(), variables['system.pullRequest.targetBranch']) -#- template: linux-deps.yml - -# See https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted-templates.md -# For now, we copy/paste the steps -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb +- script: ./.vsts/install_deps.sh displayName: 'Install dependencies' - + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - script: ./configure --with-pydebug displayName: 'Configure CPython (debug)' diff --git a/.vsts/linux-deps.yml b/.vsts/linux-deps.yml deleted file mode 100644 index d1922a7c3237..000000000000 --- a/.vsts/linux-deps.yml +++ /dev/null @@ -1,31 +0,0 @@ -# Note: this file is not currently used, but when template support comes to VSTS it -# will be referenced from the other scripts.. - -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -#parameters: - -steps: -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb - displayName: 'Install dependencies' diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml index 922fa81bc414..945ac097e5e8 100644 --- a/.vsts/linux-pr.yml +++ b/.vsts/linux-pr.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted Linux Preview + name: 'Hosted Ubuntu 1604' trigger: branches: @@ -35,36 +35,10 @@ steps: displayName: Detect doc-only changes condition: and(succeeded(), variables['system.pullRequest.targetBranch']) -#- template: linux-deps.yml - -# See https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted-templates.md -# For now, we copy/paste the steps -- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > /etc/apt/sources.list.d/python.list && sudo apt-get update - displayName: 'Update apt-get lists' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: > - sudo apt-get -yq install - build-essential - zlib1g-dev - libbz2-dev - liblzma-dev - libncurses5-dev - libreadline6-dev - libsqlite3-dev - libssl-dev - libgdbm-dev - tk-dev - lzma - lzma-dev - liblzma-dev - libffi-dev - uuid-dev - xvfb +- script: ./.vsts/install_deps.sh displayName: 'Install dependencies' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - - script: ./configure --with-pydebug displayName: 'Configure CPython (debug)' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/macos-buildbot.yml b/.vsts/macos-buildbot.yml index d9b2297283b0..f58ea1626144 100644 --- a/.vsts/macos-buildbot.yml +++ b/.vsts/macos-buildbot.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted macOS Preview + name: Hosted macOS trigger: branches: diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml index a3fd4487ed3a..c56e66b5090b 100644 --- a/.vsts/macos-pr.yml +++ b/.vsts/macos-pr.yml @@ -4,7 +4,7 @@ name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) queue: - name: Hosted macOS Preview + name: Hosted macOS trigger: branches: diff --git a/README.rst b/README.rst index c9b518bcf2cb..a2c76ac71dee 100644 --- a/README.rst +++ b/README.rst @@ -9,17 +9,17 @@ This is Python version 3.6.6+ :alt: CPython build status on Appveyor :target: https://ci.appveyor.com/project/python/cpython/branch/3.6 -.. image:: https://python.visualstudio.com/cpython/_apis/build/status/Linux-Buildbot?branchName=3.6&label=Linux +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Linux-Buildbot?branchName=3.6&label=Linux :alt: CPython build status on VSTS (Linux) - :target: https://python.visualstudio.com/cpython/_build/latest?definitionId=6&branchName=3.6 + :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=6&branchName=3.6 -.. image:: https://python.visualstudio.com/cpython/_apis/build/status/macOS-Buildbot?branchName=3.6&label=macOS +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/macOS-Buildbot?branchName=3.6&label=macOS :alt: CPython build status on VSTS (macOS) - :target: https://python.visualstudio.com/cpython/_build/latest?definitionId=5&branchName=3.6 + :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=5&branchName=3.6 -.. image:: https://python.visualstudio.com/cpython/_apis/build/status/Windows-Buildbot?branchName=3.6&label=Windows +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Windows-Buildbot?branchName=3.6&label=Windows :alt: CPython build status on VSTS (Windows) - :target: https://python.visualstudio.com/cpython/_build/latest?definitionId=4&branchName=3.6 + :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=4&branchName=3.6 .. image:: https://codecov.io/gh/python/cpython/branch/3.6/graph/badge.svg :alt: CPython code coverage on Codecov From webhook-mailer at python.org Tue Sep 11 17:24:56 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 21:24:56 -0000 Subject: [Python-checkins] bpo-34365: Update date object documentation (GH-8814) Message-ID: https://github.com/python/cpython/commit/e2b40f4ce954ea3d35a73541029b2253abd9d245 commit: e2b40f4ce954ea3d35a73541029b2253abd9d245 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T14:24:53-07:00 summary: bpo-34365: Update date object documentation (GH-8814) Python 3.x does not fall back to comparing object addresses when comparing two `dt` objects. https://bugs.python.org/issue34365 (cherry picked from commit 9c223794c754408644c16349b85dd27fdba8a926) Co-authored-by: Danish Prakash files: M Doc/library/datetime.rst diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst index 53c181c4014a..22582476cf95 100644 --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -526,10 +526,9 @@ Notes: (4) In other words, ``date1 < date2`` if and only if ``date1.toordinal() < - date2.toordinal()``. In order to stop comparison from falling back to the - default scheme of comparing object addresses, date comparison normally raises - :exc:`TypeError` if the other comparand isn't also a :class:`date` object. - However, ``NotImplemented`` is returned instead if the other comparand has a + date2.toordinal()``. Date comparison raises :exc:`TypeError` if + the other comparand isn't also a :class:`date` object. However, + ``NotImplemented`` is returned instead if the other comparand has a :meth:`timetuple` attribute. This hook gives other kinds of date objects a chance at implementing mixed-type comparison. If not, when a :class:`date` object is compared to an object of a different type, :exc:`TypeError` is raised From webhook-mailer at python.org Tue Sep 11 17:27:09 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 21:27:09 -0000 Subject: [Python-checkins] bpo-34365: Update date object documentation (GH-8814) Message-ID: https://github.com/python/cpython/commit/8a2c2d4e1b6b46a9a982e5c064ca4399ec28d55d commit: 8a2c2d4e1b6b46a9a982e5c064ca4399ec28d55d branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T14:27:06-07:00 summary: bpo-34365: Update date object documentation (GH-8814) Python 3.x does not fall back to comparing object addresses when comparing two `dt` objects. https://bugs.python.org/issue34365 (cherry picked from commit 9c223794c754408644c16349b85dd27fdba8a926) Co-authored-by: Danish Prakash files: M Doc/library/datetime.rst diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst index 89f85fc11ad4..88bc328e3a1d 100644 --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -510,10 +510,9 @@ Notes: (4) In other words, ``date1 < date2`` if and only if ``date1.toordinal() < - date2.toordinal()``. In order to stop comparison from falling back to the - default scheme of comparing object addresses, date comparison normally raises - :exc:`TypeError` if the other comparand isn't also a :class:`date` object. - However, ``NotImplemented`` is returned instead if the other comparand has a + date2.toordinal()``. Date comparison raises :exc:`TypeError` if + the other comparand isn't also a :class:`date` object. However, + ``NotImplemented`` is returned instead if the other comparand has a :meth:`timetuple` attribute. This hook gives other kinds of date objects a chance at implementing mixed-type comparison. If not, when a :class:`date` object is compared to an object of a different type, :exc:`TypeError` is raised From webhook-mailer at python.org Tue Sep 11 17:45:49 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 21:45:49 -0000 Subject: [Python-checkins] closes bpo-31902: Fix the col_offset attribute for ast.Async* nodes to point to the "async" keyword. (GH-4175) Message-ID: https://github.com/python/cpython/commit/90fc8980bbcc5c7dcced3627fe172b0bfd193a3b commit: 90fc8980bbcc5c7dcced3627fe172b0bfd193a3b branch: master author: guoci committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T14:45:45-07:00 summary: closes bpo-31902: Fix the col_offset attribute for ast.Async* nodes to point to the "async" keyword. (GH-4175) Previously, col_offset points to the keyword after "async". files: A Misc/NEWS.d/next/Core and Builtins/2017-10-30-12-44-50.bpo-31902.a07fa57.rst M Lib/test/test_ast.py M Python/ast.c diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py index 7db40e7797f7..72f8467847e8 100644 --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -1181,12 +1181,12 @@ def main(): ('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [], 0)]))]), ('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))], 0)]))]), ('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [], 0)]))]), -('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Str', (2, 1), 'async function')), ('Expr', (3, 1), ('Await', (3, 1), ('Call', (3, 7), ('Name', (3, 7), 'something', ('Load',)), [], [])))], [], None)]), -('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 7), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), -('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 7), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Str', (2, 1), 'async function')), ('Expr', (3, 1), ('Await', (3, 1), ('Call', (3, 7), ('Name', (3, 7), 'something', ('Load',)), [], [])))], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 1), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 1), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), ('Module', [('Expr', (1, 0), ('Dict', (1, 0), [None, ('Num', (1, 10), 2)], [('Dict', (1, 3), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), ('Module', [('Expr', (1, 0), ('Set', (1, 0), [('Starred', (1, 1), ('Set', (1, 2), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), -('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('ListComp', (2, 2), ('Name', (2, 2), 'i', ('Load',)), [('comprehension', ('Name', (2, 14), 'b', ('Store',)), ('Name', (2, 19), 'c', ('Load',)), [], 1)]))], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('ListComp', (2, 2), ('Name', (2, 2), 'i', ('Load',)), [('comprehension', ('Name', (2, 14), 'b', ('Store',)), ('Name', (2, 19), 'c', ('Load',)), [], 1)]))], [], None)]), ] single_results = [ ('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]), diff --git a/Misc/NEWS.d/next/Core and Builtins/2017-10-30-12-44-50.bpo-31902.a07fa57.rst b/Misc/NEWS.d/next/Core and Builtins/2017-10-30-12-44-50.bpo-31902.a07fa57.rst new file mode 100644 index 000000000000..e2b04b3ae0f5 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2017-10-30-12-44-50.bpo-31902.a07fa57.rst @@ -0,0 +1,3 @@ +Fix the ``col_offset`` attribute for ast nodes ``ast.AsyncFor``, +``ast.AsyncFunctionDef``, and ``ast.AsyncWith``. Previously, ``col_offset`` +pointed to the keyword after ``async``. diff --git a/Python/ast.c b/Python/ast.c index a91c075dae66..ef2c858ae4aa 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -600,8 +600,8 @@ static asdl_seq *ast_for_exprlist(struct compiling *, const node *, static expr_ty ast_for_testlist(struct compiling *, const node *); static stmt_ty ast_for_classdef(struct compiling *, const node *, asdl_seq *); -static stmt_ty ast_for_with_stmt(struct compiling *, const node *, int); -static stmt_ty ast_for_for_stmt(struct compiling *, const node *, int); +static stmt_ty ast_for_with_stmt(struct compiling *, const node *, bool); +static stmt_ty ast_for_for_stmt(struct compiling *, const node *, bool); /* Note different signature for ast_for_call */ static expr_ty ast_for_call(struct compiling *, const node *, expr_ty, bool); @@ -1569,10 +1569,11 @@ ast_for_decorators(struct compiling *c, const node *n) } static stmt_ty -ast_for_funcdef_impl(struct compiling *c, const node *n, - asdl_seq *decorator_seq, int is_async) +ast_for_funcdef_impl(struct compiling *c, const node *n0, + asdl_seq *decorator_seq, bool is_async) { /* funcdef: 'def' NAME parameters ['->' test] ':' suite */ + const node * const n = is_async ? CHILD(n0, 1) : n0; identifier name; arguments_ty args; asdl_seq *body; @@ -1601,7 +1602,7 @@ ast_for_funcdef_impl(struct compiling *c, const node *n, if (is_async) return AsyncFunctionDef(name, args, body, decorator_seq, returns, - LINENO(n), n->n_col_offset, c->c_arena); + LINENO(n), n0->n_col_offset, c->c_arena); else return FunctionDef(name, args, body, decorator_seq, returns, LINENO(n), n->n_col_offset, c->c_arena); @@ -1616,8 +1617,8 @@ ast_for_async_funcdef(struct compiling *c, const node *n, asdl_seq *decorator_se assert(strcmp(STR(CHILD(n, 0)), "async") == 0); REQ(CHILD(n, 1), funcdef); - return ast_for_funcdef_impl(c, CHILD(n, 1), decorator_seq, - 1 /* is_async */); + return ast_for_funcdef_impl(c, n, decorator_seq, + true /* is_async */); } static stmt_ty @@ -1625,7 +1626,7 @@ ast_for_funcdef(struct compiling *c, const node *n, asdl_seq *decorator_seq) { /* funcdef: 'def' NAME parameters ['->' test] ':' suite */ return ast_for_funcdef_impl(c, n, decorator_seq, - 0 /* is_async */); + false /* is_async */); } @@ -1639,15 +1640,15 @@ ast_for_async_stmt(struct compiling *c, const node *n) switch (TYPE(CHILD(n, 1))) { case funcdef: - return ast_for_funcdef_impl(c, CHILD(n, 1), NULL, - 1 /* is_async */); + return ast_for_funcdef_impl(c, n, NULL, + true /* is_async */); case with_stmt: - return ast_for_with_stmt(c, CHILD(n, 1), - 1 /* is_async */); + return ast_for_with_stmt(c, n, + true /* is_async */); case for_stmt: - return ast_for_for_stmt(c, CHILD(n, 1), - 1 /* is_async */); + return ast_for_for_stmt(c, n, + true /* is_async */); default: PyErr_Format(PyExc_SystemError, @@ -3681,8 +3682,9 @@ ast_for_while_stmt(struct compiling *c, const node *n) } static stmt_ty -ast_for_for_stmt(struct compiling *c, const node *n, int is_async) +ast_for_for_stmt(struct compiling *c, const node *n0, bool is_async) { + const node * const n = is_async ? CHILD(n0, 1) : n0; asdl_seq *_target, *seq = NULL, *suite_seq; expr_ty expression; expr_ty target, first; @@ -3717,7 +3719,7 @@ ast_for_for_stmt(struct compiling *c, const node *n, int is_async) if (is_async) return AsyncFor(target, expression, suite_seq, seq, - LINENO(n), n->n_col_offset, + LINENO(n), n0->n_col_offset, c->c_arena); else return For(target, expression, suite_seq, seq, @@ -3869,8 +3871,9 @@ ast_for_with_item(struct compiling *c, const node *n) /* with_stmt: 'with' with_item (',' with_item)* ':' suite */ static stmt_ty -ast_for_with_stmt(struct compiling *c, const node *n, int is_async) +ast_for_with_stmt(struct compiling *c, const node *n0, bool is_async) { + const node * const n = is_async ? CHILD(n0, 1) : n0; int i, n_items; asdl_seq *items, *body; @@ -3892,7 +3895,7 @@ ast_for_with_stmt(struct compiling *c, const node *n, int is_async) return NULL; if (is_async) - return AsyncWith(items, body, LINENO(n), n->n_col_offset, c->c_arena); + return AsyncWith(items, body, LINENO(n), n0->n_col_offset, c->c_arena); else return With(items, body, LINENO(n), n->n_col_offset, c->c_arena); } From webhook-mailer at python.org Tue Sep 11 17:49:17 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Tue, 11 Sep 2018 21:49:17 -0000 Subject: [Python-checkins] bpo-20180: convert most of itertoolsmodule.c to use Argument Clinic (GH-9164) Message-ID: https://github.com/python/cpython/commit/c4bccd3c7617018b1ce16f95840ffe1a890d44df commit: c4bccd3c7617018b1ce16f95840ffe1a890d44df branch: master author: Tal Einat committer: Raymond Hettinger date: 2018-09-11T14:49:13-07:00 summary: bpo-20180: convert most of itertoolsmodule.c to use Argument Clinic (GH-9164) files: M Modules/clinic/itertoolsmodule.c.h M Modules/itertoolsmodule.c diff --git a/Modules/clinic/itertoolsmodule.c.h b/Modules/clinic/itertoolsmodule.c.h index 68e67494bc93..94df96c0b7e4 100644 --- a/Modules/clinic/itertoolsmodule.c.h +++ b/Modules/clinic/itertoolsmodule.c.h @@ -61,4 +61,452 @@ itertools__grouper(PyTypeObject *type, PyObject *args, PyObject *kwargs) exit: return return_value; } -/*[clinic end generated code: output=82e10c91569d2b95 input=a9049054013a1b77]*/ + +PyDoc_STRVAR(itertools_teedataobject__doc__, +"teedataobject(iterable, values, next, /)\n" +"--\n" +"\n" +"Data container common to multiple tee objects."); + +static PyObject * +itertools_teedataobject_impl(PyTypeObject *type, PyObject *it, + PyObject *values, PyObject *next); + +static PyObject * +itertools_teedataobject(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyObject *it; + PyObject *values; + PyObject *next; + + if ((type == &teedataobject_type) && + !_PyArg_NoKeywords("teedataobject", kwargs)) { + goto exit; + } + if (!PyArg_ParseTuple(args, "OO!O:teedataobject", + &it, &PyList_Type, &values, &next)) { + goto exit; + } + return_value = itertools_teedataobject_impl(type, it, values, next); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools__tee__doc__, +"_tee(iterable, /)\n" +"--\n" +"\n" +"Iterator wrapped to make it copyable."); + +static PyObject * +itertools__tee_impl(PyTypeObject *type, PyObject *iterable); + +static PyObject * +itertools__tee(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyObject *iterable; + + if ((type == &tee_type) && + !_PyArg_NoKeywords("_tee", kwargs)) { + goto exit; + } + if (!PyArg_UnpackTuple(args, "_tee", + 1, 1, + &iterable)) { + goto exit; + } + return_value = itertools__tee_impl(type, iterable); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_tee__doc__, +"tee($module, iterable, n=2, /)\n" +"--\n" +"\n" +"Returns a tuple of n independent iterators."); + +#define ITERTOOLS_TEE_METHODDEF \ + {"tee", (PyCFunction)itertools_tee, METH_FASTCALL, itertools_tee__doc__}, + +static PyObject * +itertools_tee_impl(PyObject *module, PyObject *iterable, Py_ssize_t n); + +static PyObject * +itertools_tee(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *iterable; + Py_ssize_t n = 2; + + if (!_PyArg_ParseStack(args, nargs, "O|n:tee", + &iterable, &n)) { + goto exit; + } + return_value = itertools_tee_impl(module, iterable, n); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_cycle__doc__, +"cycle(iterable, /)\n" +"--\n" +"\n" +"Return elements from the iterable until it is exhausted. Then repeat the sequence indefinitely."); + +static PyObject * +itertools_cycle_impl(PyTypeObject *type, PyObject *iterable); + +static PyObject * +itertools_cycle(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyObject *iterable; + + if ((type == &cycle_type) && + !_PyArg_NoKeywords("cycle", kwargs)) { + goto exit; + } + if (!PyArg_UnpackTuple(args, "cycle", + 1, 1, + &iterable)) { + goto exit; + } + return_value = itertools_cycle_impl(type, iterable); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_dropwhile__doc__, +"dropwhile(predicate, iterable, /)\n" +"--\n" +"\n" +"Drop items from the iterable while predicate(item) is true.\n" +"\n" +"Afterwards, return every element until the iterable is exhausted."); + +static PyObject * +itertools_dropwhile_impl(PyTypeObject *type, PyObject *func, PyObject *seq); + +static PyObject * +itertools_dropwhile(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyObject *func; + PyObject *seq; + + if ((type == &dropwhile_type) && + !_PyArg_NoKeywords("dropwhile", kwargs)) { + goto exit; + } + if (!PyArg_UnpackTuple(args, "dropwhile", + 2, 2, + &func, &seq)) { + goto exit; + } + return_value = itertools_dropwhile_impl(type, func, seq); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_takewhile__doc__, +"takewhile(predicate, iterable, /)\n" +"--\n" +"\n" +"Return successive entries from an iterable as long as the predicate evaluates to true for each entry."); + +static PyObject * +itertools_takewhile_impl(PyTypeObject *type, PyObject *func, PyObject *seq); + +static PyObject * +itertools_takewhile(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyObject *func; + PyObject *seq; + + if ((type == &takewhile_type) && + !_PyArg_NoKeywords("takewhile", kwargs)) { + goto exit; + } + if (!PyArg_UnpackTuple(args, "takewhile", + 2, 2, + &func, &seq)) { + goto exit; + } + return_value = itertools_takewhile_impl(type, func, seq); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_starmap__doc__, +"starmap(function, iterable, /)\n" +"--\n" +"\n" +"Return an iterator whose values are returned from the function evaluated with an argument tuple taken from the given sequence."); + +static PyObject * +itertools_starmap_impl(PyTypeObject *type, PyObject *func, PyObject *seq); + +static PyObject * +itertools_starmap(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyObject *func; + PyObject *seq; + + if ((type == &starmap_type) && + !_PyArg_NoKeywords("starmap", kwargs)) { + goto exit; + } + if (!PyArg_UnpackTuple(args, "starmap", + 2, 2, + &func, &seq)) { + goto exit; + } + return_value = itertools_starmap_impl(type, func, seq); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_chain_from_iterable__doc__, +"from_iterable($type, iterable, /)\n" +"--\n" +"\n" +"Alternative chain() constructor taking a single iterable argument that evaluates lazily."); + +#define ITERTOOLS_CHAIN_FROM_ITERABLE_METHODDEF \ + {"from_iterable", (PyCFunction)itertools_chain_from_iterable, METH_O|METH_CLASS, itertools_chain_from_iterable__doc__}, + +PyDoc_STRVAR(itertools_combinations__doc__, +"combinations(iterable, r)\n" +"--\n" +"\n" +"Return successive r-length combinations of elements in the iterable.\n" +"\n" +"combinations(range(4), 3) --> (0,1,2), (0,1,3), (0,2,3), (1,2,3)"); + +static PyObject * +itertools_combinations_impl(PyTypeObject *type, PyObject *iterable, + Py_ssize_t r); + +static PyObject * +itertools_combinations(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"iterable", "r", NULL}; + static _PyArg_Parser _parser = {"On:combinations", _keywords, 0}; + PyObject *iterable; + Py_ssize_t r; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &iterable, &r)) { + goto exit; + } + return_value = itertools_combinations_impl(type, iterable, r); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_combinations_with_replacement__doc__, +"combinations_with_replacement(iterable, r)\n" +"--\n" +"\n" +"Return successive r-length combinations of elements in the iterable allowing individual elements to have successive repeats.\n" +"\n" +"combinations_with_replacement(\'ABC\', 2) --> AA AB AC BB BC CC\""); + +static PyObject * +itertools_combinations_with_replacement_impl(PyTypeObject *type, + PyObject *iterable, + Py_ssize_t r); + +static PyObject * +itertools_combinations_with_replacement(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"iterable", "r", NULL}; + static _PyArg_Parser _parser = {"On:combinations_with_replacement", _keywords, 0}; + PyObject *iterable; + Py_ssize_t r; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &iterable, &r)) { + goto exit; + } + return_value = itertools_combinations_with_replacement_impl(type, iterable, r); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_permutations__doc__, +"permutations(iterable, r=None)\n" +"--\n" +"\n" +"Return successive r-length permutations of elements in the iterable.\n" +"\n" +"permutations(range(3), 2) --> (0,1), (0,2), (1,0), (1,2), (2,0), (2,1)"); + +static PyObject * +itertools_permutations_impl(PyTypeObject *type, PyObject *iterable, + PyObject *robj); + +static PyObject * +itertools_permutations(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"iterable", "r", NULL}; + static _PyArg_Parser _parser = {"O|O:permutations", _keywords, 0}; + PyObject *iterable; + PyObject *robj = Py_None; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &iterable, &robj)) { + goto exit; + } + return_value = itertools_permutations_impl(type, iterable, robj); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_accumulate__doc__, +"accumulate(iterable, func=None)\n" +"--\n" +"\n" +"Return series of accumulated sums (or other binary function results)."); + +static PyObject * +itertools_accumulate_impl(PyTypeObject *type, PyObject *iterable, + PyObject *binop); + +static PyObject * +itertools_accumulate(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"iterable", "func", NULL}; + static _PyArg_Parser _parser = {"O|O:accumulate", _keywords, 0}; + PyObject *iterable; + PyObject *binop = Py_None; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &iterable, &binop)) { + goto exit; + } + return_value = itertools_accumulate_impl(type, iterable, binop); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_compress__doc__, +"compress(data, selectors)\n" +"--\n" +"\n" +"Return data elements corresponding to true selector elements.\n" +"\n" +"Forms a shorter iterator from selected data elements using the selectors to\n" +"choose the data elements."); + +static PyObject * +itertools_compress_impl(PyTypeObject *type, PyObject *seq1, PyObject *seq2); + +static PyObject * +itertools_compress(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"data", "selectors", NULL}; + static _PyArg_Parser _parser = {"OO:compress", _keywords, 0}; + PyObject *seq1; + PyObject *seq2; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &seq1, &seq2)) { + goto exit; + } + return_value = itertools_compress_impl(type, seq1, seq2); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_filterfalse__doc__, +"filterfalse(function, iterable, /)\n" +"--\n" +"\n" +"Return those items of iterable for which function(item) is false.\n" +"\n" +"If function is None, return the items that are false."); + +static PyObject * +itertools_filterfalse_impl(PyTypeObject *type, PyObject *func, PyObject *seq); + +static PyObject * +itertools_filterfalse(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyObject *func; + PyObject *seq; + + if ((type == &filterfalse_type) && + !_PyArg_NoKeywords("filterfalse", kwargs)) { + goto exit; + } + if (!PyArg_UnpackTuple(args, "filterfalse", + 2, 2, + &func, &seq)) { + goto exit; + } + return_value = itertools_filterfalse_impl(type, func, seq); + +exit: + return return_value; +} + +PyDoc_STRVAR(itertools_count__doc__, +"count(start=0, step=1)\n" +"--\n" +"\n" +"Return a count object whose .__next__() method returns consecutive values.\n" +"\n" +"Equivalent to:\n" +" def count(firstval=0, step=1):\n" +" x = firstval\n" +" while 1:\n" +" yield x\n" +" x += step"); + +static PyObject * +itertools_count_impl(PyTypeObject *type, PyObject *long_cnt, + PyObject *long_step); + +static PyObject * +itertools_count(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"start", "step", NULL}; + static _PyArg_Parser _parser = {"|OO:count", _keywords, 0}; + PyObject *long_cnt = NULL; + PyObject *long_step = NULL; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &long_cnt, &long_step)) { + goto exit; + } + return_value = itertools_count_impl(type, long_cnt, long_step); + +exit: + return return_value; +} +/*[clinic end generated code: output=d9eb9601bd3296ef input=a9049054013a1b77]*/ diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index 3ad7e5c80651..ec8f0ae14206 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -11,11 +11,39 @@ module itertools class itertools.groupby "groupbyobject *" "&groupby_type" class itertools._grouper "_grouperobject *" "&_grouper_type" +class itertools.teedataobject "teedataobject *" "&teedataobject_type" +class itertools._tee "teeobject *" "&tee_type" +class itertools.cycle "cycleobject *" "&cycle_type" +class itertools.dropwhile "dropwhileobject *" "&dropwhile_type" +class itertools.takewhile "takewhileobject *" "&takewhile_type" +class itertools.starmap "starmapobject *" "&starmap_type" +class itertools.chain "chainobject *" "&chain_type" +class itertools.combinations "combinationsobject *" "&combinations_type" +class itertools.combinations_with_replacement "cwr_object *" "&cwr_type" +class itertools.permutations "permutationsobject *" "&permutations_type" +class itertools.accumulate "accumulateobject *" "&accumulate_type" +class itertools.compress "compressobject *" "&compress_type" +class itertools.filterfalse "filterfalseobject *" "&filterfalse_type" +class itertools.count "countobject *" "&count_type" [clinic start generated code]*/ -/*[clinic end generated code: output=da39a3ee5e6b4b0d input=9d506f5bb9177570]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=ea05c93c6d94726a]*/ static PyTypeObject groupby_type; static PyTypeObject _grouper_type; +static PyTypeObject teedataobject_type; +static PyTypeObject tee_type; +static PyTypeObject cycle_type; +static PyTypeObject dropwhile_type; +static PyTypeObject takewhile_type; +static PyTypeObject starmap_type; +static PyTypeObject combinations_type; +static PyTypeObject cwr_type; +static PyTypeObject permutations_type; +static PyTypeObject accumulate_type; +static PyTypeObject compress_type; +static PyTypeObject filterfalse_type; +static PyTypeObject count_type; + #include "clinic/itertoolsmodule.c.h" @@ -539,18 +567,25 @@ teedataobject_reduce(teedataobject *tdo, PyObject *Py_UNUSED(ignored)) tdo->nextlink ? tdo->nextlink : Py_None); } -static PyTypeObject teedataobject_type; +/*[clinic input] + at classmethod +itertools.teedataobject.__new__ + iterable as it: object + values: object(subclass_of='&PyList_Type') + next: object + / +Data container common to multiple tee objects. +[clinic start generated code]*/ static PyObject * -teedataobject_new(PyTypeObject *type, PyObject *args, PyObject *kw) +itertools_teedataobject_impl(PyTypeObject *type, PyObject *it, + PyObject *values, PyObject *next) +/*[clinic end generated code: output=3343ceb07e08df5e input=be60f2fabd2b72ba]*/ { teedataobject *tdo; - PyObject *it, *values, *next; Py_ssize_t i, len; assert(type == &teedataobject_type); - if (!PyArg_ParseTuple(args, "OO!O", &it, &PyList_Type, &values, &next)) - return NULL; tdo = (teedataobject *)teedataobject_newinternal(it); if (!tdo) @@ -592,8 +627,6 @@ static PyMethodDef teedataobject_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(teedataobject_doc, "Data container common to multiple tee objects."); - static PyTypeObject teedataobject_type = { PyVarObject_HEAD_INIT(0, 0) /* Must fill in type value later */ "itertools._tee_dataobject", /* tp_name */ @@ -616,7 +649,7 @@ static PyTypeObject teedataobject_type = { 0, /* tp_setattro */ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */ - teedataobject_doc, /* tp_doc */ + itertools_teedataobject__doc__, /* tp_doc */ (traverseproc)teedataobject_traverse, /* tp_traverse */ (inquiry)teedataobject_clear, /* tp_clear */ 0, /* tp_richcompare */ @@ -633,7 +666,7 @@ static PyTypeObject teedataobject_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - teedataobject_new, /* tp_new */ + itertools_teedataobject, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -716,13 +749,18 @@ tee_fromiterable(PyObject *iterable) return (PyObject *)to; } +/*[clinic input] + at classmethod +itertools._tee.__new__ + iterable: object + / +Iterator wrapped to make it copyable. +[clinic start generated code]*/ + static PyObject * -tee_new(PyTypeObject *type, PyObject *args, PyObject *kw) +itertools__tee_impl(PyTypeObject *type, PyObject *iterable) +/*[clinic end generated code: output=b02d3fd26c810c3f input=adc0779d2afe37a2]*/ { - PyObject *iterable; - - if (!PyArg_UnpackTuple(args, "_tee", 1, 1, &iterable)) - return NULL; return tee_fromiterable(iterable); } @@ -771,9 +809,6 @@ tee_setstate(teeobject *to, PyObject *state) Py_RETURN_NONE; } -PyDoc_STRVAR(teeobject_doc, -"Iterator wrapped to make it copyable"); - static PyMethodDef tee_methods[] = { {"__copy__", (PyCFunction)tee_copy, METH_NOARGS, teecopy_doc}, {"__reduce__", (PyCFunction)tee_reduce, METH_NOARGS, reduce_doc}, @@ -803,7 +838,7 @@ static PyTypeObject tee_type = { 0, /* tp_setattro */ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */ - teeobject_doc, /* tp_doc */ + itertools__tee__doc__, /* tp_doc */ (traverseproc)tee_traverse, /* tp_traverse */ (inquiry)tee_clear, /* tp_clear */ 0, /* tp_richcompare */ @@ -820,19 +855,26 @@ static PyTypeObject tee_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - tee_new, /* tp_new */ + itertools__tee, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; +/*[clinic input] +itertools.tee + iterable: object + n: Py_ssize_t = 2 + / +Returns a tuple of n independent iterators. +[clinic start generated code]*/ + static PyObject * -tee(PyObject *self, PyObject *args) +itertools_tee_impl(PyObject *module, PyObject *iterable, Py_ssize_t n) +/*[clinic end generated code: output=1c64519cd859c2f0 input=c99a1472c425d66d]*/ { - Py_ssize_t i, n=2; - PyObject *it, *iterable, *copyable, *copyfunc, *result; + Py_ssize_t i; + PyObject *it, *copyable, *copyfunc, *result; _Py_IDENTIFIER(__copy__); - if (!PyArg_ParseTuple(args, "O|n", &iterable, &n)) - return NULL; if (n < 0) { PyErr_SetString(PyExc_ValueError, "n must be >= 0"); return NULL; @@ -885,9 +927,6 @@ tee(PyObject *self, PyObject *args) return result; } -PyDoc_STRVAR(tee_doc, -"tee(iterable, n=2) --> tuple of n independent iterators."); - /* cycle object **************************************************************/ @@ -901,20 +940,22 @@ typedef struct { static PyTypeObject cycle_type; +/*[clinic input] + at classmethod +itertools.cycle.__new__ + iterable: object + / +Return elements from the iterable until it is exhausted. Then repeat the sequence indefinitely. +[clinic start generated code]*/ + static PyObject * -cycle_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_cycle_impl(PyTypeObject *type, PyObject *iterable) +/*[clinic end generated code: output=f60e5ec17a45b35c input=9d1d84bcf66e908b]*/ { PyObject *it; - PyObject *iterable; PyObject *saved; cycleobject *lz; - if (type == &cycle_type && !_PyArg_NoKeywords("cycle", kwds)) - return NULL; - - if (!PyArg_UnpackTuple(args, "cycle", 1, 1, &iterable)) - return NULL; - /* Get iterator. */ it = PyObject_GetIter(iterable); if (it == NULL) @@ -1041,12 +1082,6 @@ static PyMethodDef cycle_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(cycle_doc, -"cycle(iterable) --> cycle object\n\ -\n\ -Return elements from the iterable until it is exhausted.\n\ -Then repeat the sequence indefinitely."); - static PyTypeObject cycle_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.cycle", /* tp_name */ @@ -1070,7 +1105,7 @@ static PyTypeObject cycle_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - cycle_doc, /* tp_doc */ + itertools_cycle__doc__, /* tp_doc */ (traverseproc)cycle_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -1087,7 +1122,7 @@ static PyTypeObject cycle_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - cycle_new, /* tp_new */ + itertools_cycle, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -1103,19 +1138,24 @@ typedef struct { static PyTypeObject dropwhile_type; +/*[clinic input] + at classmethod +itertools.dropwhile.__new__ + predicate as func: object + iterable as seq: object + / +Drop items from the iterable while predicate(item) is true. + +Afterwards, return every element until the iterable is exhausted. +[clinic start generated code]*/ + static PyObject * -dropwhile_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_dropwhile_impl(PyTypeObject *type, PyObject *func, PyObject *seq) +/*[clinic end generated code: output=92f9d0d89af149e4 input=d39737147c9f0a26]*/ { - PyObject *func, *seq; PyObject *it; dropwhileobject *lz; - if (type == &dropwhile_type && !_PyArg_NoKeywords("dropwhile", kwds)) - return NULL; - - if (!PyArg_UnpackTuple(args, "dropwhile", 2, 2, &func, &seq)) - return NULL; - /* Get iterator. */ it = PyObject_GetIter(seq); if (it == NULL) @@ -1209,12 +1249,6 @@ static PyMethodDef dropwhile_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(dropwhile_doc, -"dropwhile(predicate, iterable) --> dropwhile object\n\ -\n\ -Drop items from the iterable while predicate(item) is true.\n\ -Afterwards, return every element until the iterable is exhausted."); - static PyTypeObject dropwhile_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.dropwhile", /* tp_name */ @@ -1238,7 +1272,7 @@ static PyTypeObject dropwhile_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - dropwhile_doc, /* tp_doc */ + itertools_dropwhile__doc__, /* tp_doc */ (traverseproc)dropwhile_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -1255,7 +1289,7 @@ static PyTypeObject dropwhile_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - dropwhile_new, /* tp_new */ + itertools_dropwhile, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -1271,19 +1305,22 @@ typedef struct { static PyTypeObject takewhile_type; +/*[clinic input] + at classmethod +itertools.takewhile.__new__ + predicate as func: object + iterable as seq: object + / +Return successive entries from an iterable as long as the predicate evaluates to true for each entry. +[clinic start generated code]*/ + static PyObject * -takewhile_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_takewhile_impl(PyTypeObject *type, PyObject *func, PyObject *seq) +/*[clinic end generated code: output=bb179ea7864e2ef6 input=ba5255f7519aa119]*/ { - PyObject *func, *seq; PyObject *it; takewhileobject *lz; - if (type == &takewhile_type && !_PyArg_NoKeywords("takewhile", kwds)) - return NULL; - - if (!PyArg_UnpackTuple(args, "takewhile", 2, 2, &func, &seq)) - return NULL; - /* Get iterator. */ it = PyObject_GetIter(seq); if (it == NULL) @@ -1373,11 +1410,6 @@ static PyMethodDef takewhile_reduce_methods[] = { setstate_doc}, {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(takewhile_doc, -"takewhile(predicate, iterable) --> takewhile object\n\ -\n\ -Return successive entries from an iterable as long as the\n\ -predicate evaluates to true for each entry."); static PyTypeObject takewhile_type = { PyVarObject_HEAD_INIT(NULL, 0) @@ -1402,7 +1434,7 @@ static PyTypeObject takewhile_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - takewhile_doc, /* tp_doc */ + itertools_takewhile__doc__, /* tp_doc */ (traverseproc)takewhile_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -1419,7 +1451,7 @@ static PyTypeObject takewhile_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - takewhile_new, /* tp_new */ + itertools_takewhile, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -1693,19 +1725,22 @@ typedef struct { static PyTypeObject starmap_type; +/*[clinic input] + at classmethod +itertools.starmap.__new__ + function as func: object + iterable as seq: object + / +Return an iterator whose values are returned from the function evaluated with an argument tuple taken from the given sequence. +[clinic start generated code]*/ + static PyObject * -starmap_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_starmap_impl(PyTypeObject *type, PyObject *func, PyObject *seq) +/*[clinic end generated code: output=79eeb81d452c6e8d input=844766df6a0d4dad]*/ { - PyObject *func, *seq; PyObject *it; starmapobject *lz; - if (type == &starmap_type && !_PyArg_NoKeywords("starmap", kwds)) - return NULL; - - if (!PyArg_UnpackTuple(args, "starmap", 2, 2, &func, &seq)) - return NULL; - /* Get iterator. */ it = PyObject_GetIter(seq); if (it == NULL) @@ -1776,12 +1811,6 @@ static PyMethodDef starmap_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(starmap_doc, -"starmap(function, sequence) --> starmap object\n\ -\n\ -Return an iterator whose values are returned from the function evaluated\n\ -with an argument tuple taken from the given sequence."); - static PyTypeObject starmap_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.starmap", /* tp_name */ @@ -1805,7 +1834,7 @@ static PyTypeObject starmap_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - starmap_doc, /* tp_doc */ + itertools_starmap__doc__, /* tp_doc */ (traverseproc)starmap_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -1822,7 +1851,7 @@ static PyTypeObject starmap_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - starmap_new, /* tp_new */ + itertools_starmap, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -1868,8 +1897,17 @@ chain_new(PyTypeObject *type, PyObject *args, PyObject *kwds) return chain_new_internal(type, source); } +/*[clinic input] + at classmethod +itertools.chain.from_iterable + iterable as arg: object + / +Alternative chain() constructor taking a single iterable argument that evaluates lazily. +[clinic start generated code]*/ + static PyObject * -chain_new_from_iterable(PyTypeObject *type, PyObject *arg) +itertools_chain_from_iterable(PyTypeObject *type, PyObject *arg) +/*[clinic end generated code: output=667ae7a7f7b68654 input=72c39e3a2ca3be85]*/ { PyObject *source; @@ -1985,15 +2023,8 @@ Return a chain object whose .__next__() method returns elements from the\n\ first iterable until it is exhausted, then elements from the next\n\ iterable, until all of the iterables are exhausted."); -PyDoc_STRVAR(chain_from_iterable_doc, -"chain.from_iterable(iterable) --> chain object\n\ -\n\ -Alternate chain() constructor taking a single iterable argument\n\ -that evaluates lazily."); - static PyMethodDef chain_methods[] = { - {"from_iterable", (PyCFunction) chain_new_from_iterable, METH_O | METH_CLASS, - chain_from_iterable_doc}, + ITERTOOLS_CHAIN_FROM_ITERABLE_METHODDEF {"__reduce__", (PyCFunction)chain_reduce, METH_NOARGS, reduce_doc}, {"__setstate__", (PyCFunction)chain_setstate, METH_O, @@ -2417,21 +2448,27 @@ typedef struct { static PyTypeObject combinations_type; + +/*[clinic input] + at classmethod +itertools.combinations.__new__ + iterable: object + r: Py_ssize_t +Return successive r-length combinations of elements in the iterable. + +combinations(range(4), 3) --> (0,1,2), (0,1,3), (0,2,3), (1,2,3) +[clinic start generated code]*/ + static PyObject * -combinations_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_combinations_impl(PyTypeObject *type, PyObject *iterable, + Py_ssize_t r) +/*[clinic end generated code: output=87a689b39c40039c input=06bede09e3da20f8]*/ { combinationsobject *co; Py_ssize_t n; - Py_ssize_t r; PyObject *pool = NULL; - PyObject *iterable = NULL; Py_ssize_t *indices = NULL; Py_ssize_t i; - static char *kwargs[] = {"iterable", "r", NULL}; - - if (!PyArg_ParseTupleAndKeywords(args, kwds, "On:combinations", kwargs, - &iterable, &r)) - return NULL; pool = PySequence_Tuple(iterable); if (pool == NULL) @@ -2666,12 +2703,6 @@ static PyMethodDef combinations_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(combinations_doc, -"combinations(iterable, r) --> combinations object\n\ -\n\ -Return successive r-length combinations of elements in the iterable.\n\n\ -combinations(range(4), 3) --> (0,1,2), (0,1,3), (0,2,3), (1,2,3)"); - static PyTypeObject combinations_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.combinations", /* tp_name */ @@ -2695,7 +2726,7 @@ static PyTypeObject combinations_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - combinations_doc, /* tp_doc */ + itertools_combinations__doc__, /* tp_doc */ (traverseproc)combinations_traverse,/* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -2712,7 +2743,7 @@ static PyTypeObject combinations_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - combinations_new, /* tp_new */ + itertools_combinations, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -2756,22 +2787,27 @@ typedef struct { static PyTypeObject cwr_type; +/*[clinic input] + at classmethod +itertools.combinations_with_replacement.__new__ + iterable: object + r: Py_ssize_t +Return successive r-length combinations of elements in the iterable allowing individual elements to have successive repeats. + +combinations_with_replacement('ABC', 2) --> AA AB AC BB BC CC" +[clinic start generated code]*/ + static PyObject * -cwr_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_combinations_with_replacement_impl(PyTypeObject *type, + PyObject *iterable, + Py_ssize_t r) +/*[clinic end generated code: output=48b26856d4e659ca input=dc2a8c7ba785fad7]*/ { cwrobject *co; Py_ssize_t n; - Py_ssize_t r; PyObject *pool = NULL; - PyObject *iterable = NULL; Py_ssize_t *indices = NULL; Py_ssize_t i; - static char *kwargs[] = {"iterable", "r", NULL}; - - if (!PyArg_ParseTupleAndKeywords(args, kwds, - "On:combinations_with_replacement", - kwargs, &iterable, &r)) - return NULL; pool = PySequence_Tuple(iterable); if (pool == NULL) @@ -2996,13 +3032,6 @@ static PyMethodDef cwr_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(cwr_doc, -"combinations_with_replacement(iterable, r) --> combinations_with_replacement object\n\ -\n\ -Return successive r-length combinations of elements in the iterable\n\ -allowing individual elements to have successive repeats.\n\ -combinations_with_replacement('ABC', 2) --> AA AB AC BB BC CC"); - static PyTypeObject cwr_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.combinations_with_replacement", /* tp_name */ @@ -3026,7 +3055,7 @@ static PyTypeObject cwr_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - cwr_doc, /* tp_doc */ + itertools_combinations_with_replacement__doc__, /* tp_doc */ (traverseproc)cwr_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -3043,7 +3072,7 @@ static PyTypeObject cwr_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - cwr_new, /* tp_new */ + itertools_combinations_with_replacement, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -3085,23 +3114,28 @@ typedef struct { static PyTypeObject permutations_type; +/*[clinic input] + at classmethod +itertools.permutations.__new__ + iterable: object + r as robj: object = None +Return successive r-length permutations of elements in the iterable. + +permutations(range(3), 2) --> (0,1), (0,2), (1,0), (1,2), (2,0), (2,1) +[clinic start generated code]*/ + static PyObject * -permutations_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_permutations_impl(PyTypeObject *type, PyObject *iterable, + PyObject *robj) +/*[clinic end generated code: output=296a72fa76d620ea input=57d0170a4ac0ec7a]*/ { permutationsobject *po; Py_ssize_t n; Py_ssize_t r; - PyObject *robj = Py_None; PyObject *pool = NULL; - PyObject *iterable = NULL; Py_ssize_t *indices = NULL; Py_ssize_t *cycles = NULL; Py_ssize_t i; - static char *kwargs[] = {"iterable", "r", NULL}; - - if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|O:permutations", kwargs, - &iterable, &robj)) - return NULL; pool = PySequence_Tuple(iterable); if (pool == NULL) @@ -3389,12 +3423,6 @@ static PyMethodDef permuations_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(permutations_doc, -"permutations(iterable[, r]) --> permutations object\n\ -\n\ -Return successive r-length permutations of elements in the iterable.\n\n\ -permutations(range(3), 2) --> (0,1), (0,2), (1,0), (1,2), (2,0), (2,1)"); - static PyTypeObject permutations_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.permutations", /* tp_name */ @@ -3418,7 +3446,7 @@ static PyTypeObject permutations_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - permutations_doc, /* tp_doc */ + itertools_permutations__doc__, /* tp_doc */ (traverseproc)permutations_traverse,/* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -3435,10 +3463,11 @@ static PyTypeObject permutations_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - permutations_new, /* tp_new */ + itertools_permutations, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; + /* accumulate object ********************************************************/ typedef struct { @@ -3450,18 +3479,22 @@ typedef struct { static PyTypeObject accumulate_type; +/*[clinic input] + at classmethod +itertools.accumulate.__new__ + iterable: object + func as binop: object = None +Return series of accumulated sums (or other binary function results). +[clinic start generated code]*/ + static PyObject * -accumulate_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_accumulate_impl(PyTypeObject *type, PyObject *iterable, + PyObject *binop) +/*[clinic end generated code: output=514d0fb30ba14d55 input=6d9d16aaa1d3cbfc]*/ { - static char *kwargs[] = {"iterable", "func", NULL}; - PyObject *iterable; PyObject *it; - PyObject *binop = Py_None; accumulateobject *lz; - if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|O:accumulate", - kwargs, &iterable, &binop)) - return NULL; /* Get iterator. */ it = PyObject_GetIter(iterable); @@ -3572,11 +3605,6 @@ static PyMethodDef accumulate_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(accumulate_doc, -"accumulate(iterable[, func]) --> accumulate object\n\ -\n\ -Return series of accumulated sums (or other binary function results)."); - static PyTypeObject accumulate_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.accumulate", /* tp_name */ @@ -3600,7 +3628,7 @@ static PyTypeObject accumulate_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - accumulate_doc, /* tp_doc */ + itertools_accumulate__doc__, /* tp_doc */ (traverseproc)accumulate_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -3617,7 +3645,7 @@ static PyTypeObject accumulate_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - accumulate_new, /* tp_new */ + itertools_accumulate, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -3639,16 +3667,23 @@ typedef struct { static PyTypeObject compress_type; +/*[clinic input] + at classmethod +itertools.compress.__new__ + data as seq1: object + selectors as seq2: object +Return data elements corresponding to true selector elements. + +Forms a shorter iterator from selected data elements using the selectors to +choose the data elements. +[clinic start generated code]*/ + static PyObject * -compress_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_compress_impl(PyTypeObject *type, PyObject *seq1, PyObject *seq2) +/*[clinic end generated code: output=7e67157212ed09e0 input=79596d7cd20c77e5]*/ { - PyObject *seq1, *seq2; PyObject *data=NULL, *selectors=NULL; compressobject *lz; - static char *kwargs[] = {"data", "selectors", NULL}; - - if (!PyArg_ParseTupleAndKeywords(args, kwds, "OO:compress", kwargs, &seq1, &seq2)) - return NULL; data = PyObject_GetIter(seq1); if (data == NULL) @@ -3737,13 +3772,6 @@ static PyMethodDef compress_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(compress_doc, -"compress(data, selectors) --> iterator over selected data\n\ -\n\ -Return data elements corresponding to true selector elements.\n\ -Forms a shorter iterator from selected data elements using the\n\ -selectors to choose the data elements."); - static PyTypeObject compress_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.compress", /* tp_name */ @@ -3767,7 +3795,7 @@ static PyTypeObject compress_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - compress_doc, /* tp_doc */ + itertools_compress__doc__, /* tp_doc */ (traverseproc)compress_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -3784,7 +3812,7 @@ static PyTypeObject compress_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - compress_new, /* tp_new */ + itertools_compress, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -3799,20 +3827,24 @@ typedef struct { static PyTypeObject filterfalse_type; +/*[clinic input] + at classmethod +itertools.filterfalse.__new__ + function as func: object + iterable as seq: object + / +Return those items of iterable for which function(item) is false. + +If function is None, return the items that are false. +[clinic start generated code]*/ + static PyObject * -filterfalse_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_filterfalse_impl(PyTypeObject *type, PyObject *func, PyObject *seq) +/*[clinic end generated code: output=55f87eab9fc0484e input=2d684a2c66f99cde]*/ { - PyObject *func, *seq; PyObject *it; filterfalseobject *lz; - if (type == &filterfalse_type && - !_PyArg_NoKeywords("filterfalse", kwds)) - return NULL; - - if (!PyArg_UnpackTuple(args, "filterfalse", 2, 2, &func, &seq)) - return NULL; - /* Get iterator. */ it = PyObject_GetIter(seq); if (it == NULL) @@ -3894,12 +3926,6 @@ static PyMethodDef filterfalse_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(filterfalse_doc, -"filterfalse(function or None, sequence) --> filterfalse object\n\ -\n\ -Return those items of sequence for which function(item) is false.\n\ -If function is None, return the items that are false."); - static PyTypeObject filterfalse_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.filterfalse", /* tp_name */ @@ -3923,7 +3949,7 @@ static PyTypeObject filterfalse_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - filterfalse_doc, /* tp_doc */ + itertools_filterfalse__doc__, /* tp_doc */ (traverseproc)filterfalse_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -3940,7 +3966,7 @@ static PyTypeObject filterfalse_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - filterfalse_new, /* tp_new */ + itertools_filterfalse, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -3973,20 +3999,30 @@ slow_mode: when cnt == PY_SSIZE_T_MAX, step is not int(1), or cnt is a float. static PyTypeObject count_type; +/*[clinic input] + at classmethod +itertools.count.__new__ + start as long_cnt: object(c_default="NULL") = 0 + step as long_step: object(c_default="NULL") = 1 +Return a count object whose .__next__() method returns consecutive values. + +Equivalent to: + def count(firstval=0, step=1): + x = firstval + while 1: + yield x + x += step +[clinic start generated code]*/ + static PyObject * -count_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +itertools_count_impl(PyTypeObject *type, PyObject *long_cnt, + PyObject *long_step) +/*[clinic end generated code: output=09a9250aebd00b1c input=d7a85eec18bfcd94]*/ { countobject *lz; int fast_mode; Py_ssize_t cnt = 0; - PyObject *long_cnt = NULL; - PyObject *long_step = NULL; long step; - static char *kwlist[] = {"start", "step", 0}; - - if (!PyArg_ParseTupleAndKeywords(args, kwds, "|OO:count", - kwlist, &long_cnt, &long_step)) - return NULL; if ((long_cnt != NULL && !PyNumber_Check(long_cnt)) || (long_step != NULL && !PyNumber_Check(long_step))) { @@ -4139,17 +4175,6 @@ static PyMethodDef count_methods[] = { {NULL, NULL} /* sentinel */ }; -PyDoc_STRVAR(count_doc, - "count(start=0, step=1) --> count object\n\ -\n\ -Return a count object whose .__next__() method returns consecutive values.\n\ -Equivalent to:\n\n\ - def count(firstval=0, step=1):\n\ - x = firstval\n\ - while 1:\n\ - yield x\n\ - x += step\n"); - static PyTypeObject count_type = { PyVarObject_HEAD_INIT(NULL, 0) "itertools.count", /* tp_name */ @@ -4173,7 +4198,7 @@ static PyTypeObject count_type = { 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, /* tp_flags */ - count_doc, /* tp_doc */ + itertools_count__doc__, /* tp_doc */ (traverseproc)count_traverse, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ @@ -4190,7 +4215,7 @@ static PyTypeObject count_type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - count_new, /* tp_new */ + itertools_count, /* tp_new */ PyObject_GC_Del, /* tp_free */ }; @@ -4349,6 +4374,7 @@ static PyTypeObject repeat_type = { PyObject_GC_Del, /* tp_free */ }; + /* ziplongest object *********************************************************/ typedef struct { @@ -4619,6 +4645,7 @@ static PyTypeObject ziplongest_type = { PyObject_GC_Del, /* tp_free */ }; + /* module level code ********************************************************/ PyDoc_STRVAR(module_doc, @@ -4653,7 +4680,7 @@ combinations_with_replacement(p, r)\n\ static PyMethodDef module_methods[] = { - {"tee", (PyCFunction)tee, METH_VARARGS, tee_doc}, + ITERTOOLS_TEE_METHODDEF {NULL, NULL} /* sentinel */ }; From webhook-mailer at python.org Tue Sep 11 18:11:09 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 22:11:09 -0000 Subject: [Python-checkins] Initialize a variable to make the compiler happy. (GH-9153) Message-ID: https://github.com/python/cpython/commit/acd282fd5b3ca4de302b33c9361dbc433593c4ca commit: acd282fd5b3ca4de302b33c9361dbc433593c4ca branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-11T15:11:06-07:00 summary: Initialize a variable to make the compiler happy. (GH-9153) GCC complains: Python/pylifecycle.c: In function ?_Py_InitializeFromConfig?: Python/pylifecycle.c:900:13: warning: ?interp? may be used uninitialized in this function [-Wmaybe-uninitialized] err = _Py_InitializeMainInterpreter(interp, &main_config); ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This seems spurious since &interp is passed to _Py_InitializeCore. Anyway, we can easily initialize to quiet the warning. files: M Python/pylifecycle.c diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 33ca802fd56d..379e860b5ba4 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -886,7 +886,7 @@ _Py_InitializeMainInterpreter(PyInterpreterState *interp, _PyInitError _Py_InitializeFromConfig(const _PyCoreConfig *config) { - PyInterpreterState *interp; + PyInterpreterState *interp = NULL; _PyInitError err; err = _Py_InitializeCore(&interp, config); if (_Py_INIT_FAILED(err)) { From webhook-mailer at python.org Tue Sep 11 18:21:14 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 22:21:14 -0000 Subject: [Python-checkins] closes bpo-31902: Fix the col_offset attribute for ast.Async* nodes to point to the "async" keyword. (GH-4175) Message-ID: https://github.com/python/cpython/commit/d8bc7a666b1309aa5669980e35e7429071f3f7d3 commit: d8bc7a666b1309aa5669980e35e7429071f3f7d3 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T15:21:11-07:00 summary: closes bpo-31902: Fix the col_offset attribute for ast.Async* nodes to point to the "async" keyword. (GH-4175) Previously, col_offset points to the keyword after "async". (cherry picked from commit 90fc8980bbcc5c7dcced3627fe172b0bfd193a3b) Co-authored-by: guoci files: A Misc/NEWS.d/next/Core and Builtins/2017-10-30-12-44-50.bpo-31902.a07fa57.rst M Lib/test/test_ast.py M Python/ast.c diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py index 7db40e7797f7..72f8467847e8 100644 --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -1181,12 +1181,12 @@ def main(): ('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [], 0)]))]), ('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))], 0)]))]), ('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [], 0)]))]), -('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Str', (2, 1), 'async function')), ('Expr', (3, 1), ('Await', (3, 1), ('Call', (3, 7), ('Name', (3, 7), 'something', ('Load',)), [], [])))], [], None)]), -('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 7), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), -('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 7), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Str', (2, 1), 'async function')), ('Expr', (3, 1), ('Await', (3, 1), ('Call', (3, 7), ('Name', (3, 7), 'something', ('Load',)), [], [])))], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 1), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 1), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), ('Module', [('Expr', (1, 0), ('Dict', (1, 0), [None, ('Num', (1, 10), 2)], [('Dict', (1, 3), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), ('Module', [('Expr', (1, 0), ('Set', (1, 0), [('Starred', (1, 1), ('Set', (1, 2), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), -('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('ListComp', (2, 2), ('Name', (2, 2), 'i', ('Load',)), [('comprehension', ('Name', (2, 14), 'b', ('Store',)), ('Name', (2, 19), 'c', ('Load',)), [], 1)]))], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('ListComp', (2, 2), ('Name', (2, 2), 'i', ('Load',)), [('comprehension', ('Name', (2, 14), 'b', ('Store',)), ('Name', (2, 19), 'c', ('Load',)), [], 1)]))], [], None)]), ] single_results = [ ('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]), diff --git a/Misc/NEWS.d/next/Core and Builtins/2017-10-30-12-44-50.bpo-31902.a07fa57.rst b/Misc/NEWS.d/next/Core and Builtins/2017-10-30-12-44-50.bpo-31902.a07fa57.rst new file mode 100644 index 000000000000..e2b04b3ae0f5 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2017-10-30-12-44-50.bpo-31902.a07fa57.rst @@ -0,0 +1,3 @@ +Fix the ``col_offset`` attribute for ast nodes ``ast.AsyncFor``, +``ast.AsyncFunctionDef``, and ``ast.AsyncWith``. Previously, ``col_offset`` +pointed to the keyword after ``async``. diff --git a/Python/ast.c b/Python/ast.c index 5001ad530c8d..5f710c710acb 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -600,8 +600,8 @@ static asdl_seq *ast_for_exprlist(struct compiling *, const node *, static expr_ty ast_for_testlist(struct compiling *, const node *); static stmt_ty ast_for_classdef(struct compiling *, const node *, asdl_seq *); -static stmt_ty ast_for_with_stmt(struct compiling *, const node *, int); -static stmt_ty ast_for_for_stmt(struct compiling *, const node *, int); +static stmt_ty ast_for_with_stmt(struct compiling *, const node *, bool); +static stmt_ty ast_for_for_stmt(struct compiling *, const node *, bool); /* Note different signature for ast_for_call */ static expr_ty ast_for_call(struct compiling *, const node *, expr_ty, bool); @@ -1569,10 +1569,11 @@ ast_for_decorators(struct compiling *c, const node *n) } static stmt_ty -ast_for_funcdef_impl(struct compiling *c, const node *n, - asdl_seq *decorator_seq, int is_async) +ast_for_funcdef_impl(struct compiling *c, const node *n0, + asdl_seq *decorator_seq, bool is_async) { /* funcdef: 'def' NAME parameters ['->' test] ':' suite */ + const node * const n = is_async ? CHILD(n0, 1) : n0; identifier name; arguments_ty args; asdl_seq *body; @@ -1601,7 +1602,7 @@ ast_for_funcdef_impl(struct compiling *c, const node *n, if (is_async) return AsyncFunctionDef(name, args, body, decorator_seq, returns, - LINENO(n), n->n_col_offset, c->c_arena); + LINENO(n), n0->n_col_offset, c->c_arena); else return FunctionDef(name, args, body, decorator_seq, returns, LINENO(n), n->n_col_offset, c->c_arena); @@ -1616,8 +1617,8 @@ ast_for_async_funcdef(struct compiling *c, const node *n, asdl_seq *decorator_se assert(strcmp(STR(CHILD(n, 0)), "async") == 0); REQ(CHILD(n, 1), funcdef); - return ast_for_funcdef_impl(c, CHILD(n, 1), decorator_seq, - 1 /* is_async */); + return ast_for_funcdef_impl(c, n, decorator_seq, + true /* is_async */); } static stmt_ty @@ -1625,7 +1626,7 @@ ast_for_funcdef(struct compiling *c, const node *n, asdl_seq *decorator_seq) { /* funcdef: 'def' NAME parameters ['->' test] ':' suite */ return ast_for_funcdef_impl(c, n, decorator_seq, - 0 /* is_async */); + false /* is_async */); } @@ -1639,15 +1640,15 @@ ast_for_async_stmt(struct compiling *c, const node *n) switch (TYPE(CHILD(n, 1))) { case funcdef: - return ast_for_funcdef_impl(c, CHILD(n, 1), NULL, - 1 /* is_async */); + return ast_for_funcdef_impl(c, n, NULL, + true /* is_async */); case with_stmt: - return ast_for_with_stmt(c, CHILD(n, 1), - 1 /* is_async */); + return ast_for_with_stmt(c, n, + true /* is_async */); case for_stmt: - return ast_for_for_stmt(c, CHILD(n, 1), - 1 /* is_async */); + return ast_for_for_stmt(c, n, + true /* is_async */); default: PyErr_Format(PyExc_SystemError, @@ -3681,8 +3682,9 @@ ast_for_while_stmt(struct compiling *c, const node *n) } static stmt_ty -ast_for_for_stmt(struct compiling *c, const node *n, int is_async) +ast_for_for_stmt(struct compiling *c, const node *n0, bool is_async) { + const node * const n = is_async ? CHILD(n0, 1) : n0; asdl_seq *_target, *seq = NULL, *suite_seq; expr_ty expression; expr_ty target, first; @@ -3717,7 +3719,7 @@ ast_for_for_stmt(struct compiling *c, const node *n, int is_async) if (is_async) return AsyncFor(target, expression, suite_seq, seq, - LINENO(n), n->n_col_offset, + LINENO(n), n0->n_col_offset, c->c_arena); else return For(target, expression, suite_seq, seq, @@ -3869,8 +3871,9 @@ ast_for_with_item(struct compiling *c, const node *n) /* with_stmt: 'with' with_item (',' with_item)* ':' suite */ static stmt_ty -ast_for_with_stmt(struct compiling *c, const node *n, int is_async) +ast_for_with_stmt(struct compiling *c, const node *n0, bool is_async) { + const node * const n = is_async ? CHILD(n0, 1) : n0; int i, n_items; asdl_seq *items, *body; @@ -3892,7 +3895,7 @@ ast_for_with_stmt(struct compiling *c, const node *n, int is_async) return NULL; if (is_async) - return AsyncWith(items, body, LINENO(n), n->n_col_offset, c->c_arena); + return AsyncWith(items, body, LINENO(n), n0->n_col_offset, c->c_arena); else return With(items, body, LINENO(n), n->n_col_offset, c->c_arena); } From webhook-mailer at python.org Tue Sep 11 18:23:28 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 11 Sep 2018 22:23:28 -0000 Subject: [Python-checkins] Revert "bpo-34595: Add %T format to PyUnicode_FromFormatV() (GH-9080)" (GH-9187) Message-ID: https://github.com/python/cpython/commit/998b80636690ffbdb0a278810d9c031fad38631d commit: 998b80636690ffbdb0a278810d9c031fad38631d branch: master author: Victor Stinner committer: GitHub date: 2018-09-12T00:23:25+02:00 summary: Revert "bpo-34595: Add %T format to PyUnicode_FromFormatV() (GH-9080)" (GH-9187) This reverts commit 886483e2b9bbabf60ab769683269b873381dd5ee. files: D Misc/NEWS.d/next/C API/2018-09-06-11-17-49.bpo-34595.Hkz62y.rst M Doc/c-api/unicode.rst M Lib/test/test_unicode.py M Objects/unicodeobject.c diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 66b1efc60fde..92e22b16a4ef 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -519,9 +519,6 @@ APIs: | :attr:`%R` | PyObject\* | The result of calling | | | | :c:func:`PyObject_Repr`. | +-------------------+---------------------+--------------------------------+ - | :attr:`%T` | PyObject\* | Object type name, equivalent | - | | | to ``Py_TYPE(op)->tp_name``. | - +-------------------+---------------------+--------------------------------+ An unrecognized format character causes all the rest of the format string to be copied as-is to the result string, and any extra arguments discarded. @@ -546,9 +543,6 @@ APIs: Support width and precision formatter for ``"%s"``, ``"%A"``, ``"%U"``, ``"%V"``, ``"%S"``, ``"%R"`` added. - .. versionchanged:: 3.7 - Support for ``"%T"`` (object type name) added. - .. c:function:: PyObject* PyUnicode_FromFormatV(const char *format, va_list vargs) diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py index 73111f1c24a2..fb7bb2d523fe 100644 --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -2655,10 +2655,6 @@ def check_format(expected, format, *args): check_format(r"%A:'abc\xe9\uabcd\U0010ffff'", b'%%A:%A', 'abc\xe9\uabcd\U0010ffff') - # test %T (object type name) - check_format(r"type name: str", - b'type name: %T', 'text') - # test %V check_format('repr=abc', b'repr=%V', 'abc', b'xyz') diff --git a/Misc/NEWS.d/next/C API/2018-09-06-11-17-49.bpo-34595.Hkz62y.rst b/Misc/NEWS.d/next/C API/2018-09-06-11-17-49.bpo-34595.Hkz62y.rst deleted file mode 100644 index c054a8e229f5..000000000000 --- a/Misc/NEWS.d/next/C API/2018-09-06-11-17-49.bpo-34595.Hkz62y.rst +++ /dev/null @@ -1,4 +0,0 @@ -:c:func:`PyUnicode_FromFormatV`: add ``%T`` format to -:c:func:`PyUnicode_FromFormatV`, and so to :c:func:`PyUnicode_FromFormat` -and :c:func:`PyErr_Format`, to format an object type name: equivalent to -"%s" with ``Py_TYPE(obj)->tp_name``. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 3e61c9c37010..a797f838eb41 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -768,7 +768,8 @@ ensure_unicode(PyObject *obj) { if (!PyUnicode_Check(obj)) { PyErr_Format(PyExc_TypeError, - "must be str, not %T", obj); + "must be str, not %.100s", + Py_TYPE(obj)->tp_name); return -1; } return PyUnicode_READY(obj); @@ -2529,7 +2530,7 @@ unicode_fromformat_write_str(_PyUnicodeWriter *writer, PyObject *str, } static int -unicode_fromformat_write_utf8(_PyUnicodeWriter *writer, const char *str, +unicode_fromformat_write_cstr(_PyUnicodeWriter *writer, const char *str, Py_ssize_t width, Py_ssize_t precision) { /* UTF-8 */ @@ -2746,7 +2747,7 @@ unicode_fromformat_arg(_PyUnicodeWriter *writer, { /* UTF-8 */ const char *s = va_arg(*vargs, const char*); - if (unicode_fromformat_write_utf8(writer, s, width, precision) < 0) + if (unicode_fromformat_write_cstr(writer, s, width, precision) < 0) return NULL; break; } @@ -2772,7 +2773,7 @@ unicode_fromformat_arg(_PyUnicodeWriter *writer, } else { assert(str != NULL); - if (unicode_fromformat_write_utf8(writer, str, width, precision) < 0) + if (unicode_fromformat_write_cstr(writer, str, width, precision) < 0) return NULL; } break; @@ -2826,17 +2827,6 @@ unicode_fromformat_arg(_PyUnicodeWriter *writer, break; } - case 'T': - { - /* Object type name (tp_name) */ - PyObject *obj = va_arg(*vargs, PyObject *); - PyTypeObject *type = Py_TYPE(obj); - const char *type_name = type->tp_name; - if (unicode_fromformat_write_utf8(writer, type_name, -1, -1) < 0) { - return NULL; - } - break; - } case '%': if (_PyUnicodeWriter_WriteCharInline(writer, '%') < 0) return NULL; @@ -3034,7 +3024,8 @@ PyUnicode_FromObject(PyObject *obj) return _PyUnicode_Copy(obj); } PyErr_Format(PyExc_TypeError, - "Can't convert '%T' object to str implicitly", obj); + "Can't convert '%.100s' object to str implicitly", + Py_TYPE(obj)->tp_name); return NULL; } @@ -3070,8 +3061,8 @@ PyUnicode_FromEncodedObject(PyObject *obj, /* Retrieve a bytes buffer view through the PEP 3118 buffer interface */ if (PyObject_GetBuffer(obj, &buffer, PyBUF_SIMPLE) < 0) { PyErr_Format(PyExc_TypeError, - "decoding to str: need a bytes-like object, %T found", - obj); + "decoding to str: need a bytes-like object, %.80s found", + Py_TYPE(obj)->tp_name); return NULL; } @@ -3201,9 +3192,10 @@ PyUnicode_Decode(const char *s, goto onError; if (!PyUnicode_Check(unicode)) { PyErr_Format(PyExc_TypeError, - "'%.400s' decoder returned '%T' instead of 'str'; " + "'%.400s' decoder returned '%.400s' instead of 'str'; " "use codecs.decode() to decode to arbitrary types", - encoding, unicode); + encoding, + Py_TYPE(unicode)->tp_name); Py_DECREF(unicode); goto onError; } @@ -3263,9 +3255,10 @@ PyUnicode_AsDecodedUnicode(PyObject *unicode, goto onError; if (!PyUnicode_Check(v)) { PyErr_Format(PyExc_TypeError, - "'%.400s' decoder returned '%T' instead of 'str'; " + "'%.400s' decoder returned '%.400s' instead of 'str'; " "use codecs.decode() to decode to arbitrary types", - encoding, unicode); + encoding, + Py_TYPE(unicode)->tp_name); Py_DECREF(v); goto onError; } @@ -3496,9 +3489,10 @@ PyUnicode_AsEncodedString(PyObject *unicode, } PyErr_Format(PyExc_TypeError, - "'%.400s' encoder returned '%T' instead of 'bytes'; " + "'%.400s' encoder returned '%.400s' instead of 'bytes'; " "use codecs.encode() to encode to arbitrary types", - encoding, v); + encoding, + Py_TYPE(v)->tp_name); Py_DECREF(v); return NULL; } @@ -3529,9 +3523,10 @@ PyUnicode_AsEncodedUnicode(PyObject *unicode, goto onError; if (!PyUnicode_Check(v)) { PyErr_Format(PyExc_TypeError, - "'%.400s' encoder returned '%T' instead of 'str'; " + "'%.400s' encoder returned '%.400s' instead of 'str'; " "use codecs.encode() to encode to arbitrary types", - encoding, v); + encoding, + Py_TYPE(v)->tp_name); Py_DECREF(v); goto onError; } @@ -3703,11 +3698,9 @@ PyUnicode_FSDecoder(PyObject* arg, void* addr) if (!PyBytes_Check(path) && PyErr_WarnFormat(PyExc_DeprecationWarning, 1, - "path should be string, bytes, " - "or os.PathLike, not %T", - arg)) - { - Py_DECREF(path); + "path should be string, bytes, or os.PathLike, not %.200s", + Py_TYPE(arg)->tp_name)) { + Py_DECREF(path); return 0; } path_bytes = PyBytes_FromObject(path); @@ -3724,8 +3717,8 @@ PyUnicode_FSDecoder(PyObject* arg, void* addr) } else { PyErr_Format(PyExc_TypeError, - "path should be string, bytes, or os.PathLike, not %T", - arg); + "path should be string, bytes, or os.PathLike, not %.200s", + Py_TYPE(arg)->tp_name); Py_DECREF(path); return 0; } @@ -9893,8 +9886,9 @@ _PyUnicode_JoinArray(PyObject *separator, PyObject *const *items, Py_ssize_t seq else { if (!PyUnicode_Check(separator)) { PyErr_Format(PyExc_TypeError, - "separator: expected str instance, %T found", - separator); + "separator: expected str instance," + " %.80s found", + Py_TYPE(separator)->tp_name); goto onError; } if (PyUnicode_READY(separator)) @@ -9925,8 +9919,9 @@ _PyUnicode_JoinArray(PyObject *separator, PyObject *const *items, Py_ssize_t seq item = items[i]; if (!PyUnicode_Check(item)) { PyErr_Format(PyExc_TypeError, - "sequence item %zd: expected str instance, %T found", - i, item); + "sequence item %zd: expected str instance," + " %.80s found", + i, Py_TYPE(item)->tp_name); goto onError; } if (PyUnicode_READY(item) == -1) @@ -10741,7 +10736,7 @@ convert_uc(PyObject *obj, void *addr) if (!PyUnicode_Check(obj)) { PyErr_Format(PyExc_TypeError, "The fill character must be a unicode character, " - "not %T", obj); + "not %.100s", Py_TYPE(obj)->tp_name); return 0; } if (PyUnicode_READY(obj) < 0) @@ -11147,8 +11142,8 @@ PyUnicode_Contains(PyObject *str, PyObject *substr) if (!PyUnicode_Check(substr)) { PyErr_Format(PyExc_TypeError, - "'in ' requires string as left operand, not %T", - substr); + "'in ' requires string as left operand, not %.100s", + Py_TYPE(substr)->tp_name); return -1; } if (PyUnicode_READY(substr) == -1) @@ -12853,7 +12848,9 @@ unicode_split_impl(PyObject *self, PyObject *sep, Py_ssize_t maxsplit) if (PyUnicode_Check(sep)) return split(self, sep, maxsplit); - PyErr_Format(PyExc_TypeError, "must be str or None, not %T", sep); + PyErr_Format(PyExc_TypeError, + "must be str or None, not %.100s", + Py_TYPE(sep)->tp_name); return NULL; } @@ -13039,7 +13036,9 @@ unicode_rsplit_impl(PyObject *self, PyObject *sep, Py_ssize_t maxsplit) if (PyUnicode_Check(sep)) return rsplit(self, sep, maxsplit); - PyErr_Format(PyExc_TypeError, "must be str or None, not %T", sep); + PyErr_Format(PyExc_TypeError, + "must be str or None, not %.100s", + Py_TYPE(sep)->tp_name); return NULL; } @@ -13334,8 +13333,8 @@ unicode_startswith(PyObject *self, if (!PyUnicode_Check(substring)) { PyErr_Format(PyExc_TypeError, "tuple for startswith must only contain str, " - "not %T", - substring); + "not %.100s", + Py_TYPE(substring)->tp_name); return NULL; } result = tailmatch(self, substring, start, end, -1); @@ -13351,7 +13350,7 @@ unicode_startswith(PyObject *self, if (!PyUnicode_Check(subobj)) { PyErr_Format(PyExc_TypeError, "startswith first arg must be str or " - "a tuple of str, not %T", subobj); + "a tuple of str, not %.100s", Py_TYPE(subobj)->tp_name); return NULL; } result = tailmatch(self, subobj, start, end, -1); @@ -13388,8 +13387,8 @@ unicode_endswith(PyObject *self, if (!PyUnicode_Check(substring)) { PyErr_Format(PyExc_TypeError, "tuple for endswith must only contain str, " - "not %T", - substring); + "not %.100s", + Py_TYPE(substring)->tp_name); return NULL; } result = tailmatch(self, substring, start, end, +1); @@ -13404,7 +13403,7 @@ unicode_endswith(PyObject *self, if (!PyUnicode_Check(subobj)) { PyErr_Format(PyExc_TypeError, "endswith first arg must be str or " - "a tuple of str, not %T", subobj); + "a tuple of str, not %.100s", Py_TYPE(subobj)->tp_name); return NULL; } result = tailmatch(self, subobj, start, end, +1); @@ -14314,13 +14313,15 @@ mainformatlong(PyObject *v, case 'x': case 'X': PyErr_Format(PyExc_TypeError, - "%%%c format: an integer is required, not %T", - type, v); + "%%%c format: an integer is required, " + "not %.200s", + type, Py_TYPE(v)->tp_name); break; default: PyErr_Format(PyExc_TypeError, - "%%%c format: a number is required, not %T", - type, v); + "%%%c format: a number is required, " + "not %.200s", + type, Py_TYPE(v)->tp_name); break; } return -1; From webhook-mailer at python.org Tue Sep 11 18:30:00 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 22:30:00 -0000 Subject: [Python-checkins] Make sure the line comes from the same node as the col offset. (GH-9189) Message-ID: https://github.com/python/cpython/commit/d13e59c1b512069d90efe7ee9b613d3913e79c56 commit: d13e59c1b512069d90efe7ee9b613d3913e79c56 branch: master author: Benjamin Peterson committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T15:29:57-07:00 summary: Make sure the line comes from the same node as the col offset. (GH-9189) Followup to 90fc8980bbcc5c7dcced3627fe172b0bfd193a3b. files: M Python/ast.c diff --git a/Python/ast.c b/Python/ast.c index ef2c858ae4aa..94962e00c7d3 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -1602,7 +1602,7 @@ ast_for_funcdef_impl(struct compiling *c, const node *n0, if (is_async) return AsyncFunctionDef(name, args, body, decorator_seq, returns, - LINENO(n), n0->n_col_offset, c->c_arena); + LINENO(n0), n0->n_col_offset, c->c_arena); else return FunctionDef(name, args, body, decorator_seq, returns, LINENO(n), n->n_col_offset, c->c_arena); @@ -3719,7 +3719,7 @@ ast_for_for_stmt(struct compiling *c, const node *n0, bool is_async) if (is_async) return AsyncFor(target, expression, suite_seq, seq, - LINENO(n), n0->n_col_offset, + LINENO(n0), n0->n_col_offset, c->c_arena); else return For(target, expression, suite_seq, seq, @@ -3895,7 +3895,7 @@ ast_for_with_stmt(struct compiling *c, const node *n0, bool is_async) return NULL; if (is_async) - return AsyncWith(items, body, LINENO(n), n0->n_col_offset, c->c_arena); + return AsyncWith(items, body, LINENO(n0), n0->n_col_offset, c->c_arena); else return With(items, body, LINENO(n), n->n_col_offset, c->c_arena); } From webhook-mailer at python.org Tue Sep 11 18:45:38 2018 From: webhook-mailer at python.org (Ned Deily) Date: Tue, 11 Sep 2018 22:45:38 -0000 Subject: [Python-checkins] bpo-34405: Update to OpenSSL 1.0.2p for macOS installer builds (GH-9191) Message-ID: https://github.com/python/cpython/commit/31912b43c903aafad09350899ed6a9dec7c43421 commit: 31912b43c903aafad09350899ed6a9dec7c43421 branch: 3.6 author: Ned Deily committer: GitHub date: 2018-09-11T15:45:36-07:00 summary: bpo-34405: Update to OpenSSL 1.0.2p for macOS installer builds (GH-9191) files: A Misc/NEWS.d/next/macOS/2018-09-11-08-47-50.bpo-34405.f1-fT5.rst M Mac/BuildScript/build-installer.py diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index d76b2c096764..b97d55bb0306 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -162,7 +162,8 @@ def getTargetCompilers(): CC, CXX = getTargetCompilers() -PYTHON_3 = getVersionMajorMinor() >= (3, 0) +PYTHON_2 = getVersionMajorMinor()[0] == 2 +PYTHON_3 = getVersionMajorMinor()[0] == 3 USAGE = textwrap.dedent("""\ Usage: build_python [options] @@ -210,9 +211,9 @@ def library_recipes(): result.extend([ dict( - name="OpenSSL 1.0.2o", - url="https://www.openssl.org/source/openssl-1.0.2o.tar.gz", - checksum='44279b8557c3247cbe324e2322ecd114', + name="OpenSSL 1.0.2p", + url="https://www.openssl.org/source/openssl-1.0.2p.tar.gz", + checksum='ac5eb30bf5798aa14b1ae6d0e7da58df', buildrecipe=build_universal_openssl, configure=None, install=None, @@ -607,9 +608,10 @@ def checkEnvironment(): base_path = base_path + ':' + OLD_DEVELOPER_TOOLS os.environ['PATH'] = base_path print("Setting default PATH: %s"%(os.environ['PATH'])) - # Ensure we have access to sphinx-build. - # You may have to create a link in /usr/bin for it. - runCommand('sphinx-build --version') + if PYTHON_2: + # Ensure we have access to sphinx-build. + # You may have to define SDK_TOOLS_BIN and link to it there, + runCommand('sphinx-build --version') def parseOptions(args=None): """ @@ -822,6 +824,13 @@ def build_openssl_arch(archbase, arch): ] if no_asm: configure_opts.append("no-asm") + # OpenSSL 1.0.2o broke the Configure test for whether the compiler + # in use supports dependency rule generation (cc -M) with gcc-4.2 + # used for the 10.6+ installer builds. Patch Configure here to + # force use of "cc -M" rather than "makedepend". + runCommand( + """sed -i "" 's|my $cc_as_makedepend = 0|my $cc_as_makedepend = 1|g' Configure""") + runCommand(" ".join(["perl", "Configure"] + arch_opts[arch] + configure_opts)) runCommand("make depend") @@ -1055,9 +1064,14 @@ def buildPythonDocs(): curDir = os.getcwd() os.chdir(buildDir) runCommand('make clean') - # Create virtual environment for docs builds with blurb and sphinx - runCommand('make venv') - runCommand('make html PYTHON=venv/bin/python') + if PYTHON_2: + # Python 2 doc builds do not use blurb nor do they have a venv target. + # Assume sphinx-build is on our PATH, checked in checkEnvironment + runCommand('make html') + else: + # Create virtual environment for docs builds with blurb and sphinx + runCommand('make venv') + runCommand('make html PYTHON=venv/bin/python') os.chdir(curDir) if not os.path.exists(docdir): os.mkdir(docdir) diff --git a/Misc/NEWS.d/next/macOS/2018-09-11-08-47-50.bpo-34405.f1-fT5.rst b/Misc/NEWS.d/next/macOS/2018-09-11-08-47-50.bpo-34405.f1-fT5.rst new file mode 100644 index 000000000000..e9237004d236 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2018-09-11-08-47-50.bpo-34405.f1-fT5.rst @@ -0,0 +1 @@ +Update to OpenSSL 1.0.2p for macOS installer builds. From webhook-mailer at python.org Tue Sep 11 18:48:02 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 22:48:02 -0000 Subject: [Python-checkins] bpo-34636: Use fast path for more chars in SRE category macros. (GH-9170) Message-ID: https://github.com/python/cpython/commit/ec014a101a7f6243b95dfc08acfe1542b9fa5d39 commit: ec014a101a7f6243b95dfc08acfe1542b9fa5d39 branch: master author: Sergey Fedoseev committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T15:47:59-07:00 summary: bpo-34636: Use fast path for more chars in SRE category macros. (GH-9170) When handling \s, \d, or \w (and their inverse) escapes in bytes regexes this a small but measurable performance improvement. https://bugs.python.org/issue34636 files: A Misc/NEWS.d/next/Library/2018-09-11-15-04-05.bpo-34636.capCmt.rst M Modules/_sre.c diff --git a/Misc/NEWS.d/next/Library/2018-09-11-15-04-05.bpo-34636.capCmt.rst b/Misc/NEWS.d/next/Library/2018-09-11-15-04-05.bpo-34636.capCmt.rst new file mode 100644 index 000000000000..c982b0a4cda0 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-11-15-04-05.bpo-34636.capCmt.rst @@ -0,0 +1,2 @@ +Speed up re scanning of many non-matching characters for \s \w and \d within +bytes objects. (microoptimization) diff --git a/Modules/_sre.c b/Modules/_sre.c index d67083037e51..483cf5e9ff9c 100644 --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -87,13 +87,13 @@ static const char copyright[] = /* search engine state */ #define SRE_IS_DIGIT(ch)\ - ((ch) < 128 && Py_ISDIGIT(ch)) + ((ch) <= '9' && Py_ISDIGIT(ch)) #define SRE_IS_SPACE(ch)\ - ((ch) < 128 && Py_ISSPACE(ch)) + ((ch) <= ' ' && Py_ISSPACE(ch)) #define SRE_IS_LINEBREAK(ch)\ ((ch) == '\n') #define SRE_IS_WORD(ch)\ - ((ch) < 128 && (Py_ISALNUM(ch) || (ch) == '_')) + ((ch) <= 'z' && (Py_ISALNUM(ch) || (ch) == '_')) static unsigned int sre_lower_ascii(unsigned int ch) { From webhook-mailer at python.org Tue Sep 11 19:14:03 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 11 Sep 2018 23:14:03 -0000 Subject: [Python-checkins] closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) Message-ID: https://github.com/python/cpython/commit/4859ba0d2ce4506fddc3f55f90f8dce031b3804f commit: 4859ba0d2ce4506fddc3f55f90f8dce031b3804f branch: master author: Max B?langer committer: Benjamin Peterson date: 2018-09-11T16:14:00-07:00 summary: closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) files: A Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst M Modules/_scproxy.c diff --git a/Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst b/Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst new file mode 100644 index 000000000000..3788112cd730 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst @@ -0,0 +1,2 @@ +In :mod:`_scproxy`, drop the GIL when calling into ``SystemConfiguration`` to avoid +deadlocks. diff --git a/Modules/_scproxy.c b/Modules/_scproxy.c index 8861dc456d75..dbee3f7367ed 100644 --- a/Modules/_scproxy.c +++ b/Modules/_scproxy.c @@ -62,7 +62,10 @@ get_proxy_settings(PyObject* Py_UNUSED(mod), PyObject *Py_UNUSED(ignored)) PyObject* v; int r; + Py_BEGIN_ALLOW_THREADS proxyDict = SCDynamicStoreCopyProxies(NULL); + Py_END_ALLOW_THREADS + if (!proxyDict) { Py_RETURN_NONE; } @@ -172,7 +175,10 @@ get_proxies(PyObject* Py_UNUSED(mod), PyObject *Py_UNUSED(ignored)) int r; CFDictionaryRef proxyDict = NULL; + Py_BEGIN_ALLOW_THREADS proxyDict = SCDynamicStoreCopyProxies(NULL); + Py_END_ALLOW_THREADS + if (proxyDict == NULL) { return PyDict_New(); } From webhook-mailer at python.org Tue Sep 11 19:18:04 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 23:18:04 -0000 Subject: [Python-checkins] bpo-1621: Avoid signed integer overflow in set_table_resize(). (GH-9059) Message-ID: https://github.com/python/cpython/commit/6c7d67ce83a62b5f0fe5c53a6df602827451bf7f commit: 6c7d67ce83a62b5f0fe5c53a6df602827451bf7f branch: master author: Sergey Fedoseev committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T16:18:01-07:00 summary: bpo-1621: Avoid signed integer overflow in set_table_resize(). (GH-9059) Address a C undefined behavior signed integer overflow issue in set object table resizing. Our -fwrapv compiler flag and practical reasons why sets are unlikely to get this large should mean this was never an issue but it was incorrect code that generates code analysis warnings. https://bugs.python.org/issue1621 files: A Misc/NEWS.d/next/Core and Builtins/2018-09-11-15-19-37.bpo-1621.7o19yG.rst M Objects/setobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-11-15-19-37.bpo-1621.7o19yG.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-11-15-19-37.bpo-1621.7o19yG.rst new file mode 100644 index 000000000000..4047ff3bfe86 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-11-15-19-37.bpo-1621.7o19yG.rst @@ -0,0 +1,2 @@ +Do not assume signed integer overflow behavior (C undefined behavior) when +performing set hash table resizing. diff --git a/Objects/setobject.c b/Objects/setobject.c index 82b583820816..e7a528888ef0 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -302,7 +302,6 @@ actually be smaller than the old one. static int set_table_resize(PySetObject *so, Py_ssize_t minused) { - Py_ssize_t newsize; setentry *oldtable, *newtable, *entry; Py_ssize_t oldmask = so->mask; size_t newmask; @@ -313,13 +312,9 @@ set_table_resize(PySetObject *so, Py_ssize_t minused) /* Find the smallest table size > minused. */ /* XXX speed-up with intrinsics */ - for (newsize = PySet_MINSIZE; - newsize <= minused && newsize > 0; - newsize <<= 1) - ; - if (newsize <= 0) { - PyErr_NoMemory(); - return -1; + size_t newsize = PySet_MINSIZE; + while (newsize <= (size_t)minused) { + newsize <<= 1; // The largest possible value is PY_SSIZE_T_MAX + 1. } /* Get space for a new table. */ From webhook-mailer at python.org Tue Sep 11 19:30:07 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 23:30:07 -0000 Subject: [Python-checkins] Remove configure check LOG1P_DROPS_ZERO_SIGN. (GH-9193) Message-ID: https://github.com/python/cpython/commit/50c99d917c0eded2afcad0a3d7e49475a76440a3 commit: 50c99d917c0eded2afcad0a3d7e49475a76440a3 branch: master author: Benjamin Peterson committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T16:30:04-07:00 summary: Remove configure check LOG1P_DROPS_ZERO_SIGN. (GH-9193) It is unused. files: M aclocal.m4 M configure M configure.ac M pyconfig.h.in diff --git a/aclocal.m4 b/aclocal.m4 index 69205776ed4c..6a24d8e6b9c0 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -1,6 +1,6 @@ -# generated automatically by aclocal 1.15 -*- Autoconf -*- +# generated automatically by aclocal 1.15.1 -*- Autoconf -*- -# Copyright (C) 1996-2014 Free Software Foundation, Inc. +# Copyright (C) 1996-2017 Free Software Foundation, Inc. # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, diff --git a/configure b/configure index b4a70e6736bb..9520446d5d45 100755 --- a/configure +++ b/configure @@ -13688,7 +13688,6 @@ fi - # checks for system services # (none yet) @@ -14209,56 +14208,6 @@ then $as_echo "#define TANH_PRESERVES_ZERO_SIGN 1" >>confdefs.h fi - -if test "$ac_cv_func_log1p" = yes -then - # On some versions of AIX, log1p(-0.) returns 0. instead of - # -0. See issue #9920. - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether log1p drops the sign of negative zero" >&5 -$as_echo_n "checking whether log1p drops the sign of negative zero... " >&6; } - if ${ac_cv_log1p_drops_zero_sign+:} false; then : - $as_echo_n "(cached) " >&6 -else - - if test "$cross_compiling" = yes; then : - ac_cv_log1p_drops_zero_sign=no -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - - #include - #include - int main() { - /* Fail if the signs of log1p(-0.) and -0. can be - distinguished. */ - if (atan2(log1p(-0.), -1.) == atan2(-0., -1.)) - return 0; - else - return 1; - } - -_ACEOF -if ac_fn_c_try_run "$LINENO"; then : - ac_cv_log1p_drops_zero_sign=no -else - ac_cv_log1p_drops_zero_sign=yes -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ - conftest.$ac_objext conftest.beam conftest.$ac_ext -fi - -fi - - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_log1p_drops_zero_sign" >&5 -$as_echo "$ac_cv_log1p_drops_zero_sign" >&6; } -fi -if test "$ac_cv_log1p_drops_zero_sign" = yes -then - -$as_echo "#define LOG1P_DROPS_ZERO_SIGN 1" >>confdefs.h - -fi - LIBS=$LIBS_SAVE # For multiprocessing module, check that sem_open diff --git a/configure.ac b/configure.ac index e47586bd403c..4b34bad35a1d 100644 --- a/configure.ac +++ b/configure.ac @@ -4384,36 +4384,6 @@ then AC_DEFINE(TANH_PRESERVES_ZERO_SIGN, 1, [Define if tanh(-0.) is -0., or if platform doesn't have signed zeros]) fi - -if test "$ac_cv_func_log1p" = yes -then - # On some versions of AIX, log1p(-0.) returns 0. instead of - # -0. See issue #9920. - AC_MSG_CHECKING(whether log1p drops the sign of negative zero) - AC_CACHE_VAL(ac_cv_log1p_drops_zero_sign, [ - AC_RUN_IFELSE([AC_LANG_SOURCE([[ - #include - #include - int main() { - /* Fail if the signs of log1p(-0.) and -0. can be - distinguished. */ - if (atan2(log1p(-0.), -1.) == atan2(-0., -1.)) - return 0; - else - return 1; - } - ]])], - [ac_cv_log1p_drops_zero_sign=no], - [ac_cv_log1p_drops_zero_sign=yes], - [ac_cv_log1p_drops_zero_sign=no])]) - AC_MSG_RESULT($ac_cv_log1p_drops_zero_sign) -fi -if test "$ac_cv_log1p_drops_zero_sign" = yes -then - AC_DEFINE(LOG1P_DROPS_ZERO_SIGN, 1, - [Define if log1p(-0.) is 0. rather than -0.]) -fi - LIBS=$LIBS_SAVE # For multiprocessing module, check that sem_open diff --git a/pyconfig.h.in b/pyconfig.h.in index 4d3a4b995fe3..51e9213826ae 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -1285,9 +1285,6 @@ /* Define to 1 if you have the `_getpty' function. */ #undef HAVE__GETPTY -/* Define if log1p(-0.) is 0. rather than -0. */ -#undef LOG1P_DROPS_ZERO_SIGN - /* Define to 1 if `major', `minor', and `makedev' are declared in . */ #undef MAJOR_IN_MKDEV From webhook-mailer at python.org Tue Sep 11 19:32:57 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 23:32:57 -0000 Subject: [Python-checkins] Make sure the line comes from the same node as the col offset. (GH-9189) Message-ID: https://github.com/python/cpython/commit/4007e4e207bb6cf934864a9012a14022474e36df commit: 4007e4e207bb6cf934864a9012a14022474e36df branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T16:32:52-07:00 summary: Make sure the line comes from the same node as the col offset. (GH-9189) Followup to 90fc8980bbcc5c7dcced3627fe172b0bfd193a3b. (cherry picked from commit d13e59c1b512069d90efe7ee9b613d3913e79c56) Co-authored-by: Benjamin Peterson files: M Python/ast.c diff --git a/Python/ast.c b/Python/ast.c index 5f710c710acb..07227c238ef4 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -1602,7 +1602,7 @@ ast_for_funcdef_impl(struct compiling *c, const node *n0, if (is_async) return AsyncFunctionDef(name, args, body, decorator_seq, returns, - LINENO(n), n0->n_col_offset, c->c_arena); + LINENO(n0), n0->n_col_offset, c->c_arena); else return FunctionDef(name, args, body, decorator_seq, returns, LINENO(n), n->n_col_offset, c->c_arena); @@ -3719,7 +3719,7 @@ ast_for_for_stmt(struct compiling *c, const node *n0, bool is_async) if (is_async) return AsyncFor(target, expression, suite_seq, seq, - LINENO(n), n0->n_col_offset, + LINENO(n0), n0->n_col_offset, c->c_arena); else return For(target, expression, suite_seq, seq, @@ -3895,7 +3895,7 @@ ast_for_with_stmt(struct compiling *c, const node *n0, bool is_async) return NULL; if (is_async) - return AsyncWith(items, body, LINENO(n), n0->n_col_offset, c->c_arena); + return AsyncWith(items, body, LINENO(n0), n0->n_col_offset, c->c_arena); else return With(items, body, LINENO(n), n->n_col_offset, c->c_arena); } From webhook-mailer at python.org Tue Sep 11 19:40:09 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 23:40:09 -0000 Subject: [Python-checkins] bpo-34605: Replace "pliant children" with "helpers" (GH-9195) Message-ID: https://github.com/python/cpython/commit/fa7dfae3171914f91d629a64c6e829788b485b06 commit: fa7dfae3171914f91d629a64c6e829788b485b06 branch: master author: Victor Stinner committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T16:40:06-07:00 summary: bpo-34605: Replace "pliant children" with "helpers" (GH-9195) In distutils.command.install, replace "pliant children" (previously, it was "pliant slaves") with "helpers". https://bugs.python.org/issue34605 files: M Lib/distutils/command/install.py diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py index a1d1a1ea37ad..c625c95bf7e8 100644 --- a/Lib/distutils/command/install.py +++ b/Lib/distutils/command/install.py @@ -223,7 +223,7 @@ def initialize_options(self): def finalize_options(self): """Finalizes options.""" - # This method (and its pliant children, like 'finalize_unix()', + # This method (and its helpers, like 'finalize_unix()', # 'finalize_other()', and 'select_scheme()') is where the default # installation directories for modules, extension modules, and # anything else we care to install from a Python module From webhook-mailer at python.org Tue Sep 11 19:53:24 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 11 Sep 2018 23:53:24 -0000 Subject: [Python-checkins] [2.7] bpo-34405: Update to OpenSSL 1.0.2p for macOS installer builds (GH-9191) (GH-9196) Message-ID: https://github.com/python/cpython/commit/aa02ab15aed9d0b70f619c186326594c0d50bdd6 commit: aa02ab15aed9d0b70f619c186326594c0d50bdd6 branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T16:53:20-07:00 summary: [2.7] bpo-34405: Update to OpenSSL 1.0.2p for macOS installer builds (GH-9191) (GH-9196) (cherry picked from commit 31912b43c903aafad09350899ed6a9dec7c43421) Co-authored-by: Ned Deily https://bugs.python.org/issue34405 files: A Misc/NEWS.d/next/macOS/2018-09-11-08-47-50.bpo-34405.f1-fT5.rst M Mac/BuildScript/build-installer.py diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index 038e1917c4e1..b97d55bb0306 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -211,9 +211,9 @@ def library_recipes(): result.extend([ dict( - name="OpenSSL 1.0.2o", - url="https://www.openssl.org/source/openssl-1.0.2o.tar.gz", - checksum='44279b8557c3247cbe324e2322ecd114', + name="OpenSSL 1.0.2p", + url="https://www.openssl.org/source/openssl-1.0.2p.tar.gz", + checksum='ac5eb30bf5798aa14b1ae6d0e7da58df', buildrecipe=build_universal_openssl, configure=None, install=None, @@ -824,6 +824,13 @@ def build_openssl_arch(archbase, arch): ] if no_asm: configure_opts.append("no-asm") + # OpenSSL 1.0.2o broke the Configure test for whether the compiler + # in use supports dependency rule generation (cc -M) with gcc-4.2 + # used for the 10.6+ installer builds. Patch Configure here to + # force use of "cc -M" rather than "makedepend". + runCommand( + """sed -i "" 's|my $cc_as_makedepend = 0|my $cc_as_makedepend = 1|g' Configure""") + runCommand(" ".join(["perl", "Configure"] + arch_opts[arch] + configure_opts)) runCommand("make depend") diff --git a/Misc/NEWS.d/next/macOS/2018-09-11-08-47-50.bpo-34405.f1-fT5.rst b/Misc/NEWS.d/next/macOS/2018-09-11-08-47-50.bpo-34405.f1-fT5.rst new file mode 100644 index 000000000000..e9237004d236 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2018-09-11-08-47-50.bpo-34405.f1-fT5.rst @@ -0,0 +1 @@ +Update to OpenSSL 1.0.2p for macOS installer builds. From webhook-mailer at python.org Tue Sep 11 20:10:40 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Wed, 12 Sep 2018 00:10:40 -0000 Subject: [Python-checkins] bpo-33649: Cleanup asyncio/streams and asyncio/synchronization docs (GH-9192) Message-ID: https://github.com/python/cpython/commit/8be876e44b2dffb96b551029454cbfb16d697992 commit: 8be876e44b2dffb96b551029454cbfb16d697992 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-11T17:10:37-07:00 summary: bpo-33649: Cleanup asyncio/streams and asyncio/synchronization docs (GH-9192) files: M Doc/library/asyncio-stream.rst M Doc/library/asyncio-sync.rst M Doc/whatsnew/3.7.rst diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 27b5205f1c5b..0cfecda91e99 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -7,10 +7,10 @@ Streams ======= Streams are high-level async/await-ready primitives to work with -network connections. Streams allow send and receive data without +network connections. Streams allow sending and receiving data without using callbacks or low-level protocols and transports. -Here's an example of a TCP echo client written using asyncio +Here is an example of a TCP echo client written using asyncio streams:: import asyncio @@ -31,6 +31,9 @@ streams:: asyncio.run(tcp_echo_client('Hello World!')) +See also the `Examples`_ section below. + + .. rubric:: Stream Functions The following top-level asyncio functions can be used to create @@ -43,7 +46,7 @@ and work with streams: server_hostname=None, ssl_handshake_timeout=None) Establish a network connection and return a pair of - ``(reader, writer)``. + ``(reader, writer)`` objects. The returned *reader* and *writer* objects are instances of :class:`StreamReader` and :class:`StreamWriter` classes. @@ -52,7 +55,8 @@ and work with streams: automatically when this method is awaited from a coroutine. *limit* determines the buffer size limit used by the - returned :class:`StreamReader` instance. + returned :class:`StreamReader` instance. By default the *limit* + is set to 64 KiB. The rest of the arguments are passed directly to :meth:`loop.create_connection`. @@ -84,7 +88,8 @@ and work with streams: automatically when this method is awaited from a coroutine. *limit* determines the buffer size limit used by the - returned :class:`StreamReader` instance. + returned :class:`StreamReader` instance. By default the *limit* + is set to 64 KiB. The rest of the arguments are passed directly to :meth:`loop.create_server`. @@ -93,6 +98,9 @@ and work with streams: The *ssl_handshake_timeout* and *start_serving* parameters. + +.. rubric:: Unix Sockets + .. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, \ limit=None, ssl=None, sock=None, \ server_hostname=None, ssl_handshake_timeout=None) @@ -114,6 +122,7 @@ and work with streams: The *path* parameter can now be a :term:`path-like object` + .. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \ \*, loop=None, limit=None, sock=None, \ backlog=100, ssl=None, ssl_handshake_timeout=None, \ @@ -121,7 +130,7 @@ and work with streams: Start a UNIX socket server. - Similar to :func:`start_server` but operates on UNIX sockets. + Similar to :func:`start_server` but works with UNIX sockets. See also the documentation of :meth:`loop.create_unix_server`. @@ -136,67 +145,47 @@ and work with streams: The *path* parameter can now be a :term:`path-like object`. -.. rubric:: Contents - -* `StreamReader`_ and `StreamWriter`_ -* `StreamReaderProtocol`_ -* `Examples`_ +--------- StreamReader ============ -.. class:: StreamReader(limit=_DEFAULT_LIMIT, loop=None) - - This class is :ref:`not thread safe `. - - The *limit* argument's default value is set to _DEFAULT_LIMIT which is 2**16 (64 KiB) - - .. method:: exception() - - Get the exception. - - .. method:: feed_eof() +.. class:: StreamReader - Acknowledge the EOF. + Represents a reader object that provides APIs to read data + from the IO stream. - .. method:: feed_data(data) - - Feed *data* bytes in the internal buffer. Any operations waiting - for the data will be resumed. - - .. method:: set_exception(exc) - - Set the exception. - - .. method:: set_transport(transport) - - Set the transport. + It is not recommended to instantiate *StreamReader* objects + directly; use :func:`open_connection` and :func:`start_server` + instead. .. coroutinemethod:: read(n=-1) Read up to *n* bytes. If *n* is not provided, or set to ``-1``, read until EOF and return all read bytes. - If the EOF was received and the internal buffer is empty, + If an EOF was received and the internal buffer is empty, return an empty ``bytes`` object. .. coroutinemethod:: readline() - Read one line, where "line" is a sequence of bytes ending with ``\n``. + Read one line, where "line" is a sequence of bytes + ending with ``\n``. - If EOF is received, and ``\n`` was not found, the method will - return the partial read bytes. + If an EOF is received and ``\n`` was not found, the method + returns partially read data. - If the EOF was received and the internal buffer is empty, + If an EOF is received and the internal buffer is empty, return an empty ``bytes`` object. .. coroutinemethod:: readexactly(n) - Read exactly *n* bytes. Raise an :exc:`IncompleteReadError` if the end of - the stream is reached before *n* can be read, the - :attr:`IncompleteReadError.partial` attribute of the exception contains - the partial read bytes. + Read exactly *n* bytes. + + Raise an :exc:`IncompleteReadError` if an EOF reached before *n* + can be read. Use the :attr:`IncompleteReadError.partial` + attribute to get the partially read data. .. coroutinemethod:: readuntil(separator=b'\\n') @@ -231,105 +220,76 @@ StreamReader StreamWriter ============ -.. class:: StreamWriter(transport, protocol, reader, loop) +.. class:: StreamWriter - Wraps a Transport. + Represents a writer object that provides APIs to write data + to the IO stream. - This exposes :meth:`write`, :meth:`writelines`, :meth:`can_write_eof()`, - :meth:`write_eof`, :meth:`get_extra_info` and :meth:`close`. It adds - :meth:`drain` which returns an optional :class:`Future` on which you can - wait for flow control. It also adds a transport attribute which references - the :class:`Transport` directly. + It is not recommended to instantiate *StreamWriter* objects + directly; use :func:`open_connection` and :func:`start_server` + instead. - This class is :ref:`not thread safe `. + .. method:: write(data) - .. attribute:: transport + Write *data* to the stream. - Transport. + .. method:: writelines(data) - .. method:: can_write_eof() + Write a list (or any iterable) of bytes to the stream. + + .. coroutinemethod:: drain() + + Wait until it is appropriate to resume writing to the stream. + E.g.:: + + writer.write(data) + await writer.drain() - Return :const:`True` if the transport supports :meth:`write_eof`, - :const:`False` if not. See :meth:`WriteTransport.can_write_eof`. + This is a flow-control method that interacts with the underlying + IO write buffer. When the size of the buffer reaches + the high-water limit, *drain()* blocks until the size of the + buffer is drained down to the low-water limit and writing can + be resumed. When there is nothing to wait for, the :meth:`drain` + returns immediately. .. method:: close() - Close the transport: see :meth:`BaseTransport.close`. + Close the stream. .. method:: is_closing() - Return ``True`` if the writer is closing or is closed. + Return ``True`` if the stream is closed or in the process of + being closed. .. versionadded:: 3.7 .. coroutinemethod:: wait_closed() - Wait until the writer is closed. + Wait until the stream is closed. - Should be called after :meth:`close` to wait until the underlying - connection (and the associated transport/protocol pair) is closed. + Should be called after :meth:`close` to wait until the underlying + connection is closed. .. versionadded:: 3.7 - .. coroutinemethod:: drain() - - Let the write buffer of the underlying transport a chance to be flushed. - - The intended use is to write:: - - w.write(data) - await w.drain() - - When the size of the transport buffer reaches the high-water limit (the - protocol is paused), block until the size of the buffer is drained down - to the low-water limit and the protocol is resumed. When there is nothing - to wait for, the yield-from continues immediately. - - Yielding from :meth:`drain` gives the opportunity for the loop to - schedule the write operation and flush the buffer. It should especially - be used when a possibly large amount of data is written to the transport, - and the coroutine does not yield-from between calls to :meth:`write`. - - This method is a :ref:`coroutine `. - - .. method:: get_extra_info(name, default=None) - - Return optional transport information: see - :meth:`BaseTransport.get_extra_info`. - - .. method:: write(data) - - Write some *data* bytes to the transport: see - :meth:`WriteTransport.write`. - - .. method:: writelines(data) + .. method:: can_write_eof() - Write a list (or any iterable) of data bytes to the transport: - see :meth:`WriteTransport.writelines`. + Return *True* if the underlying transport supports + the :meth:`write_eof` method, *False* otherwise. .. method:: write_eof() - Close the write end of the transport after flushing buffered data: - see :meth:`WriteTransport.write_eof`. - - -StreamReaderProtocol -==================== + Close the write end of the stream after the buffered write + data is flushed. -.. class:: StreamReaderProtocol(stream_reader, client_connected_cb=None, \ - loop=None) + .. attribute:: transport - Trivial helper class to adapt between :class:`Protocol` and - :class:`StreamReader`. Subclass of :class:`Protocol`. + Return the underlying asyncio transport. - *stream_reader* is a :class:`StreamReader` instance, *client_connected_cb* - is an optional function called with (stream_reader, stream_writer) when a - connection is made, *loop* is the event loop instance to use. + .. method:: get_extra_info(name, default=None) - (This is a helper class instead of making :class:`StreamReader` itself a - :class:`Protocol` subclass, because the :class:`StreamReader` has other - potential uses, and to prevent the user of the :class:`StreamReader` from - accidentally calling inappropriate methods of the protocol.) + Access optional transport information; see + :meth:`BaseTransport.get_extra_info` for details. Examples diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index 574f70f069b3..8e01ca944083 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -1,172 +1,204 @@ .. currentmodule:: asyncio + .. _asyncio-sync: -Synchronization primitives ========================== +Synchronization Primitives +========================== + +asyncio synchronization primitives are designed to be similar to +those of the :mod:`threading` module with two important caveats: + +* asyncio primitives are not thread-safe, therefore they should not + be used for OS threads synchronization (use :mod:`threading` for + that); -**Source code:** :source:`Lib/asyncio/locks.py` +* methods of synchronization objects do not accept the *timeout* + argument; use the :func:`asyncio.wait_for` function to perform + operations with timeouts. -Locks: +asyncio has the following basic primitives: * :class:`Lock` * :class:`Event` * :class:`Condition` - -Semaphores: - * :class:`Semaphore` * :class:`BoundedSemaphore` -asyncio lock API was designed to be close to classes of the :mod:`threading` -module (:class:`~threading.Lock`, :class:`~threading.Event`, -:class:`~threading.Condition`, :class:`~threading.Semaphore`, -:class:`~threading.BoundedSemaphore`), but it has no *timeout* parameter. The -:func:`asyncio.wait_for` function can be used to cancel a task after a timeout. + +--------- Lock ----- +==== .. class:: Lock(\*, loop=None) - Primitive lock objects. + Implements a mutex lock for asyncio tasks. Not thread-safe. - A primitive lock is a synchronization primitive that is not owned by a - particular coroutine when locked. A primitive lock is in one of two states, - 'locked' or 'unlocked'. + An asyncio lock can be used to guarantee exclusive access to a + shared resource. - The lock is created in the unlocked state. - It has two basic methods, :meth:`acquire` and :meth:`release`. - When the state is unlocked, acquire() changes the state to - locked and returns immediately. When the state is locked, acquire() blocks - until a call to release() in another coroutine changes it to unlocked, then - the acquire() call resets it to locked and returns. The release() method - should only be called in the locked state; it changes the state to unlocked - and returns immediately. If an attempt is made to release an unlocked lock, - a :exc:`RuntimeError` will be raised. + The preferred way to use a Lock is an :keyword:`async with` + statement:: - When more than one coroutine is blocked in acquire() waiting for the state - to turn to unlocked, only one coroutine proceeds when a release() call - resets the state to unlocked; first coroutine which is blocked in acquire() - is being processed. + lock = asyncio.Lock() - :meth:`acquire` is a coroutine and should be called with ``await``. + # ... later + async with lock: + # access shared state - Locks support the :ref:`context management protocol `. + which is equivalent to:: - This class is :ref:`not thread safe `. + lock = asyncio.Lock() - .. method:: locked() - - Return ``True`` if the lock is acquired. + # ... later + await lock.acquire() + try: + # access shared state + finally: + lock.release() .. coroutinemethod:: acquire() - Acquire a lock. - - This method blocks until the lock is unlocked, then sets it to locked and - returns ``True``. + Acquire the lock. - This method is a :ref:`coroutine `. + This method waits until the lock is *unlocked*, sets it to + *locked* and returns ``True``. .. method:: release() - Release a lock. + Release the lock. - When the lock is locked, reset it to unlocked, and return. If any other - coroutines are blocked waiting for the lock to become unlocked, allow - exactly one of them to proceed. + When the lock is *locked*, reset it to *unlocked* and return. - When invoked on an unlocked lock, a :exc:`RuntimeError` is raised. + If the lock is *unlocked* a :exc:`RuntimeError` is raised. - There is no return value. + .. method:: locked() + + Return ``True`` if the lock is *locked*. Event ------ +===== .. class:: Event(\*, loop=None) - An Event implementation, asynchronous equivalent to :class:`threading.Event`. + An event object. Not thread-safe. - Class implementing event objects. An event manages a flag that can be set to - true with the :meth:`set` method and reset to false with the :meth:`clear` - method. The :meth:`wait` method blocks until the flag is true. The flag is - initially false. + An asyncio event can be used to notify multiple asyncio tasks + that some event has happened. - This class is :ref:`not thread safe `. + An Event object manages an internal flag that can be set to *true* + with the :meth:`set` method and reset to *false* with the + :meth:`clear` method. The :meth:`wait` method blocks until the + flag is set to *true*. The flag is set to *false* initially. - .. method:: clear() + Example:: - Reset the internal flag to false. Subsequently, coroutines calling - :meth:`wait` will block until :meth:`set` is called to set the internal - flag to true again. + async def waiter(event): + print('waiting ...') + await event.wait() + print('... got it!') - .. method:: is_set() + async def main(): + # Create an Event object. + event = asyncio.Event() - Return ``True`` if and only if the internal flag is true. + # Spawn a Task to wait until 'event' is set. + waiter_task = asyncio.create_task(waiter(event)) - .. method:: set() + # Sleep for 1 second and set the event. + await asyncio.sleep(1) + event.set() + + # Wait until the waiter task is finished. + await waiter_task - Set the internal flag to true. All coroutines waiting for it to become - true are awakened. Coroutine that call :meth:`wait` once the flag is true - will not block at all. + asyncio.run(main()) .. coroutinemethod:: wait() - Block until the internal flag is true. + Wait until the event is set. + + If the event is set, return ``True`` immediately. + Otherwise block until another task calls :meth:`set`. + + .. method:: set() + + Set the event. - If the internal flag is true on entry, return ``True`` immediately. - Otherwise, block until another coroutine calls :meth:`set` to set the - flag to true, then return ``True``. + All tasks waiting for event to be set will be immediately + awakened. + + .. method:: clear() - This method is a :ref:`coroutine `. + Clear (unset) the event. + + Tasks awaiting on :meth:`wait` will now block until the + :meth:`set` method is called again. + + .. method:: is_set() + + Return ``True`` if the event is set. Condition ---------- +========= .. class:: Condition(lock=None, \*, loop=None) - A Condition implementation, asynchronous equivalent to - :class:`threading.Condition`. + A Condition object. Not thread-safe. - This class implements condition variable objects. A condition variable - allows one or more coroutines to wait until they are notified by another - coroutine. + An asyncio condition primitive can be used by a task to wait for + some event to happen and then get an exclusive access to a shared + resource. - If the *lock* argument is given and not ``None``, it must be a :class:`Lock` - object, and it is used as the underlying lock. Otherwise, - a new :class:`Lock` object is created and used as the underlying lock. + In essence, a Condition object combines the functionality + of :class:`Event` and :class:`Lock`. It is possible to have many + Condition objects sharing one Lock, which allows to coordinate + exclusive access to a shared resource between different tasks + interested in particular states of that shared resource. - Conditions support the :ref:`context management protocol - `. + The optional *lock* argument must be a :class:`Lock` object or + ``None``. In the latter case a new Lock object is created + automatically. - This class is :ref:`not thread safe `. + The preferred way to use a Condition is an :keyword:`async with` + statement:: - .. coroutinemethod:: acquire() + cond = asyncio.Condition() - Acquire the underlying lock. + # ... later + async with cond: + await cond.wait() - This method blocks until the lock is unlocked, then sets it to locked and - returns ``True``. + which is equivalent to:: - This method is a :ref:`coroutine `. + cond = asyncio.Condition() - .. method:: notify(n=1) + # ... later + await lock.acquire() + try: + await cond.wait() + finally: + lock.release() - By default, wake up one coroutine waiting on this condition, if any. - If the calling coroutine has not acquired the lock when this method is - called, a :exc:`RuntimeError` is raised. + .. coroutinemethod:: acquire() - This method wakes up at most *n* of the coroutines waiting for the - condition variable; it is a no-op if no coroutines are waiting. + Acquire the underlying lock. + + This method waits until the underlying lock is *unlocked*, + sets it to *locked* and returns ``True``. + + .. method:: notify(n=1) - .. note:: + Wake up at most *n* tasks (1 by default) waiting on this + condition. The method is no-op if no tasks are waiting. - An awakened coroutine does not actually return from its :meth:`wait` - call until it can reacquire the lock. Since :meth:`notify` does not - release the lock, its caller should. + The lock must be acquired before this method is called and + released shortly after. If called with an *unlocked* lock + a :exc:`RuntimeError` error is raised. .. method:: locked() @@ -174,78 +206,87 @@ Condition .. method:: notify_all() - Wake up all coroutines waiting on this condition. This method acts like - :meth:`notify`, but wakes up all waiting coroutines instead of one. If the - calling coroutine has not acquired the lock when this method is called, a - :exc:`RuntimeError` is raised. + Wake up all tasks waiting on this condition. - .. method:: release() + This method acts like :meth:`notify`, but wakes up all waiting + tasks. - Release the underlying lock. + The lock must be acquired before this method is called and + released shortly after. If called with an *unlocked* lock + a :exc:`RuntimeError` error is raised. - When the lock is locked, reset it to unlocked, and return. If any other - coroutines are blocked waiting for the lock to become unlocked, allow - exactly one of them to proceed. + .. method:: release() - When invoked on an unlocked lock, a :exc:`RuntimeError` is raised. + Release the underlying lock. - There is no return value. + When invoked on an unlocked lock, a :exc:`RuntimeError` is + raised. .. coroutinemethod:: wait() Wait until notified. - If the calling coroutine has not acquired the lock when this method is + If the calling task has not acquired the lock when this method is called, a :exc:`RuntimeError` is raised. - This method releases the underlying lock, and then blocks until it is - awakened by a :meth:`notify` or :meth:`notify_all` call for the same - condition variable in another coroutine. Once awakened, it re-acquires - the lock and returns ``True``. - - This method is a :ref:`coroutine `. + This method releases the underlying lock, and then blocks until + it is awakened by a :meth:`notify` or :meth:`notify_all` call. + Once awakened, the Condition re-acquires its lock and this method + returns ``True``. .. coroutinemethod:: wait_for(predicate) - Wait until a predicate becomes true. - - The predicate should be a callable which result will be interpreted as a - boolean value. The final predicate value is the return value. + Wait until a predicate becomes *true*. - This method is a :ref:`coroutine `. + The predicate must be a callable which result will be + interpreted as a boolean value. The final value is the + return value. Semaphore ---------- +========= .. class:: Semaphore(value=1, \*, loop=None) - A Semaphore implementation. + A Semaphore object. Not thread-safe. A semaphore manages an internal counter which is decremented by each - :meth:`acquire` call and incremented by each :meth:`release` call. The - counter can never go below zero; when :meth:`acquire` finds that it is zero, - it blocks, waiting until some other coroutine calls :meth:`release`. + :meth:`acquire` call and incremented by each :meth:`release` call. + The counter can never go below zero; when :meth:`acquire` finds + that it is zero, it blocks, waiting until some task calls + :meth:`release`. + + The optional *value* argument gives the initial value for the + internal counter (``1`` by default). If the given value is + less than ``0`` a :exc:`ValueError` is raised. + + The preferred way to use a Semaphore is an :keyword:`async with` + statement:: + + sem = asyncio.Semaphore(10) - The optional argument gives the initial value for the internal counter; it - defaults to ``1``. If the value given is less than ``0``, :exc:`ValueError` - is raised. + # ... later + async with sem: + # work with shared resource - Semaphores support the :ref:`context management protocol - `. + which is equivalent to:: - This class is :ref:`not thread safe `. + sem = asyncio.Semaphore(10) + + # ... later + await sem.acquire() + try: + # work with shared resource + finally: + sem.release() .. coroutinemethod:: acquire() Acquire a semaphore. - If the internal counter is larger than zero on entry, decrement it by one - and return ``True`` immediately. If it is zero on entry, block, waiting - until some other coroutine has called :meth:`release` to make it larger - than ``0``, and then return ``True``. - - This method is a :ref:`coroutine `. + If the internal counter is greater than zero, decrement + it by one and return ``True`` immediately. If it is zero wait + until a :meth:`release` is called and return ``True``. .. method:: locked() @@ -253,53 +294,30 @@ Semaphore .. method:: release() - Release a semaphore, incrementing the internal counter by one. When it - was zero on entry and another coroutine is waiting for it to become - larger than zero again, wake up that coroutine. + Release a semaphore, incrementing the internal counter by one. + Can wake up a task waiting to acquire the semaphore. + + Unlike :class:`BoundedSemaphore`, :class:`Semaphore` allows + to make more ``release()`` calls than ``acquire()`` calls. BoundedSemaphore ----------------- +================ .. class:: BoundedSemaphore(value=1, \*, loop=None) - A bounded semaphore implementation. Inherit from :class:`Semaphore`. - - This raises :exc:`ValueError` in :meth:`~Semaphore.release` if it would - increase the value above the initial value. + A bounded semaphore object. Not thread-safe. - Bounded semaphores support the :ref:`context management - protocol `. + Bounded Semaphore is a version of :class:`Semaphore` that raises + a :exc:`ValueError` in :meth:`~Semaphore.release` if it + increases the internal counter above the initial *value*. - This class is :ref:`not thread safe `. +--------- -.. _async-with-locks: - -Using locks, conditions and semaphores in the :keyword:`async with` statement ------------------------------------------------------------------------------ - -:class:`Lock`, :class:`Condition`, :class:`Semaphore`, and -:class:`BoundedSemaphore` objects can be used in :keyword:`async with` -statements. - -The :meth:`acquire` method will be called when the block is entered, -and :meth:`release` will be called when the block is exited. Hence, -the following snippet:: - - async with lock: - # do something... - -is equivalent to:: - - await lock.acquire() - try: - # do something... - finally: - lock.release() .. deprecated:: 3.7 - Lock acquiring using ``await lock`` or ``yield from lock`` and + Acquiring a lock using ``await lock`` or ``yield from lock`` and/or :keyword:`with` statement (``with await lock``, ``with (yield from - lock)``) are deprecated. + lock)``) is deprecated. Use ``async with lock`` instead. diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index fbaa2cf15f94..be4fef181917 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1923,8 +1923,7 @@ asyncio Support for directly ``await``-ing instances of :class:`asyncio.Lock` and other asyncio synchronization primitives has been deprecated. An asynchronous context manager must be used in order to acquire and release -the synchronization resource. See :ref:`async-with-locks` for more -information. +the synchronization resource. (Contributed by Andrew Svetlov in :issue:`32253`.) The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` From webhook-mailer at python.org Tue Sep 11 20:17:42 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 00:17:42 -0000 Subject: [Python-checkins] Delete old expat comment. (GH-9197) Message-ID: https://github.com/python/cpython/commit/b9bf9d025e659b5a1963027eb73690e57cb35dd0 commit: b9bf9d025e659b5a1963027eb73690e57cb35dd0 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-11T17:17:39-07:00 summary: Delete old expat comment. (GH-9197) files: M Modules/Setup diff --git a/Modules/Setup b/Modules/Setup index e791c8cf8ac6..9787ea2d2c48 100644 --- a/Modules/Setup +++ b/Modules/Setup @@ -335,14 +335,6 @@ _symtable symtablemodule.c #zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz # Interface to the Expat XML parser -# -# Expat was written by James Clark and is now maintained by a group of -# developers on SourceForge; see www.libexpat.org for more -# information. The pyexpat module was written by Paul Prescod after a -# prototype by Jack Jansen. Source of Expat 1.95.2 is included in -# Modules/expat/. Usage of a system shared libexpat.so/expat.dll is -# not advised. -# # More information on Expat can be found at www.libexpat.org. # #pyexpat expat/xmlparse.c expat/xmlrole.c expat/xmltok.c pyexpat.c -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI From webhook-mailer at python.org Tue Sep 11 20:32:18 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 00:32:18 -0000 Subject: [Python-checkins] closes bpo-25041: Document AF_PACKET socket address format. (GH-4092) Message-ID: https://github.com/python/cpython/commit/731ff68eeef58babdf2b32dc9a73b141760c2be9 commit: 731ff68eeef58babdf2b32dc9a73b141760c2be9 branch: master author: Cheryl Sabella committer: Benjamin Peterson date: 2018-09-11T17:32:15-07:00 summary: closes bpo-25041: Document AF_PACKET socket address format. (GH-4092) files: A Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst M Doc/library/socket.rst M Modules/socketmodule.c diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index cea017557771..32e7c5e6c079 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -173,10 +173,25 @@ created. Socket addresses are represented as follows: .. versionadded:: 3.7 -- Certain other address families (:const:`AF_PACKET`, :const:`AF_CAN`) - support specific representations. - - .. XXX document them! +- :const:`AF_PACKET` is a low-level interface directly to network devices. + The packets are represented by the tuple + ``(ifname, proto[, pkttype[, hatype[, addr]]])`` where: + + - *ifname* - String specifying the device name. + - *proto* - An in network-byte-order integer specifying the Ethernet + protocol number. + - *pkttype* - Optional integer specifying the packet type: + + - ``PACKET_HOST`` (the default) - Packet addressed to the local host. + - ``PACKET_BROADCAST`` - Physical-layer broadcast packet. + - ``PACKET_MULTIHOST`` - Packet sent to a physical-layer multicast address. + - ``PACKET_OTHERHOST`` - Packet to some other host that has been caught by + a device driver in promiscuous mode. + - ``PACKET_OUTGOING`` - Packet originating from the local host that is + looped back to a packet socket. + - *hatype* - Optional integer specifying the ARP hardware address type. + - *addr* - Optional bytes-like object specifying the hardware physical + address, whose interpretation depends on the device. If you use a hostname in the *host* portion of IPv4/v6 socket address, the program may show a nondeterministic behavior, as Python uses the first address @@ -376,6 +391,16 @@ Constants .. versionadded:: 3.7 +.. data:: AF_PACKET + PF_PACKET + PACKET_* + + Many constants of these forms, documented in the Linux documentation, are + also defined in the socket module. + + Availability: Linux >= 2.2. + + .. data:: AF_RDS PF_RDS SOL_RDS @@ -469,12 +494,12 @@ The following functions all create :ref:`socket objects `. Create a new socket using the given address family, socket type and protocol number. The address family should be :const:`AF_INET` (the default), - :const:`AF_INET6`, :const:`AF_UNIX`, :const:`AF_CAN` or :const:`AF_RDS`. The - socket type should be :const:`SOCK_STREAM` (the default), - :const:`SOCK_DGRAM`, :const:`SOCK_RAW` or perhaps one of the other ``SOCK_`` - constants. The protocol number is usually zero and may be omitted or in the - case where the address family is :const:`AF_CAN` the protocol should be one - of :const:`CAN_RAW`, :const:`CAN_BCM` or :const:`CAN_ISOTP` + :const:`AF_INET6`, :const:`AF_UNIX`, :const:`AF_CAN`, :const:`AF_PACKET`, + or :const:`AF_RDS`. The socket type should be :const:`SOCK_STREAM` (the + default), :const:`SOCK_DGRAM`, :const:`SOCK_RAW` or perhaps one of the other + ``SOCK_`` constants. The protocol number is usually zero and may be omitted + or in the case where the address family is :const:`AF_CAN` the protocol + should be one of :const:`CAN_RAW`, :const:`CAN_BCM` or :const:`CAN_ISOTP`. If *fileno* is specified, the values for *family*, *type*, and *proto* are auto-detected from the specified file descriptor. Auto-detection can be diff --git a/Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst b/Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst new file mode 100644 index 000000000000..5bb6e25db9de --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst @@ -0,0 +1 @@ +Document ``AF_PACKET`` in the :mod:`socket` module. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 8aadc780ffeb..adaefad556ca 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -1901,7 +1901,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, const char *interfaceName; int protoNumber; int hatype = 0; - int pkttype = 0; + int pkttype = PACKET_HOST; Py_buffer haddr = {NULL, NULL}; if (!PyTuple_Check(args)) { @@ -1943,7 +1943,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (protoNumber < 0 || protoNumber > 0xffff) { PyErr_Format( PyExc_OverflowError, - "%s(): protoNumber must be 0-65535.", caller); + "%s(): proto must be 0-65535.", caller); PyBuffer_Release(&haddr); return 0; } @@ -2979,7 +2979,7 @@ PyDoc_STRVAR(bind_doc, \n\ Bind the socket to a local address. For IP sockets, the address is a\n\ pair (host, port); the host must refer to the local host. For raw packet\n\ -sockets the address is a tuple (ifname, proto [,pkttype [,hatype]])"); +sockets the address is a tuple (ifname, proto [,pkttype [,hatype [,addr]]])"); /* s.close() method. From webhook-mailer at python.org Tue Sep 11 21:02:44 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 01:02:44 -0000 Subject: [Python-checkins] [2.7] bpo-34405: Updated to OpenSSL 1.0.2p for Windows builds. (GH-8776) (GH-9202) Message-ID: https://github.com/python/cpython/commit/e351fe241c1fcb8cd739b08e21d2bb2f2b4c8c89 commit: e351fe241c1fcb8cd739b08e21d2bb2f2b4c8c89 branch: 2.7 author: Steve Dower committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-11T18:02:40-07:00 summary: [2.7] bpo-34405: Updated to OpenSSL 1.0.2p for Windows builds. (GH-8776) (GH-9202) https://bugs.python.org/issue34405 files: A Misc/NEWS.d/next/Security/2018-08-15-12-14-03.bpo-34405.R4IZGw.rst M PCbuild/get_externals.bat M PCbuild/python.props diff --git a/Misc/NEWS.d/next/Security/2018-08-15-12-14-03.bpo-34405.R4IZGw.rst b/Misc/NEWS.d/next/Security/2018-08-15-12-14-03.bpo-34405.R4IZGw.rst new file mode 100644 index 000000000000..b79eddbc038e --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-08-15-12-14-03.bpo-34405.R4IZGw.rst @@ -0,0 +1 @@ +Updated to OpenSSL 1.0.2p for Windows builds. \ No newline at end of file diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat index 6edae2e8e4e4..8bc19e678ce7 100644 --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -47,7 +47,7 @@ rem files in both this dir and PC\VS9.0 set libraries= set libraries=%libraries% bzip2-1.0.6 if NOT "%IncludeBsddb%"=="false" set libraries=%libraries% bsddb-4.7.25.0 -if NOT "%IncludeSSL%"=="false" set libraries=%libraries% openssl-1.0.2o +if NOT "%IncludeSSL%"=="false" set libraries=%libraries% openssl-1.0.2p set libraries=%libraries% sqlite-3.14.2.0 if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tcl-8.5.19.0 if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tk-8.5.19.0 diff --git a/PCbuild/python.props b/PCbuild/python.props index d2b19915581f..cebb86f147a5 100644 --- a/PCbuild/python.props +++ b/PCbuild/python.props @@ -35,7 +35,7 @@ $(ExternalsDir)sqlite-3.14.2.0\ $(ExternalsDir)bzip2-1.0.6\ $(ExternalsDir)bsddb-4.7.25.0 - $(ExternalsDir)openssl-1.0.2o\ + $(ExternalsDir)openssl-1.0.2p\ $(opensslDir)include32 $(opensslDir)include64 $(ExternalsDir)\nasm-2.11.06\ From webhook-mailer at python.org Tue Sep 11 21:04:36 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 01:04:36 -0000 Subject: [Python-checkins] Delete old expat comment. (GH-9197) Message-ID: https://github.com/python/cpython/commit/01ce535ba97b8a1e96853b3822ebca75ddcbebb3 commit: 01ce535ba97b8a1e96853b3822ebca75ddcbebb3 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T18:04:33-07:00 summary: Delete old expat comment. (GH-9197) (cherry picked from commit b9bf9d025e659b5a1963027eb73690e57cb35dd0) Co-authored-by: Benjamin Peterson files: M Modules/Setup.dist diff --git a/Modules/Setup.dist b/Modules/Setup.dist index da2404a44a26..f0eff5d2d0c2 100644 --- a/Modules/Setup.dist +++ b/Modules/Setup.dist @@ -338,14 +338,6 @@ _symtable symtablemodule.c #zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz # Interface to the Expat XML parser -# -# Expat was written by James Clark and is now maintained by a group of -# developers on SourceForge; see www.libexpat.org for more -# information. The pyexpat module was written by Paul Prescod after a -# prototype by Jack Jansen. Source of Expat 1.95.2 is included in -# Modules/expat/. Usage of a system shared libexpat.so/expat.dll is -# not advised. -# # More information on Expat can be found at www.libexpat.org. # #pyexpat expat/xmlparse.c expat/xmlrole.c expat/xmltok.c pyexpat.c -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI From webhook-mailer at python.org Tue Sep 11 21:13:49 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 01:13:49 -0000 Subject: [Python-checkins] Delete old expat comment. (GH-9197) Message-ID: https://github.com/python/cpython/commit/6539b9136f4ffe73a8942a835d84fc218fb97967 commit: 6539b9136f4ffe73a8942a835d84fc218fb97967 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-11T18:13:46-07:00 summary: Delete old expat comment. (GH-9197) (cherry picked from commit b9bf9d025e659b5a1963027eb73690e57cb35dd0) Co-authored-by: Benjamin Peterson files: M Modules/Setup.dist diff --git a/Modules/Setup.dist b/Modules/Setup.dist index 735bacb1b4df..acf4433cfe6f 100644 --- a/Modules/Setup.dist +++ b/Modules/Setup.dist @@ -366,14 +366,6 @@ _symtable symtablemodule.c #zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz # Interface to the Expat XML parser -# -# Expat was written by James Clark and is now maintained by a group of -# developers on SourceForge; see www.libexpat.org for more -# information. The pyexpat module was written by Paul Prescod after a -# prototype by Jack Jansen. Source of Expat 1.95.2 is included in -# Modules/expat/. Usage of a system shared libexpat.so/expat.dll is -# not advised. -# # More information on Expat can be found at www.libexpat.org. # #pyexpat expat/xmlparse.c expat/xmlrole.c expat/xmltok.c pyexpat.c -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI From webhook-mailer at python.org Tue Sep 11 21:28:38 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 01:28:38 -0000 Subject: [Python-checkins] [2.7] Delete old expat comment. (GH-9205) Message-ID: https://github.com/python/cpython/commit/b663205ca90d4c2755695bef18f1d42faba08c80 commit: b663205ca90d4c2755695bef18f1d42faba08c80 branch: 2.7 author: Benjamin Peterson committer: GitHub date: 2018-09-11T18:28:35-07:00 summary: [2.7] Delete old expat comment. (GH-9205) (cherry picked from commit b9bf9d025e659b5a1963027eb73690e57cb35dd0) Co-authored-by: Benjamin Peterson files: M Modules/Setup.dist diff --git a/Modules/Setup.dist b/Modules/Setup.dist index 523e05a0636b..bd8b52e1de23 100644 --- a/Modules/Setup.dist +++ b/Modules/Setup.dist @@ -467,14 +467,6 @@ GLHACK=-Dclear=__GLclear #zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz # Interface to the Expat XML parser -# -# Expat was written by James Clark and is now maintained by a group of -# developers on SourceForge; see www.libexpat.org for more -# information. The pyexpat module was written by Paul Prescod after a -# prototype by Jack Jansen. Source of Expat 1.95.2 is included in -# Modules/expat/. Usage of a system shared libexpat.so/expat.dll is -# not advised. -# # More information on Expat can be found at www.libexpat.org. # #pyexpat expat/xmlparse.c expat/xmlrole.c expat/xmltok.c pyexpat.c -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI From webhook-mailer at python.org Tue Sep 11 21:46:39 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 01:46:39 -0000 Subject: [Python-checkins] [3.7] closes bpo-25041: Document AF_PACKET socket address format. (GH-9207) Message-ID: https://github.com/python/cpython/commit/cadb66e173ded86ebc85b54889db48f66e7af592 commit: cadb66e173ded86ebc85b54889db48f66e7af592 branch: 3.7 author: Benjamin Peterson committer: GitHub date: 2018-09-11T18:46:36-07:00 summary: [3.7] closes bpo-25041: Document AF_PACKET socket address format. (GH-9207) (cherry picked from commit 731ff68eeef58babdf2b32dc9a73b141760c2be9) Co-authored-by: Cheryl Sabella files: A Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst M Doc/library/socket.rst M Modules/socketmodule.c diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index cea017557771..32e7c5e6c079 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -173,10 +173,25 @@ created. Socket addresses are represented as follows: .. versionadded:: 3.7 -- Certain other address families (:const:`AF_PACKET`, :const:`AF_CAN`) - support specific representations. - - .. XXX document them! +- :const:`AF_PACKET` is a low-level interface directly to network devices. + The packets are represented by the tuple + ``(ifname, proto[, pkttype[, hatype[, addr]]])`` where: + + - *ifname* - String specifying the device name. + - *proto* - An in network-byte-order integer specifying the Ethernet + protocol number. + - *pkttype* - Optional integer specifying the packet type: + + - ``PACKET_HOST`` (the default) - Packet addressed to the local host. + - ``PACKET_BROADCAST`` - Physical-layer broadcast packet. + - ``PACKET_MULTIHOST`` - Packet sent to a physical-layer multicast address. + - ``PACKET_OTHERHOST`` - Packet to some other host that has been caught by + a device driver in promiscuous mode. + - ``PACKET_OUTGOING`` - Packet originating from the local host that is + looped back to a packet socket. + - *hatype* - Optional integer specifying the ARP hardware address type. + - *addr* - Optional bytes-like object specifying the hardware physical + address, whose interpretation depends on the device. If you use a hostname in the *host* portion of IPv4/v6 socket address, the program may show a nondeterministic behavior, as Python uses the first address @@ -376,6 +391,16 @@ Constants .. versionadded:: 3.7 +.. data:: AF_PACKET + PF_PACKET + PACKET_* + + Many constants of these forms, documented in the Linux documentation, are + also defined in the socket module. + + Availability: Linux >= 2.2. + + .. data:: AF_RDS PF_RDS SOL_RDS @@ -469,12 +494,12 @@ The following functions all create :ref:`socket objects `. Create a new socket using the given address family, socket type and protocol number. The address family should be :const:`AF_INET` (the default), - :const:`AF_INET6`, :const:`AF_UNIX`, :const:`AF_CAN` or :const:`AF_RDS`. The - socket type should be :const:`SOCK_STREAM` (the default), - :const:`SOCK_DGRAM`, :const:`SOCK_RAW` or perhaps one of the other ``SOCK_`` - constants. The protocol number is usually zero and may be omitted or in the - case where the address family is :const:`AF_CAN` the protocol should be one - of :const:`CAN_RAW`, :const:`CAN_BCM` or :const:`CAN_ISOTP` + :const:`AF_INET6`, :const:`AF_UNIX`, :const:`AF_CAN`, :const:`AF_PACKET`, + or :const:`AF_RDS`. The socket type should be :const:`SOCK_STREAM` (the + default), :const:`SOCK_DGRAM`, :const:`SOCK_RAW` or perhaps one of the other + ``SOCK_`` constants. The protocol number is usually zero and may be omitted + or in the case where the address family is :const:`AF_CAN` the protocol + should be one of :const:`CAN_RAW`, :const:`CAN_BCM` or :const:`CAN_ISOTP`. If *fileno* is specified, the values for *family*, *type*, and *proto* are auto-detected from the specified file descriptor. Auto-detection can be diff --git a/Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst b/Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst new file mode 100644 index 000000000000..5bb6e25db9de --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst @@ -0,0 +1 @@ +Document ``AF_PACKET`` in the :mod:`socket` module. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index f40bd89e577f..eeade2ecad64 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -1881,7 +1881,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, const char *interfaceName; int protoNumber; int hatype = 0; - int pkttype = 0; + int pkttype = PACKET_HOST; Py_buffer haddr = {NULL, NULL}; if (!PyTuple_Check(args)) { @@ -1912,7 +1912,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (protoNumber < 0 || protoNumber > 0xffff) { PyErr_SetString( PyExc_OverflowError, - "getsockaddrarg: protoNumber must be 0-65535."); + "getsockaddrarg: proto must be 0-65535."); PyBuffer_Release(&haddr); return 0; } @@ -2925,7 +2925,7 @@ PyDoc_STRVAR(bind_doc, \n\ Bind the socket to a local address. For IP sockets, the address is a\n\ pair (host, port); the host must refer to the local host. For raw packet\n\ -sockets the address is a tuple (ifname, proto [,pkttype [,hatype]])"); +sockets the address is a tuple (ifname, proto [,pkttype [,hatype [,addr]]])"); /* s.close() method. From webhook-mailer at python.org Wed Sep 12 01:36:14 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 05:36:14 -0000 Subject: [Python-checkins] [3.6] closes bpo-25041: Document AF_PACKET socket address format. (GH-9209) Message-ID: https://github.com/python/cpython/commit/a00de685c11cf8bdca1f8efa7aab80552d80ddfb commit: a00de685c11cf8bdca1f8efa7aab80552d80ddfb branch: 3.6 author: Benjamin Peterson committer: GitHub date: 2018-09-11T22:36:09-07:00 summary: [3.6] closes bpo-25041: Document AF_PACKET socket address format. (GH-9209) (cherry picked from commit 731ff68eeef58babdf2b32dc9a73b141760c2be9) Co-authored-by: Cheryl Sabella files: A Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst M Doc/library/socket.rst M Modules/socketmodule.c diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index 974df4c6d94b..f4e5af9d5eb5 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -156,10 +156,25 @@ created. Socket addresses are represented as follows: .. versionadded:: 3.6 -- Certain other address families (:const:`AF_PACKET`, :const:`AF_CAN`) - support specific representations. - - .. XXX document them! +- :const:`AF_PACKET` is a low-level interface directly to network devices. + The packets are represented by the tuple + ``(ifname, proto[, pkttype[, hatype[, addr]]])`` where: + + - *ifname* - String specifying the device name. + - *proto* - An in network-byte-order integer specifying the Ethernet + protocol number. + - *pkttype* - Optional integer specifying the packet type: + + - ``PACKET_HOST`` (the default) - Packet addressed to the local host. + - ``PACKET_BROADCAST`` - Physical-layer broadcast packet. + - ``PACKET_MULTIHOST`` - Packet sent to a physical-layer multicast address. + - ``PACKET_OTHERHOST`` - Packet to some other host that has been caught by + a device driver in promiscuous mode. + - ``PACKET_OUTGOING`` - Packet originating from the local host that is + looped back to a packet socket. + - *hatype* - Optional integer specifying the ARP hardware address type. + - *addr* - Optional bytes-like object specifying the hardware physical + address, whose interpretation depends on the device. If you use a hostname in the *host* portion of IPv4/v6 socket address, the program may show a nondeterministic behavior, as Python uses the first address @@ -343,6 +358,17 @@ Constants .. versionadded:: 3.5 + +.. data:: AF_PACKET + PF_PACKET + PACKET_* + + Many constants of these forms, documented in the Linux documentation, are + also defined in the socket module. + + Availability: Linux >= 2.2. + + .. data:: AF_RDS PF_RDS SOL_RDS @@ -424,16 +450,16 @@ The following functions all create :ref:`socket objects `. Create a new socket using the given address family, socket type and protocol number. The address family should be :const:`AF_INET` (the default), - :const:`AF_INET6`, :const:`AF_UNIX`, :const:`AF_CAN` or :const:`AF_RDS`. The - socket type should be :const:`SOCK_STREAM` (the default), - :const:`SOCK_DGRAM`, :const:`SOCK_RAW` or perhaps one of the other ``SOCK_`` - constants. The protocol number is usually zero and may be omitted or in the - case where the address family is :const:`AF_CAN` the protocol should be one - of :const:`CAN_RAW` or :const:`CAN_BCM`. If *fileno* is specified, the other - arguments are ignored, causing the socket with the specified file descriptor - to return. Unlike :func:`socket.fromfd`, *fileno* will return the same - socket and not a duplicate. This may help close a detached socket using - :meth:`socket.close()`. + :const:`AF_INET6`, :const:`AF_UNIX`, :const:`AF_CAN`, :const:`AF_PACKET`, or + :const:`AF_RDS`. The socket type should be :const:`SOCK_STREAM` (the + default), :const:`SOCK_DGRAM`, :const:`SOCK_RAW` or perhaps one of the other + ``SOCK_`` constants. The protocol number is usually zero and may be omitted + or in the case where the address family is :const:`AF_CAN` the protocol + should be one of :const:`CAN_RAW` or :const:`CAN_BCM`. If *fileno* is + specified, the other arguments are ignored, causing the socket with the + specified file descriptor to return. Unlike :func:`socket.fromfd`, *fileno* + will return the same socket and not a duplicate. This may help close a + detached socket using :meth:`socket.close()`. The newly created socket is :ref:`non-inheritable `. diff --git a/Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst b/Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst new file mode 100644 index 000000000000..5bb6e25db9de --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2017-10-23-13-41-12.bpo-25041.iAo2gW.rst @@ -0,0 +1 @@ +Document ``AF_PACKET`` in the :mod:`socket` module. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index ff73a3fbfa2b..ed3166740278 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -1834,7 +1834,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, const char *interfaceName; int protoNumber; int hatype = 0; - int pkttype = 0; + int pkttype = PACKET_HOST; Py_buffer haddr = {NULL, NULL}; if (!PyTuple_Check(args)) { @@ -1865,7 +1865,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, if (protoNumber < 0 || protoNumber > 0xffff) { PyErr_SetString( PyExc_OverflowError, - "getsockaddrarg: protoNumber must be 0-65535."); + "getsockaddrarg: proto must be 0-65535."); PyBuffer_Release(&haddr); return 0; } @@ -2742,7 +2742,7 @@ PyDoc_STRVAR(bind_doc, \n\ Bind the socket to a local address. For IP sockets, the address is a\n\ pair (host, port); the host must refer to the local host. For raw packet\n\ -sockets the address is a tuple (ifname, proto [,pkttype [,hatype]])"); +sockets the address is a tuple (ifname, proto [,pkttype [,hatype [,addr]]])"); /* s.close() method. From solipsis at pitrou.net Wed Sep 12 05:16:06 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 12 Sep 2018 09:16:06 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=-1 Message-ID: <20180912091606.1.AB0538BF1B91415E@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [0, -7, 1] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [2, 0, -1] memory blocks, sum=1 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogwLD56k', '--timeout', '7200'] From webhook-mailer at python.org Wed Sep 12 07:28:24 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Wed, 12 Sep 2018 11:28:24 -0000 Subject: [Python-checkins] [2.7] bpo-28994: Remove mistakenly backported atexitmodule.c (GH-9214) Message-ID: https://github.com/python/cpython/commit/b36567bef80202f53ebe924dd183270c276497f8 commit: b36567bef80202f53ebe924dd183270c276497f8 branch: 2.7 author: Zackery Spytz committer: Serhiy Storchaka date: 2018-09-12T14:28:18+03:00 summary: [2.7] bpo-28994: Remove mistakenly backported atexitmodule.c (GH-9214) It was backported in 0cc43df05e36655220468953e838169966b94ebd. files: D Modules/atexitmodule.c M Misc/NEWS.d/2.7.15rc1.rst diff --git a/Misc/NEWS.d/2.7.15rc1.rst b/Misc/NEWS.d/2.7.15rc1.rst index a01bfd6b8f53..de5f61c59766 100644 --- a/Misc/NEWS.d/2.7.15rc1.rst +++ b/Misc/NEWS.d/2.7.15rc1.rst @@ -854,16 +854,6 @@ Prohibited the '=' character in environment variable names in .. -.. bpo: 28994 -.. date: 056 -.. nonce: 9vzun1 -.. section: Library - -The traceback no longer displayed for SystemExit raised in a callback -registered by atexit. - -.. - .. bpo: 30418 .. date: 055 .. nonce: EwISQm diff --git a/Modules/atexitmodule.c b/Modules/atexitmodule.c deleted file mode 100644 index 35ebf08ecd3c..000000000000 --- a/Modules/atexitmodule.c +++ /dev/null @@ -1,346 +0,0 @@ -/* - * atexit - allow programmer to define multiple exit functions to be executed - * upon normal program termination. - * - * Translated from atexit.py by Collin Winter. - + Copyright 2007 Python Software Foundation. - */ - -#include "Python.h" - -/* Forward declaration (for atexit_cleanup) */ -static PyObject *atexit_clear(PyObject*, PyObject*); -/* Forward declaration of module object */ -static struct PyModuleDef atexitmodule; - -/* ===================================================================== */ -/* Callback machinery. */ - -typedef struct { - PyObject *func; - PyObject *args; - PyObject *kwargs; -} atexit_callback; - -typedef struct { - atexit_callback **atexit_callbacks; - int ncallbacks; - int callback_len; -} atexitmodule_state; - -#define GET_ATEXIT_STATE(mod) ((atexitmodule_state*)PyModule_GetState(mod)) - - -static void -atexit_delete_cb(atexitmodule_state *modstate, int i) -{ - atexit_callback *cb; - - cb = modstate->atexit_callbacks[i]; - modstate->atexit_callbacks[i] = NULL; - Py_DECREF(cb->func); - Py_DECREF(cb->args); - Py_XDECREF(cb->kwargs); - PyMem_Free(cb); -} - -/* Clear all callbacks without calling them */ -static void -atexit_cleanup(atexitmodule_state *modstate) -{ - atexit_callback *cb; - int i; - for (i = 0; i < modstate->ncallbacks; i++) { - cb = modstate->atexit_callbacks[i]; - if (cb == NULL) - continue; - - atexit_delete_cb(modstate, i); - } - modstate->ncallbacks = 0; -} - -/* Installed into pylifecycle.c's atexit mechanism */ - -static void -atexit_callfuncs(void) -{ - PyObject *exc_type = NULL, *exc_value, *exc_tb, *r; - atexit_callback *cb; - PyObject *module; - atexitmodule_state *modstate; - int i; - - module = PyState_FindModule(&atexitmodule); - if (module == NULL) - return; - modstate = GET_ATEXIT_STATE(module); - - if (modstate->ncallbacks == 0) - return; - - - for (i = modstate->ncallbacks - 1; i >= 0; i--) - { - cb = modstate->atexit_callbacks[i]; - if (cb == NULL) - continue; - - r = PyObject_Call(cb->func, cb->args, cb->kwargs); - Py_XDECREF(r); - if (r == NULL) { - /* Maintain the last exception, but don't leak if there are - multiple exceptions. */ - if (exc_type) { - Py_DECREF(exc_type); - Py_XDECREF(exc_value); - Py_XDECREF(exc_tb); - } - PyErr_Fetch(&exc_type, &exc_value, &exc_tb); - if (!PyErr_GivenExceptionMatches(exc_type, PyExc_SystemExit)) { - PySys_WriteStderr("Error in atexit._run_exitfuncs:\n"); - PyErr_NormalizeException(&exc_type, &exc_value, &exc_tb); - PyErr_Display(exc_type, exc_value, exc_tb); - } - } - } - - atexit_cleanup(modstate); - - if (exc_type) - PyErr_Restore(exc_type, exc_value, exc_tb); -} - -/* ===================================================================== */ -/* Module methods. */ - -PyDoc_STRVAR(atexit_register__doc__, -"register(func, *args, **kwargs) -> func\n\ -\n\ -Register a function to be executed upon normal program termination\n\ -\n\ - func - function to be called at exit\n\ - args - optional arguments to pass to func\n\ - kwargs - optional keyword arguments to pass to func\n\ -\n\ - func is returned to facilitate usage as a decorator."); - -static PyObject * -atexit_register(PyObject *self, PyObject *args, PyObject *kwargs) -{ - atexitmodule_state *modstate; - atexit_callback *new_callback; - PyObject *func = NULL; - - modstate = GET_ATEXIT_STATE(self); - - if (modstate->ncallbacks >= modstate->callback_len) { - atexit_callback **r; - modstate->callback_len += 16; - r = (atexit_callback**)PyMem_Realloc(modstate->atexit_callbacks, - sizeof(atexit_callback*) * modstate->callback_len); - if (r == NULL) - return PyErr_NoMemory(); - modstate->atexit_callbacks = r; - } - - if (PyTuple_GET_SIZE(args) == 0) { - PyErr_SetString(PyExc_TypeError, - "register() takes at least 1 argument (0 given)"); - return NULL; - } - - func = PyTuple_GET_ITEM(args, 0); - if (!PyCallable_Check(func)) { - PyErr_SetString(PyExc_TypeError, - "the first argument must be callable"); - return NULL; - } - - new_callback = PyMem_Malloc(sizeof(atexit_callback)); - if (new_callback == NULL) - return PyErr_NoMemory(); - - new_callback->args = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); - if (new_callback->args == NULL) { - PyMem_Free(new_callback); - return NULL; - } - new_callback->func = func; - new_callback->kwargs = kwargs; - Py_INCREF(func); - Py_XINCREF(kwargs); - - modstate->atexit_callbacks[modstate->ncallbacks++] = new_callback; - - Py_INCREF(func); - return func; -} - -PyDoc_STRVAR(atexit_run_exitfuncs__doc__, -"_run_exitfuncs() -> None\n\ -\n\ -Run all registered exit functions."); - -static PyObject * -atexit_run_exitfuncs(PyObject *self, PyObject *unused) -{ - atexit_callfuncs(); - if (PyErr_Occurred()) - return NULL; - Py_RETURN_NONE; -} - -PyDoc_STRVAR(atexit_clear__doc__, -"_clear() -> None\n\ -\n\ -Clear the list of previously registered exit functions."); - -static PyObject * -atexit_clear(PyObject *self, PyObject *unused) -{ - atexit_cleanup(GET_ATEXIT_STATE(self)); - Py_RETURN_NONE; -} - -PyDoc_STRVAR(atexit_ncallbacks__doc__, -"_ncallbacks() -> int\n\ -\n\ -Return the number of registered exit functions."); - -static PyObject * -atexit_ncallbacks(PyObject *self, PyObject *unused) -{ - atexitmodule_state *modstate; - - modstate = GET_ATEXIT_STATE(self); - - return PyLong_FromSsize_t(modstate->ncallbacks); -} - -static int -atexit_m_traverse(PyObject *self, visitproc visit, void *arg) -{ - int i; - atexitmodule_state *modstate; - - modstate = GET_ATEXIT_STATE(self); - for (i = 0; i < modstate->ncallbacks; i++) { - atexit_callback *cb = modstate->atexit_callbacks[i]; - if (cb == NULL) - continue; - Py_VISIT(cb->func); - Py_VISIT(cb->args); - Py_VISIT(cb->kwargs); - } - return 0; -} - -static int -atexit_m_clear(PyObject *self) -{ - atexitmodule_state *modstate; - modstate = GET_ATEXIT_STATE(self); - atexit_cleanup(modstate); - return 0; -} - -static void -atexit_free(PyObject *m) -{ - atexitmodule_state *modstate; - modstate = GET_ATEXIT_STATE(m); - atexit_cleanup(modstate); - PyMem_Free(modstate->atexit_callbacks); -} - -PyDoc_STRVAR(atexit_unregister__doc__, -"unregister(func) -> None\n\ -\n\ -Unregister an exit function which was previously registered using\n\ -atexit.register\n\ -\n\ - func - function to be unregistered"); - -static PyObject * -atexit_unregister(PyObject *self, PyObject *func) -{ - atexitmodule_state *modstate; - atexit_callback *cb; - int i, eq; - - modstate = GET_ATEXIT_STATE(self); - - for (i = 0; i < modstate->ncallbacks; i++) - { - cb = modstate->atexit_callbacks[i]; - if (cb == NULL) - continue; - - eq = PyObject_RichCompareBool(cb->func, func, Py_EQ); - if (eq < 0) - return NULL; - if (eq) - atexit_delete_cb(modstate, i); - } - Py_RETURN_NONE; -} - -static PyMethodDef atexit_methods[] = { - {"register", (PyCFunction) atexit_register, METH_VARARGS|METH_KEYWORDS, - atexit_register__doc__}, - {"_clear", (PyCFunction) atexit_clear, METH_NOARGS, - atexit_clear__doc__}, - {"unregister", (PyCFunction) atexit_unregister, METH_O, - atexit_unregister__doc__}, - {"_run_exitfuncs", (PyCFunction) atexit_run_exitfuncs, METH_NOARGS, - atexit_run_exitfuncs__doc__}, - {"_ncallbacks", (PyCFunction) atexit_ncallbacks, METH_NOARGS, - atexit_ncallbacks__doc__}, - {NULL, NULL} /* sentinel */ -}; - -/* ===================================================================== */ -/* Initialization function. */ - -PyDoc_STRVAR(atexit__doc__, -"allow programmer to define multiple exit functions to be executed\ -upon normal program termination.\n\ -\n\ -Two public functions, register and unregister, are defined.\n\ -"); - - -static struct PyModuleDef atexitmodule = { - PyModuleDef_HEAD_INIT, - "atexit", - atexit__doc__, - sizeof(atexitmodule_state), - atexit_methods, - NULL, - atexit_m_traverse, - atexit_m_clear, - (freefunc)atexit_free -}; - -PyMODINIT_FUNC -PyInit_atexit(void) -{ - PyObject *m; - atexitmodule_state *modstate; - - m = PyModule_Create(&atexitmodule); - if (m == NULL) - return NULL; - - modstate = GET_ATEXIT_STATE(m); - modstate->callback_len = 32; - modstate->ncallbacks = 0; - modstate->atexit_callbacks = PyMem_New(atexit_callback*, - modstate->callback_len); - if (modstate->atexit_callbacks == NULL) - return NULL; - - _Py_PyAtExit(atexit_callfuncs); - return m; -} From webhook-mailer at python.org Wed Sep 12 08:49:16 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Wed, 12 Sep 2018 12:49:16 -0000 Subject: [Python-checkins] Simplified implementation of _sre.ascii_iscased(). (GH-9097) Message-ID: https://github.com/python/cpython/commit/7f0d59f3a83cf1a98398b8e5bdfb97c7a71216bd commit: 7f0d59f3a83cf1a98398b8e5bdfb97c7a71216bd branch: master author: Sergey Fedoseev committer: Serhiy Storchaka date: 2018-09-12T15:49:09+03:00 summary: Simplified implementation of _sre.ascii_iscased(). (GH-9097) files: M Modules/_sre.c diff --git a/Modules/_sre.c b/Modules/_sre.c index 483cf5e9ff9c..c3509796f3a1 100644 --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -100,11 +100,6 @@ static unsigned int sre_lower_ascii(unsigned int ch) return ((ch) < 128 ? Py_TOLOWER(ch) : ch); } -static unsigned int sre_upper_ascii(unsigned int ch) -{ - return ((ch) < 128 ? Py_TOUPPER(ch) : ch); -} - /* locale-specific character predicates */ /* !(c & ~N) == (c < N+1) for any unsigned c, this avoids * warnings when c's type supports only numbers < N+1 */ @@ -293,7 +288,7 @@ _sre_ascii_iscased_impl(PyObject *module, int character) /*[clinic end generated code: output=4f454b630fbd19a2 input=9f0bd952812c7ed3]*/ { unsigned int ch = (unsigned int)character; - return ch != sre_lower_ascii(ch) || ch != sre_upper_ascii(ch); + return ch < 128 && Py_ISALPHA(ch); } /*[clinic input] From webhook-mailer at python.org Wed Sep 12 09:51:28 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 13:51:28 -0000 Subject: [Python-checkins] closes bpo-34640: Remove the TANH_PRESERVES_ZERO_SIGN configure check. (GH-9206) Message-ID: https://github.com/python/cpython/commit/865c17fb28f8c3275fd94da6ee4ac51472ec874a commit: 865c17fb28f8c3275fd94da6ee4ac51472ec874a branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-12T06:51:18-07:00 summary: closes bpo-34640: Remove the TANH_PRESERVES_ZERO_SIGN configure check. (GH-9206) files: M Lib/test/test_cmath.py M Lib/test/test_math.py M configure M configure.ac M pyconfig.h.in diff --git a/Lib/test/test_cmath.py b/Lib/test/test_cmath.py index 0451fb0aa22e..43a074b4b663 100644 --- a/Lib/test/test_cmath.py +++ b/Lib/test/test_cmath.py @@ -577,8 +577,6 @@ def test_isinf(self): self.assertTrue(cmath.isinf(complex(INF, NAN))) @requires_IEEE_754 - @unittest.skipIf(sysconfig.get_config_var('TANH_PRESERVES_ZERO_SIGN') == 0, - "system tanh() function doesn't copy the sign") def testTanhSign(self): for z in complex_zeros: self.assertComplexIdentical(cmath.tanh(z), z) diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py index 608789f7202f..9b2f55e1f410 100644 --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -1355,8 +1355,6 @@ def testTanh(self): self.assertTrue(math.isnan(math.tanh(NAN))) @requires_IEEE_754 - @unittest.skipIf(sysconfig.get_config_var('TANH_PRESERVES_ZERO_SIGN') == 0, - "system tanh() function doesn't copy the sign") def testTanhSign(self): # check that tanh(-0.) == -0. on IEEE 754 systems self.assertEqual(math.tanh(-0.), -0.) diff --git a/configure b/configure index 9520446d5d45..fe642c439a4c 100755 --- a/configure +++ b/configure @@ -14163,53 +14163,6 @@ cat >>confdefs.h <<_ACEOF _ACEOF -# On FreeBSD 6.2, it appears that tanh(-0.) returns 0. instead of -# -0. on some architectures. -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether tanh preserves the sign of zero" >&5 -$as_echo_n "checking whether tanh preserves the sign of zero... " >&6; } -if ${ac_cv_tanh_preserves_zero_sign+:} false; then : - $as_echo_n "(cached) " >&6 -else - -if test "$cross_compiling" = yes; then : - ac_cv_tanh_preserves_zero_sign=no -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -#include -#include -int main() { - /* return 0 if either negative zeros don't exist - on this platform or if negative zeros exist - and tanh(-0.) == -0. */ - if (atan2(0., -1.) == atan2(-0., -1.) || - atan2(tanh(-0.), -1.) == atan2(-0., -1.)) exit(0); - else exit(1); -} - -_ACEOF -if ac_fn_c_try_run "$LINENO"; then : - ac_cv_tanh_preserves_zero_sign=yes -else - ac_cv_tanh_preserves_zero_sign=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ - conftest.$ac_objext conftest.beam conftest.$ac_ext -fi - -fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_tanh_preserves_zero_sign" >&5 -$as_echo "$ac_cv_tanh_preserves_zero_sign" >&6; } -if test "$ac_cv_tanh_preserves_zero_sign" = yes -then - -$as_echo "#define TANH_PRESERVES_ZERO_SIGN 1" >>confdefs.h - -fi -LIBS=$LIBS_SAVE - # For multiprocessing module, check that sem_open # actually works. For FreeBSD versions <= 7.2, # the kernel module that provides POSIX semaphores diff --git a/configure.ac b/configure.ac index 4b34bad35a1d..a985ec2b000b 100644 --- a/configure.ac +++ b/configure.ac @@ -4359,33 +4359,6 @@ AC_CHECK_FUNCS([acosh asinh atanh copysign erf erfc expm1 finite gamma]) AC_CHECK_FUNCS([hypot lgamma log1p log2 round tgamma]) AC_CHECK_DECLS([isinf, isnan, isfinite], [], [], [[#include ]]) -# On FreeBSD 6.2, it appears that tanh(-0.) returns 0. instead of -# -0. on some architectures. -AC_MSG_CHECKING(whether tanh preserves the sign of zero) -AC_CACHE_VAL(ac_cv_tanh_preserves_zero_sign, [ -AC_RUN_IFELSE([AC_LANG_SOURCE([[ -#include -#include -int main() { - /* return 0 if either negative zeros don't exist - on this platform or if negative zeros exist - and tanh(-0.) == -0. */ - if (atan2(0., -1.) == atan2(-0., -1.) || - atan2(tanh(-0.), -1.) == atan2(-0., -1.)) exit(0); - else exit(1); -} -]])], -[ac_cv_tanh_preserves_zero_sign=yes], -[ac_cv_tanh_preserves_zero_sign=no], -[ac_cv_tanh_preserves_zero_sign=no])]) -AC_MSG_RESULT($ac_cv_tanh_preserves_zero_sign) -if test "$ac_cv_tanh_preserves_zero_sign" = yes -then - AC_DEFINE(TANH_PRESERVES_ZERO_SIGN, 1, - [Define if tanh(-0.) is -0., or if platform doesn't have signed zeros]) -fi -LIBS=$LIBS_SAVE - # For multiprocessing module, check that sem_open # actually works. For FreeBSD versions <= 7.2, # the kernel module that provides POSIX semaphores diff --git a/pyconfig.h.in b/pyconfig.h.in index 51e9213826ae..360f79994faf 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -1419,9 +1419,6 @@ (which you can't on SCO ODT 3.0). */ #undef SYS_SELECT_WITH_SYS_TIME -/* Define if tanh(-0.) is -0., or if platform doesn't have signed zeros */ -#undef TANH_PRESERVES_ZERO_SIGN - /* Library needed by timemodule.c: librt may be needed for clock_gettime() */ #undef TIMEMODULE_LIB From webhook-mailer at python.org Wed Sep 12 11:37:43 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 15:37:43 -0000 Subject: [Python-checkins] closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) Message-ID: https://github.com/python/cpython/commit/72c34cf6dd5fb206fe4d407ff603468af466fd2e commit: 72c34cf6dd5fb206fe4d407ff603468af466fd2e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T08:37:30-07:00 summary: closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) (cherry picked from commit 4859ba0d2ce4506fddc3f55f90f8dce031b3804f) Co-authored-by: Max B?langer files: A Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst M Modules/_scproxy.c diff --git a/Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst b/Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst new file mode 100644 index 000000000000..3788112cd730 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst @@ -0,0 +1,2 @@ +In :mod:`_scproxy`, drop the GIL when calling into ``SystemConfiguration`` to avoid +deadlocks. diff --git a/Modules/_scproxy.c b/Modules/_scproxy.c index 0e3b02879b83..c5660673c635 100644 --- a/Modules/_scproxy.c +++ b/Modules/_scproxy.c @@ -62,7 +62,10 @@ get_proxy_settings(PyObject* mod __attribute__((__unused__))) PyObject* v; int r; + Py_BEGIN_ALLOW_THREADS proxyDict = SCDynamicStoreCopyProxies(NULL); + Py_END_ALLOW_THREADS + if (!proxyDict) { Py_RETURN_NONE; } @@ -172,7 +175,10 @@ get_proxies(PyObject* mod __attribute__((__unused__))) int r; CFDictionaryRef proxyDict = NULL; + Py_BEGIN_ALLOW_THREADS proxyDict = SCDynamicStoreCopyProxies(NULL); + Py_END_ALLOW_THREADS + if (proxyDict == NULL) { return PyDict_New(); } From webhook-mailer at python.org Wed Sep 12 11:38:39 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 15:38:39 -0000 Subject: [Python-checkins] closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) Message-ID: https://github.com/python/cpython/commit/aa12534acff4868f6dce9e2364a66fbb0ebcb3ca commit: aa12534acff4868f6dce9e2364a66fbb0ebcb3ca branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T08:38:36-07:00 summary: closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) (cherry picked from commit 4859ba0d2ce4506fddc3f55f90f8dce031b3804f) Co-authored-by: Max B?langer files: A Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst M Modules/_scproxy.c diff --git a/Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst b/Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst new file mode 100644 index 000000000000..3788112cd730 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2017-11-01-16-53-12.bpo-31903.K6jCVG.rst @@ -0,0 +1,2 @@ +In :mod:`_scproxy`, drop the GIL when calling into ``SystemConfiguration`` to avoid +deadlocks. diff --git a/Modules/_scproxy.c b/Modules/_scproxy.c index 1ce4b776f3e2..e37515ee6552 100644 --- a/Modules/_scproxy.c +++ b/Modules/_scproxy.c @@ -62,7 +62,10 @@ get_proxy_settings(PyObject* mod __attribute__((__unused__))) PyObject* v; int r; + Py_BEGIN_ALLOW_THREADS proxyDict = SCDynamicStoreCopyProxies(NULL); + Py_END_ALLOW_THREADS + if (!proxyDict) { Py_INCREF(Py_None); return Py_None; @@ -173,7 +176,10 @@ get_proxies(PyObject* mod __attribute__((__unused__))) int r; CFDictionaryRef proxyDict = NULL; + Py_BEGIN_ALLOW_THREADS proxyDict = SCDynamicStoreCopyProxies(NULL); + Py_END_ALLOW_THREADS + if (proxyDict == NULL) { return PyDict_New(); } From webhook-mailer at python.org Wed Sep 12 13:00:42 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 17:00:42 -0000 Subject: [Python-checkins] bpo-33437: add __new__ vs __init__ example (GH-9145) Message-ID: https://github.com/python/cpython/commit/f52237400b9960d434c5d0676a3479b8c1e8c869 commit: f52237400b9960d434c5d0676a3479b8c1e8c869 branch: master author: Ethan Furman committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-12T10:00:30-07:00 summary: bpo-33437: add __new__ vs __init__ example (GH-9145) Improve Enum docs. https://bugs.python.org/issue33437 files: M Doc/library/enum.rst M Misc/ACKS diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 3f17f9947641..6408c0106040 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -736,6 +736,37 @@ Some rules: type's :meth:`__format__`. If the :class:`Enum` class's :func:`str` or :func:`repr` is desired, use the `!s` or `!r` format codes. +When to use :meth:`__new__` vs. :meth:`__init__` +------------------------------------------------ + +:meth:`__new__` must be used whenever you want to customize the actual value of +the :class:`Enum` member. Any other modifications may go in either +:meth:`__new__` or :meth:`__init__`, with :meth:`__init__` being preferred. + +For example, if you want to pass several items to the constructor, but only +want one of them to be the value:: + + >>> class Coordinate(bytes, Enum): + ... """ + ... Coordinate with binary codes that can be indexed by the int code. + ... """ + ... def __new__(cls, value, label, unit): + ... obj = bytes.__new__(cls, [value]) + ... obj._value_ = value + ... obj.label = label + ... obj.unit = unit + ... return obj + ... PX = (0, 'P.X', 'km') + ... PY = (1, 'P.Y', 'km') + ... VX = (2, 'V.X', 'km/s') + ... VY = (3, 'V.Y', 'km/s') + ... + + >>> print(Coordinate['PY']) + Coordinate.PY + + >>> print(Coordinate(3)) + Coordinate.VY Interesting examples -------------------- diff --git a/Misc/ACKS b/Misc/ACKS index 75047d89010e..0d9431d24290 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -68,6 +68,7 @@ Ammar Askar Neil Aspinall Chris AtLee Aymeric Augustin +Andres Ayala Cathy Avery John Aycock Donovan Baarda From webhook-mailer at python.org Wed Sep 12 13:28:58 2018 From: webhook-mailer at python.org (Ethan Furman) Date: Wed, 12 Sep 2018 17:28:58 -0000 Subject: [Python-checkins] bpo-34282: Fix Enum._convert shadowing members named _convert (GH-8568) Message-ID: https://github.com/python/cpython/commit/0fb9fadd3b3e9e3698647e0b92d49b0b7aacd979 commit: 0fb9fadd3b3e9e3698647e0b92d49b0b7aacd979 branch: master author: orlnub123 committer: Ethan Furman date: 2018-09-12T10:28:53-07:00 summary: bpo-34282: Fix Enum._convert shadowing members named _convert (GH-8568) * Fix enum members getting shadowed by parent attributes * Move Enum._convert to EnumMeta._convert_ * Deprecate _convert files: A Misc/NEWS.d/next/Library/2018-08-31-06-28-03.bpo-34282.ztyXH8.rst M Lib/enum.py M Lib/signal.py M Lib/socket.py M Lib/ssl.py M Lib/test/test_enum.py M Misc/ACKS diff --git a/Lib/enum.py b/Lib/enum.py index 9d1aef372c12..0839671cca00 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -165,9 +165,11 @@ def __new__(metacls, cls, bases, classdict): enum_class._member_map_ = {} # name->value map enum_class._member_type_ = member_type - # save attributes from super classes so we know if we can take - # the shortcut of storing members in the class dict - base_attributes = {a for b in enum_class.mro() for a in b.__dict__} + # save DynamicClassAttribute attributes from super classes so we know + # if we can take the shortcut of storing members in the class dict + dynamic_attributes = {k for c in enum_class.mro() + for k, v in c.__dict__.items() + if isinstance(v, DynamicClassAttribute)} # Reverse value->name map for hashable values. enum_class._value2member_map_ = {} @@ -227,7 +229,7 @@ def __new__(metacls, cls, bases, classdict): enum_class._member_names_.append(member_name) # performance boost for any member that would not shadow # a DynamicClassAttribute - if member_name not in base_attributes: + if member_name not in dynamic_attributes: setattr(enum_class, member_name, enum_member) # now add to _member_map_ enum_class._member_map_[member_name] = enum_member @@ -428,6 +430,45 @@ def _create_(cls, class_name, names, *, module=None, qualname=None, type=None, s return enum_class + def _convert_(cls, name, module, filter, source=None): + """ + Create a new Enum subclass that replaces a collection of global constants + """ + # convert all constants from source (or module) that pass filter() to + # a new Enum called name, and export the enum and its members back to + # module; + # also, replace the __reduce_ex__ method so unpickling works in + # previous Python versions + module_globals = vars(sys.modules[module]) + if source: + source = vars(source) + else: + source = module_globals + # _value2member_map_ is populated in the same order every time + # for a consistent reverse mapping of number to name when there + # are multiple names for the same number. + members = [ + (name, value) + for name, value in source.items() + if filter(name)] + try: + # sort by value + members.sort(key=lambda t: (t[1], t[0])) + except TypeError: + # unless some values aren't comparable, in which case sort by name + members.sort(key=lambda t: t[0]) + cls = cls(name, members, module=module) + cls.__reduce_ex__ = _reduce_ex_by_name + module_globals.update(cls.__members__) + module_globals[name] = cls + return cls + + def _convert(cls, *args, **kwargs): + import warnings + warnings.warn("_convert is deprecated and will be removed in 3.9, use " + "_convert_ instead.", DeprecationWarning, stacklevel=2) + return cls._convert_(*args, **kwargs) + @staticmethod def _get_mixins_(bases): """Returns the type for creating enum members, and the first inherited @@ -613,40 +654,6 @@ def value(self): """The value of the Enum member.""" return self._value_ - @classmethod - def _convert(cls, name, module, filter, source=None): - """ - Create a new Enum subclass that replaces a collection of global constants - """ - # convert all constants from source (or module) that pass filter() to - # a new Enum called name, and export the enum and its members back to - # module; - # also, replace the __reduce_ex__ method so unpickling works in - # previous Python versions - module_globals = vars(sys.modules[module]) - if source: - source = vars(source) - else: - source = module_globals - # _value2member_map_ is populated in the same order every time - # for a consistent reverse mapping of number to name when there - # are multiple names for the same number. - members = [ - (name, value) - for name, value in source.items() - if filter(name)] - try: - # sort by value - members.sort(key=lambda t: (t[1], t[0])) - except TypeError: - # unless some values aren't comparable, in which case sort by name - members.sort(key=lambda t: t[0]) - cls = cls(name, members, module=module) - cls.__reduce_ex__ = _reduce_ex_by_name - module_globals.update(cls.__members__) - module_globals[name] = cls - return cls - class IntEnum(int, Enum): """Enum where members are also (and must be) ints""" diff --git a/Lib/signal.py b/Lib/signal.py index 826b62cf596c..d4a6d6fe2ada 100644 --- a/Lib/signal.py +++ b/Lib/signal.py @@ -5,19 +5,19 @@ _globals = globals() -_IntEnum._convert( +_IntEnum._convert_( 'Signals', __name__, lambda name: name.isupper() and (name.startswith('SIG') and not name.startswith('SIG_')) or name.startswith('CTRL_')) -_IntEnum._convert( +_IntEnum._convert_( 'Handlers', __name__, lambda name: name in ('SIG_DFL', 'SIG_IGN')) if 'pthread_sigmask' in _globals: - _IntEnum._convert( + _IntEnum._convert_( 'Sigmasks', __name__, lambda name: name in ('SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK')) diff --git a/Lib/socket.py b/Lib/socket.py index cfa605a22ada..385844b58532 100644 --- a/Lib/socket.py +++ b/Lib/socket.py @@ -70,22 +70,22 @@ # in this module understands the enums and translates them back from integers # where needed (e.g. .family property of a socket object). -IntEnum._convert( +IntEnum._convert_( 'AddressFamily', __name__, lambda C: C.isupper() and C.startswith('AF_')) -IntEnum._convert( +IntEnum._convert_( 'SocketKind', __name__, lambda C: C.isupper() and C.startswith('SOCK_')) -IntFlag._convert( +IntFlag._convert_( 'MsgFlag', __name__, lambda C: C.isupper() and C.startswith('MSG_')) -IntFlag._convert( +IntFlag._convert_( 'AddressInfo', __name__, lambda C: C.isupper() and C.startswith('AI_')) diff --git a/Lib/ssl.py b/Lib/ssl.py index fdd161574434..fa7c152ade91 100644 --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -119,32 +119,32 @@ from _ssl import _DEFAULT_CIPHERS, _OPENSSL_API_VERSION -_IntEnum._convert( +_IntEnum._convert_( '_SSLMethod', __name__, lambda name: name.startswith('PROTOCOL_') and name != 'PROTOCOL_SSLv23', source=_ssl) -_IntFlag._convert( +_IntFlag._convert_( 'Options', __name__, lambda name: name.startswith('OP_'), source=_ssl) -_IntEnum._convert( +_IntEnum._convert_( 'AlertDescription', __name__, lambda name: name.startswith('ALERT_DESCRIPTION_'), source=_ssl) -_IntEnum._convert( +_IntEnum._convert_( 'SSLErrorNumber', __name__, lambda name: name.startswith('SSL_ERROR_'), source=_ssl) -_IntFlag._convert( +_IntFlag._convert_( 'VerifyFlags', __name__, lambda name: name.startswith('VERIFY_'), source=_ssl) -_IntEnum._convert( +_IntEnum._convert_( 'VerifyMode', __name__, lambda name: name.startswith('CERT_'), source=_ssl) diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 68483e654254..c04d03f37523 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1,6 +1,7 @@ import enum import inspect import pydoc +import sys import unittest import threading from collections import OrderedDict @@ -1511,6 +1512,23 @@ class MoreColor(Color): yellow = 6 self.assertEqual(MoreColor.magenta.hex(), '5 hexlified!') + def test_subclass_duplicate_name(self): + class Base(Enum): + def test(self): + pass + class Test(Base): + test = 1 + self.assertIs(type(Test.test), Test) + + def test_subclass_duplicate_name_dynamic(self): + from types import DynamicClassAttribute + class Base(Enum): + @DynamicClassAttribute + def test(self): + return 'dynamic' + class Test(Base): + test = 1 + self.assertEqual(Test.test.test, 'dynamic') def test_no_duplicates(self): class UniqueEnum(Enum): @@ -2668,7 +2686,7 @@ def test__all__(self): class TestIntEnumConvert(unittest.TestCase): def test_convert_value_lookup_priority(self): - test_type = enum.IntEnum._convert( + test_type = enum.IntEnum._convert_( 'UnittestConvert', ('test.test_enum', '__main__')[__name__=='__main__'], filter=lambda x: x.startswith('CONVERT_TEST_')) @@ -2678,7 +2696,7 @@ def test_convert_value_lookup_priority(self): self.assertEqual(test_type(5).name, 'CONVERT_TEST_NAME_A') def test_convert(self): - test_type = enum.IntEnum._convert( + test_type = enum.IntEnum._convert_( 'UnittestConvert', ('test.test_enum', '__main__')[__name__=='__main__'], filter=lambda x: x.startswith('CONVERT_TEST_')) @@ -2694,6 +2712,24 @@ def test_convert(self): if name[0:2] not in ('CO', '__')], [], msg='Names other than CONVERT_TEST_* found.') + @unittest.skipUnless(sys.version_info[:2] == (3, 8), + '_convert was deprecated in 3.8') + def test_convert_warn(self): + with self.assertWarns(DeprecationWarning): + enum.IntEnum._convert( + 'UnittestConvert', + ('test.test_enum', '__main__')[__name__=='__main__'], + filter=lambda x: x.startswith('CONVERT_TEST_')) + + @unittest.skipUnless(sys.version_info >= (3, 9), + '_convert was removed in 3.9') + def test_convert_raise(self): + with self.assertRaises(AttributeError): + enum.IntEnum._convert( + 'UnittestConvert', + ('test.test_enum', '__main__')[__name__=='__main__'], + filter=lambda x: x.startswith('CONVERT_TEST_')) + if __name__ == '__main__': unittest.main() diff --git a/Misc/ACKS b/Misc/ACKS index 0d9431d24290..96985358e236 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1178,6 +1178,7 @@ Piet van Oostrum Tomas Oppelstrup Jason Orendorff Bastien Orivel +orlnub123 Douglas Orr William Orr Michele Orr? diff --git a/Misc/NEWS.d/next/Library/2018-08-31-06-28-03.bpo-34282.ztyXH8.rst b/Misc/NEWS.d/next/Library/2018-08-31-06-28-03.bpo-34282.ztyXH8.rst new file mode 100644 index 000000000000..79f56f124a3f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-08-31-06-28-03.bpo-34282.ztyXH8.rst @@ -0,0 +1,2 @@ +Move ``Enum._convert`` to ``EnumMeta._convert_`` and fix enum members getting +shadowed by parent attributes. From webhook-mailer at python.org Wed Sep 12 13:54:10 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Wed, 12 Sep 2018 17:54:10 -0000 Subject: [Python-checkins] bpo-34637: Make the *start* argument for *sum()* visible as a keyword argument. (GH-9208) Message-ID: https://github.com/python/cpython/commit/9dfa0fe587eae3626ffc973680c6a17f35de3864 commit: 9dfa0fe587eae3626ffc973680c6a17f35de3864 branch: master author: Raymond Hettinger committer: GitHub date: 2018-09-12T10:54:06-07:00 summary: bpo-34637: Make the *start* argument for *sum()* visible as a keyword argument. (GH-9208) files: A Misc/NEWS.d/next/Core and Builtins/2018-09-11-17-25-44.bpo-34637.HSLqY4.rst M Doc/library/functions.rst M Lib/test/test_builtin.py M Python/bltinmodule.c M Python/clinic/bltinmodule.c.h diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 48cd9b04750b..17960eb9c10e 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1488,6 +1488,9 @@ are always available. They are listed here in alphabetical order. see :func:`math.fsum`\. To concatenate a series of iterables, consider using :func:`itertools.chain`. + .. versionchanged:: 3.8 + The *start* parameter can be specified as a keyword argument. + .. function:: super([type[, object-or-type]]) Return a proxy object that delegates method calls to a parent or sibling diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py index 233c57935653..46cf2d328362 100644 --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -1297,6 +1297,9 @@ def test_sum(self): self.assertEqual(sum(iter(Squares(10))), 285) self.assertEqual(sum([[1], [2], [3]], []), [1, 2, 3]) + self.assertEqual(sum(range(10), 1000), 1045) + self.assertEqual(sum(range(10), start=1000), 1045) + self.assertRaises(TypeError, sum) self.assertRaises(TypeError, sum, 42) self.assertRaises(TypeError, sum, ['a', 'b', 'c']) diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-11-17-25-44.bpo-34637.HSLqY4.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-11-17-25-44.bpo-34637.HSLqY4.rst new file mode 100644 index 000000000000..c22359cc3780 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-11-17-25-44.bpo-34637.HSLqY4.rst @@ -0,0 +1 @@ +Make the *start* argument to *sum()* visible as a keyword argument. diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c index a23bdc1078c9..8001c603939a 100644 --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -2272,8 +2272,8 @@ With an argument, equivalent to object.__dict__."); sum as builtin_sum iterable: object - start: object(c_default="NULL") = 0 / + start: object(c_default="NULL") = 0 Return the sum of a 'start' value (default: 0) plus an iterable of numbers @@ -2284,7 +2284,7 @@ reject non-numeric types. static PyObject * builtin_sum_impl(PyObject *module, PyObject *iterable, PyObject *start) -/*[clinic end generated code: output=df758cec7d1d302f input=3b5b7a9d7611c73a]*/ +/*[clinic end generated code: output=df758cec7d1d302f input=162b50765250d222]*/ { PyObject *result = start; PyObject *temp, *item, *iter; diff --git a/Python/clinic/bltinmodule.c.h b/Python/clinic/bltinmodule.c.h index 285caf94b77d..121bbd4e65c1 100644 --- a/Python/clinic/bltinmodule.c.h +++ b/Python/clinic/bltinmodule.c.h @@ -608,7 +608,7 @@ builtin_round(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObjec } PyDoc_STRVAR(builtin_sum__doc__, -"sum($module, iterable, start=0, /)\n" +"sum($module, iterable, /, start=0)\n" "--\n" "\n" "Return the sum of a \'start\' value (default: 0) plus an iterable of numbers\n" @@ -618,20 +618,21 @@ PyDoc_STRVAR(builtin_sum__doc__, "reject non-numeric types."); #define BUILTIN_SUM_METHODDEF \ - {"sum", (PyCFunction)builtin_sum, METH_FASTCALL, builtin_sum__doc__}, + {"sum", (PyCFunction)builtin_sum, METH_FASTCALL|METH_KEYWORDS, builtin_sum__doc__}, static PyObject * builtin_sum_impl(PyObject *module, PyObject *iterable, PyObject *start); static PyObject * -builtin_sum(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +builtin_sum(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", "start", NULL}; + static _PyArg_Parser _parser = {"O|O:sum", _keywords, 0}; PyObject *iterable; PyObject *start = NULL; - if (!_PyArg_UnpackStack(args, nargs, "sum", - 1, 2, + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, &iterable, &start)) { goto exit; } @@ -710,4 +711,4 @@ builtin_issubclass(PyObject *module, PyObject *const *args, Py_ssize_t nargs) exit: return return_value; } -/*[clinic end generated code: output=9f17c7a87d740374 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=eb6d08a32e7c83b6 input=a9049054013a1b77]*/ From webhook-mailer at python.org Wed Sep 12 14:32:01 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Wed, 12 Sep 2018 18:32:01 -0000 Subject: [Python-checkins] bpo-33649: Fix doc to reflect changes in 47cd10d (or bpo-23347) (GH-9219) Message-ID: https://github.com/python/cpython/commit/aca819fb494d4801b3e5b5b507b17cab772c1b40 commit: aca819fb494d4801b3e5b5b507b17cab772c1b40 branch: master author: Bumsik Kim committer: Yury Selivanov date: 2018-09-12T11:31:56-07:00 summary: bpo-33649: Fix doc to reflect changes in 47cd10d (or bpo-23347) (GH-9219) files: M Doc/library/asyncio-protocol.rst diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst index 348ced57772f..84a275e0d0b6 100644 --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -406,7 +406,6 @@ Subprocess Transports .. method:: SubprocessTransport.terminate() Ask the subprocess to stop, as in :meth:`subprocess.Popen.terminate`. - This method is an alias for the :meth:`close` method. On POSIX systems, this method sends SIGTERM to the subprocess. On Windows, the Windows API function TerminateProcess() is called to @@ -414,7 +413,7 @@ Subprocess Transports .. method:: SubprocessTransport.close() - Ask the subprocess to stop by calling the :meth:`terminate` method + Kill the subprocess by calling the :meth:`kill` method if the subprocess hasn't returned yet, and close transports of all pipes (*stdin*, *stdout* and *stderr*). From webhook-mailer at python.org Wed Sep 12 14:43:07 2018 From: webhook-mailer at python.org (Andrew Svetlov) Date: Wed, 12 Sep 2018 18:43:07 -0000 Subject: [Python-checkins] bpo-34638: Store a weak reference to stream reader to break strong references loop (GH-9201) Message-ID: https://github.com/python/cpython/commit/a5d1eb8d8b7add31b5f5d9bbb31cee1a491b2c08 commit: a5d1eb8d8b7add31b5f5d9bbb31cee1a491b2c08 branch: master author: Andrew Svetlov committer: GitHub date: 2018-09-12T11:43:04-07:00 summary: bpo-34638: Store a weak reference to stream reader to break strong references loop (GH-9201) Store a weak reference to stream readerfor breaking strong references It breaks the strong reference loop between reader and protocol and allows to detect and close the socket if the stream is deleted (garbage collected) files: A Misc/NEWS.d/next/Library/2018-09-12-10-33-44.bpo-34638.xaeZX5.rst M Lib/asyncio/streams.py M Lib/asyncio/subprocess.py M Lib/test/test_asyncio/test_streams.py diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py index 9dab49b35e46..e7fb22ee5d1a 100644 --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -3,6 +3,8 @@ 'open_connection', 'start_server') import socket +import sys +import weakref if hasattr(socket, 'AF_UNIX'): __all__ += ('open_unix_connection', 'start_unix_server') @@ -10,6 +12,7 @@ from . import coroutines from . import events from . import exceptions +from . import format_helpers from . import protocols from .log import logger from .tasks import sleep @@ -186,46 +189,106 @@ class StreamReaderProtocol(FlowControlMixin, protocols.Protocol): call inappropriate methods of the protocol.) """ + _source_traceback = None + def __init__(self, stream_reader, client_connected_cb=None, loop=None): super().__init__(loop=loop) - self._stream_reader = stream_reader + if stream_reader is not None: + self._stream_reader_wr = weakref.ref(stream_reader, + self._on_reader_gc) + self._source_traceback = stream_reader._source_traceback + else: + self._stream_reader_wr = None + if client_connected_cb is not None: + # This is a stream created by the `create_server()` function. + # Keep a strong reference to the reader until a connection + # is established. + self._strong_reader = stream_reader + self._reject_connection = False self._stream_writer = None + self._transport = None self._client_connected_cb = client_connected_cb self._over_ssl = False self._closed = self._loop.create_future() + def _on_reader_gc(self, wr): + transport = self._transport + if transport is not None: + # connection_made was called + context = { + 'message': ('An open stream object is being garbage ' + 'collected; call "stream.close()" explicitly.') + } + if self._source_traceback: + context['source_traceback'] = self._source_traceback + self._loop.call_exception_handler(context) + transport.abort() + else: + self._reject_connection = True + self._stream_reader_wr = None + + def _untrack_reader(self): + self._stream_reader_wr = None + + @property + def _stream_reader(self): + if self._stream_reader_wr is None: + return None + return self._stream_reader_wr() + def connection_made(self, transport): - self._stream_reader.set_transport(transport) + if self._reject_connection: + context = { + 'message': ('An open stream was garbage collected prior to ' + 'establishing network connection; ' + 'call "stream.close()" explicitly.') + } + if self._source_traceback: + context['source_traceback'] = self._source_traceback + self._loop.call_exception_handler(context) + transport.abort() + return + self._transport = transport + reader = self._stream_reader + if reader is not None: + reader.set_transport(transport) self._over_ssl = transport.get_extra_info('sslcontext') is not None if self._client_connected_cb is not None: self._stream_writer = StreamWriter(transport, self, - self._stream_reader, + reader, self._loop) - res = self._client_connected_cb(self._stream_reader, + res = self._client_connected_cb(reader, self._stream_writer) if coroutines.iscoroutine(res): self._loop.create_task(res) + self._strong_reader = None def connection_lost(self, exc): - if self._stream_reader is not None: + reader = self._stream_reader + if reader is not None: if exc is None: - self._stream_reader.feed_eof() + reader.feed_eof() else: - self._stream_reader.set_exception(exc) + reader.set_exception(exc) if not self._closed.done(): if exc is None: self._closed.set_result(None) else: self._closed.set_exception(exc) super().connection_lost(exc) - self._stream_reader = None + self._stream_reader_wr = None self._stream_writer = None + self._transport = None def data_received(self, data): - self._stream_reader.feed_data(data) + reader = self._stream_reader + if reader is not None: + reader.feed_data(data) def eof_received(self): - self._stream_reader.feed_eof() + reader = self._stream_reader + if reader is not None: + reader.feed_eof() if self._over_ssl: # Prevent a warning in SSLProtocol.eof_received: # "returning true from eof_received() @@ -282,6 +345,9 @@ def can_write_eof(self): return self._transport.can_write_eof() def close(self): + # a reader can be garbage collected + # after connection closing + self._protocol._untrack_reader() return self._transport.close() def is_closing(self): @@ -318,6 +384,8 @@ def get_extra_info(self, name, default=None): class StreamReader: + _source_traceback = None + def __init__(self, limit=_DEFAULT_LIMIT, loop=None): # The line length limit is a security feature; # it also doubles as half the buffer limit. @@ -336,6 +404,9 @@ def __init__(self, limit=_DEFAULT_LIMIT, loop=None): self._exception = None self._transport = None self._paused = False + if self._loop.get_debug(): + self._source_traceback = format_helpers.extract_stack( + sys._getframe(1)) def __repr__(self): info = ['StreamReader'] diff --git a/Lib/asyncio/subprocess.py b/Lib/asyncio/subprocess.py index 90fc00de8339..c86de3d08702 100644 --- a/Lib/asyncio/subprocess.py +++ b/Lib/asyncio/subprocess.py @@ -36,6 +36,11 @@ def __repr__(self): info.append(f'stderr={self.stderr!r}') return '<{}>'.format(' '.join(info)) + def _untrack_reader(self): + # StreamWriter.close() expects the protocol + # to have this method defined. + pass + def connection_made(self, transport): self._transport = transport diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py index 66d18738b316..67ac9d91a0b1 100644 --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -46,6 +46,8 @@ def test_ctor_global_loop(self, m_events): self.assertIs(stream._loop, m_events.get_event_loop.return_value) def _basetest_open_connection(self, open_connection_fut): + messages = [] + self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx)) reader, writer = self.loop.run_until_complete(open_connection_fut) writer.write(b'GET / HTTP/1.0\r\n\r\n') f = reader.readline() @@ -55,6 +57,7 @@ def _basetest_open_connection(self, open_connection_fut): data = self.loop.run_until_complete(f) self.assertTrue(data.endswith(b'\r\n\r\nTest message')) writer.close() + self.assertEqual(messages, []) def test_open_connection(self): with test_utils.run_test_server() as httpd: @@ -70,6 +73,8 @@ def test_open_unix_connection(self): self._basetest_open_connection(conn_fut) def _basetest_open_connection_no_loop_ssl(self, open_connection_fut): + messages = [] + self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx)) try: reader, writer = self.loop.run_until_complete(open_connection_fut) finally: @@ -80,6 +85,7 @@ def _basetest_open_connection_no_loop_ssl(self, open_connection_fut): self.assertTrue(data.endswith(b'\r\n\r\nTest message')) writer.close() + self.assertEqual(messages, []) @unittest.skipIf(ssl is None, 'No ssl module') def test_open_connection_no_loop_ssl(self): @@ -104,6 +110,8 @@ def test_open_unix_connection_no_loop_ssl(self): self._basetest_open_connection_no_loop_ssl(conn_fut) def _basetest_open_connection_error(self, open_connection_fut): + messages = [] + self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx)) reader, writer = self.loop.run_until_complete(open_connection_fut) writer._protocol.connection_lost(ZeroDivisionError()) f = reader.read() @@ -111,6 +119,7 @@ def _basetest_open_connection_error(self, open_connection_fut): self.loop.run_until_complete(f) writer.close() test_utils.run_briefly(self.loop) + self.assertEqual(messages, []) def test_open_connection_error(self): with test_utils.run_test_server() as httpd: @@ -621,6 +630,9 @@ def stop(self): writer.close() return msgback + messages = [] + self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx)) + # test the server variant with a coroutine as client handler server = MyServer(self.loop) addr = server.start() @@ -637,6 +649,8 @@ def stop(self): server.stop() self.assertEqual(msg, b"hello world!\n") + self.assertEqual(messages, []) + @support.skip_unless_bind_unix_socket def test_start_unix_server(self): @@ -685,6 +699,9 @@ def stop(self): writer.close() return msgback + messages = [] + self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx)) + # test the server variant with a coroutine as client handler with test_utils.unix_socket_path() as path: server = MyServer(self.loop, path) @@ -703,6 +720,8 @@ def stop(self): server.stop() self.assertEqual(msg, b"hello world!\n") + self.assertEqual(messages, []) + @unittest.skipIf(sys.platform == 'win32', "Don't have pipes") def test_read_all_from_pipe_reader(self): # See asyncio issue 168. This test is derived from the example @@ -893,6 +912,58 @@ def test_wait_closed_on_close_with_unread_data(self): wr.close() self.loop.run_until_complete(wr.wait_closed()) + def test_del_stream_before_sock_closing(self): + messages = [] + self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx)) + + with test_utils.run_test_server() as httpd: + rd, wr = self.loop.run_until_complete( + asyncio.open_connection(*httpd.address, loop=self.loop)) + sock = wr.get_extra_info('socket') + self.assertNotEqual(sock.fileno(), -1) + + wr.write(b'GET / HTTP/1.0\r\n\r\n') + f = rd.readline() + data = self.loop.run_until_complete(f) + self.assertEqual(data, b'HTTP/1.0 200 OK\r\n') + + # drop refs to reader/writer + del rd + del wr + gc.collect() + # make a chance to close the socket + test_utils.run_briefly(self.loop) + + self.assertEqual(1, len(messages)) + self.assertEqual(sock.fileno(), -1) + + self.assertEqual(1, len(messages)) + self.assertEqual('An open stream object is being garbage ' + 'collected; call "stream.close()" explicitly.', + messages[0]['message']) + + def test_del_stream_before_connection_made(self): + messages = [] + self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx)) + + with test_utils.run_test_server() as httpd: + rd = asyncio.StreamReader(loop=self.loop) + pr = asyncio.StreamReaderProtocol(rd, loop=self.loop) + del rd + gc.collect() + tr, _ = self.loop.run_until_complete( + self.loop.create_connection( + lambda: pr, *httpd.address)) + + sock = tr.get_extra_info('socket') + self.assertEqual(sock.fileno(), -1) + + self.assertEqual(1, len(messages)) + self.assertEqual('An open stream was garbage collected prior to ' + 'establishing network connection; ' + 'call "stream.close()" explicitly.', + messages[0]['message']) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS.d/next/Library/2018-09-12-10-33-44.bpo-34638.xaeZX5.rst b/Misc/NEWS.d/next/Library/2018-09-12-10-33-44.bpo-34638.xaeZX5.rst new file mode 100644 index 000000000000..13b3952a98eb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-12-10-33-44.bpo-34638.xaeZX5.rst @@ -0,0 +1,3 @@ +Store a weak reference to stream reader to break strong references loop +between reader and protocol. It allows to detect and close the socket if +the stream is deleted (garbage collected) without ``close()`` call. From webhook-mailer at python.org Wed Sep 12 14:43:37 2018 From: webhook-mailer at python.org (Ethan Furman) Date: Wed, 12 Sep 2018 18:43:37 -0000 Subject: [Python-checkins] bpo-34536: raise error for invalid _missing_ results (GH-9147) Message-ID: https://github.com/python/cpython/commit/019f0a0cb85ebc234356415f3638b9bd77528e55 commit: 019f0a0cb85ebc234356415f3638b9bd77528e55 branch: master author: Ethan Furman committer: GitHub date: 2018-09-12T11:43:34-07:00 summary: bpo-34536: raise error for invalid _missing_ results (GH-9147) * raise exception if _missing_ returns None or invalid type files: A Misc/NEWS.d/next/Library/2018-09-11-15-49-09.bpo-34536.3IPIH5.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 0839671cca00..02405c865b06 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -585,7 +585,25 @@ def __new__(cls, value): if member._value_ == value: return member # still not found -- try _missing_ hook - return cls._missing_(value) + try: + exc = None + result = cls._missing_(value) + except Exception as e: + exc = e + result = None + if isinstance(result, cls): + return result + else: + ve_exc = ValueError("%r is not a valid %s" % (value, cls.__name__)) + if result is None and exc is None: + raise ve_exc + elif exc is None: + exc = TypeError( + 'error in %s._missing_: returned %r instead of None or a valid member' + % (cls.__name__, result) + ) + exc.__context__ = ve_exc + raise exc def _generate_next_value_(name, start, count, last_values): for last_value in reversed(last_values): diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index c04d03f37523..b8efb835ce74 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -3,6 +3,7 @@ import pydoc import sys import unittest +import sys import threading from collections import OrderedDict from enum import Enum, IntEnum, EnumMeta, Flag, IntFlag, unique, auto @@ -1697,6 +1698,38 @@ class Dupes(Enum): third = auto() self.assertEqual([Dupes.first, Dupes.second, Dupes.third], list(Dupes)) + def test_missing(self): + class Color(Enum): + red = 1 + green = 2 + blue = 3 + @classmethod + def _missing_(cls, item): + if item == 'three': + return cls.blue + elif item == 'bad return': + # trigger internal error + return 5 + elif item == 'error out': + raise ZeroDivisionError + else: + # trigger not found + return None + self.assertIs(Color('three'), Color.blue) + self.assertRaises(ValueError, Color, 7) + try: + Color('bad return') + except TypeError as exc: + self.assertTrue(isinstance(exc.__context__, ValueError)) + else: + raise Exception('Exception not raised.') + try: + Color('error out') + except ZeroDivisionError as exc: + self.assertTrue(isinstance(exc.__context__, ValueError)) + else: + raise Exception('Exception not raised.') + class TestOrder(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2018-09-11-15-49-09.bpo-34536.3IPIH5.rst b/Misc/NEWS.d/next/Library/2018-09-11-15-49-09.bpo-34536.3IPIH5.rst new file mode 100644 index 000000000000..be45eb57cad5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-11-15-49-09.bpo-34536.3IPIH5.rst @@ -0,0 +1,2 @@ +`Enum._missing_`: raise `ValueError` if None returned and `TypeError` if +non-member is returned. From webhook-mailer at python.org Wed Sep 12 15:06:46 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 19:06:46 -0000 Subject: [Python-checkins] closes bpo-34646: Remove PyAPI_* macros from declarations. (GH-9218) Message-ID: https://github.com/python/cpython/commit/e5024517811ee990b770fca0ba7058742d00e032 commit: e5024517811ee990b770fca0ba7058742d00e032 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-12T12:06:42-07:00 summary: closes bpo-34646: Remove PyAPI_* macros from declarations. (GH-9218) files: M Modules/_hashopenssl.c M Modules/_io/winconsoleio.c M Modules/posixmodule.c M Objects/methodobject.c M Objects/object.c M Parser/printgrammar.c M Python/_warnings.c M Python/compile.c M Python/graminit.c M Python/import.c M Python/pystrhex.c M Python/pystrtod.c M Python/pythonrun.c diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c index 40cd6327312d..7121224e2ff1 100644 --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -245,7 +245,7 @@ EVP_hexdigest(EVPobject *self, PyObject *unused) EVP_MD_CTX_free(temp_ctx); - return _Py_strhex((const char *)digest, digest_size); + return _Py_strhex((const char *)digest, (Py_ssize_t)digest_size); } PyDoc_STRVAR(EVP_update__doc__, diff --git a/Modules/_io/winconsoleio.c b/Modules/_io/winconsoleio.c index f08406643f03..13342ec239d6 100644 --- a/Modules/_io/winconsoleio.c +++ b/Modules/_io/winconsoleio.c @@ -1170,6 +1170,6 @@ PyTypeObject PyWindowsConsoleIO_Type = { 0, /* tp_finalize */ }; -PyAPI_DATA(PyObject *) _PyWindowsConsoleIO_Type = (PyObject*)&PyWindowsConsoleIO_Type; +PyObject * _PyWindowsConsoleIO_Type = (PyObject*)&PyWindowsConsoleIO_Type; #endif /* MS_WINDOWS */ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index c5d9a17b002e..48bd5dc39415 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -488,8 +488,8 @@ PyOS_AfterFork(void) #ifdef MS_WINDOWS /* defined in fileutils.c */ -PyAPI_FUNC(void) _Py_time_t_to_FILE_TIME(time_t, int, FILETIME *); -PyAPI_FUNC(void) _Py_attribute_data_to_stat(BY_HANDLE_FILE_INFORMATION *, +void _Py_time_t_to_FILE_TIME(time_t, int, FILETIME *); +void _Py_attribute_data_to_stat(BY_HANDLE_FILE_INFORMATION *, ULONG, struct _Py_stat_struct *); #endif diff --git a/Objects/methodobject.c b/Objects/methodobject.c index a7042ca39e38..5ad283104ff3 100644 --- a/Objects/methodobject.c +++ b/Objects/methodobject.c @@ -18,7 +18,7 @@ static int numfree = 0; /* undefine macro trampoline to PyCFunction_NewEx */ #undef PyCFunction_New -PyAPI_FUNC(PyObject *) +PyObject * PyCFunction_New(PyMethodDef *ml, PyObject *self) { return PyCFunction_NewEx(ml, self, NULL); diff --git a/Objects/object.c b/Objects/object.c index 6498756c92bd..607f047d147d 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -2180,7 +2180,7 @@ _PyTrash_thread_destroy_chain(void) /* For Py_LIMITED_API, we need an out-of-line version of _Py_Dealloc. Define this here, so we can undefine the macro. */ #undef _Py_Dealloc -PyAPI_FUNC(void) _Py_Dealloc(PyObject *); +void _Py_Dealloc(PyObject *); void _Py_Dealloc(PyObject *op) { diff --git a/Parser/printgrammar.c b/Parser/printgrammar.c index 7311e55170de..1a8b0e176f2a 100644 --- a/Parser/printgrammar.c +++ b/Parser/printgrammar.c @@ -18,7 +18,7 @@ printgrammar(grammar *g, FILE *fp) fprintf(fp, "/* Generated by Parser/pgen */\n\n"); fprintf(fp, "#include \"pgenheaders.h\"\n"); fprintf(fp, "#include \"grammar.h\"\n"); - fprintf(fp, "PyAPI_DATA(grammar) _PyParser_Grammar;\n"); + fprintf(fp, "grammar _PyParser_Grammar;\n"); printdfas(g, fp); printlabels(g, fp); fprintf(fp, "grammar _PyParser_Grammar = {\n"); diff --git a/Python/_warnings.c b/Python/_warnings.c index 2229206b25c9..8e1c01d0b295 100644 --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -1004,7 +1004,7 @@ PyErr_WarnEx(PyObject *category, const char *text, Py_ssize_t stack_level) #undef PyErr_Warn -PyAPI_FUNC(int) +int PyErr_Warn(PyObject *category, const char *text) { return PyErr_WarnEx(category, text, 1); diff --git a/Python/compile.c b/Python/compile.c index 3528670ef675..c6a667c29efe 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -5550,7 +5550,7 @@ assemble(struct compiler *c, int addNone) } #undef PyAST_Compile -PyAPI_FUNC(PyCodeObject *) +PyCodeObject * PyAST_Compile(mod_ty mod, const char *filename, PyCompilerFlags *flags, PyArena *arena) { diff --git a/Python/graminit.c b/Python/graminit.c index 8e89ccea3bab..5770e8f6a941 100644 --- a/Python/graminit.c +++ b/Python/graminit.c @@ -2,7 +2,7 @@ #include "pgenheaders.h" #include "grammar.h" -PyAPI_DATA(grammar) _PyParser_Grammar; +grammar _PyParser_Grammar; static arc arcs_0_0[3] = { {2, 1}, {3, 1}, diff --git a/Python/import.c b/Python/import.c index 319b661f1f9b..2a9a5766583f 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1138,7 +1138,7 @@ get_path_importer(PyObject *path_importer_cache, PyObject *path_hooks, return importer; } -PyAPI_FUNC(PyObject *) +PyObject * PyImport_GetImporter(PyObject *path) { PyObject *importer=NULL, *path_importer_cache=NULL, *path_hooks=NULL; diff --git a/Python/pystrhex.c b/Python/pystrhex.c index 6dbf32dcc4f9..028f187c707c 100644 --- a/Python/pystrhex.c +++ b/Python/pystrhex.c @@ -2,6 +2,8 @@ #include "Python.h" +#include "pystrhex.h" + static PyObject *_Py_strhex_impl(const char* argbuf, const Py_ssize_t arglen, int return_bytes) { @@ -48,14 +50,14 @@ static PyObject *_Py_strhex_impl(const char* argbuf, const Py_ssize_t arglen, return retval; } -PyAPI_FUNC(PyObject *) _Py_strhex(const char* argbuf, const Py_ssize_t arglen) +PyObject * _Py_strhex(const char* argbuf, const Py_ssize_t arglen) { return _Py_strhex_impl(argbuf, arglen, 0); } /* Same as above but returns a bytes() instead of str() to avoid the * need to decode the str() when bytes are needed. */ -PyAPI_FUNC(PyObject *) _Py_strhex_bytes(const char* argbuf, const Py_ssize_t arglen) +PyObject * _Py_strhex_bytes(const char* argbuf, const Py_ssize_t arglen) { return _Py_strhex_impl(argbuf, arglen, 1); } diff --git a/Python/pystrtod.c b/Python/pystrtod.c index 461e8dcb5e0c..98aa9ba48bda 100644 --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -791,7 +791,7 @@ _PyOS_ascii_formatd(char *buffer, /* The fallback code to use if _Py_dg_dtoa is not available. */ -PyAPI_FUNC(char *) PyOS_double_to_string(double val, +char * PyOS_double_to_string(double val, char format_code, int precision, int flags, @@ -1238,7 +1238,7 @@ format_float_short(double d, char format_code, } -PyAPI_FUNC(char *) PyOS_double_to_string(double val, +char * PyOS_double_to_string(double val, char format_code, int precision, int flags, diff --git a/Python/pythonrun.c b/Python/pythonrun.c index 273eae25e2e1..43a98c96cd5c 100644 --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -1492,56 +1492,56 @@ PyOS_CheckStack(void) /* Deprecated C API functions still provided for binary compatibility */ #undef PyParser_SimpleParseFile -PyAPI_FUNC(node *) +node * PyParser_SimpleParseFile(FILE *fp, const char *filename, int start) { return PyParser_SimpleParseFileFlags(fp, filename, start, 0); } #undef PyParser_SimpleParseString -PyAPI_FUNC(node *) +node * PyParser_SimpleParseString(const char *str, int start) { return PyParser_SimpleParseStringFlags(str, start, 0); } #undef PyRun_AnyFile -PyAPI_FUNC(int) +int PyRun_AnyFile(FILE *fp, const char *name) { return PyRun_AnyFileExFlags(fp, name, 0, NULL); } #undef PyRun_AnyFileEx -PyAPI_FUNC(int) +int PyRun_AnyFileEx(FILE *fp, const char *name, int closeit) { return PyRun_AnyFileExFlags(fp, name, closeit, NULL); } #undef PyRun_AnyFileFlags -PyAPI_FUNC(int) +int PyRun_AnyFileFlags(FILE *fp, const char *name, PyCompilerFlags *flags) { return PyRun_AnyFileExFlags(fp, name, 0, flags); } #undef PyRun_File -PyAPI_FUNC(PyObject *) +PyObject * PyRun_File(FILE *fp, const char *p, int s, PyObject *g, PyObject *l) { return PyRun_FileExFlags(fp, p, s, g, l, 0, NULL); } #undef PyRun_FileEx -PyAPI_FUNC(PyObject *) +PyObject * PyRun_FileEx(FILE *fp, const char *p, int s, PyObject *g, PyObject *l, int c) { return PyRun_FileExFlags(fp, p, s, g, l, c, NULL); } #undef PyRun_FileFlags -PyAPI_FUNC(PyObject *) +PyObject * PyRun_FileFlags(FILE *fp, const char *p, int s, PyObject *g, PyObject *l, PyCompilerFlags *flags) { @@ -1549,14 +1549,14 @@ PyRun_FileFlags(FILE *fp, const char *p, int s, PyObject *g, PyObject *l, } #undef PyRun_SimpleFile -PyAPI_FUNC(int) +int PyRun_SimpleFile(FILE *f, const char *p) { return PyRun_SimpleFileExFlags(f, p, 0, NULL); } #undef PyRun_SimpleFileEx -PyAPI_FUNC(int) +int PyRun_SimpleFileEx(FILE *f, const char *p, int c) { return PyRun_SimpleFileExFlags(f, p, c, NULL); @@ -1564,28 +1564,28 @@ PyRun_SimpleFileEx(FILE *f, const char *p, int c) #undef PyRun_String -PyAPI_FUNC(PyObject *) +PyObject * PyRun_String(const char *str, int s, PyObject *g, PyObject *l) { return PyRun_StringFlags(str, s, g, l, NULL); } #undef PyRun_SimpleString -PyAPI_FUNC(int) +int PyRun_SimpleString(const char *s) { return PyRun_SimpleStringFlags(s, NULL); } #undef Py_CompileString -PyAPI_FUNC(PyObject *) +PyObject * Py_CompileString(const char *str, const char *p, int s) { return Py_CompileStringExFlags(str, p, s, NULL, -1); } #undef Py_CompileStringFlags -PyAPI_FUNC(PyObject *) +PyObject * Py_CompileStringFlags(const char *str, const char *p, int s, PyCompilerFlags *flags) { @@ -1593,14 +1593,14 @@ Py_CompileStringFlags(const char *str, const char *p, int s, } #undef PyRun_InteractiveOne -PyAPI_FUNC(int) +int PyRun_InteractiveOne(FILE *f, const char *p) { return PyRun_InteractiveOneFlags(f, p, NULL); } #undef PyRun_InteractiveLoop -PyAPI_FUNC(int) +int PyRun_InteractiveLoop(FILE *f, const char *p) { return PyRun_InteractiveLoopFlags(f, p, NULL); From webhook-mailer at python.org Wed Sep 12 15:14:38 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Wed, 12 Sep 2018 19:14:38 -0000 Subject: [Python-checkins] bpo-31577: Fix a crash in os.utime() in case of a bad ns argument. (GH-3752) Message-ID: https://github.com/python/cpython/commit/0bd1a2dcfdf36b181385ae61361e7692f4ebb0fd commit: 0bd1a2dcfdf36b181385ae61361e7692f4ebb0fd branch: master author: Oren Milman committer: Serhiy Storchaka date: 2018-09-12T22:14:35+03:00 summary: bpo-31577: Fix a crash in os.utime() in case of a bad ns argument. (GH-3752) files: A Misc/NEWS.d/next/Core and Builtins/2017-09-25-20-36-24.bpo-31577.jgYsSA.rst M Lib/test/test_os.py M Modules/posixmodule.c diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index 6658a61ea2a7..0a8ba5cf95f1 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -635,6 +635,22 @@ def test_utime_invalid_arguments(self): with self.assertRaises(ValueError): os.utime(self.fname, (5, 5), ns=(5, 5)) + @support.cpython_only + def test_issue31577(self): + # The interpreter shouldn't crash in case utime() received a bad + # ns argument. + def get_bad_int(divmod_ret_val): + class BadInt: + def __divmod__(*args): + return divmod_ret_val + return BadInt() + with self.assertRaises(TypeError): + os.utime(self.fname, ns=(get_bad_int(42), 1)) + with self.assertRaises(TypeError): + os.utime(self.fname, ns=(get_bad_int(()), 1)) + with self.assertRaises(TypeError): + os.utime(self.fname, ns=(get_bad_int((1, 2, 3)), 1)) + from test import mapping_tests diff --git a/Misc/NEWS.d/next/Core and Builtins/2017-09-25-20-36-24.bpo-31577.jgYsSA.rst b/Misc/NEWS.d/next/Core and Builtins/2017-09-25-20-36-24.bpo-31577.jgYsSA.rst new file mode 100644 index 000000000000..81428828af2a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2017-09-25-20-36-24.bpo-31577.jgYsSA.rst @@ -0,0 +1,2 @@ +Fix a crash in `os.utime()` in case of a bad ns argument. Patch by Oren +Milman. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 48bd5dc39415..7c02351a4661 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4630,6 +4630,12 @@ split_py_long_to_s_and_ns(PyObject *py_long, time_t *s, long *ns) divmod = PyNumber_Divmod(py_long, billion); if (!divmod) goto exit; + if (!PyTuple_Check(divmod) || PyTuple_GET_SIZE(divmod) != 2) { + PyErr_Format(PyExc_TypeError, + "%.200s.__divmod__() must return a 2-tuple, not %.200s", + Py_TYPE(py_long)->tp_name, Py_TYPE(divmod)->tp_name); + goto exit; + } *s = _PyLong_AsTime_t(PyTuple_GET_ITEM(divmod, 0)); if ((*s == -1) && PyErr_Occurred()) goto exit; From webhook-mailer at python.org Wed Sep 12 15:46:33 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 19:46:33 -0000 Subject: [Python-checkins] bpo-31577: Fix a crash in os.utime() in case of a bad ns argument. (GH-3752) Message-ID: https://github.com/python/cpython/commit/329ea4ef7cc3a907a64c6f0702fc93206b6744de commit: 329ea4ef7cc3a907a64c6f0702fc93206b6744de branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T12:46:30-07:00 summary: bpo-31577: Fix a crash in os.utime() in case of a bad ns argument. (GH-3752) (cherry picked from commit 0bd1a2dcfdf36b181385ae61361e7692f4ebb0fd) Co-authored-by: Oren Milman files: A Misc/NEWS.d/next/Core and Builtins/2017-09-25-20-36-24.bpo-31577.jgYsSA.rst M Lib/test/test_os.py M Modules/posixmodule.c diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index a140ae0d29ea..79ddc48eeb7d 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -635,6 +635,22 @@ def test_utime_invalid_arguments(self): with self.assertRaises(ValueError): os.utime(self.fname, (5, 5), ns=(5, 5)) + @support.cpython_only + def test_issue31577(self): + # The interpreter shouldn't crash in case utime() received a bad + # ns argument. + def get_bad_int(divmod_ret_val): + class BadInt: + def __divmod__(*args): + return divmod_ret_val + return BadInt() + with self.assertRaises(TypeError): + os.utime(self.fname, ns=(get_bad_int(42), 1)) + with self.assertRaises(TypeError): + os.utime(self.fname, ns=(get_bad_int(()), 1)) + with self.assertRaises(TypeError): + os.utime(self.fname, ns=(get_bad_int((1, 2, 3)), 1)) + from test import mapping_tests diff --git a/Misc/NEWS.d/next/Core and Builtins/2017-09-25-20-36-24.bpo-31577.jgYsSA.rst b/Misc/NEWS.d/next/Core and Builtins/2017-09-25-20-36-24.bpo-31577.jgYsSA.rst new file mode 100644 index 000000000000..81428828af2a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2017-09-25-20-36-24.bpo-31577.jgYsSA.rst @@ -0,0 +1,2 @@ +Fix a crash in `os.utime()` in case of a bad ns argument. Patch by Oren +Milman. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index d487e5386e89..bbbff1eca8ff 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4563,6 +4563,12 @@ split_py_long_to_s_and_ns(PyObject *py_long, time_t *s, long *ns) divmod = PyNumber_Divmod(py_long, billion); if (!divmod) goto exit; + if (!PyTuple_Check(divmod) || PyTuple_GET_SIZE(divmod) != 2) { + PyErr_Format(PyExc_TypeError, + "%.200s.__divmod__() must return a 2-tuple, not %.200s", + Py_TYPE(py_long)->tp_name, Py_TYPE(divmod)->tp_name); + goto exit; + } *s = _PyLong_AsTime_t(PyTuple_GET_ITEM(divmod, 0)); if ((*s == -1) && PyErr_Occurred()) goto exit; From webhook-mailer at python.org Wed Sep 12 16:48:07 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 20:48:07 -0000 Subject: [Python-checkins] closes bpo-34004: Skip lock interruption tests on musl. (GH-9224) Message-ID: https://github.com/python/cpython/commit/5b10d5111d7a855297654af9045f8907b7d3dd08 commit: 5b10d5111d7a855297654af9045f8907b7d3dd08 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-12T13:48:03-07:00 summary: closes bpo-34004: Skip lock interruption tests on musl. (GH-9224) Returning EINTR from pthread semaphore or lock acquisition is an optional POSIX feature. musl does not provide this feature, so some threadsignal tests fail when Python is built against it. There's no good way to test for musl, so we skip if we're on Linux and not using glibc pthreads. Also, hedge in the threading documentation about when we can provide interrupts from lock acquisition. files: M Doc/library/threading.rst M Lib/test/test_threadsignals.py diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index b94021b4eb8f..e6185c5b3a21 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -400,7 +400,8 @@ All methods are executed atomically. The *timeout* parameter is new. .. versionchanged:: 3.2 - Lock acquires can now be interrupted by signals on POSIX. + Lock acquisition can now be interrupted by signals on POSIX if the + underlying threading implementation supports it. .. method:: release() diff --git a/Lib/test/test_threadsignals.py b/Lib/test/test_threadsignals.py index 1ad6c63fea2e..7e13b1720f91 100644 --- a/Lib/test/test_threadsignals.py +++ b/Lib/test/test_threadsignals.py @@ -78,6 +78,10 @@ def alarm_interrupt(self, sig, frame): @unittest.skipIf(USING_PTHREAD_COND, 'POSIX condition variables cannot be interrupted') + @unittest.skipIf(sys.platform.startswith('linux') and + not sys.thread_info.version, + 'Issue 34004: musl does not allow interruption of locks ' + 'by signals.') # Issue #20564: sem_timedwait() cannot be interrupted on OpenBSD @unittest.skipIf(sys.platform.startswith('openbsd'), 'lock cannot be interrupted on OpenBSD') @@ -105,6 +109,10 @@ def test_lock_acquire_interruption(self): @unittest.skipIf(USING_PTHREAD_COND, 'POSIX condition variables cannot be interrupted') + @unittest.skipIf(sys.platform.startswith('linux') and + not sys.thread_info.version, + 'Issue 34004: musl does not allow interruption of locks ' + 'by signals.') # Issue #20564: sem_timedwait() cannot be interrupted on OpenBSD @unittest.skipIf(sys.platform.startswith('openbsd'), 'lock cannot be interrupted on OpenBSD') From webhook-mailer at python.org Wed Sep 12 17:03:57 2018 From: webhook-mailer at python.org (Andrew Svetlov) Date: Wed, 12 Sep 2018 21:03:57 -0000 Subject: [Python-checkins] bpo-34630: Skip logging SSL certificate errors by asyncio code (GH-9169) Message-ID: https://github.com/python/cpython/commit/0dd71807a98c4a86ece2aea869ea99f09204b16b commit: 0dd71807a98c4a86ece2aea869ea99f09204b16b branch: master author: Andrew Svetlov committer: GitHub date: 2018-09-12T14:03:54-07:00 summary: bpo-34630: Skip logging SSL certificate errors by asyncio code (GH-9169) files: A Misc/NEWS.d/next/Library/2018-09-11-10-00-53.bpo-34630.YbqUS6.rst M Lib/asyncio/base_events.py diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index 046743864fdd..492e377d09e0 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -62,6 +62,9 @@ _FATAL_ERROR_IGNORE = (BrokenPipeError, ConnectionResetError, ConnectionAbortedError) +if ssl is not None: + _FATAL_ERROR_IGNORE = _FATAL_ERROR_IGNORE + (ssl.SSLCertVerificationError,) + _HAS_IPv6 = hasattr(socket, 'AF_INET6') # Maximum timeout passed to select to avoid OS limitations diff --git a/Misc/NEWS.d/next/Library/2018-09-11-10-00-53.bpo-34630.YbqUS6.rst b/Misc/NEWS.d/next/Library/2018-09-11-10-00-53.bpo-34630.YbqUS6.rst new file mode 100644 index 000000000000..452bcb6142a0 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-11-10-00-53.bpo-34630.YbqUS6.rst @@ -0,0 +1,2 @@ +Don't log SSL certificate errors in asyncio code (connection error logging +is skipped already). From webhook-mailer at python.org Wed Sep 12 17:05:23 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Wed, 12 Sep 2018 21:05:23 -0000 Subject: [Python-checkins] bpo-34649: Add missing NULL checks to _encoded_const() (GH-9225) Message-ID: https://github.com/python/cpython/commit/6f82bffd2df63a4072b3f0483cdbe93ddedb87e9 commit: 6f82bffd2df63a4072b3f0483cdbe93ddedb87e9 branch: master author: Alexey Izbyshev committer: Berker Peksag date: 2018-09-13T00:05:20+03:00 summary: bpo-34649: Add missing NULL checks to _encoded_const() (GH-9225) Reported by Svace static analyzer. files: M Modules/_json.c diff --git a/Modules/_json.c b/Modules/_json.c index 5a9464e34fb7..ac6e017a4eaf 100644 --- a/Modules/_json.c +++ b/Modules/_json.c @@ -1374,7 +1374,7 @@ _encoded_const(PyObject *obj) if (s_null == NULL) { s_null = PyUnicode_InternFromString("null"); } - Py_INCREF(s_null); + Py_XINCREF(s_null); return s_null; } else if (obj == Py_True) { @@ -1382,7 +1382,7 @@ _encoded_const(PyObject *obj) if (s_true == NULL) { s_true = PyUnicode_InternFromString("true"); } - Py_INCREF(s_true); + Py_XINCREF(s_true); return s_true; } else if (obj == Py_False) { @@ -1390,7 +1390,7 @@ _encoded_const(PyObject *obj) if (s_false == NULL) { s_false = PyUnicode_InternFromString("false"); } - Py_INCREF(s_false); + Py_XINCREF(s_false); return s_false; } else { From webhook-mailer at python.org Wed Sep 12 17:10:59 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 21:10:59 -0000 Subject: [Python-checkins] closes bpo-34004: Skip lock interruption tests on musl. (GH-9224) Message-ID: https://github.com/python/cpython/commit/b608fcd444c00ff37a19d34e4eeadb1221fb6436 commit: b608fcd444c00ff37a19d34e4eeadb1221fb6436 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T14:10:56-07:00 summary: closes bpo-34004: Skip lock interruption tests on musl. (GH-9224) Returning EINTR from pthread semaphore or lock acquisition is an optional POSIX feature. musl does not provide this feature, so some threadsignal tests fail when Python is built against it. There's no good way to test for musl, so we skip if we're on Linux and not using glibc pthreads. Also, hedge in the threading documentation about when we can provide interrupts from lock acquisition. (cherry picked from commit 5b10d5111d7a855297654af9045f8907b7d3dd08) Co-authored-by: Benjamin Peterson files: M Doc/library/threading.rst M Lib/test/test_threadsignals.py diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index b94021b4eb8f..e6185c5b3a21 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -400,7 +400,8 @@ All methods are executed atomically. The *timeout* parameter is new. .. versionchanged:: 3.2 - Lock acquires can now be interrupted by signals on POSIX. + Lock acquisition can now be interrupted by signals on POSIX if the + underlying threading implementation supports it. .. method:: release() diff --git a/Lib/test/test_threadsignals.py b/Lib/test/test_threadsignals.py index 1ad6c63fea2e..7e13b1720f91 100644 --- a/Lib/test/test_threadsignals.py +++ b/Lib/test/test_threadsignals.py @@ -78,6 +78,10 @@ def alarm_interrupt(self, sig, frame): @unittest.skipIf(USING_PTHREAD_COND, 'POSIX condition variables cannot be interrupted') + @unittest.skipIf(sys.platform.startswith('linux') and + not sys.thread_info.version, + 'Issue 34004: musl does not allow interruption of locks ' + 'by signals.') # Issue #20564: sem_timedwait() cannot be interrupted on OpenBSD @unittest.skipIf(sys.platform.startswith('openbsd'), 'lock cannot be interrupted on OpenBSD') @@ -105,6 +109,10 @@ def test_lock_acquire_interruption(self): @unittest.skipIf(USING_PTHREAD_COND, 'POSIX condition variables cannot be interrupted') + @unittest.skipIf(sys.platform.startswith('linux') and + not sys.thread_info.version, + 'Issue 34004: musl does not allow interruption of locks ' + 'by signals.') # Issue #20564: sem_timedwait() cannot be interrupted on OpenBSD @unittest.skipIf(sys.platform.startswith('openbsd'), 'lock cannot be interrupted on OpenBSD') From webhook-mailer at python.org Wed Sep 12 17:11:48 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 21:11:48 -0000 Subject: [Python-checkins] closes bpo-34004: Skip lock interruption tests on musl. (GH-9224) Message-ID: https://github.com/python/cpython/commit/5a435eac1b83f080c9dfceff0de0d639541e4bcb commit: 5a435eac1b83f080c9dfceff0de0d639541e4bcb branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T14:11:45-07:00 summary: closes bpo-34004: Skip lock interruption tests on musl. (GH-9224) Returning EINTR from pthread semaphore or lock acquisition is an optional POSIX feature. musl does not provide this feature, so some threadsignal tests fail when Python is built against it. There's no good way to test for musl, so we skip if we're on Linux and not using glibc pthreads. Also, hedge in the threading documentation about when we can provide interrupts from lock acquisition. (cherry picked from commit 5b10d5111d7a855297654af9045f8907b7d3dd08) Co-authored-by: Benjamin Peterson files: M Doc/library/threading.rst M Lib/test/test_threadsignals.py diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index 26e6a35bfba2..063d2c066366 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -400,7 +400,8 @@ All methods are executed atomically. The *timeout* parameter is new. .. versionchanged:: 3.2 - Lock acquires can now be interrupted by signals on POSIX. + Lock acquisition can now be interrupted by signals on POSIX if the + underlying threading implementation supports it. .. method:: release() diff --git a/Lib/test/test_threadsignals.py b/Lib/test/test_threadsignals.py index 99b60cd9e6b1..9083e419d1d5 100644 --- a/Lib/test/test_threadsignals.py +++ b/Lib/test/test_threadsignals.py @@ -78,6 +78,10 @@ def alarm_interrupt(self, sig, frame): @unittest.skipIf(USING_PTHREAD_COND, 'POSIX condition variables cannot be interrupted') + @unittest.skipIf(sys.platform.startswith('linux') and + not sys.thread_info.version, + 'Issue 34004: musl does not allow interruption of locks ' + 'by signals.') # Issue #20564: sem_timedwait() cannot be interrupted on OpenBSD @unittest.skipIf(sys.platform.startswith('openbsd'), 'lock cannot be interrupted on OpenBSD') @@ -105,6 +109,10 @@ def test_lock_acquire_interruption(self): @unittest.skipIf(USING_PTHREAD_COND, 'POSIX condition variables cannot be interrupted') + @unittest.skipIf(sys.platform.startswith('linux') and + not sys.thread_info.version, + 'Issue 34004: musl does not allow interruption of locks ' + 'by signals.') # Issue #20564: sem_timedwait() cannot be interrupted on OpenBSD @unittest.skipIf(sys.platform.startswith('openbsd'), 'lock cannot be interrupted on OpenBSD') From webhook-mailer at python.org Wed Sep 12 17:46:44 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Wed, 12 Sep 2018 21:46:44 -0000 Subject: [Python-checkins] bpo-34649: Add missing NULL checks to _encoded_const() (GH-9225) Message-ID: https://github.com/python/cpython/commit/ec4d099b9f1951b08d1a53181c278173ff6a0cc1 commit: ec4d099b9f1951b08d1a53181c278173ff6a0cc1 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Berker Peksag date: 2018-09-13T00:46:40+03:00 summary: bpo-34649: Add missing NULL checks to _encoded_const() (GH-9225) Reported by Svace static analyzer. (cherry picked from commit 6f82bffd2df63a4072b3f0483cdbe93ddedb87e9) Co-authored-by: Alexey Izbyshev files: M Modules/_json.c diff --git a/Modules/_json.c b/Modules/_json.c index 5a9464e34fb7..ac6e017a4eaf 100644 --- a/Modules/_json.c +++ b/Modules/_json.c @@ -1374,7 +1374,7 @@ _encoded_const(PyObject *obj) if (s_null == NULL) { s_null = PyUnicode_InternFromString("null"); } - Py_INCREF(s_null); + Py_XINCREF(s_null); return s_null; } else if (obj == Py_True) { @@ -1382,7 +1382,7 @@ _encoded_const(PyObject *obj) if (s_true == NULL) { s_true = PyUnicode_InternFromString("true"); } - Py_INCREF(s_true); + Py_XINCREF(s_true); return s_true; } else if (obj == Py_False) { @@ -1390,7 +1390,7 @@ _encoded_const(PyObject *obj) if (s_false == NULL) { s_false = PyUnicode_InternFromString("false"); } - Py_INCREF(s_false); + Py_XINCREF(s_false); return s_false; } else { From webhook-mailer at python.org Wed Sep 12 18:00:14 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Wed, 12 Sep 2018 22:00:14 -0000 Subject: [Python-checkins] bpo-34649: Add missing NULL checks to _encoded_const() (GH-9225) Message-ID: https://github.com/python/cpython/commit/6d726868cd1743623a28b8e048e31b9c3c52a399 commit: 6d726868cd1743623a28b8e048e31b9c3c52a399 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Berker Peksag date: 2018-09-13T01:00:11+03:00 summary: bpo-34649: Add missing NULL checks to _encoded_const() (GH-9225) Reported by Svace static analyzer. (cherry picked from commit 6f82bffd2df63a4072b3f0483cdbe93ddedb87e9) Co-authored-by: Alexey Izbyshev files: M Modules/_json.c diff --git a/Modules/_json.c b/Modules/_json.c index 8f1743bada87..dad6b01f822f 100644 --- a/Modules/_json.c +++ b/Modules/_json.c @@ -1376,7 +1376,7 @@ _encoded_const(PyObject *obj) if (s_null == NULL) { s_null = PyUnicode_InternFromString("null"); } - Py_INCREF(s_null); + Py_XINCREF(s_null); return s_null; } else if (obj == Py_True) { @@ -1384,7 +1384,7 @@ _encoded_const(PyObject *obj) if (s_true == NULL) { s_true = PyUnicode_InternFromString("true"); } - Py_INCREF(s_true); + Py_XINCREF(s_true); return s_true; } else if (obj == Py_False) { @@ -1392,7 +1392,7 @@ _encoded_const(PyObject *obj) if (s_false == NULL) { s_false = PyUnicode_InternFromString("false"); } - Py_INCREF(s_false); + Py_XINCREF(s_false); return s_false; } else { From webhook-mailer at python.org Wed Sep 12 18:00:59 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Wed, 12 Sep 2018 22:00:59 -0000 Subject: [Python-checkins] bpo-34649: Add missing NULL checks to _encoded_const() (GH-9225) Message-ID: https://github.com/python/cpython/commit/669429fb583031c7c87392e30b065e99a2d8ccda commit: 669429fb583031c7c87392e30b065e99a2d8ccda branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Berker Peksag date: 2018-09-13T01:00:56+03:00 summary: bpo-34649: Add missing NULL checks to _encoded_const() (GH-9225) Reported by Svace static analyzer. (cherry picked from commit 6f82bffd2df63a4072b3f0483cdbe93ddedb87e9) Co-authored-by: Alexey Izbyshev files: M Modules/_json.c diff --git a/Modules/_json.c b/Modules/_json.c index 28c0b3f4570e..3a88882f0c98 100644 --- a/Modules/_json.c +++ b/Modules/_json.c @@ -1893,7 +1893,7 @@ _encoded_const(PyObject *obj) if (s_null == NULL) { s_null = PyString_InternFromString("null"); } - Py_INCREF(s_null); + Py_XINCREF(s_null); return s_null; } else if (obj == Py_True) { @@ -1901,7 +1901,7 @@ _encoded_const(PyObject *obj) if (s_true == NULL) { s_true = PyString_InternFromString("true"); } - Py_INCREF(s_true); + Py_XINCREF(s_true); return s_true; } else if (obj == Py_False) { @@ -1909,7 +1909,7 @@ _encoded_const(PyObject *obj) if (s_false == NULL) { s_false = PyString_InternFromString("false"); } - Py_INCREF(s_false); + Py_XINCREF(s_false); return s_false; } else { From webhook-mailer at python.org Wed Sep 12 18:12:27 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 22:12:27 -0000 Subject: [Python-checkins] closes bpo-34650: Check if sched_getscheduler returns ENOSYS before declaring it supported. (GH-9228) Message-ID: https://github.com/python/cpython/commit/c7042224b8a67748f125c22836862483f81a87a6 commit: c7042224b8a67748f125c22836862483f81a87a6 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-12T15:12:24-07:00 summary: closes bpo-34650: Check if sched_getscheduler returns ENOSYS before declaring it supported. (GH-9228) musl doesn't support the scheduler API, but declares stubs that alway return ENOSYS. files: M Lib/test/test_posix.py diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py index 7a2fc263cb27..d402d4fb088c 100644 --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -25,6 +25,18 @@ requires_32b = unittest.skipUnless(sys.maxsize < 2**32, 'test is only meaningful on 32-bit builds') +def _supports_sched(): + if not hasattr(posix, 'sched_getscheduler'): + return False + try: + posix.sched_getscheduler(0) + except OSError as e: + if e.errno == errno.ENOSYS: + return False + return True + +requires_sched = unittest.skipUnless(_supports_sched(), 'requires POSIX scheduler API') + class PosixTester(unittest.TestCase): def setUp(self): @@ -1273,7 +1285,7 @@ def test_sched_priority(self): self.assertRaises(OSError, posix.sched_get_priority_min, -23) self.assertRaises(OSError, posix.sched_get_priority_max, -23) - @unittest.skipUnless(hasattr(posix, 'sched_setscheduler'), "can't change scheduler") + @requires_sched def test_get_and_set_scheduler_and_param(self): possible_schedulers = [sched for name, sched in posix.__dict__.items() if name.startswith("SCHED_")] @@ -1646,7 +1658,7 @@ def test_setsigdef_wrong_type(self): [sys.executable, "-c", "pass"], os.environ, setsigdef=[signal.NSIG, signal.NSIG+1]) - @unittest.skipUnless(hasattr(posix, 'sched_setscheduler'), "can't change scheduler") + @requires_sched def test_setscheduler_only_param(self): policy = os.sched_getscheduler(0) priority = os.sched_get_priority_min(policy) @@ -1664,7 +1676,7 @@ def test_setscheduler_only_param(self): ) self.assertEqual(os.waitpid(pid, 0), (pid, 0)) - @unittest.skipUnless(hasattr(posix, 'sched_setscheduler'), "can't change scheduler") + @requires_sched def test_setscheduler_with_policy(self): policy = os.sched_getscheduler(0) priority = os.sched_get_priority_min(policy) From webhook-mailer at python.org Wed Sep 12 18:21:19 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Wed, 12 Sep 2018 22:21:19 -0000 Subject: [Python-checkins] bpo-32933: Implement __iter__ method on mock_open() (GH-5974) Message-ID: https://github.com/python/cpython/commit/2087023fdec2c89070bd14f384a3c308c548a94a commit: 2087023fdec2c89070bd14f384a3c308c548a94a branch: master author: Tony Flury committer: Berker Peksag date: 2018-09-13T01:21:16+03:00 summary: bpo-32933: Implement __iter__ method on mock_open() (GH-5974) files: A Misc/NEWS.d/next/Library/2018-04-30-22-43-31.bpo-32933.M3iI_y.rst M Doc/library/unittest.mock.rst M Lib/unittest/mock.py M Lib/unittest/test/testmock/testmock.py M Lib/unittest/test/testmock/testwith.py diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index fd4e067546e5..d1b18d08f797 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -2095,6 +2095,10 @@ mock_open .. versionchanged:: 3.5 *read_data* is now reset on each call to the *mock*. + .. versionchanged:: 3.8 + Added :meth:`__iter__` to implementation so that iteration (such as in for + loops) correctly consumes *read_data*. + Using :func:`open` as a context manager is a great way to ensure your file handles are closed properly and is becoming common:: diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py index db1e642c00b7..83026e6f3bd3 100644 --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -2358,14 +2358,16 @@ def _read_side_effect(*args, **kwargs): return type(read_data)().join(_state[0]) def _readline_side_effect(): + yield from _iter_side_effect() + while True: + yield type(read_data)() + + def _iter_side_effect(): if handle.readline.return_value is not None: while True: yield handle.readline.return_value for line in _state[0]: yield line - while True: - yield type(read_data)() - global file_spec if file_spec is None: @@ -2389,6 +2391,7 @@ def _readline_side_effect(): _state[1] = _readline_side_effect() handle.readline.side_effect = _state[1] handle.readlines.side_effect = _readlines_side_effect + handle.__iter__.side_effect = _iter_side_effect def reset_data(*args, **kwargs): _state[0] = _iterate_read_data(read_data) diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py index b64c8663d212..c7bfa277b511 100644 --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -1450,6 +1450,16 @@ def test_mock_open_reuse_issue_21750(self): f2_data = f2.read() self.assertEqual(f1_data, f2_data) + def test_mock_open_dunder_iter_issue(self): + # Test dunder_iter method generates the expected result and + # consumes the iterator. + mocked_open = mock.mock_open(read_data='Remarkable\nNorwegian Blue') + f1 = mocked_open('a-name') + lines = [line for line in f1] + self.assertEqual(lines[0], 'Remarkable\n') + self.assertEqual(lines[1], 'Norwegian Blue') + self.assertEqual(list(f1), []) + def test_mock_open_write(self): # Test exception in file writing write() mock_namedtemp = mock.mock_open(mock.MagicMock(name='JLV')) diff --git a/Lib/unittest/test/testmock/testwith.py b/Lib/unittest/test/testmock/testwith.py index a7bee7300301..43b36a119952 100644 --- a/Lib/unittest/test/testmock/testwith.py +++ b/Lib/unittest/test/testmock/testwith.py @@ -188,6 +188,7 @@ def test_read_data(self): def test_readline_data(self): # Check that readline will return all the lines from the fake file + # And that once fully consumed, readline will return an empty string. mock = mock_open(read_data='foo\nbar\nbaz\n') with patch('%s.open' % __name__, mock, create=True): h = open('bar') @@ -197,6 +198,7 @@ def test_readline_data(self): self.assertEqual(line1, 'foo\n') self.assertEqual(line2, 'bar\n') self.assertEqual(line3, 'baz\n') + self.assertEqual(h.readline(), '') # Check that we properly emulate a file that doesn't end in a newline mock = mock_open(read_data='foo') @@ -204,6 +206,19 @@ def test_readline_data(self): h = open('bar') result = h.readline() self.assertEqual(result, 'foo') + self.assertEqual(h.readline(), '') + + + def test_dunder_iter_data(self): + # Check that dunder_iter will return all the lines from the fake file. + mock = mock_open(read_data='foo\nbar\nbaz\n') + with patch('%s.open' % __name__, mock, create=True): + h = open('bar') + lines = [l for l in h] + self.assertEqual(lines[0], 'foo\n') + self.assertEqual(lines[1], 'bar\n') + self.assertEqual(lines[2], 'baz\n') + self.assertEqual(h.readline(), '') def test_readlines_data(self): diff --git a/Misc/NEWS.d/next/Library/2018-04-30-22-43-31.bpo-32933.M3iI_y.rst b/Misc/NEWS.d/next/Library/2018-04-30-22-43-31.bpo-32933.M3iI_y.rst new file mode 100644 index 000000000000..4de7a8f927d5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-04-30-22-43-31.bpo-32933.M3iI_y.rst @@ -0,0 +1,2 @@ +:func:`unittest.mock.mock_open` now supports iteration over the file +contents. Patch by Tony Flury. From webhook-mailer at python.org Wed Sep 12 18:52:43 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 22:52:43 -0000 Subject: [Python-checkins] closes bpo-34652: Always disable lchmod on Linux. (GH-9234) Message-ID: https://github.com/python/cpython/commit/40caa05fa4d1810a1a6bfc34e0ec930c351089b7 commit: 40caa05fa4d1810a1a6bfc34e0ec930c351089b7 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-12T15:52:40-07:00 summary: closes bpo-34652: Always disable lchmod on Linux. (GH-9234) files: A Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst M configure M configure.ac diff --git a/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst b/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst new file mode 100644 index 000000000000..cbdd7e0ec6ce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst @@ -0,0 +1 @@ +Ensure :func:`os.lchmod` is never defined on Linux. diff --git a/configure b/configure index fe642c439a4c..d341453443fc 100755 --- a/configure +++ b/configure @@ -11285,6 +11285,17 @@ fi done +# Force lchmod off for Linux. Linux disallows changing the mode of symbolic +# links. Some libc implementations have a stub lchmod implementation that always +# returns an error. +if test "$MACHDEP" != linux; then + ac_fn_c_check_func "$LINENO" "lchmod" "ac_cv_func_lchmod" +if test "x$ac_cv_func_lchmod" = xyes; then : + +fi + +fi + ac_fn_c_check_decl "$LINENO" "dirfd" "ac_cv_have_decl_dirfd" "#include #include " diff --git a/configure.ac b/configure.ac index a985ec2b000b..cf7328c4032a 100644 --- a/configure.ac +++ b/configure.ac @@ -3454,6 +3454,13 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ truncate uname unlinkat unsetenv utimensat utimes waitid waitpid wait3 wait4 \ wcscoll wcsftime wcsxfrm wmemcmp writev _getpty) +# Force lchmod off for Linux. Linux disallows changing the mode of symbolic +# links. Some libc implementations have a stub lchmod implementation that always +# returns an error. +if test "$MACHDEP" != linux; then + AC_CHECK_FUNC(lchmod) +fi + AC_CHECK_DECL(dirfd, AC_DEFINE(HAVE_DIRFD, 1, Define if you have the 'dirfd' function or macro.), , From webhook-mailer at python.org Wed Sep 12 19:00:09 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 23:00:09 -0000 Subject: [Python-checkins] [3.7] closes bpo-34650: Check if sched_getscheduler returns ENOSYS before declaring it supported. (GH-9236) Message-ID: https://github.com/python/cpython/commit/0aef909d630ff40614cde0c58796a8bdf66c67eb commit: 0aef909d630ff40614cde0c58796a8bdf66c67eb branch: 3.7 author: Benjamin Peterson committer: GitHub date: 2018-09-12T16:00:06-07:00 summary: [3.7] closes bpo-34650: Check if sched_getscheduler returns ENOSYS before declaring it supported. (GH-9236) musl doesn't support the scheduler API, but declares stubs that alway return ENOSYS.. (cherry picked from commit c7042224b8a67748f125c22836862483f81a87a6) Co-authored-by: Benjamin Peterson files: M Lib/test/test_posix.py diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py index d2cb1c9840ea..11180b7278c9 100644 --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -23,6 +23,18 @@ requires_32b = unittest.skipUnless(sys.maxsize < 2**32, 'test is only meaningful on 32-bit builds') +def _supports_sched(): + if not hasattr(posix, 'sched_getscheduler'): + return False + try: + posix.sched_getscheduler(0) + except OSError as e: + if e.errno == errno.ENOSYS: + return False + return True + +requires_sched = unittest.skipUnless(_supports_sched(), 'requires POSIX scheduler API') + class PosixTester(unittest.TestCase): def setUp(self): @@ -1276,7 +1288,7 @@ def test_sched_priority(self): self.assertRaises(OSError, posix.sched_get_priority_min, -23) self.assertRaises(OSError, posix.sched_get_priority_max, -23) - @unittest.skipUnless(hasattr(posix, 'sched_setscheduler'), "can't change scheduler") + @requires_sched def test_get_and_set_scheduler_and_param(self): possible_schedulers = [sched for name, sched in posix.__dict__.items() if name.startswith("SCHED_")] From webhook-mailer at python.org Wed Sep 12 19:00:30 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 23:00:30 -0000 Subject: [Python-checkins] [3.6] closes bpo-34650: Check if sched_getscheduler returns ENOSYS before declaring it supported. (GH-9237) Message-ID: https://github.com/python/cpython/commit/e105e551dbf072d773aaa1fd6f8cbda218c273f0 commit: e105e551dbf072d773aaa1fd6f8cbda218c273f0 branch: 3.6 author: Benjamin Peterson committer: GitHub date: 2018-09-12T16:00:27-07:00 summary: [3.6] closes bpo-34650: Check if sched_getscheduler returns ENOSYS before declaring it supported. (GH-9237) musl doesn't support the scheduler API, but declares stubs that alway return ENOSYS.. (cherry picked from commit c7042224b8a67748f125c22836862483f81a87a6) Co-authored-by: Benjamin Peterson files: M Lib/test/test_posix.py diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py index a204377d9af4..2dae473327f1 100644 --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -22,6 +22,18 @@ requires_32b = unittest.skipUnless(sys.maxsize < 2**32, 'test is only meaningful on 32-bit builds') +def _supports_sched(): + if not hasattr(posix, 'sched_getscheduler'): + return False + try: + posix.sched_getscheduler(0) + except OSError as e: + if e.errno == errno.ENOSYS: + return False + return True + +requires_sched = unittest.skipUnless(_supports_sched(), 'requires POSIX scheduler API') + class PosixTester(unittest.TestCase): def setUp(self): @@ -1136,7 +1148,7 @@ def test_sched_priority(self): self.assertRaises(OSError, posix.sched_get_priority_min, -23) self.assertRaises(OSError, posix.sched_get_priority_max, -23) - @unittest.skipUnless(hasattr(posix, 'sched_setscheduler'), "can't change scheduler") + @requires_sched def test_get_and_set_scheduler_and_param(self): possible_schedulers = [sched for name, sched in posix.__dict__.items() if name.startswith("SCHED_")] From webhook-mailer at python.org Wed Sep 12 19:21:42 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 23:21:42 -0000 Subject: [Python-checkins] closes bpo-34654: Tolerate + at the beginning of large years. (GH-9238) Message-ID: https://github.com/python/cpython/commit/e1a34ceb541ef87e03bb428630097dacc9c658e5 commit: e1a34ceb541ef87e03bb428630097dacc9c658e5 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-12T16:21:36-07:00 summary: closes bpo-34654: Tolerate + at the beginning of large years. (GH-9238) files: M Lib/test/test_time.py diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py index 9acd1d497ee8..62abd891aafa 100644 --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -703,9 +703,9 @@ def test_year(self, fmt=None, func=None): self.assertEqual(func(9999), fmt % 9999) def test_large_year(self): - self.assertEqual(self.yearstr(12345), '12345') - self.assertEqual(self.yearstr(123456789), '123456789') - self.assertEqual(self.yearstr(TIME_MAXYEAR), str(TIME_MAXYEAR)) + self.assertEqual(self.yearstr(12345).lstrip('+'), '12345') + self.assertEqual(self.yearstr(123456789).lstrip('+'), '123456789') + self.assertEqual(self.yearstr(TIME_MAXYEAR).lstrip('+'), str(TIME_MAXYEAR)) self.assertRaises(OverflowError, self.yearstr, TIME_MAXYEAR + 1) def test_negative(self): From webhook-mailer at python.org Wed Sep 12 19:31:21 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 12 Sep 2018 23:31:21 -0000 Subject: [Python-checkins] [2.7] closes bpo-34652: Always disable lchmod on Linux. (GH-9242) Message-ID: https://github.com/python/cpython/commit/69e96910153219b0b15a18323b917bd74336d229 commit: 69e96910153219b0b15a18323b917bd74336d229 branch: 2.7 author: Benjamin Peterson committer: GitHub date: 2018-09-12T16:31:17-07:00 summary: [2.7] closes bpo-34652: Always disable lchmod on Linux. (GH-9242) (cherry picked from commit 40caa05fa4d1810a1a6bfc34e0ec930c351089b7) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst M aclocal.m4 M configure M configure.ac M pyconfig.h.in diff --git a/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst b/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst new file mode 100644 index 000000000000..cbdd7e0ec6ce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst @@ -0,0 +1 @@ +Ensure :func:`os.lchmod` is never defined on Linux. diff --git a/aclocal.m4 b/aclocal.m4 index 5fadcb1e4457..5e29449ae2a1 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -1,6 +1,6 @@ -# generated automatically by aclocal 1.15 -*- Autoconf -*- +# generated automatically by aclocal 1.15.1 -*- Autoconf -*- -# Copyright (C) 1996-2014 Free Software Foundation, Inc. +# Copyright (C) 1996-2017 Free Software Foundation, Inc. # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -12,9 +12,9 @@ # PARTICULAR PURPOSE. m4_ifndef([AC_CONFIG_MACRO_DIRS], [m4_defun([_AM_CONFIG_MACRO_DIRS], [])m4_defun([AC_CONFIG_MACRO_DIRS], [_AM_CONFIG_MACRO_DIRS($@)])]) -# pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*- -# serial 11 (pkg-config-0.29.1) - +dnl pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*- +dnl serial 11 (pkg-config-0.29.1) +dnl dnl Copyright ? 2004 Scott James Remnant . dnl Copyright ? 2012-2015 Dan Nicholson dnl @@ -288,71 +288,3 @@ AS_VAR_COPY([$1], [pkg_cv_][$1]) AS_VAR_IF([$1], [""], [$5], [$4])dnl ])dnl PKG_CHECK_VAR -dnl PKG_WITH_MODULES(VARIABLE-PREFIX, MODULES, -dnl [ACTION-IF-FOUND],[ACTION-IF-NOT-FOUND], -dnl [DESCRIPTION], [DEFAULT]) -dnl ------------------------------------------ -dnl -dnl Prepare a "--with-" configure option using the lowercase -dnl [VARIABLE-PREFIX] name, merging the behaviour of AC_ARG_WITH and -dnl PKG_CHECK_MODULES in a single macro. -AC_DEFUN([PKG_WITH_MODULES], -[ -m4_pushdef([with_arg], m4_tolower([$1])) - -m4_pushdef([description], - [m4_default([$5], [build with ]with_arg[ support])]) - -m4_pushdef([def_arg], [m4_default([$6], [auto])]) -m4_pushdef([def_action_if_found], [AS_TR_SH([with_]with_arg)=yes]) -m4_pushdef([def_action_if_not_found], [AS_TR_SH([with_]with_arg)=no]) - -m4_case(def_arg, - [yes],[m4_pushdef([with_without], [--without-]with_arg)], - [m4_pushdef([with_without],[--with-]with_arg)]) - -AC_ARG_WITH(with_arg, - AS_HELP_STRING(with_without, description[ @<:@default=]def_arg[@:>@]),, - [AS_TR_SH([with_]with_arg)=def_arg]) - -AS_CASE([$AS_TR_SH([with_]with_arg)], - [yes],[PKG_CHECK_MODULES([$1],[$2],$3,$4)], - [auto],[PKG_CHECK_MODULES([$1],[$2], - [m4_n([def_action_if_found]) $3], - [m4_n([def_action_if_not_found]) $4])]) - -m4_popdef([with_arg]) -m4_popdef([description]) -m4_popdef([def_arg]) - -])dnl PKG_WITH_MODULES - -dnl PKG_HAVE_WITH_MODULES(VARIABLE-PREFIX, MODULES, -dnl [DESCRIPTION], [DEFAULT]) -dnl ----------------------------------------------- -dnl -dnl Convenience macro to trigger AM_CONDITIONAL after PKG_WITH_MODULES -dnl check._[VARIABLE-PREFIX] is exported as make variable. -AC_DEFUN([PKG_HAVE_WITH_MODULES], -[ -PKG_WITH_MODULES([$1],[$2],,,[$3],[$4]) - -AM_CONDITIONAL([HAVE_][$1], - [test "$AS_TR_SH([with_]m4_tolower([$1]))" = "yes"]) -])dnl PKG_HAVE_WITH_MODULES - -dnl PKG_HAVE_DEFINE_WITH_MODULES(VARIABLE-PREFIX, MODULES, -dnl [DESCRIPTION], [DEFAULT]) -dnl ------------------------------------------------------ -dnl -dnl Convenience macro to run AM_CONDITIONAL and AC_DEFINE after -dnl PKG_WITH_MODULES check. HAVE_[VARIABLE-PREFIX] is exported as make -dnl and preprocessor variable. -AC_DEFUN([PKG_HAVE_DEFINE_WITH_MODULES], -[ -PKG_HAVE_WITH_MODULES([$1],[$2],[$3],[$4]) - -AS_IF([test "$AS_TR_SH([with_]m4_tolower([$1]))" = "yes"], - [AC_DEFINE([HAVE_][$1], 1, [Enable ]m4_tolower([$1])[ support])]) -])dnl PKG_HAVE_DEFINE_WITH_MODULES - diff --git a/configure b/configure index 03b73c6f1356..505f8bbc045d 100755 --- a/configure +++ b/configure @@ -769,6 +769,7 @@ infodir docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -880,6 +881,7 @@ datadir='${datarootdir}' sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1132,6 +1134,15 @@ do | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1269,7 +1280,7 @@ fi for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1422,6 +1433,7 @@ Fine tuning of the installation directories: --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -10593,7 +10605,7 @@ for ac_func in alarm setitimer getitimer bind_textdomain_codeset chown \ gai_strerror getgroups getlogin getloadavg getpeername getpgid getpid \ getentropy \ getpriority getresuid getresgid getpwent getspnam getspent getsid getwd \ - initgroups kill killpg lchmod lchown lstat mkfifo mknod mktime mmap \ + initgroups kill killpg lchown lstat mkfifo mknod mktime mmap \ mremap nice pathconf pause plock poll pthread_init \ putenv readlink realpath \ select sem_open sem_timedwait sem_getvalue sem_unlink setegid seteuid \ @@ -10616,6 +10628,17 @@ fi done +# Force lchmod off for Linux. Linux disallows changing the mode of symbolic +# links. Some libc implementations have a stub lchmod implementation that always +# returns an error. +if test "$MACHDEP" != linux; then + ac_fn_c_check_func "$LINENO" "lchmod" "ac_cv_func_lchmod" +if test "x$ac_cv_func_lchmod" = xyes; then : + +fi + +fi + # For some functions, having a definition is not sufficient, since # we want to take their address. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for chroot" >&5 diff --git a/configure.ac b/configure.ac index 257c8eb21e54..33870961b3a8 100644 --- a/configure.ac +++ b/configure.ac @@ -3122,7 +3122,7 @@ AC_CHECK_FUNCS(alarm setitimer getitimer bind_textdomain_codeset chown \ gai_strerror getgroups getlogin getloadavg getpeername getpgid getpid \ getentropy \ getpriority getresuid getresgid getpwent getspnam getspent getsid getwd \ - initgroups kill killpg lchmod lchown lstat mkfifo mknod mktime mmap \ + initgroups kill killpg lchown lstat mkfifo mknod mktime mmap \ mremap nice pathconf pause plock poll pthread_init \ putenv readlink realpath \ select sem_open sem_timedwait sem_getvalue sem_unlink setegid seteuid \ @@ -3134,6 +3134,13 @@ AC_CHECK_FUNCS(alarm setitimer getitimer bind_textdomain_codeset chown \ sysconf tcgetpgrp tcsetpgrp tempnam timegm times tmpfile tmpnam tmpnam_r \ truncate uname unsetenv utimes waitpid wait3 wait4 wcscoll _getpty) +# Force lchmod off for Linux. Linux disallows changing the mode of symbolic +# links. Some libc implementations have a stub lchmod implementation that always +# returns an error. +if test "$MACHDEP" != linux; then + AC_CHECK_FUNC(lchmod) +fi + # For some functions, having a definition is not sufficient, since # we want to take their address. AC_MSG_CHECKING(for chroot) diff --git a/pyconfig.h.in b/pyconfig.h.in index 11c4a66873c1..f828677dda01 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -439,9 +439,6 @@ /* Define to 1 if you have the 'lchflags' function. */ #undef HAVE_LCHFLAGS -/* Define to 1 if you have the `lchmod' function. */ -#undef HAVE_LCHMOD - /* Define to 1 if you have the `lchown' function. */ #undef HAVE_LCHOWN From webhook-mailer at python.org Wed Sep 12 19:36:09 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 23:36:09 -0000 Subject: [Python-checkins] closes bpo-34652: Always disable lchmod on Linux. (GH-9234) Message-ID: https://github.com/python/cpython/commit/98344a4f2f09ddae295352ec54fae6388446eeaf commit: 98344a4f2f09ddae295352ec54fae6388446eeaf branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T16:36:05-07:00 summary: closes bpo-34652: Always disable lchmod on Linux. (GH-9234) (cherry picked from commit 40caa05fa4d1810a1a6bfc34e0ec930c351089b7) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst M configure M configure.ac diff --git a/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst b/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst new file mode 100644 index 000000000000..cbdd7e0ec6ce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst @@ -0,0 +1 @@ +Ensure :func:`os.lchmod` is never defined on Linux. diff --git a/configure b/configure index 7e39458727d0..1219cb9ffa91 100755 --- a/configure +++ b/configure @@ -11354,6 +11354,17 @@ fi done +# Force lchmod off for Linux. Linux disallows changing the mode of symbolic +# links. Some libc implementations have a stub lchmod implementation that always +# returns an error. +if test "$MACHDEP" != linux; then + ac_fn_c_check_func "$LINENO" "lchmod" "ac_cv_func_lchmod" +if test "x$ac_cv_func_lchmod" = xyes; then : + +fi + +fi + ac_fn_c_check_decl "$LINENO" "dirfd" "ac_cv_have_decl_dirfd" "#include #include " diff --git a/configure.ac b/configure.ac index 2b8aa4d592f9..d7e0de3a32d3 100644 --- a/configure.ac +++ b/configure.ac @@ -3519,6 +3519,13 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ truncate uname unlinkat unsetenv utimensat utimes waitid waitpid wait3 wait4 \ wcscoll wcsftime wcsxfrm wmemcmp writev _getpty) +# Force lchmod off for Linux. Linux disallows changing the mode of symbolic +# links. Some libc implementations have a stub lchmod implementation that always +# returns an error. +if test "$MACHDEP" != linux; then + AC_CHECK_FUNC(lchmod) +fi + AC_CHECK_DECL(dirfd, AC_DEFINE(HAVE_DIRFD, 1, Define if you have the 'dirfd' function or macro.), , From webhook-mailer at python.org Wed Sep 12 19:43:23 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 23:43:23 -0000 Subject: [Python-checkins] closes bpo-34652: Always disable lchmod on Linux. (GH-9234) Message-ID: https://github.com/python/cpython/commit/bdace2ea53c578184235729a4cf383891084cc70 commit: bdace2ea53c578184235729a4cf383891084cc70 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T16:43:19-07:00 summary: closes bpo-34652: Always disable lchmod on Linux. (GH-9234) (cherry picked from commit 40caa05fa4d1810a1a6bfc34e0ec930c351089b7) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst M configure M configure.ac diff --git a/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst b/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst new file mode 100644 index 000000000000..cbdd7e0ec6ce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-12-14-46-51.bpo-34652.Rt1m1b.rst @@ -0,0 +1 @@ +Ensure :func:`os.lchmod` is never defined on Linux. diff --git a/configure b/configure index 5eebb0510e5c..1944a7259172 100755 --- a/configure +++ b/configure @@ -11377,6 +11377,17 @@ fi done +# Force lchmod off for Linux. Linux disallows changing the mode of symbolic +# links. Some libc implementations have a stub lchmod implementation that always +# returns an error. +if test "$MACHDEP" != linux; then + ac_fn_c_check_func "$LINENO" "lchmod" "ac_cv_func_lchmod" +if test "x$ac_cv_func_lchmod" = xyes; then : + +fi + +fi + ac_fn_c_check_decl "$LINENO" "dirfd" "ac_cv_have_decl_dirfd" "#include #include " diff --git a/configure.ac b/configure.ac index 7ee66343ff84..0eb0bcfa3f9e 100644 --- a/configure.ac +++ b/configure.ac @@ -3495,6 +3495,13 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ truncate uname unlinkat unsetenv utimensat utimes waitid waitpid wait3 wait4 \ wcscoll wcsftime wcsxfrm wmemcmp writev _getpty) +# Force lchmod off for Linux. Linux disallows changing the mode of symbolic +# links. Some libc implementations have a stub lchmod implementation that always +# returns an error. +if test "$MACHDEP" != linux; then + AC_CHECK_FUNC(lchmod) +fi + AC_CHECK_DECL(dirfd, AC_DEFINE(HAVE_DIRFD, 1, Define if you have the 'dirfd' function or macro.), , From webhook-mailer at python.org Wed Sep 12 19:54:14 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 12 Sep 2018 23:54:14 -0000 Subject: [Python-checkins] closes bpo-34654: Tolerate + at the beginning of large years. (GH-9238) Message-ID: https://github.com/python/cpython/commit/ea2fcd3db65b796ad3eda1619a903cefb1aa9363 commit: ea2fcd3db65b796ad3eda1619a903cefb1aa9363 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T16:54:10-07:00 summary: closes bpo-34654: Tolerate + at the beginning of large years. (GH-9238) (cherry picked from commit e1a34ceb541ef87e03bb428630097dacc9c658e5) Co-authored-by: Benjamin Peterson files: M Lib/test/test_time.py diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py index 9acd1d497ee8..62abd891aafa 100644 --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -703,9 +703,9 @@ def test_year(self, fmt=None, func=None): self.assertEqual(func(9999), fmt % 9999) def test_large_year(self): - self.assertEqual(self.yearstr(12345), '12345') - self.assertEqual(self.yearstr(123456789), '123456789') - self.assertEqual(self.yearstr(TIME_MAXYEAR), str(TIME_MAXYEAR)) + self.assertEqual(self.yearstr(12345).lstrip('+'), '12345') + self.assertEqual(self.yearstr(123456789).lstrip('+'), '123456789') + self.assertEqual(self.yearstr(TIME_MAXYEAR).lstrip('+'), str(TIME_MAXYEAR)) self.assertRaises(OverflowError, self.yearstr, TIME_MAXYEAR + 1) def test_negative(self): From webhook-mailer at python.org Wed Sep 12 20:05:20 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Thu, 13 Sep 2018 00:05:20 -0000 Subject: [Python-checkins] bpo-33649: Edit asyncio eventloop doc - second pass (GH-9233) Message-ID: https://github.com/python/cpython/commit/5b7cbd602e57265604d6c099fd174b1c7917f861 commit: 5b7cbd602e57265604d6c099fd174b1c7917f861 branch: master author: Carol Willing committer: Yury Selivanov date: 2018-09-12T17:05:17-07:00 summary: bpo-33649: Edit asyncio eventloop doc - second pass (GH-9233) files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index c200844385c1..baa5234d3e51 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -8,18 +8,18 @@ Event Loop .. rubric:: Preface -An event loop is the central component of every asyncio application. +The event loop is a central component of every asyncio application. Event loops run asynchronous tasks and callbacks, perform network -IO operations, run subprocesses, etc. +IO operations, and run subprocesses. -In general, it is *not recommended* to use event loops directly at -the application-level asyncio code. They should only be accessed -in low-level code in libraries and frameworks. - -High-level asyncio applications should not need to work with event -loops and should use the :func:`asyncio.run` function to initialize -and run asynchronous code. +Application developers will typically use high-level asyncio functions +to interact with the event loop. In general, high-level asyncio applications +should not need to work directly with event loops and, instead, should use +the :func:`asyncio.run` function to initialize, manage the event loop, and +run asynchronous code. +Alternatively, developers of low-level code, such as libraries and +framework, may need access to the event loop. .. rubric:: Accessing Event Loop @@ -42,13 +42,13 @@ an event loop: been called, asyncio will create a new event loop and set it as the current one. - Because this function has a rather complex behavior (especially - when custom event loop policies are in use), it is recommended - to use the :func:`get_running_loop` function in coroutines and - callbacks instead. + Because this function has rather complex behavior (especially + when custom event loop policies are in use), using the + :func:`get_running_loop` function is preferred to :func:`get_event_loop` + in coroutines and callbacks. - Consider also using the :func:`asyncio.run` function instead of - manually creating and closing an event loop. + Consider also using the :func:`asyncio.run` function instead of using + lower level functions to manually create and close an event loop. .. function:: set_event_loop(loop) @@ -67,14 +67,14 @@ and :func:`new_event_loop` functions can be altered by This documentation page contains the following sections: -* The `Event Loop Methods`_ section is a reference documentation of +* The `Event Loop Methods`_ section is the reference documentation of event loop APIs; * The `Callback Handles`_ section documents the :class:`Handle` and - :class:`TimerHandle`, instances of which are returned from functions - :meth:`loop.call_soon`, :meth:`loop.call_later`, etc; + :class:`TimerHandle`, instances which are returned from functions, such + as :meth:`loop.call_soon` and :meth:`loop.call_later`; -* The `Server Objects`_ sections documents types returned from +* The `Server Objects`_ sections document types returned from event loop methods like :meth:`loop.create_server`; * The `Event Loops Implementations`_ section documents the @@ -89,7 +89,7 @@ This documentation page contains the following sections: Event Loop Methods ================== -Event loops provide the following **low-level** APIs: +Event loops have **low-level** APIs for the following: .. contents:: :depth: 1 @@ -120,8 +120,8 @@ Running and stopping the loop If :meth:`stop` is called while :meth:`run_forever` is running, the loop will run the current batch of callbacks and then exit. - Note that callbacks scheduled by callbacks will not run in that - case; they will run the next time :meth:`run_forever` or + Note that callbacks scheduled by callbacks will not run in this + case; instead, they will run the next time :meth:`run_forever` or :meth:`run_until_complete` is called. .. method:: loop.stop() @@ -140,7 +140,7 @@ Running and stopping the loop Close the event loop. - The loop cannot not be running when this function is called. + The loop must be running when this function is called. Any pending callbacks will be discarded. This method clears all queues and shuts down the executor, but does @@ -154,8 +154,9 @@ Running and stopping the loop Schedule all currently open :term:`asynchronous generator` objects to close with an :meth:`~agen.aclose()` call. After calling this method, the event loop will issue a warning if a new asynchronous generator - is iterated. Should be used to reliably finalize all scheduled - asynchronous generators, e.g.: + is iterated. This should be used to reliably finalize all scheduled + asynchronous generators, e.g.:: + try: loop.run_forever() @@ -173,7 +174,7 @@ Scheduling callbacks .. method:: loop.call_soon(callback, *args, context=None) - Schedule *callback* to be called with *args* arguments at + Schedule a *callback* to be called with *args* arguments at the next iteration of the event loop. Callbacks are called in the order in which they are registered. @@ -184,7 +185,9 @@ Scheduling callbacks The current context is used when no *context* is provided. An instance of :class:`asyncio.Handle` is returned, which can be - used to cancel the callback. + used later to cancel the callback. + + This method is not thread-safe. .. method:: loop.call_soon_threadsafe(callback, *args, context=None) @@ -200,11 +203,11 @@ Scheduling callbacks .. note:: - Most :mod:`asyncio` scheduling functions don't allow to pass + Most :mod:`asyncio` scheduling functions don't allow passing keyword arguments. To do that, use :func:`functools.partial`, e.g.:: - # will schedule "print("Hello", flush=True)": + # will schedule "print("Hello", flush=True)" loop.call_soon( functools.partial(print, "Hello", flush=True)) @@ -232,7 +235,7 @@ clocks to track time. be used to cancel the callback. *callback* will be called exactly once. If two callbacks are - scheduled for exactly the same time, it is undefined which will + scheduled for exactly the same time, it is undefined which one will be called first. The optional positional *args* will be passed to the callback when @@ -284,8 +287,8 @@ Creating Futures and Tasks Create an :class:`asyncio.Future` object attached to the event loop. - This is the preferred way to create Futures in asyncio, that lets - third-party event loops to provide alternative implementations of + This is the preferred way to create Futures in asyncio. This lets + third-party event loops provide alternative implementations of the Future object (with better performance or instrumentation). .. versionadded:: 3.5.2 @@ -397,7 +400,7 @@ Opening network connections * *local_addr*, if given, is a ``(local_host, local_port)`` tuple used to bind the socket to locally. The *local_host* and *local_port* - are looked up using getaddrinfo(), similarly to *host* and *port*. + are looked up using ``getaddrinfo()``, similarly to *host* and *port*. * *ssl_handshake_timeout* is (for an SSL connection) the time in seconds to wait for the SSL handshake to complete before aborting the connection. @@ -532,16 +535,17 @@ Creating network servers Arguments: - * The *host* parameter can be a string, in that case the TCP server is - bound to *host* and *port*. The *host* parameter can also be a sequence - of strings and in that case the TCP server is bound to all hosts of the - sequence. If *host* is an empty string or ``None``, all interfaces are - assumed and a list of multiple sockets will be returned (most likely one - for IPv4 and another one for IPv6). + * The *host* parameter can be set to several types which determine behavior: + - If *host* is a string, the TCP server is bound to *host* and *port*. + - if *host* is a sequence of strings, the TCP server is bound to all + hosts of the sequence. + - If *host* is an empty string or ``None``, all interfaces are + assumed and a list of multiple sockets will be returned (most likely + one for IPv4 and another one for IPv6). * *family* can be set to either :data:`socket.AF_INET` or :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. - If not set it will be determined from host name + If not set, the *family* will be determined from host name (defaults to :data:`~socket.AF_UNSPEC`). * *flags* is a bitmask for :meth:`getaddrinfo`. @@ -721,16 +725,16 @@ Watching file descriptors .. method:: loop.add_reader(fd, callback, \*args) - Start watching the file descriptor for read availability and + Start watching the file descriptor *fd* for read availability and call the *callback* with specified arguments. .. method:: loop.remove_reader(fd) - Stop watching the file descriptor for read availability. + Stop watching the file descriptor *fd* for read availability. .. method:: loop.add_writer(fd, callback, \*args) - Start watching the file descriptor for write availability and then + Start watching the file descriptor *fd* for write availability and then call the *callback* with specified arguments. Use :func:`functools.partial` :ref:`to pass keywords @@ -738,7 +742,7 @@ Watching file descriptors .. method:: loop.remove_writer(fd) - Stop watching the file descriptor for write availability. + Stop watching the file descriptor *fd* for write availability. See also :ref:`Platform Support ` section for some limitations of these methods. @@ -747,10 +751,10 @@ for some limitations of these methods. Working with socket objects directly ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In general, protocols implementations that use transport-based APIs +In general, protocol implementations that use transport-based APIs such as :meth:`loop.create_connection` and :meth:`loop.create_server` are faster than implementations that work with sockets directly. -However, there are use cases when performance is not critical and +However, there are some use cases when performance is not critical, and working with :class:`~socket.socket` objects directly is more convenient. @@ -765,8 +769,8 @@ convenient. The socket *sock* must be non-blocking. .. versionchanged:: 3.7 - Even though the method was always documented as a coroutine - method, before Python 3.7 it returned a :class:`Future`. + Even though this method was always documented as a coroutine + method, releases before Python 3.7 returned a :class:`Future`. Since Python 3.7 this is an ``async def`` method. .. coroutinemethod:: loop.sock_recv_into(sock, buf) @@ -785,10 +789,11 @@ convenient. Send data to the socket. Asynchronous version of :meth:`socket.sendall() `. - This method continues to send data from *data* until either all data has - been sent or an error occurs. ``None`` is returned on success. On error, - an exception is raised, and there is no way to determine how much data, if - any, was successfully processed by the receiving end of the connection. + This method continues to send data from *data* to the socket until either + all data in *data* has been sent or an error occurs. ``None`` is returned + on success. On error, an exception is raised. Additionally, there is no way + to determine how much data, if any, was successfully processed by the + receiving end of the connection. The socket *sock* must be non-blocking. @@ -860,7 +865,7 @@ convenient. ` can be used to figure out the number of bytes which were sent. - *fallback* set to ``True`` makes asyncio to manually read and send + *fallback*, when set to ``True``, makes asyncio manually read and send the file when the platform does not support the sendfile syscall (e.g. Windows or SSL socket on Unix). @@ -927,7 +932,7 @@ Working with pipes .. note:: :class:`SelectorEventLoop` does not support the above methods on - Windows. Use :class:`ProactorEventLoop` instead. + Windows. Use :class:`ProactorEventLoop` instead for Windows. .. seealso:: @@ -1028,8 +1033,8 @@ Allows customizing how exceptions are handled in the event loop. Default exception handler. This is called when an exception occurs and no exception - handler is set, and can be called by a custom exception - handler that wants to defer to the default behavior. + handler is set. This can be called by a custom exception + handler that wants to defer to the default handler behavior. *context* parameter has the same meaning as in :meth:`call_exception_handler`. @@ -1051,7 +1056,7 @@ Allows customizing how exceptions are handled in the event loop. .. note:: - Note: this method should not be overloaded in subclassed + This method should not be overloaded in subclassed event loops. For any custom exception handling, use :meth:`set_exception_handler()` method. @@ -1104,7 +1109,7 @@ async/await code consider using high-level convenient :ref:`filesystem encoding `. The first string specifies the program to execute, - and the remaining strings specify the arguments. Together string + and the remaining strings specify the arguments. Together, string arguments form the ``argv`` of the program. This is similar to the standard library :class:`subprocess.Popen` @@ -1238,8 +1243,7 @@ Do not instantiate the class directly. async with srv: # some code - # At this point, srv is closed and no longer accepts new - connections. + # At this point, srv is closed and no longer accepts new connections. .. versionchanged:: 3.7 @@ -1408,6 +1412,7 @@ event loop:: import asyncio def hello_world(loop): + """A callback to print 'Hello World' and stop the event loop""" print('Hello World') loop.stop() @@ -1478,6 +1483,7 @@ Wait until a file descriptor received some data using the # Create a pair of connected file descriptors rsock, wsock = socketpair() + loop = asyncio.get_event_loop() def reader(): @@ -1500,7 +1506,7 @@ Wait until a file descriptor received some data using the # Run the event loop loop.run_forever() finally: - # We are done, close sockets and the event loop + # We are done. Close sockets and the event loop. rsock.close() wsock.close() loop.close() @@ -1519,7 +1525,7 @@ Wait until a file descriptor received some data using the Set signal handlers for SIGINT and SIGTERM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -(This example only works on UNIX.) +(This ``signals`` example only works on UNIX.) Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM` using the :meth:`loop.add_signal_handler` method:: From webhook-mailer at python.org Wed Sep 12 20:06:56 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 00:06:56 -0000 Subject: [Python-checkins] closes bpo-34654: Tolerate + at the beginning of large years. (GH-9238) Message-ID: https://github.com/python/cpython/commit/21a808230aa33741f1dd9bf38f0e283c1b72210e commit: 21a808230aa33741f1dd9bf38f0e283c1b72210e branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T17:06:53-07:00 summary: closes bpo-34654: Tolerate + at the beginning of large years. (GH-9238) (cherry picked from commit e1a34ceb541ef87e03bb428630097dacc9c658e5) Co-authored-by: Benjamin Peterson files: M Lib/test/test_time.py diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py index b22546dd5c9f..6e57b6010f77 100644 --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -603,9 +603,9 @@ def test_year(self, fmt=None, func=None): self.assertEqual(func(9999), fmt % 9999) def test_large_year(self): - self.assertEqual(self.yearstr(12345), '12345') - self.assertEqual(self.yearstr(123456789), '123456789') - self.assertEqual(self.yearstr(TIME_MAXYEAR), str(TIME_MAXYEAR)) + self.assertEqual(self.yearstr(12345).lstrip('+'), '12345') + self.assertEqual(self.yearstr(123456789).lstrip('+'), '123456789') + self.assertEqual(self.yearstr(TIME_MAXYEAR).lstrip('+'), str(TIME_MAXYEAR)) self.assertRaises(OverflowError, self.yearstr, TIME_MAXYEAR + 1) def test_negative(self): From webhook-mailer at python.org Wed Sep 12 20:09:11 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Thu, 13 Sep 2018 00:09:11 -0000 Subject: [Python-checkins] edit async policy doc - second pass (GH-9235) Message-ID: https://github.com/python/cpython/commit/6d9767fb263d286dc383ece01ca5b0b25117bd94 commit: 6d9767fb263d286dc383ece01ca5b0b25117bd94 branch: master author: Carol Willing committer: Yury Selivanov date: 2018-09-12T17:09:08-07:00 summary: edit async policy doc - second pass (GH-9235) files: M Doc/library/asyncio-policy.rst diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst index 7bb1c1587b7a..61727d40afab 100644 --- a/Doc/library/asyncio-policy.rst +++ b/Doc/library/asyncio-policy.rst @@ -11,9 +11,9 @@ An event loop policy, a global per-process object, controls management of the event loop. Each event loop has a default policy, which can be changed and customized using the API. -A policy defines the notion of context and manages a +A policy defines the notion of *context* and manages a separate event loop per context. The default policy -defines context to be the current thread. +defines *context* to be the current thread. By using a custom event loop policy, the behavior of :func:`get_event_loop`, :func:`set_event_loop`, and @@ -94,8 +94,8 @@ asyncio ships with the following built-in policies: The default asyncio policy. Uses :class:`SelectorEventLoop` on both Unix and Windows platforms. - There is no need to install the default policy manually; asyncio - is configured to use it automatically. + There is no need to install the default policy manually. asyncio + is configured to use the default policy automatically. .. class:: WindowsProactorEventLoopPolicy @@ -125,7 +125,7 @@ by default) and :class:`FastChildWatcher`. See also the :ref:`Subprocess and Threads ` section. -The following two functions can be used to customize the watcher +The following two functions can be used to customize the child process watcher implementation used by the asyncio event loop: .. function:: get_child_watcher() @@ -189,7 +189,7 @@ implementation used by the asyncio event loop: handling a big number of processes (*O(n)* each time a :py:data:`SIGCHLD` is received). - asyncio uses this implementation by default. + asyncio uses this safe implementation by default. .. class:: FastChildWatcher From webhook-mailer at python.org Wed Sep 12 20:14:42 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 13 Sep 2018 00:14:42 -0000 Subject: [Python-checkins] closes bpo-34641: Further restrict the LHS of keyword argument function call syntax. (GH-9212) Message-ID: https://github.com/python/cpython/commit/c9a71dd223c4ad200a97aa320aef6bd3d45f8897 commit: c9a71dd223c4ad200a97aa320aef6bd3d45f8897 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-12T17:14:39-07:00 summary: closes bpo-34641: Further restrict the LHS of keyword argument function call syntax. (GH-9212) files: A Misc/NEWS.d/next/Core and Builtins/2018-09-11-23-12-33.bpo-34641.gFBCc9.rst M Doc/whatsnew/3.8.rst M Lib/test/test_syntax.py M Python/ast.c diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 259dcf65f2e0..b2475c7df33e 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -94,6 +94,10 @@ Other Language Changes * Added support of ``\N{name}`` escapes in :mod:`regular expressions `. (Contributed by Jonathan Eunice and Serhiy Storchaka in :issue:`30688`.) +* The syntax allowed for keyword names in function calls was further + restricted. In particular, ``f((keyword)=arg)`` is no longer allowed. It was + never intended to permit more than a bare name on the left-hand side of a + keyword argument assignment term. See :issue:`34641`. New Modules =========== diff --git a/Lib/test/test_syntax.py b/Lib/test/test_syntax.py index fa1e7aa5d4f2..c5b2496e010d 100644 --- a/Lib/test/test_syntax.py +++ b/Lib/test/test_syntax.py @@ -269,6 +269,9 @@ >>> f(x.y=1) Traceback (most recent call last): SyntaxError: keyword can't be an expression +>>> f((x)=2) +Traceback (most recent call last): +SyntaxError: keyword can't be an expression More set_context(): diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-11-23-12-33.bpo-34641.gFBCc9.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-11-23-12-33.bpo-34641.gFBCc9.rst new file mode 100644 index 000000000000..9b6eb24c138c --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-11-23-12-33.bpo-34641.gFBCc9.rst @@ -0,0 +1,2 @@ +Further restrict the syntax of the left-hand side of keyword arguments in +function calls. In particular, ``f((keyword)=arg)`` is now disallowed. diff --git a/Python/ast.c b/Python/ast.c index 94962e00c7d3..b93eb88dae7f 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -2815,29 +2815,56 @@ ast_for_call(struct compiling *c, const node *n, expr_ty func, bool allowgen) identifier key, tmp; int k; - /* chch is test, but must be an identifier? */ - e = ast_for_expr(c, chch); - if (!e) - return NULL; - /* f(lambda x: x[0] = 3) ends up getting parsed with - * LHS test = lambda x: x[0], and RHS test = 3. - * SF bug 132313 points out that complaining about a keyword - * then is very confusing. - */ - if (e->kind == Lambda_kind) { + // To remain LL(1), the grammar accepts any test (basically, any + // expression) in the keyword slot of a call site. So, we need + // to manually enforce that the keyword is a NAME here. + static const int name_tree[] = { + test, + or_test, + and_test, + not_test, + comparison, + expr, + xor_expr, + and_expr, + shift_expr, + arith_expr, + term, + factor, + power, + atom_expr, + atom, + 0, + }; + node *expr_node = chch; + for (int i = 0; name_tree[i]; i++) { + if (TYPE(expr_node) != name_tree[i]) + break; + if (NCH(expr_node) != 1) + break; + expr_node = CHILD(expr_node, 0); + } + if (TYPE(expr_node) == lambdef) { + // f(lambda x: x[0] = 3) ends up getting parsed with LHS + // test = lambda x: x[0], and RHS test = 3. Issue #132313 + // points out that complaining about a keyword then is very + // confusing. ast_error(c, chch, "lambda cannot contain assignment"); return NULL; } - else if (e->kind != Name_kind) { + else if (TYPE(expr_node) != NAME) { ast_error(c, chch, - "keyword can't be an expression"); + "keyword can't be an expression"); + return NULL; + } + key = new_identifier(STR(expr_node), c); + if (key == NULL) { return NULL; } - else if (forbidden_name(c, e->v.Name.id, ch, 1)) { + if (forbidden_name(c, key, chch, 1)) { return NULL; } - key = e->v.Name.id; for (k = 0; k < nkeywords; k++) { tmp = ((keyword_ty)asdl_seq_GET(keywords, k))->arg; if (tmp && !PyUnicode_Compare(tmp, key)) { From webhook-mailer at python.org Wed Sep 12 20:22:14 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 13 Sep 2018 00:22:14 -0000 Subject: [Python-checkins] bpo-34652: Remove lchmod from the big func checking block. (GH-9247) Message-ID: https://github.com/python/cpython/commit/ed709d5699716bf7237856dc20aba321e2dfff6d commit: ed709d5699716bf7237856dc20aba321e2dfff6d branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-12T17:22:11-07:00 summary: bpo-34652: Remove lchmod from the big func checking block. (GH-9247) A fix for 883702ebb8bbfa749ef0040d1b58d6222bf589ee. files: M configure M configure.ac diff --git a/configure b/configure index d341453443fc..ddfd1c4434c2 100755 --- a/configure +++ b/configure @@ -11258,7 +11258,7 @@ for ac_func in alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \ getpriority getresuid getresgid getpwent getpwnam_r getpwuid_r getspnam getspent getsid getwd \ if_nameindex \ - initgroups kill killpg lchmod lchown lockf linkat lstat lutimes mmap \ + initgroups kill killpg lchown lockf linkat lstat lutimes mmap \ memrchr mbrtowc mkdirat mkfifo \ mkfifoat mknod mknodat mktime mremap nice openat pathconf pause pipe2 plock poll \ posix_fallocate posix_fadvise posix_spawn pread preadv preadv2 \ @@ -11289,10 +11289,16 @@ done # links. Some libc implementations have a stub lchmod implementation that always # returns an error. if test "$MACHDEP" != linux; then + for ac_func in lchmod +do : ac_fn_c_check_func "$LINENO" "lchmod" "ac_cv_func_lchmod" if test "x$ac_cv_func_lchmod" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_LCHMOD 1 +_ACEOF fi +done fi diff --git a/configure.ac b/configure.ac index cf7328c4032a..cc1bba4b384d 100644 --- a/configure.ac +++ b/configure.ac @@ -3438,7 +3438,7 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \ getpriority getresuid getresgid getpwent getpwnam_r getpwuid_r getspnam getspent getsid getwd \ if_nameindex \ - initgroups kill killpg lchmod lchown lockf linkat lstat lutimes mmap \ + initgroups kill killpg lchown lockf linkat lstat lutimes mmap \ memrchr mbrtowc mkdirat mkfifo \ mkfifoat mknod mknodat mktime mremap nice openat pathconf pause pipe2 plock poll \ posix_fallocate posix_fadvise posix_spawn pread preadv preadv2 \ @@ -3458,7 +3458,7 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ # links. Some libc implementations have a stub lchmod implementation that always # returns an error. if test "$MACHDEP" != linux; then - AC_CHECK_FUNC(lchmod) + AC_CHECK_FUNCS(lchmod) fi AC_CHECK_DECL(dirfd, From webhook-mailer at python.org Wed Sep 12 20:37:50 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 13 Sep 2018 00:37:50 -0000 Subject: [Python-checkins] [3.6] bpo-34652: Remove lchmod from the big func checking block. (GH-9250) Message-ID: https://github.com/python/cpython/commit/2184fe5ba48a58cb51be144230622b84b372644c commit: 2184fe5ba48a58cb51be144230622b84b372644c branch: 3.6 author: Benjamin Peterson committer: GitHub date: 2018-09-12T17:37:46-07:00 summary: [3.6] bpo-34652: Remove lchmod from the big func checking block. (GH-9250) A fix for 883702ebb8bbfa749ef0040d1b58d6222bf589ee.. (cherry picked from commit ed709d5699716bf7237856dc20aba321e2dfff6d) files: M configure M configure.ac diff --git a/configure b/configure index 1944a7259172..1d93d683a6c4 100755 --- a/configure +++ b/configure @@ -784,6 +784,7 @@ infodir docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -895,6 +896,7 @@ datadir='${datarootdir}' sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1147,6 +1149,15 @@ do | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1284,7 +1295,7 @@ fi for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1437,6 +1448,7 @@ Fine tuning of the installation directories: --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -11350,7 +11362,7 @@ for ac_func in alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ futimens futimes gai_strerror getentropy \ getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \ getpriority getresuid getresgid getpwent getspnam getspent getsid getwd \ - initgroups kill killpg lchmod lchown linkat lstat lutimes mmap \ + initgroups kill killpg lchown linkat lstat lutimes mmap \ memrchr mbrtowc mkdirat mkfifo \ mkfifoat mknod mknodat mktime mremap nice openat pathconf pause pipe2 plock poll \ posix_fallocate posix_fadvise pread \ @@ -11381,10 +11393,16 @@ done # links. Some libc implementations have a stub lchmod implementation that always # returns an error. if test "$MACHDEP" != linux; then + for ac_func in lchmod +do : ac_fn_c_check_func "$LINENO" "lchmod" "ac_cv_func_lchmod" if test "x$ac_cv_func_lchmod" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_LCHMOD 1 +_ACEOF fi +done fi diff --git a/configure.ac b/configure.ac index 0eb0bcfa3f9e..7ec62c2dd7eb 100644 --- a/configure.ac +++ b/configure.ac @@ -3479,7 +3479,7 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ futimens futimes gai_strerror getentropy \ getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \ getpriority getresuid getresgid getpwent getspnam getspent getsid getwd \ - initgroups kill killpg lchmod lchown linkat lstat lutimes mmap \ + initgroups kill killpg lchown linkat lstat lutimes mmap \ memrchr mbrtowc mkdirat mkfifo \ mkfifoat mknod mknodat mktime mremap nice openat pathconf pause pipe2 plock poll \ posix_fallocate posix_fadvise pread \ @@ -3499,7 +3499,7 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ # links. Some libc implementations have a stub lchmod implementation that always # returns an error. if test "$MACHDEP" != linux; then - AC_CHECK_FUNC(lchmod) + AC_CHECK_FUNCS(lchmod) fi AC_CHECK_DECL(dirfd, From webhook-mailer at python.org Wed Sep 12 20:58:44 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 00:58:44 -0000 Subject: [Python-checkins] bpo-34200: Fix non-determinism of test_pkg (GH-9248) Message-ID: https://github.com/python/cpython/commit/4ae8ece5cd4c5853b625381db13429f25512108d commit: 4ae8ece5cd4c5853b625381db13429f25512108d branch: master author: Gregory P. Smith committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-12T17:58:40-07:00 summary: bpo-34200: Fix non-determinism of test_pkg (GH-9248) This causes the tearDown code to only unimport the test modules specifically created as part of each test via the self.mkhier method rather than abusing test.support.modules_setup() and the scary test.support.modules_cleanup() code. https://bugs.python.org/issue34200 files: A Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst M Lib/test/test_pkg.py diff --git a/Lib/test/test_pkg.py b/Lib/test/test_pkg.py index 8130eab50a93..eed0fd1c6b73 100644 --- a/Lib/test/test_pkg.py +++ b/Lib/test/test_pkg.py @@ -5,7 +5,6 @@ import tempfile import textwrap import unittest -from test import support # Helpers to create and destroy hierarchies. @@ -50,11 +49,13 @@ def setUp(self): self.root = None self.pkgname = None self.syspath = list(sys.path) - self.modules_before = support.modules_setup() + self.modules_to_cleanup = set() # Populated by mkhier(). def tearDown(self): sys.path[:] = self.syspath - support.modules_cleanup(*self.modules_before) + for modulename in self.modules_to_cleanup: + if modulename in sys.modules: + del sys.modules[modulename] if self.root: # Only clean if the test was actually run cleanout(self.root) @@ -75,17 +76,17 @@ def mkhier(self, descr): os.mkdir(root) for name, contents in descr: comps = name.split() + self.modules_to_cleanup.add('.'.join(comps)) fullname = root for c in comps: fullname = os.path.join(fullname, c) if contents is None: os.mkdir(fullname) else: - f = open(fullname, "w") - f.write(contents) - if contents and contents[-1] != '\n': - f.write('\n') - f.close() + with open(fullname, "w") as f: + f.write(contents) + if not contents.endswith('\n'): + f.write('\n') self.root = root # package name is the name of the first item self.pkgname = descr[0][0] diff --git a/Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst b/Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst new file mode 100644 index 000000000000..b53339c5856c --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst @@ -0,0 +1,3 @@ +Fixed non-deterministic flakiness of test_pkg by not using the scary +test.support.module_cleanup() logic to save and restore sys.modules contents +between test cases. From webhook-mailer at python.org Wed Sep 12 21:29:13 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 01:29:13 -0000 Subject: [Python-checkins] bpo-34200: Fix non-determinism of test_pkg (GH-9248) Message-ID: https://github.com/python/cpython/commit/90f7d455b7f4bd53d3af11cb951347c9c8230399 commit: 90f7d455b7f4bd53d3af11cb951347c9c8230399 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T18:29:08-07:00 summary: bpo-34200: Fix non-determinism of test_pkg (GH-9248) This causes the tearDown code to only unimport the test modules specifically created as part of each test via the self.mkhier method rather than abusing test.support.modules_setup() and the scary test.support.modules_cleanup() code. https://bugs.python.org/issue34200 (cherry picked from commit 4ae8ece5cd4c5853b625381db13429f25512108d) Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst M Lib/test/test_pkg.py diff --git a/Lib/test/test_pkg.py b/Lib/test/test_pkg.py index 8130eab50a93..eed0fd1c6b73 100644 --- a/Lib/test/test_pkg.py +++ b/Lib/test/test_pkg.py @@ -5,7 +5,6 @@ import tempfile import textwrap import unittest -from test import support # Helpers to create and destroy hierarchies. @@ -50,11 +49,13 @@ def setUp(self): self.root = None self.pkgname = None self.syspath = list(sys.path) - self.modules_before = support.modules_setup() + self.modules_to_cleanup = set() # Populated by mkhier(). def tearDown(self): sys.path[:] = self.syspath - support.modules_cleanup(*self.modules_before) + for modulename in self.modules_to_cleanup: + if modulename in sys.modules: + del sys.modules[modulename] if self.root: # Only clean if the test was actually run cleanout(self.root) @@ -75,17 +76,17 @@ def mkhier(self, descr): os.mkdir(root) for name, contents in descr: comps = name.split() + self.modules_to_cleanup.add('.'.join(comps)) fullname = root for c in comps: fullname = os.path.join(fullname, c) if contents is None: os.mkdir(fullname) else: - f = open(fullname, "w") - f.write(contents) - if contents and contents[-1] != '\n': - f.write('\n') - f.close() + with open(fullname, "w") as f: + f.write(contents) + if not contents.endswith('\n'): + f.write('\n') self.root = root # package name is the name of the first item self.pkgname = descr[0][0] diff --git a/Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst b/Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst new file mode 100644 index 000000000000..b53339c5856c --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst @@ -0,0 +1,3 @@ +Fixed non-deterministic flakiness of test_pkg by not using the scary +test.support.module_cleanup() logic to save and restore sys.modules contents +between test cases. From webhook-mailer at python.org Wed Sep 12 21:29:36 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 01:29:36 -0000 Subject: [Python-checkins] bpo-34200: Fix non-determinism of test_pkg (GH-9248) Message-ID: https://github.com/python/cpython/commit/3e3d4a4b55cbef481486aaa68ee255f4928b9c3f commit: 3e3d4a4b55cbef481486aaa68ee255f4928b9c3f branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T18:29:33-07:00 summary: bpo-34200: Fix non-determinism of test_pkg (GH-9248) This causes the tearDown code to only unimport the test modules specifically created as part of each test via the self.mkhier method rather than abusing test.support.modules_setup() and the scary test.support.modules_cleanup() code. https://bugs.python.org/issue34200 (cherry picked from commit 4ae8ece5cd4c5853b625381db13429f25512108d) Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst M Lib/test/test_pkg.py diff --git a/Lib/test/test_pkg.py b/Lib/test/test_pkg.py index 8130eab50a93..eed0fd1c6b73 100644 --- a/Lib/test/test_pkg.py +++ b/Lib/test/test_pkg.py @@ -5,7 +5,6 @@ import tempfile import textwrap import unittest -from test import support # Helpers to create and destroy hierarchies. @@ -50,11 +49,13 @@ def setUp(self): self.root = None self.pkgname = None self.syspath = list(sys.path) - self.modules_before = support.modules_setup() + self.modules_to_cleanup = set() # Populated by mkhier(). def tearDown(self): sys.path[:] = self.syspath - support.modules_cleanup(*self.modules_before) + for modulename in self.modules_to_cleanup: + if modulename in sys.modules: + del sys.modules[modulename] if self.root: # Only clean if the test was actually run cleanout(self.root) @@ -75,17 +76,17 @@ def mkhier(self, descr): os.mkdir(root) for name, contents in descr: comps = name.split() + self.modules_to_cleanup.add('.'.join(comps)) fullname = root for c in comps: fullname = os.path.join(fullname, c) if contents is None: os.mkdir(fullname) else: - f = open(fullname, "w") - f.write(contents) - if contents and contents[-1] != '\n': - f.write('\n') - f.close() + with open(fullname, "w") as f: + f.write(contents) + if not contents.endswith('\n'): + f.write('\n') self.root = root # package name is the name of the first item self.pkgname = descr[0][0] diff --git a/Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst b/Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst new file mode 100644 index 000000000000..b53339c5856c --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-12-17-00-34.bpo-34200.dfxYQK.rst @@ -0,0 +1,3 @@ +Fixed non-deterministic flakiness of test_pkg by not using the scary +test.support.module_cleanup() logic to save and restore sys.modules contents +between test cases. From webhook-mailer at python.org Thu Sep 13 01:33:08 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 05:33:08 -0000 Subject: [Python-checkins] bpo-34652: Remove lchmod from the big func checking block. (GH-9247) Message-ID: https://github.com/python/cpython/commit/35c94c765bf46e93a0d84e39dbc6eaaba40b1f36 commit: 35c94c765bf46e93a0d84e39dbc6eaaba40b1f36 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-12T22:33:02-07:00 summary: bpo-34652: Remove lchmod from the big func checking block. (GH-9247) A fix for 883702ebb8bbfa749ef0040d1b58d6222bf589ee. (cherry picked from commit ed709d5699716bf7237856dc20aba321e2dfff6d) Co-authored-by: Benjamin Peterson files: M configure M configure.ac diff --git a/configure b/configure index 1219cb9ffa91..ee08da97b430 100755 --- a/configure +++ b/configure @@ -11327,7 +11327,7 @@ for ac_func in alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \ getpriority getresuid getresgid getpwent getspnam getspent getsid getwd \ if_nameindex \ - initgroups kill killpg lchmod lchown lockf linkat lstat lutimes mmap \ + initgroups kill killpg lchown lockf linkat lstat lutimes mmap \ memrchr mbrtowc mkdirat mkfifo \ mkfifoat mknod mknodat mktime mremap nice openat pathconf pause pipe2 plock poll \ posix_fallocate posix_fadvise posix_spawn pread preadv preadv2 \ @@ -11358,10 +11358,16 @@ done # links. Some libc implementations have a stub lchmod implementation that always # returns an error. if test "$MACHDEP" != linux; then + for ac_func in lchmod +do : ac_fn_c_check_func "$LINENO" "lchmod" "ac_cv_func_lchmod" if test "x$ac_cv_func_lchmod" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_LCHMOD 1 +_ACEOF fi +done fi diff --git a/configure.ac b/configure.ac index d7e0de3a32d3..0cd1b78cf77f 100644 --- a/configure.ac +++ b/configure.ac @@ -3503,7 +3503,7 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \ getpriority getresuid getresgid getpwent getspnam getspent getsid getwd \ if_nameindex \ - initgroups kill killpg lchmod lchown lockf linkat lstat lutimes mmap \ + initgroups kill killpg lchown lockf linkat lstat lutimes mmap \ memrchr mbrtowc mkdirat mkfifo \ mkfifoat mknod mknodat mktime mremap nice openat pathconf pause pipe2 plock poll \ posix_fallocate posix_fadvise posix_spawn pread preadv preadv2 \ @@ -3523,7 +3523,7 @@ AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ # links. Some libc implementations have a stub lchmod implementation that always # returns an error. if test "$MACHDEP" != linux; then - AC_CHECK_FUNC(lchmod) + AC_CHECK_FUNCS(lchmod) fi AC_CHECK_DECL(dirfd, From webhook-mailer at python.org Thu Sep 13 01:40:41 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Thu, 13 Sep 2018 05:40:41 -0000 Subject: [Python-checkins] Polish doc as part of asyncio doc improvement (GH-9185) Message-ID: https://github.com/python/cpython/commit/1abba455d1e703b7050c0d183e40c6a9d45798c5 commit: 1abba455d1e703b7050c0d183e40c6a9d45798c5 branch: master author: Carol Willing committer: Yury Selivanov date: 2018-09-12T22:40:37-07:00 summary: Polish doc as part of asyncio doc improvement (GH-9185) files: M Doc/library/asyncio-dev.rst diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index cb574c308ad1..e4986ba5eb26 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -98,9 +98,11 @@ Concurrency and multithreading An event loop runs in a thread (typically the main thread) and executes all callbacks and tasks in its thread. While a task is running in the -event loop, no other task is running in the same thread. When a task -executes an ``await`` expression, the task gets suspended and the -event loop executes the next task. +event loop, no other tasks may run in the same thread. When a task +executes an ``await`` expression, the running task gets suspended, and the +event loop executes the next task. Prior to suspending the task, the awaiting +chain is checked, and if the chain ends with a future, the running task is +not suspended. To schedule a callback from a different thread, the :meth:`loop.call_soon_threadsafe` method should be used. Example:: @@ -123,9 +125,9 @@ To schedule a coroutine object from a different thread, the future = asyncio.run_coroutine_threadsafe(coro_func(), loop) result = future.result(timeout) # Wait for the result with a timeout -The :meth:`loop.run_in_executor` method can be used with a thread pool -executor to execute a callback in different thread to not block the thread of -the event loop. +The :meth:`loop.run_in_executor` method can be used with a +:class:`concurrent.futures.ThreadPoolExecutor` to execute a callback in +different thread so as not to block the event loop's main thread. .. seealso:: @@ -183,9 +185,10 @@ Detect coroutine objects never scheduled ---------------------------------------- When a coroutine function is called and its result is not passed to -:func:`ensure_future` or to the :meth:`loop.create_task` method, -the execution of the coroutine object will never be scheduled which is -probably a bug. :ref:`Enable the debug mode of asyncio ` +:meth:`asyncio.create_task` the execution of the coroutine object will +never be scheduled which is probably a bug. Using ``asyncio.create_task`` is +preferred to the low level :func:`ensure_future` and :meth:`loop.create_task` +methods. :ref:`Enable the debug mode of asyncio ` to :ref:`log a warning ` to detect it. Example with the bug:: @@ -204,8 +207,9 @@ Output in debug mode:: File "test.py", line 7, in test() -The fix is to call the :func:`ensure_future` function or the -:meth:`loop.create_task` method with the coroutine object. +The fix is to call the :meth:`asyncio.create_task` function. Using +``asyncio.create_task`` is preferred to the low level :func:`ensure_future` and +:meth:`loop.create_task` methods. .. seealso:: @@ -280,14 +284,9 @@ coroutine in another coroutine and use classic try/except:: loop.run_forever() loop.close() -Another option is to use the :meth:`loop.run_until_complete` -function:: +Another option is to use the :meth:`asyncio.run` function:: - task = asyncio.ensure_future(bug()) - try: - loop.run_until_complete(task) - except Exception: - print("exception consumed") + asyncio.run(bug()) .. seealso:: From solipsis at pitrou.net Thu Sep 13 05:09:57 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 13 Sep 2018 09:09:57 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=-29 Message-ID: <20180913090957.1.5864AB802B43BD39@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [-7, 1, 0] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_logging leaked [-26, 1, 0] memory blocks, sum=-25 test_multiprocessing_fork leaked [0, 0, -2] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogWfAt1A', '--timeout', '7200'] From webhook-mailer at python.org Thu Sep 13 07:30:15 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 11:30:15 -0000 Subject: [Python-checkins] bpo-34658: Fix rare subprocess prexec_fn fork error. (GH-9255) Message-ID: https://github.com/python/cpython/commit/a20b6adb5a5880fd22c099961eb9f9787739cefe commit: a20b6adb5a5880fd22c099961eb9f9787739cefe branch: master author: Gregory P. Smith committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-13T04:30:10-07:00 summary: bpo-34658: Fix rare subprocess prexec_fn fork error. (GH-9255) [bpo-34658](https://www.bugs.python.org/issue34658): Fix a rare interpreter unhandled exception state SystemError only seen when using subprocess with a preexec_fn while an after_parent handler has been registered with os.register_at_fork and the fork system call fails. https://bugs.python.org/issue34658 files: A Misc/NEWS.d/next/Library/2018-09-13-03-59-43.bpo-34658.ykZ-ia.rst M Modules/_posixsubprocess.c diff --git a/Misc/NEWS.d/next/Library/2018-09-13-03-59-43.bpo-34658.ykZ-ia.rst b/Misc/NEWS.d/next/Library/2018-09-13-03-59-43.bpo-34658.ykZ-ia.rst new file mode 100644 index 000000000000..35375a088300 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-13-03-59-43.bpo-34658.ykZ-ia.rst @@ -0,0 +1,3 @@ +Fix a rare interpreter unhandled exception state SystemError only seen when +using subprocess with a preexec_fn while an after_parent handler has been +registered with os.register_at_fork and the fork system call fails. diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index aeb10f9ecfe4..dd69b9eb1ff6 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -564,6 +564,7 @@ subprocess_fork_exec(PyObject* self, PyObject *args) char *const *exec_array, *const *argv = NULL, *const *envp = NULL; Py_ssize_t arg_num; int need_after_fork = 0; + int saved_errno = 0; if (!PyArg_ParseTuple( args, "OOpO!OOiiiiiiiiiiO:fork_exec", @@ -700,14 +701,14 @@ subprocess_fork_exec(PyObject* self, PyObject *args) _exit(255); return NULL; /* Dead code to avoid a potential compiler warning. */ } - Py_XDECREF(cwd_obj2); - + /* Parent (original) process */ if (pid == -1) { - /* Capture the errno exception before errno can be clobbered. */ - PyErr_SetFromErrno(PyExc_OSError); + /* Capture errno for the exception. */ + saved_errno = errno; } - /* Parent process */ + Py_XDECREF(cwd_obj2); + if (need_after_fork) PyOS_AfterFork_Parent(); if (envp) @@ -723,8 +724,13 @@ subprocess_fork_exec(PyObject* self, PyObject *args) Py_XDECREF(preexec_fn_args_tuple); Py_XDECREF(gc_module); - if (pid == -1) - return NULL; /* fork() failed. Exception set earlier. */ + if (pid == -1) { + errno = saved_errno; + /* We can't call this above as PyOS_AfterFork_Parent() calls back + * into Python code which would see the unreturned error. */ + PyErr_SetFromErrno(PyExc_OSError); + return NULL; /* fork() failed. */ + } return PyLong_FromPid(pid); From webhook-mailer at python.org Thu Sep 13 12:35:01 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 16:35:01 -0000 Subject: [Python-checkins] bpo-34653: Removed unused function PyParser_SimpleParseStringFilename. (GH-9260) Message-ID: https://github.com/python/cpython/commit/53c427e839d3e55d4791dca687a2d47534465a7a commit: 53c427e839d3e55d4791dca687a2d47534465a7a branch: master author: Eric V. Smith committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-13T09:34:55-07:00 summary: bpo-34653: Removed unused function PyParser_SimpleParseStringFilename. (GH-9260) This function was not in any .h file and was not used by Python, so removing it is safe. https://bugs.python.org/issue34653 files: A Misc/NEWS.d/next/Core and Builtins/2018-09-13-12-06-09.bpo-34653.z8NE-i.rst M Python/pythonrun.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-13-12-06-09.bpo-34653.z8NE-i.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-13-12-06-09.bpo-34653.z8NE-i.rst new file mode 100644 index 000000000000..30193742a320 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-13-12-06-09.bpo-34653.z8NE-i.rst @@ -0,0 +1 @@ +Remove unused function PyParser_SimpleParseStringFilename. diff --git a/Python/pythonrun.c b/Python/pythonrun.c index 43a98c96cd5c..276c5d102858 100644 --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -1295,12 +1295,6 @@ PyParser_SimpleParseStringFlagsFilename(const char *str, const char *filename, return n; } -node * -PyParser_SimpleParseStringFilename(const char *str, const char *filename, int start) -{ - return PyParser_SimpleParseStringFlagsFilename(str, filename, start, 0); -} - /* May want to move a more generalized form of this to parsetok.c or even parser modules. */ From webhook-mailer at python.org Thu Sep 13 13:08:49 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 13 Sep 2018 17:08:49 -0000 Subject: [Python-checkins] closes bpo-34661: Fix test_shutil if unzip doesn't support -t. (GH-9262) Message-ID: https://github.com/python/cpython/commit/a710ebd21b09efe902dde84d4862ce5c6427f7af commit: a710ebd21b09efe902dde84d4862ce5c6427f7af branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-13T10:08:46-07:00 summary: closes bpo-34661: Fix test_shutil if unzip doesn't support -t. (GH-9262) files: A Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst M Lib/test/test_shutil.py diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 7e0a3292e0f8..797228db4d51 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1181,6 +1181,8 @@ def test_unzip_zipfile(self): subprocess.check_output(zip_cmd, stderr=subprocess.STDOUT) except subprocess.CalledProcessError as exc: details = exc.output.decode(errors="replace") + if 'unrecognized option: t' in details: + self.skip("unzip doesn't support -t") msg = "{}\n\n**Unzip Output**\n{}" self.fail(msg.format(exc, details)) diff --git a/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst b/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst new file mode 100644 index 000000000000..fc2b8e90ead5 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst @@ -0,0 +1 @@ +Fix test_shutil if unzip doesn't support -t. From webhook-mailer at python.org Thu Sep 13 13:11:03 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 17:11:03 -0000 Subject: [Python-checkins] bpo-34658: Fix rare subprocess prexec_fn fork error. (GH-9255) Message-ID: https://github.com/python/cpython/commit/b2ff9a9f9298761985aa85aa3f42421ce6effb9e commit: b2ff9a9f9298761985aa85aa3f42421ce6effb9e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-13T10:11:00-07:00 summary: bpo-34658: Fix rare subprocess prexec_fn fork error. (GH-9255) [bpo-34658](https://www.bugs.python.org/issue34658): Fix a rare interpreter unhandled exception state SystemError only seen when using subprocess with a preexec_fn while an after_parent handler has been registered with os.register_at_fork and the fork system call fails. https://bugs.python.org/issue34658 (cherry picked from commit a20b6adb5a5880fd22c099961eb9f9787739cefe) Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Library/2018-09-13-03-59-43.bpo-34658.ykZ-ia.rst M Modules/_posixsubprocess.c diff --git a/Misc/NEWS.d/next/Library/2018-09-13-03-59-43.bpo-34658.ykZ-ia.rst b/Misc/NEWS.d/next/Library/2018-09-13-03-59-43.bpo-34658.ykZ-ia.rst new file mode 100644 index 000000000000..35375a088300 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-13-03-59-43.bpo-34658.ykZ-ia.rst @@ -0,0 +1,3 @@ +Fix a rare interpreter unhandled exception state SystemError only seen when +using subprocess with a preexec_fn while an after_parent handler has been +registered with os.register_at_fork and the fork system call fails. diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index aeb10f9ecfe4..dd69b9eb1ff6 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -564,6 +564,7 @@ subprocess_fork_exec(PyObject* self, PyObject *args) char *const *exec_array, *const *argv = NULL, *const *envp = NULL; Py_ssize_t arg_num; int need_after_fork = 0; + int saved_errno = 0; if (!PyArg_ParseTuple( args, "OOpO!OOiiiiiiiiiiO:fork_exec", @@ -700,14 +701,14 @@ subprocess_fork_exec(PyObject* self, PyObject *args) _exit(255); return NULL; /* Dead code to avoid a potential compiler warning. */ } - Py_XDECREF(cwd_obj2); - + /* Parent (original) process */ if (pid == -1) { - /* Capture the errno exception before errno can be clobbered. */ - PyErr_SetFromErrno(PyExc_OSError); + /* Capture errno for the exception. */ + saved_errno = errno; } - /* Parent process */ + Py_XDECREF(cwd_obj2); + if (need_after_fork) PyOS_AfterFork_Parent(); if (envp) @@ -723,8 +724,13 @@ subprocess_fork_exec(PyObject* self, PyObject *args) Py_XDECREF(preexec_fn_args_tuple); Py_XDECREF(gc_module); - if (pid == -1) - return NULL; /* fork() failed. Exception set earlier. */ + if (pid == -1) { + errno = saved_errno; + /* We can't call this above as PyOS_AfterFork_Parent() calls back + * into Python code which would see the unreturned error. */ + PyErr_SetFromErrno(PyExc_OSError); + return NULL; /* fork() failed. */ + } return PyLong_FromPid(pid); From webhook-mailer at python.org Thu Sep 13 13:27:55 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 17:27:55 -0000 Subject: [Python-checkins] closes bpo-34661: Fix test_shutil if unzip doesn't support -t. (GH-9262) Message-ID: https://github.com/python/cpython/commit/1550b7311216082748ddcdc048ee0374ce004513 commit: 1550b7311216082748ddcdc048ee0374ce004513 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-13T10:27:52-07:00 summary: closes bpo-34661: Fix test_shutil if unzip doesn't support -t. (GH-9262) (cherry picked from commit a710ebd21b09efe902dde84d4862ce5c6427f7af) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst M Lib/test/test_shutil.py diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 2cb2f14643e1..ee23ac806283 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1124,6 +1124,8 @@ def test_unzip_zipfile(self): subprocess.check_output(zip_cmd, stderr=subprocess.STDOUT) except subprocess.CalledProcessError as exc: details = exc.output.decode(errors="replace") + if 'unrecognized option: t' in details: + self.skip("unzip doesn't support -t") msg = "{}\n\n**Unzip Output**\n{}" self.fail(msg.format(exc, details)) diff --git a/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst b/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst new file mode 100644 index 000000000000..fc2b8e90ead5 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst @@ -0,0 +1 @@ +Fix test_shutil if unzip doesn't support -t. From webhook-mailer at python.org Thu Sep 13 13:32:48 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 17:32:48 -0000 Subject: [Python-checkins] closes bpo-34661: Fix test_shutil if unzip doesn't support -t. (GH-9262) Message-ID: https://github.com/python/cpython/commit/7eeb80b1e0034fd4e6b8d03b2c0fc5c94fe8d8fa commit: 7eeb80b1e0034fd4e6b8d03b2c0fc5c94fe8d8fa branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-13T10:32:44-07:00 summary: closes bpo-34661: Fix test_shutil if unzip doesn't support -t. (GH-9262) (cherry picked from commit a710ebd21b09efe902dde84d4862ce5c6427f7af) Co-authored-by: Benjamin Peterson files: A Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst M Lib/test/test_shutil.py diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 588e7d27941c..8b5c35ae9ad5 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1127,6 +1127,8 @@ def test_unzip_zipfile(self): subprocess.check_output(zip_cmd, stderr=subprocess.STDOUT) except subprocess.CalledProcessError as exc: details = exc.output.decode(errors="replace") + if 'unrecognized option: t' in details: + self.skip("unzip doesn't support -t") msg = "{}\n\n**Unzip Output**\n{}" self.fail(msg.format(exc, details)) diff --git a/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst b/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst new file mode 100644 index 000000000000..fc2b8e90ead5 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst @@ -0,0 +1 @@ +Fix test_shutil if unzip doesn't support -t. From webhook-mailer at python.org Thu Sep 13 13:57:26 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 13 Sep 2018 17:57:26 -0000 Subject: [Python-checkins] bpo-34661: Fix test skipping call. (GH-9266) Message-ID: https://github.com/python/cpython/commit/e78734d579439861f6d7e12f35d268836b2c1e24 commit: e78734d579439861f6d7e12f35d268836b2c1e24 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-13T10:57:23-07:00 summary: bpo-34661: Fix test skipping call. (GH-9266) files: M Lib/test/test_shutil.py diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 797228db4d51..a169c3684f2d 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1182,7 +1182,7 @@ def test_unzip_zipfile(self): except subprocess.CalledProcessError as exc: details = exc.output.decode(errors="replace") if 'unrecognized option: t' in details: - self.skip("unzip doesn't support -t") + self.skipTest("unzip doesn't support -t") msg = "{}\n\n**Unzip Output**\n{}" self.fail(msg.format(exc, details)) From webhook-mailer at python.org Thu Sep 13 14:24:10 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 13 Sep 2018 18:24:10 -0000 Subject: [Python-checkins] [2.7] closes bpo-34661: Fix test_shutil if unzip doesn't support -t. (GH-9267) Message-ID: https://github.com/python/cpython/commit/18e21883a4ec2a36a02054eb2ff47e3ba9bd1d33 commit: 18e21883a4ec2a36a02054eb2ff47e3ba9bd1d33 branch: 2.7 author: Benjamin Peterson committer: GitHub date: 2018-09-13T11:24:07-07:00 summary: [2.7] closes bpo-34661: Fix test_shutil if unzip doesn't support -t. (GH-9267) (cherry picked from commit a710ebd21b09efe902dde84d4862ce5c6427f7af) files: A Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst M Lib/test/test_shutil.py diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 8338712da2b7..ca89fe509b23 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -543,6 +543,8 @@ def test_unzip_zipfile(self): subprocess.check_output(zip_cmd, stderr=subprocess.STDOUT) except subprocess.CalledProcessError as exc: details = exc.output + if 'unrecognized option: t' in details: + self.skipTest("unzip doesn't support -t") msg = "{}\n\n**Unzip Output**\n{}" self.fail(msg.format(exc, details)) diff --git a/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst b/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst new file mode 100644 index 000000000000..fc2b8e90ead5 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-13-09-53-15.bpo-34661.bdTamP.rst @@ -0,0 +1 @@ +Fix test_shutil if unzip doesn't support -t. From webhook-mailer at python.org Thu Sep 13 14:35:25 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 18:35:25 -0000 Subject: [Python-checkins] bpo-34661: Fix test skipping call. (GH-9266) Message-ID: https://github.com/python/cpython/commit/218b4bf47410ff8ab46b68461267fe10db18aee1 commit: 218b4bf47410ff8ab46b68461267fe10db18aee1 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-13T11:35:18-07:00 summary: bpo-34661: Fix test skipping call. (GH-9266) (cherry picked from commit e78734d579439861f6d7e12f35d268836b2c1e24) Co-authored-by: Benjamin Peterson files: M Lib/test/test_shutil.py diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index ee23ac806283..6e2b1004d309 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1125,7 +1125,7 @@ def test_unzip_zipfile(self): except subprocess.CalledProcessError as exc: details = exc.output.decode(errors="replace") if 'unrecognized option: t' in details: - self.skip("unzip doesn't support -t") + self.skipTest("unzip doesn't support -t") msg = "{}\n\n**Unzip Output**\n{}" self.fail(msg.format(exc, details)) From webhook-mailer at python.org Thu Sep 13 14:47:50 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 18:47:50 -0000 Subject: [Python-checkins] bpo-34661: Fix test skipping call. (GH-9266) Message-ID: https://github.com/python/cpython/commit/81361cdff847f14f6258c0ead17817d6f9e04022 commit: 81361cdff847f14f6258c0ead17817d6f9e04022 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-13T11:47:47-07:00 summary: bpo-34661: Fix test skipping call. (GH-9266) (cherry picked from commit e78734d579439861f6d7e12f35d268836b2c1e24) Co-authored-by: Benjamin Peterson files: M Lib/test/test_shutil.py diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 8b5c35ae9ad5..81457c8f09c9 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1128,7 +1128,7 @@ def test_unzip_zipfile(self): except subprocess.CalledProcessError as exc: details = exc.output.decode(errors="replace") if 'unrecognized option: t' in details: - self.skip("unzip doesn't support -t") + self.skipTest("unzip doesn't support -t") msg = "{}\n\n**Unzip Output**\n{}" self.fail(msg.format(exc, details)) From webhook-mailer at python.org Thu Sep 13 14:49:50 2018 From: webhook-mailer at python.org (Ned Deily) Date: Thu, 13 Sep 2018 18:49:50 -0000 Subject: [Python-checkins] bpo-34247: add porting note to 3.7 What's New (GH-9223) Message-ID: https://github.com/python/cpython/commit/66755cbb1e529f54c9066639ebbbac81add0affd commit: 66755cbb1e529f54c9066639ebbbac81add0affd branch: master author: Ned Deily committer: GitHub date: 2018-09-13T11:49:47-07:00 summary: bpo-34247: add porting note to 3.7 What's New (GH-9223) files: M Doc/whatsnew/3.7.rst diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index be4fef181917..f53a0268738a 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2484,3 +2484,13 @@ The current exception state has been moved from the frame object to the co-routi This simplified the interpreter and fixed a couple of obscure bugs caused by having swap exception state when entering or exiting a generator. (Contributed by Mark Shannon in :issue:`25612`.) + +Notable changes in Python 3.7.1 +=============================== + +Starting in 3.7.1, :c:func:`Py_Initialize` now consistently reads and respects +all of the same environment settings as :c:func:`Py_Main` (in earlier Python +versions, it respected an ill-defined subset of those environment variables, +while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If +this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before +calling :c:func:`Py_Initialize`. From webhook-mailer at python.org Thu Sep 13 15:00:19 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 13 Sep 2018 19:00:19 -0000 Subject: [Python-checkins] closes bpo-34664: Only check file permission bits of newly created directories. (GH-9273) Message-ID: https://github.com/python/cpython/commit/84db4a9978069a98978e9cd7951d1a01d47e5286 commit: 84db4a9978069a98978e9cd7951d1a01d47e5286 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-13T12:00:14-07:00 summary: closes bpo-34664: Only check file permission bits of newly created directories. (GH-9273) files: M Lib/test/test_os.py diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index 0a8ba5cf95f1..6dbc25561237 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1131,8 +1131,8 @@ def test_mode(self): self.assertTrue(os.path.exists(path)) self.assertTrue(os.path.isdir(path)) if os.name != 'nt': - self.assertEqual(stat.S_IMODE(os.stat(path).st_mode), 0o555) - self.assertEqual(stat.S_IMODE(os.stat(parent).st_mode), 0o775) + self.assertEqual(os.stat(path).st_mode & 0o777, 0o555) + self.assertEqual(os.stat(parent).st_mode & 0o777, 0o775) def test_exist_ok_existing_directory(self): path = os.path.join(support.TESTFN, 'dir1') From webhook-mailer at python.org Thu Sep 13 15:14:49 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 19:14:49 -0000 Subject: [Python-checkins] bpo-34247: add porting note to 3.7 What's New (GH-9223) Message-ID: https://github.com/python/cpython/commit/305056494d7e1debec3df268b8925725b0110293 commit: 305056494d7e1debec3df268b8925725b0110293 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-13T12:14:46-07:00 summary: bpo-34247: add porting note to 3.7 What's New (GH-9223) (cherry picked from commit 66755cbb1e529f54c9066639ebbbac81add0affd) Co-authored-by: Ned Deily files: M Doc/whatsnew/3.7.rst diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index cb4865e48403..2a2e23cc6f91 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2485,3 +2485,13 @@ The current exception state has been moved from the frame object to the co-routi This simplified the interpreter and fixed a couple of obscure bugs caused by having swap exception state when entering or exiting a generator. (Contributed by Mark Shannon in :issue:`25612`.) + +Notable changes in Python 3.7.1 +=============================== + +Starting in 3.7.1, :c:func:`Py_Initialize` now consistently reads and respects +all of the same environment settings as :c:func:`Py_Main` (in earlier Python +versions, it respected an ill-defined subset of those environment variables, +while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If +this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before +calling :c:func:`Py_Initialize`. From webhook-mailer at python.org Thu Sep 13 15:42:31 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 19:42:31 -0000 Subject: [Python-checkins] closes bpo-34664: Only check file permission bits of newly created directories. (GH-9273) Message-ID: https://github.com/python/cpython/commit/b2a6aa32f34b6d77c15f175a9df7271a05519bf9 commit: b2a6aa32f34b6d77c15f175a9df7271a05519bf9 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-13T12:42:26-07:00 summary: closes bpo-34664: Only check file permission bits of newly created directories. (GH-9273) (cherry picked from commit 84db4a9978069a98978e9cd7951d1a01d47e5286) Co-authored-by: Benjamin Peterson files: M Lib/test/test_os.py diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index 79ddc48eeb7d..cef4a09d7de9 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1131,8 +1131,8 @@ def test_mode(self): self.assertTrue(os.path.exists(path)) self.assertTrue(os.path.isdir(path)) if os.name != 'nt': - self.assertEqual(stat.S_IMODE(os.stat(path).st_mode), 0o555) - self.assertEqual(stat.S_IMODE(os.stat(parent).st_mode), 0o775) + self.assertEqual(os.stat(path).st_mode & 0o777, 0o555) + self.assertEqual(os.stat(parent).st_mode & 0o777, 0o775) def test_exist_ok_existing_directory(self): path = os.path.join(support.TESTFN, 'dir1') From webhook-mailer at python.org Thu Sep 13 17:53:13 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 21:53:13 -0000 Subject: [Python-checkins] bpo-31132: Remove prlimit permission test. (GH-9280) Message-ID: https://github.com/python/cpython/commit/01e0afa994c2e840f85e2de103e72a2c0ddf1b1f commit: 01e0afa994c2e840f85e2de103e72a2c0ddf1b1f branch: master author: Benjamin Peterson committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-13T14:53:09-07:00 summary: bpo-31132: Remove prlimit permission test. (GH-9280) This test is doesn't work when the test process is privledged, which is hard to detect. https://bugs.python.org/issue34668 files: M Lib/test/test_resource.py diff --git a/Lib/test/test_resource.py b/Lib/test/test_resource.py index 4b852789be7e..b07eb73b2aeb 100644 --- a/Lib/test/test_resource.py +++ b/Lib/test/test_resource.py @@ -147,9 +147,6 @@ def test_freebsd_contants(self): @support.requires_linux_version(2, 6, 36) def test_prlimit(self): self.assertRaises(TypeError, resource.prlimit) - if os.geteuid() != 0: - self.assertRaises(PermissionError, resource.prlimit, - 1, resource.RLIMIT_AS) self.assertRaises(ProcessLookupError, resource.prlimit, -1, resource.RLIMIT_AS) limit = resource.getrlimit(resource.RLIMIT_AS) From webhook-mailer at python.org Thu Sep 13 18:13:59 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 22:13:59 -0000 Subject: [Python-checkins] bpo-31132: Remove prlimit permission test. (GH-9280) Message-ID: https://github.com/python/cpython/commit/2b162941c0793e0dd5533ee3090901cba9d8cf7d commit: 2b162941c0793e0dd5533ee3090901cba9d8cf7d branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-13T15:13:55-07:00 summary: bpo-31132: Remove prlimit permission test. (GH-9280) This test is doesn't work when the test process is privledged, which is hard to detect. https://bugs.python.org/issue34668 (cherry picked from commit 01e0afa994c2e840f85e2de103e72a2c0ddf1b1f) Co-authored-by: Benjamin Peterson files: M Lib/test/test_resource.py diff --git a/Lib/test/test_resource.py b/Lib/test/test_resource.py index 4b852789be7e..b07eb73b2aeb 100644 --- a/Lib/test/test_resource.py +++ b/Lib/test/test_resource.py @@ -147,9 +147,6 @@ def test_freebsd_contants(self): @support.requires_linux_version(2, 6, 36) def test_prlimit(self): self.assertRaises(TypeError, resource.prlimit) - if os.geteuid() != 0: - self.assertRaises(PermissionError, resource.prlimit, - 1, resource.RLIMIT_AS) self.assertRaises(ProcessLookupError, resource.prlimit, -1, resource.RLIMIT_AS) limit = resource.getrlimit(resource.RLIMIT_AS) From webhook-mailer at python.org Thu Sep 13 18:22:19 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 13 Sep 2018 22:22:19 -0000 Subject: [Python-checkins] bpo-31132: Remove prlimit permission test. (GH-9280) Message-ID: https://github.com/python/cpython/commit/f79d74d5a3ceb06b63996e8ecf23e419e16e4be2 commit: f79d74d5a3ceb06b63996e8ecf23e419e16e4be2 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-13T15:22:16-07:00 summary: bpo-31132: Remove prlimit permission test. (GH-9280) This test is doesn't work when the test process is privledged, which is hard to detect. https://bugs.python.org/issue34668 (cherry picked from commit 01e0afa994c2e840f85e2de103e72a2c0ddf1b1f) Co-authored-by: Benjamin Peterson files: M Lib/test/test_resource.py diff --git a/Lib/test/test_resource.py b/Lib/test/test_resource.py index cc9c57024dee..b405f0169d6e 100644 --- a/Lib/test/test_resource.py +++ b/Lib/test/test_resource.py @@ -148,9 +148,6 @@ def test_freebsd_contants(self): @support.requires_linux_version(2, 6, 36) def test_prlimit(self): self.assertRaises(TypeError, resource.prlimit) - if os.geteuid() != 0: - self.assertRaises(PermissionError, resource.prlimit, - 1, resource.RLIMIT_AS) self.assertRaises(ProcessLookupError, resource.prlimit, -1, resource.RLIMIT_AS) limit = resource.getrlimit(resource.RLIMIT_AS) From webhook-mailer at python.org Thu Sep 13 19:14:44 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Thu, 13 Sep 2018 23:14:44 -0000 Subject: [Python-checkins] polish exceptions and platforms (GH-9272) Message-ID: https://github.com/python/cpython/commit/a3c8ba723530ceb3328d184d3e0020534b522778 commit: a3c8ba723530ceb3328d184d3e0020534b522778 branch: master author: Carol Willing committer: Yury Selivanov date: 2018-09-13T16:14:41-07:00 summary: polish exceptions and platforms (GH-9272) files: M Doc/library/asyncio-exceptions.rst M Doc/library/asyncio-platforms.rst diff --git a/Doc/library/asyncio-exceptions.rst b/Doc/library/asyncio-exceptions.rst index bcd3599477f7..7f3ed857943d 100644 --- a/Doc/library/asyncio-exceptions.rst +++ b/Doc/library/asyncio-exceptions.rst @@ -19,11 +19,12 @@ Exceptions The operation has been cancelled. - This exception can be caught to perform custom operations on + This exception can be caught to perform custom operations when asyncio Tasks are cancelled. In almost all situations the exception must always be re-raised. .. note:: + This exception is a subclass of :exc:`Exception`, so it can be accidentally suppressed by ``try..except`` block:: @@ -54,7 +55,7 @@ Exceptions .. exception:: SendfileNotAvailableError - The "sendfile" syscall for is not available for the given + The "sendfile" syscall is not available for the given socket or file type. A subclass of :exc:`RuntimeError`. diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst index afdbce67cdd4..2f0f53495223 100644 --- a/Doc/library/asyncio-platforms.rst +++ b/Doc/library/asyncio-platforms.rst @@ -9,7 +9,8 @@ Platforms Support ================= The :mod:`asyncio` module has been designed to be portable, -but some platforms have subtle differences and limitations. +but some platforms have subtle differences and limitations +due to the platforms' underlying architecture and capabilities. All Platforms @@ -26,7 +27,7 @@ All event loops on Windows do not support the following methods: * :meth:`loop.create_unix_connection` and :meth:`loop.create_unix_server` are not supported. - The :data:`socket.AF_UNIX` socket family is specific to UNIX/ + The :data:`socket.AF_UNIX` socket family is specific to UNIX. * :meth:`loop.add_signal_handler` and :meth:`loop.remove_signal_handler` are not supported. @@ -66,8 +67,8 @@ Windows configuration. Subprocess Support on Windows ----------------------------- -:class:`SelectorEventLoop` on Windows does not support subproceses, -so :class:`ProactorEventLoop` should be used instead:: +:class:`SelectorEventLoop` on Windows does not support subproceses. +On Windows, :class:`ProactorEventLoop` should be used instead:: import asyncio From webhook-mailer at python.org Thu Sep 13 19:16:44 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 13 Sep 2018 23:16:44 -0000 Subject: [Python-checkins] [2.7] bpo-20047: Remove Objects/bytesobject.c from 2.7 (GH-9268) Message-ID: https://github.com/python/cpython/commit/0b9fe1734168d45861d6dc3022492387dec5a4a2 commit: 0b9fe1734168d45861d6dc3022492387dec5a4a2 branch: 2.7 author: Zackery Spytz committer: Berker Peksag date: 2018-09-14T02:16:41+03:00 summary: [2.7] bpo-20047: Remove Objects/bytesobject.c from 2.7 (GH-9268) It was accidentally added in 107f3cc791d223dc06b7c80f0de672e88ae6a8d1. files: D Objects/bytesobject.c diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c deleted file mode 100644 index 7ba90aa307b8..000000000000 --- a/Objects/bytesobject.c +++ /dev/null @@ -1,3432 +0,0 @@ -/* bytes object implementation */ - -#define PY_SSIZE_T_CLEAN - -#include "Python.h" -#include "internal/mem.h" -#include "internal/pystate.h" - -#include "bytes_methods.h" -#include "pystrhex.h" -#include - -/*[clinic input] -class bytes "PyBytesObject *" "&PyBytes_Type" -[clinic start generated code]*/ -/*[clinic end generated code: output=da39a3ee5e6b4b0d input=7a238f965d64892b]*/ - -#include "clinic/bytesobject.c.h" - -#ifdef COUNT_ALLOCS -Py_ssize_t null_strings, one_strings; -#endif - -static PyBytesObject *characters[UCHAR_MAX + 1]; -static PyBytesObject *nullstring; - -/* PyBytesObject_SIZE gives the basic size of a string; any memory allocation - for a string of length n should request PyBytesObject_SIZE + n bytes. - - Using PyBytesObject_SIZE instead of sizeof(PyBytesObject) saves - 3 bytes per string allocation on a typical system. -*/ -#define PyBytesObject_SIZE (offsetof(PyBytesObject, ob_sval) + 1) - -/* Forward declaration */ -Py_LOCAL_INLINE(Py_ssize_t) _PyBytesWriter_GetSize(_PyBytesWriter *writer, - char *str); - -/* - For PyBytes_FromString(), the parameter `str' points to a null-terminated - string containing exactly `size' bytes. - - For PyBytes_FromStringAndSize(), the parameter `str' is - either NULL or else points to a string containing at least `size' bytes. - For PyBytes_FromStringAndSize(), the string in the `str' parameter does - not have to be null-terminated. (Therefore it is safe to construct a - substring by calling `PyBytes_FromStringAndSize(origstring, substrlen)'.) - If `str' is NULL then PyBytes_FromStringAndSize() will allocate `size+1' - bytes (setting the last byte to the null terminating character) and you can - fill in the data yourself. If `str' is non-NULL then the resulting - PyBytes object must be treated as immutable and you must not fill in nor - alter the data yourself, since the strings may be shared. - - The PyObject member `op->ob_size', which denotes the number of "extra - items" in a variable-size object, will contain the number of bytes - allocated for string data, not counting the null terminating character. - It is therefore equal to the `size' parameter (for - PyBytes_FromStringAndSize()) or the length of the string in the `str' - parameter (for PyBytes_FromString()). -*/ -static PyObject * -_PyBytes_FromSize(Py_ssize_t size, int use_calloc) -{ - PyBytesObject *op; - assert(size >= 0); - - if (size == 0 && (op = nullstring) != NULL) { -#ifdef COUNT_ALLOCS - null_strings++; -#endif - Py_INCREF(op); - return (PyObject *)op; - } - - if ((size_t)size > (size_t)PY_SSIZE_T_MAX - PyBytesObject_SIZE) { - PyErr_SetString(PyExc_OverflowError, - "byte string is too large"); - return NULL; - } - - /* Inline PyObject_NewVar */ - if (use_calloc) - op = (PyBytesObject *)PyObject_Calloc(1, PyBytesObject_SIZE + size); - else - op = (PyBytesObject *)PyObject_Malloc(PyBytesObject_SIZE + size); - if (op == NULL) - return PyErr_NoMemory(); - (void)PyObject_INIT_VAR(op, &PyBytes_Type, size); - op->ob_shash = -1; - if (!use_calloc) - op->ob_sval[size] = '\0'; - /* empty byte string singleton */ - if (size == 0) { - nullstring = op; - Py_INCREF(op); - } - return (PyObject *) op; -} - -PyObject * -PyBytes_FromStringAndSize(const char *str, Py_ssize_t size) -{ - PyBytesObject *op; - if (size < 0) { - PyErr_SetString(PyExc_SystemError, - "Negative size passed to PyBytes_FromStringAndSize"); - return NULL; - } - if (size == 1 && str != NULL && - (op = characters[*str & UCHAR_MAX]) != NULL) - { -#ifdef COUNT_ALLOCS - one_strings++; -#endif - Py_INCREF(op); - return (PyObject *)op; - } - - op = (PyBytesObject *)_PyBytes_FromSize(size, 0); - if (op == NULL) - return NULL; - if (str == NULL) - return (PyObject *) op; - - memcpy(op->ob_sval, str, size); - /* share short strings */ - if (size == 1) { - characters[*str & UCHAR_MAX] = op; - Py_INCREF(op); - } - return (PyObject *) op; -} - -PyObject * -PyBytes_FromString(const char *str) -{ - size_t size; - PyBytesObject *op; - - assert(str != NULL); - size = strlen(str); - if (size > PY_SSIZE_T_MAX - PyBytesObject_SIZE) { - PyErr_SetString(PyExc_OverflowError, - "byte string is too long"); - return NULL; - } - if (size == 0 && (op = nullstring) != NULL) { -#ifdef COUNT_ALLOCS - null_strings++; -#endif - Py_INCREF(op); - return (PyObject *)op; - } - if (size == 1 && (op = characters[*str & UCHAR_MAX]) != NULL) { -#ifdef COUNT_ALLOCS - one_strings++; -#endif - Py_INCREF(op); - return (PyObject *)op; - } - - /* Inline PyObject_NewVar */ - op = (PyBytesObject *)PyObject_MALLOC(PyBytesObject_SIZE + size); - if (op == NULL) - return PyErr_NoMemory(); - (void)PyObject_INIT_VAR(op, &PyBytes_Type, size); - op->ob_shash = -1; - memcpy(op->ob_sval, str, size+1); - /* share short strings */ - if (size == 0) { - nullstring = op; - Py_INCREF(op); - } else if (size == 1) { - characters[*str & UCHAR_MAX] = op; - Py_INCREF(op); - } - return (PyObject *) op; -} - -PyObject * -PyBytes_FromFormatV(const char *format, va_list vargs) -{ - char *s; - const char *f; - const char *p; - Py_ssize_t prec; - int longflag; - int size_tflag; - /* Longest 64-bit formatted numbers: - - "18446744073709551615\0" (21 bytes) - - "-9223372036854775808\0" (21 bytes) - Decimal takes the most space (it isn't enough for octal.) - - Longest 64-bit pointer representation: - "0xffffffffffffffff\0" (19 bytes). */ - char buffer[21]; - _PyBytesWriter writer; - - _PyBytesWriter_Init(&writer); - - s = _PyBytesWriter_Alloc(&writer, strlen(format)); - if (s == NULL) - return NULL; - writer.overallocate = 1; - -#define WRITE_BYTES(str) \ - do { \ - s = _PyBytesWriter_WriteBytes(&writer, s, (str), strlen(str)); \ - if (s == NULL) \ - goto error; \ - } while (0) - - for (f = format; *f; f++) { - if (*f != '%') { - *s++ = *f; - continue; - } - - p = f++; - - /* ignore the width (ex: 10 in "%10s") */ - while (Py_ISDIGIT(*f)) - f++; - - /* parse the precision (ex: 10 in "%.10s") */ - prec = 0; - if (*f == '.') { - f++; - for (; Py_ISDIGIT(*f); f++) { - prec = (prec * 10) + (*f - '0'); - } - } - - while (*f && *f != '%' && !Py_ISALPHA(*f)) - f++; - - /* handle the long flag ('l'), but only for %ld and %lu. - others can be added when necessary. */ - longflag = 0; - if (*f == 'l' && (f[1] == 'd' || f[1] == 'u')) { - longflag = 1; - ++f; - } - - /* handle the size_t flag ('z'). */ - size_tflag = 0; - if (*f == 'z' && (f[1] == 'd' || f[1] == 'u')) { - size_tflag = 1; - ++f; - } - - /* subtract bytes preallocated for the format string - (ex: 2 for "%s") */ - writer.min_size -= (f - p + 1); - - switch (*f) { - case 'c': - { - int c = va_arg(vargs, int); - if (c < 0 || c > 255) { - PyErr_SetString(PyExc_OverflowError, - "PyBytes_FromFormatV(): %c format " - "expects an integer in range [0; 255]"); - goto error; - } - writer.min_size++; - *s++ = (unsigned char)c; - break; - } - - case 'd': - if (longflag) - sprintf(buffer, "%ld", va_arg(vargs, long)); - else if (size_tflag) - sprintf(buffer, "%" PY_FORMAT_SIZE_T "d", - va_arg(vargs, Py_ssize_t)); - else - sprintf(buffer, "%d", va_arg(vargs, int)); - assert(strlen(buffer) < sizeof(buffer)); - WRITE_BYTES(buffer); - break; - - case 'u': - if (longflag) - sprintf(buffer, "%lu", - va_arg(vargs, unsigned long)); - else if (size_tflag) - sprintf(buffer, "%" PY_FORMAT_SIZE_T "u", - va_arg(vargs, size_t)); - else - sprintf(buffer, "%u", - va_arg(vargs, unsigned int)); - assert(strlen(buffer) < sizeof(buffer)); - WRITE_BYTES(buffer); - break; - - case 'i': - sprintf(buffer, "%i", va_arg(vargs, int)); - assert(strlen(buffer) < sizeof(buffer)); - WRITE_BYTES(buffer); - break; - - case 'x': - sprintf(buffer, "%x", va_arg(vargs, int)); - assert(strlen(buffer) < sizeof(buffer)); - WRITE_BYTES(buffer); - break; - - case 's': - { - Py_ssize_t i; - - p = va_arg(vargs, const char*); - i = strlen(p); - if (prec > 0 && i > prec) - i = prec; - s = _PyBytesWriter_WriteBytes(&writer, s, p, i); - if (s == NULL) - goto error; - break; - } - - case 'p': - sprintf(buffer, "%p", va_arg(vargs, void*)); - assert(strlen(buffer) < sizeof(buffer)); - /* %p is ill-defined: ensure leading 0x. */ - if (buffer[1] == 'X') - buffer[1] = 'x'; - else if (buffer[1] != 'x') { - memmove(buffer+2, buffer, strlen(buffer)+1); - buffer[0] = '0'; - buffer[1] = 'x'; - } - WRITE_BYTES(buffer); - break; - - case '%': - writer.min_size++; - *s++ = '%'; - break; - - default: - if (*f == 0) { - /* fix min_size if we reached the end of the format string */ - writer.min_size++; - } - - /* invalid format string: copy unformatted string and exit */ - WRITE_BYTES(p); - return _PyBytesWriter_Finish(&writer, s); - } - } - -#undef WRITE_BYTES - - return _PyBytesWriter_Finish(&writer, s); - - error: - _PyBytesWriter_Dealloc(&writer); - return NULL; -} - -PyObject * -PyBytes_FromFormat(const char *format, ...) -{ - PyObject* ret; - va_list vargs; - -#ifdef HAVE_STDARG_PROTOTYPES - va_start(vargs, format); -#else - va_start(vargs); -#endif - ret = PyBytes_FromFormatV(format, vargs); - va_end(vargs); - return ret; -} - -/* Helpers for formatstring */ - -Py_LOCAL_INLINE(PyObject *) -getnextarg(PyObject *args, Py_ssize_t arglen, Py_ssize_t *p_argidx) -{ - Py_ssize_t argidx = *p_argidx; - if (argidx < arglen) { - (*p_argidx)++; - if (arglen < 0) - return args; - else - return PyTuple_GetItem(args, argidx); - } - PyErr_SetString(PyExc_TypeError, - "not enough arguments for format string"); - return NULL; -} - -/* Format codes - * F_LJUST '-' - * F_SIGN '+' - * F_BLANK ' ' - * F_ALT '#' - * F_ZERO '0' - */ -#define F_LJUST (1<<0) -#define F_SIGN (1<<1) -#define F_BLANK (1<<2) -#define F_ALT (1<<3) -#define F_ZERO (1<<4) - -/* Returns a new reference to a PyBytes object, or NULL on failure. */ - -static char* -formatfloat(PyObject *v, int flags, int prec, int type, - PyObject **p_result, _PyBytesWriter *writer, char *str) -{ - char *p; - PyObject *result; - double x; - size_t len; - - x = PyFloat_AsDouble(v); - if (x == -1.0 && PyErr_Occurred()) { - PyErr_Format(PyExc_TypeError, "float argument required, " - "not %.200s", Py_TYPE(v)->tp_name); - return NULL; - } - - if (prec < 0) - prec = 6; - - p = PyOS_double_to_string(x, type, prec, - (flags & F_ALT) ? Py_DTSF_ALT : 0, NULL); - - if (p == NULL) - return NULL; - - len = strlen(p); - if (writer != NULL) { - str = _PyBytesWriter_Prepare(writer, str, len); - if (str == NULL) - return NULL; - memcpy(str, p, len); - PyMem_Free(p); - str += len; - return str; - } - - result = PyBytes_FromStringAndSize(p, len); - PyMem_Free(p); - *p_result = result; - return str; -} - -static PyObject * -formatlong(PyObject *v, int flags, int prec, int type) -{ - PyObject *result, *iobj; - if (type == 'i') - type = 'd'; - if (PyLong_Check(v)) - return _PyUnicode_FormatLong(v, flags & F_ALT, prec, type); - if (PyNumber_Check(v)) { - /* make sure number is a type of integer for o, x, and X */ - if (type == 'o' || type == 'x' || type == 'X') - iobj = PyNumber_Index(v); - else - iobj = PyNumber_Long(v); - if (iobj == NULL) { - if (!PyErr_ExceptionMatches(PyExc_TypeError)) - return NULL; - } - else if (!PyLong_Check(iobj)) - Py_CLEAR(iobj); - if (iobj != NULL) { - result = _PyUnicode_FormatLong(iobj, flags & F_ALT, prec, type); - Py_DECREF(iobj); - return result; - } - } - PyErr_Format(PyExc_TypeError, - "%%%c format: %s is required, not %.200s", type, - (type == 'o' || type == 'x' || type == 'X') ? "an integer" - : "a number", - Py_TYPE(v)->tp_name); - return NULL; -} - -static int -byte_converter(PyObject *arg, char *p) -{ - if (PyBytes_Check(arg) && PyBytes_GET_SIZE(arg) == 1) { - *p = PyBytes_AS_STRING(arg)[0]; - return 1; - } - else if (PyByteArray_Check(arg) && PyByteArray_GET_SIZE(arg) == 1) { - *p = PyByteArray_AS_STRING(arg)[0]; - return 1; - } - else { - PyObject *iobj; - long ival; - int overflow; - /* make sure number is a type of integer */ - if (PyLong_Check(arg)) { - ival = PyLong_AsLongAndOverflow(arg, &overflow); - } - else { - iobj = PyNumber_Index(arg); - if (iobj == NULL) { - if (!PyErr_ExceptionMatches(PyExc_TypeError)) - return 0; - goto onError; - } - ival = PyLong_AsLongAndOverflow(iobj, &overflow); - Py_DECREF(iobj); - } - if (!overflow && ival == -1 && PyErr_Occurred()) - goto onError; - if (overflow || !(0 <= ival && ival <= 255)) { - PyErr_SetString(PyExc_OverflowError, - "%c arg not in range(256)"); - return 0; - } - *p = (char)ival; - return 1; - } - onError: - PyErr_SetString(PyExc_TypeError, - "%c requires an integer in range(256) or a single byte"); - return 0; -} - -static PyObject *_PyBytes_FromBuffer(PyObject *x); - -static PyObject * -format_obj(PyObject *v, const char **pbuf, Py_ssize_t *plen) -{ - PyObject *func, *result; - _Py_IDENTIFIER(__bytes__); - /* is it a bytes object? */ - if (PyBytes_Check(v)) { - *pbuf = PyBytes_AS_STRING(v); - *plen = PyBytes_GET_SIZE(v); - Py_INCREF(v); - return v; - } - if (PyByteArray_Check(v)) { - *pbuf = PyByteArray_AS_STRING(v); - *plen = PyByteArray_GET_SIZE(v); - Py_INCREF(v); - return v; - } - /* does it support __bytes__? */ - func = _PyObject_LookupSpecial(v, &PyId___bytes__); - if (func != NULL) { - result = _PyObject_CallNoArg(func); - Py_DECREF(func); - if (result == NULL) - return NULL; - if (!PyBytes_Check(result)) { - PyErr_Format(PyExc_TypeError, - "__bytes__ returned non-bytes (type %.200s)", - Py_TYPE(result)->tp_name); - Py_DECREF(result); - return NULL; - } - *pbuf = PyBytes_AS_STRING(result); - *plen = PyBytes_GET_SIZE(result); - return result; - } - /* does it support buffer protocol? */ - if (PyObject_CheckBuffer(v)) { - /* maybe we can avoid making a copy of the buffer object here? */ - result = _PyBytes_FromBuffer(v); - if (result == NULL) - return NULL; - *pbuf = PyBytes_AS_STRING(result); - *plen = PyBytes_GET_SIZE(result); - return result; - } - PyErr_Format(PyExc_TypeError, - "%%b requires a bytes-like object, " - "or an object that implements __bytes__, not '%.100s'", - Py_TYPE(v)->tp_name); - return NULL; -} - -/* fmt%(v1,v2,...) is roughly equivalent to sprintf(fmt, v1, v2, ...) */ - -PyObject * -_PyBytes_FormatEx(const char *format, Py_ssize_t format_len, - PyObject *args, int use_bytearray) -{ - const char *fmt; - char *res; - Py_ssize_t arglen, argidx; - Py_ssize_t fmtcnt; - int args_owned = 0; - PyObject *dict = NULL; - _PyBytesWriter writer; - - if (args == NULL) { - PyErr_BadInternalCall(); - return NULL; - } - fmt = format; - fmtcnt = format_len; - - _PyBytesWriter_Init(&writer); - writer.use_bytearray = use_bytearray; - - res = _PyBytesWriter_Alloc(&writer, fmtcnt); - if (res == NULL) - return NULL; - if (!use_bytearray) - writer.overallocate = 1; - - if (PyTuple_Check(args)) { - arglen = PyTuple_GET_SIZE(args); - argidx = 0; - } - else { - arglen = -1; - argidx = -2; - } - if (Py_TYPE(args)->tp_as_mapping && Py_TYPE(args)->tp_as_mapping->mp_subscript && - !PyTuple_Check(args) && !PyBytes_Check(args) && !PyUnicode_Check(args) && - !PyByteArray_Check(args)) { - dict = args; - } - - while (--fmtcnt >= 0) { - if (*fmt != '%') { - Py_ssize_t len; - char *pos; - - pos = (char *)memchr(fmt + 1, '%', fmtcnt); - if (pos != NULL) - len = pos - fmt; - else - len = fmtcnt + 1; - assert(len != 0); - - memcpy(res, fmt, len); - res += len; - fmt += len; - fmtcnt -= (len - 1); - } - else { - /* Got a format specifier */ - int flags = 0; - Py_ssize_t width = -1; - int prec = -1; - int c = '\0'; - int fill; - PyObject *v = NULL; - PyObject *temp = NULL; - const char *pbuf = NULL; - int sign; - Py_ssize_t len = 0; - char onechar; /* For byte_converter() */ - Py_ssize_t alloc; -#ifdef Py_DEBUG - char *before; -#endif - - fmt++; - if (*fmt == '%') { - *res++ = '%'; - fmt++; - fmtcnt--; - continue; - } - if (*fmt == '(') { - const char *keystart; - Py_ssize_t keylen; - PyObject *key; - int pcount = 1; - - if (dict == NULL) { - PyErr_SetString(PyExc_TypeError, - "format requires a mapping"); - goto error; - } - ++fmt; - --fmtcnt; - keystart = fmt; - /* Skip over balanced parentheses */ - while (pcount > 0 && --fmtcnt >= 0) { - if (*fmt == ')') - --pcount; - else if (*fmt == '(') - ++pcount; - fmt++; - } - keylen = fmt - keystart - 1; - if (fmtcnt < 0 || pcount > 0) { - PyErr_SetString(PyExc_ValueError, - "incomplete format key"); - goto error; - } - key = PyBytes_FromStringAndSize(keystart, - keylen); - if (key == NULL) - goto error; - if (args_owned) { - Py_DECREF(args); - args_owned = 0; - } - args = PyObject_GetItem(dict, key); - Py_DECREF(key); - if (args == NULL) { - goto error; - } - args_owned = 1; - arglen = -1; - argidx = -2; - } - - /* Parse flags. Example: "%+i" => flags=F_SIGN. */ - while (--fmtcnt >= 0) { - switch (c = *fmt++) { - case '-': flags |= F_LJUST; continue; - case '+': flags |= F_SIGN; continue; - case ' ': flags |= F_BLANK; continue; - case '#': flags |= F_ALT; continue; - case '0': flags |= F_ZERO; continue; - } - break; - } - - /* Parse width. Example: "%10s" => width=10 */ - if (c == '*') { - v = getnextarg(args, arglen, &argidx); - if (v == NULL) - goto error; - if (!PyLong_Check(v)) { - PyErr_SetString(PyExc_TypeError, - "* wants int"); - goto error; - } - width = PyLong_AsSsize_t(v); - if (width == -1 && PyErr_Occurred()) - goto error; - if (width < 0) { - flags |= F_LJUST; - width = -width; - } - if (--fmtcnt >= 0) - c = *fmt++; - } - else if (c >= 0 && isdigit(c)) { - width = c - '0'; - while (--fmtcnt >= 0) { - c = Py_CHARMASK(*fmt++); - if (!isdigit(c)) - break; - if (width > (PY_SSIZE_T_MAX - ((int)c - '0')) / 10) { - PyErr_SetString( - PyExc_ValueError, - "width too big"); - goto error; - } - width = width*10 + (c - '0'); - } - } - - /* Parse precision. Example: "%.3f" => prec=3 */ - if (c == '.') { - prec = 0; - if (--fmtcnt >= 0) - c = *fmt++; - if (c == '*') { - v = getnextarg(args, arglen, &argidx); - if (v == NULL) - goto error; - if (!PyLong_Check(v)) { - PyErr_SetString( - PyExc_TypeError, - "* wants int"); - goto error; - } - prec = _PyLong_AsInt(v); - if (prec == -1 && PyErr_Occurred()) - goto error; - if (prec < 0) - prec = 0; - if (--fmtcnt >= 0) - c = *fmt++; - } - else if (c >= 0 && isdigit(c)) { - prec = c - '0'; - while (--fmtcnt >= 0) { - c = Py_CHARMASK(*fmt++); - if (!isdigit(c)) - break; - if (prec > (INT_MAX - ((int)c - '0')) / 10) { - PyErr_SetString( - PyExc_ValueError, - "prec too big"); - goto error; - } - prec = prec*10 + (c - '0'); - } - } - } /* prec */ - if (fmtcnt >= 0) { - if (c == 'h' || c == 'l' || c == 'L') { - if (--fmtcnt >= 0) - c = *fmt++; - } - } - if (fmtcnt < 0) { - PyErr_SetString(PyExc_ValueError, - "incomplete format"); - goto error; - } - v = getnextarg(args, arglen, &argidx); - if (v == NULL) - goto error; - - if (fmtcnt < 0) { - /* last writer: disable writer overallocation */ - writer.overallocate = 0; - } - - sign = 0; - fill = ' '; - switch (c) { - case 'r': - // %r is only for 2/3 code; 3 only code should use %a - case 'a': - temp = PyObject_ASCII(v); - if (temp == NULL) - goto error; - assert(PyUnicode_IS_ASCII(temp)); - pbuf = (const char *)PyUnicode_1BYTE_DATA(temp); - len = PyUnicode_GET_LENGTH(temp); - if (prec >= 0 && len > prec) - len = prec; - break; - - case 's': - // %s is only for 2/3 code; 3 only code should use %b - case 'b': - temp = format_obj(v, &pbuf, &len); - if (temp == NULL) - goto error; - if (prec >= 0 && len > prec) - len = prec; - break; - - case 'i': - case 'd': - case 'u': - case 'o': - case 'x': - case 'X': - if (PyLong_CheckExact(v) - && width == -1 && prec == -1 - && !(flags & (F_SIGN | F_BLANK)) - && c != 'X') - { - /* Fast path */ - int alternate = flags & F_ALT; - int base; - - switch(c) - { - default: - Py_UNREACHABLE(); - case 'd': - case 'i': - case 'u': - base = 10; - break; - case 'o': - base = 8; - break; - case 'x': - case 'X': - base = 16; - break; - } - - /* Fast path */ - writer.min_size -= 2; /* size preallocated for "%d" */ - res = _PyLong_FormatBytesWriter(&writer, res, - v, base, alternate); - if (res == NULL) - goto error; - continue; - } - - temp = formatlong(v, flags, prec, c); - if (!temp) - goto error; - assert(PyUnicode_IS_ASCII(temp)); - pbuf = (const char *)PyUnicode_1BYTE_DATA(temp); - len = PyUnicode_GET_LENGTH(temp); - sign = 1; - if (flags & F_ZERO) - fill = '0'; - break; - - case 'e': - case 'E': - case 'f': - case 'F': - case 'g': - case 'G': - if (width == -1 && prec == -1 - && !(flags & (F_SIGN | F_BLANK))) - { - /* Fast path */ - writer.min_size -= 2; /* size preallocated for "%f" */ - res = formatfloat(v, flags, prec, c, NULL, &writer, res); - if (res == NULL) - goto error; - continue; - } - - if (!formatfloat(v, flags, prec, c, &temp, NULL, res)) - goto error; - pbuf = PyBytes_AS_STRING(temp); - len = PyBytes_GET_SIZE(temp); - sign = 1; - if (flags & F_ZERO) - fill = '0'; - break; - - case 'c': - pbuf = &onechar; - len = byte_converter(v, &onechar); - if (!len) - goto error; - if (width == -1) { - /* Fast path */ - *res++ = onechar; - continue; - } - break; - - default: - PyErr_Format(PyExc_ValueError, - "unsupported format character '%c' (0x%x) " - "at index %zd", - c, c, - (Py_ssize_t)(fmt - 1 - format)); - goto error; - } - - if (sign) { - if (*pbuf == '-' || *pbuf == '+') { - sign = *pbuf++; - len--; - } - else if (flags & F_SIGN) - sign = '+'; - else if (flags & F_BLANK) - sign = ' '; - else - sign = 0; - } - if (width < len) - width = len; - - alloc = width; - if (sign != 0 && len == width) - alloc++; - /* 2: size preallocated for %s */ - if (alloc > 2) { - res = _PyBytesWriter_Prepare(&writer, res, alloc - 2); - if (res == NULL) - goto error; - } -#ifdef Py_DEBUG - before = res; -#endif - - /* Write the sign if needed */ - if (sign) { - if (fill != ' ') - *res++ = sign; - if (width > len) - width--; - } - - /* Write the numeric prefix for "x", "X" and "o" formats - if the alternate form is used. - For example, write "0x" for the "%#x" format. */ - if ((flags & F_ALT) && (c == 'o' || c == 'x' || c == 'X')) { - assert(pbuf[0] == '0'); - assert(pbuf[1] == c); - if (fill != ' ') { - *res++ = *pbuf++; - *res++ = *pbuf++; - } - width -= 2; - if (width < 0) - width = 0; - len -= 2; - } - - /* Pad left with the fill character if needed */ - if (width > len && !(flags & F_LJUST)) { - memset(res, fill, width - len); - res += (width - len); - width = len; - } - - /* If padding with spaces: write sign if needed and/or numeric - prefix if the alternate form is used */ - if (fill == ' ') { - if (sign) - *res++ = sign; - if ((flags & F_ALT) && (c == 'o' || c == 'x' || c == 'X')) { - assert(pbuf[0] == '0'); - assert(pbuf[1] == c); - *res++ = *pbuf++; - *res++ = *pbuf++; - } - } - - /* Copy bytes */ - memcpy(res, pbuf, len); - res += len; - - /* Pad right with the fill character if needed */ - if (width > len) { - memset(res, ' ', width - len); - res += (width - len); - } - - if (dict && (argidx < arglen)) { - PyErr_SetString(PyExc_TypeError, - "not all arguments converted during bytes formatting"); - Py_XDECREF(temp); - goto error; - } - Py_XDECREF(temp); - -#ifdef Py_DEBUG - /* check that we computed the exact size for this write */ - assert((res - before) == alloc); -#endif - } /* '%' */ - - /* If overallocation was disabled, ensure that it was the last - write. Otherwise, we missed an optimization */ - assert(writer.overallocate || fmtcnt < 0 || use_bytearray); - } /* until end */ - - if (argidx < arglen && !dict) { - PyErr_SetString(PyExc_TypeError, - "not all arguments converted during bytes formatting"); - goto error; - } - - if (args_owned) { - Py_DECREF(args); - } - return _PyBytesWriter_Finish(&writer, res); - - error: - _PyBytesWriter_Dealloc(&writer); - if (args_owned) { - Py_DECREF(args); - } - return NULL; -} - -/* =-= */ - -static void -bytes_dealloc(PyObject *op) -{ - Py_TYPE(op)->tp_free(op); -} - -/* Unescape a backslash-escaped string. If unicode is non-zero, - the string is a u-literal. If recode_encoding is non-zero, - the string is UTF-8 encoded and should be re-encoded in the - specified encoding. */ - -static char * -_PyBytes_DecodeEscapeRecode(const char **s, const char *end, - const char *errors, const char *recode_encoding, - _PyBytesWriter *writer, char *p) -{ - PyObject *u, *w; - const char* t; - - t = *s; - /* Decode non-ASCII bytes as UTF-8. */ - while (t < end && (*t & 0x80)) - t++; - u = PyUnicode_DecodeUTF8(*s, t - *s, errors); - if (u == NULL) - return NULL; - - /* Recode them in target encoding. */ - w = PyUnicode_AsEncodedString(u, recode_encoding, errors); - Py_DECREF(u); - if (w == NULL) - return NULL; - assert(PyBytes_Check(w)); - - /* Append bytes to output buffer. */ - writer->min_size--; /* subtract 1 preallocated byte */ - p = _PyBytesWriter_WriteBytes(writer, p, - PyBytes_AS_STRING(w), - PyBytes_GET_SIZE(w)); - Py_DECREF(w); - if (p == NULL) - return NULL; - - *s = t; - return p; -} - -PyObject *_PyBytes_DecodeEscape(const char *s, - Py_ssize_t len, - const char *errors, - Py_ssize_t unicode, - const char *recode_encoding, - const char **first_invalid_escape) -{ - int c; - char *p; - const char *end; - _PyBytesWriter writer; - - _PyBytesWriter_Init(&writer); - - p = _PyBytesWriter_Alloc(&writer, len); - if (p == NULL) - return NULL; - writer.overallocate = 1; - - *first_invalid_escape = NULL; - - end = s + len; - while (s < end) { - if (*s != '\\') { - non_esc: - if (!(recode_encoding && (*s & 0x80))) { - *p++ = *s++; - } - else { - /* non-ASCII character and need to recode */ - p = _PyBytes_DecodeEscapeRecode(&s, end, - errors, recode_encoding, - &writer, p); - if (p == NULL) - goto failed; - } - continue; - } - - s++; - if (s == end) { - PyErr_SetString(PyExc_ValueError, - "Trailing \\ in string"); - goto failed; - } - - switch (*s++) { - /* XXX This assumes ASCII! */ - case '\n': break; - case '\\': *p++ = '\\'; break; - case '\'': *p++ = '\''; break; - case '\"': *p++ = '\"'; break; - case 'b': *p++ = '\b'; break; - case 'f': *p++ = '\014'; break; /* FF */ - case 't': *p++ = '\t'; break; - case 'n': *p++ = '\n'; break; - case 'r': *p++ = '\r'; break; - case 'v': *p++ = '\013'; break; /* VT */ - case 'a': *p++ = '\007'; break; /* BEL, not classic C */ - case '0': case '1': case '2': case '3': - case '4': case '5': case '6': case '7': - c = s[-1] - '0'; - if (s < end && '0' <= *s && *s <= '7') { - c = (c<<3) + *s++ - '0'; - if (s < end && '0' <= *s && *s <= '7') - c = (c<<3) + *s++ - '0'; - } - *p++ = c; - break; - case 'x': - if (s+1 < end) { - int digit1, digit2; - digit1 = _PyLong_DigitValue[Py_CHARMASK(s[0])]; - digit2 = _PyLong_DigitValue[Py_CHARMASK(s[1])]; - if (digit1 < 16 && digit2 < 16) { - *p++ = (unsigned char)((digit1 << 4) + digit2); - s += 2; - break; - } - } - /* invalid hexadecimal digits */ - - if (!errors || strcmp(errors, "strict") == 0) { - PyErr_Format(PyExc_ValueError, - "invalid \\x escape at position %d", - s - 2 - (end - len)); - goto failed; - } - if (strcmp(errors, "replace") == 0) { - *p++ = '?'; - } else if (strcmp(errors, "ignore") == 0) - /* do nothing */; - else { - PyErr_Format(PyExc_ValueError, - "decoding error; unknown " - "error handling code: %.400s", - errors); - goto failed; - } - /* skip \x */ - if (s < end && Py_ISXDIGIT(s[0])) - s++; /* and a hexdigit */ - break; - - default: - if (*first_invalid_escape == NULL) { - *first_invalid_escape = s-1; /* Back up one char, since we've - already incremented s. */ - } - *p++ = '\\'; - s--; - goto non_esc; /* an arbitrary number of unescaped - UTF-8 bytes may follow. */ - } - } - - return _PyBytesWriter_Finish(&writer, p); - - failed: - _PyBytesWriter_Dealloc(&writer); - return NULL; -} - -PyObject *PyBytes_DecodeEscape(const char *s, - Py_ssize_t len, - const char *errors, - Py_ssize_t unicode, - const char *recode_encoding) -{ - const char* first_invalid_escape; - PyObject *result = _PyBytes_DecodeEscape(s, len, errors, unicode, - recode_encoding, - &first_invalid_escape); - if (result == NULL) - return NULL; - if (first_invalid_escape != NULL) { - if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, - "invalid escape sequence '\\%c'", - (unsigned char)*first_invalid_escape) < 0) { - Py_DECREF(result); - return NULL; - } - } - return result; - -} -/* -------------------------------------------------------------------- */ -/* object api */ - -Py_ssize_t -PyBytes_Size(PyObject *op) -{ - if (!PyBytes_Check(op)) { - PyErr_Format(PyExc_TypeError, - "expected bytes, %.200s found", Py_TYPE(op)->tp_name); - return -1; - } - return Py_SIZE(op); -} - -char * -PyBytes_AsString(PyObject *op) -{ - if (!PyBytes_Check(op)) { - PyErr_Format(PyExc_TypeError, - "expected bytes, %.200s found", Py_TYPE(op)->tp_name); - return NULL; - } - return ((PyBytesObject *)op)->ob_sval; -} - -int -PyBytes_AsStringAndSize(PyObject *obj, - char **s, - Py_ssize_t *len) -{ - if (s == NULL) { - PyErr_BadInternalCall(); - return -1; - } - - if (!PyBytes_Check(obj)) { - PyErr_Format(PyExc_TypeError, - "expected bytes, %.200s found", Py_TYPE(obj)->tp_name); - return -1; - } - - *s = PyBytes_AS_STRING(obj); - if (len != NULL) - *len = PyBytes_GET_SIZE(obj); - else if (strlen(*s) != (size_t)PyBytes_GET_SIZE(obj)) { - PyErr_SetString(PyExc_ValueError, - "embedded null byte"); - return -1; - } - return 0; -} - -/* -------------------------------------------------------------------- */ -/* Methods */ - -#include "stringlib/stringdefs.h" - -#include "stringlib/fastsearch.h" -#include "stringlib/count.h" -#include "stringlib/find.h" -#include "stringlib/join.h" -#include "stringlib/partition.h" -#include "stringlib/split.h" -#include "stringlib/ctype.h" - -#include "stringlib/transmogrify.h" - -PyObject * -PyBytes_Repr(PyObject *obj, int smartquotes) -{ - PyBytesObject* op = (PyBytesObject*) obj; - Py_ssize_t i, length = Py_SIZE(op); - Py_ssize_t newsize, squotes, dquotes; - PyObject *v; - unsigned char quote, *s, *p; - - /* Compute size of output string */ - squotes = dquotes = 0; - newsize = 3; /* b'' */ - s = (unsigned char*)op->ob_sval; - for (i = 0; i < length; i++) { - Py_ssize_t incr = 1; - switch(s[i]) { - case '\'': squotes++; break; - case '"': dquotes++; break; - case '\\': case '\t': case '\n': case '\r': - incr = 2; break; /* \C */ - default: - if (s[i] < ' ' || s[i] >= 0x7f) - incr = 4; /* \xHH */ - } - if (newsize > PY_SSIZE_T_MAX - incr) - goto overflow; - newsize += incr; - } - quote = '\''; - if (smartquotes && squotes && !dquotes) - quote = '"'; - if (squotes && quote == '\'') { - if (newsize > PY_SSIZE_T_MAX - squotes) - goto overflow; - newsize += squotes; - } - - v = PyUnicode_New(newsize, 127); - if (v == NULL) { - return NULL; - } - p = PyUnicode_1BYTE_DATA(v); - - *p++ = 'b', *p++ = quote; - for (i = 0; i < length; i++) { - unsigned char c = op->ob_sval[i]; - if (c == quote || c == '\\') - *p++ = '\\', *p++ = c; - else if (c == '\t') - *p++ = '\\', *p++ = 't'; - else if (c == '\n') - *p++ = '\\', *p++ = 'n'; - else if (c == '\r') - *p++ = '\\', *p++ = 'r'; - else if (c < ' ' || c >= 0x7f) { - *p++ = '\\'; - *p++ = 'x'; - *p++ = Py_hexdigits[(c & 0xf0) >> 4]; - *p++ = Py_hexdigits[c & 0xf]; - } - else - *p++ = c; - } - *p++ = quote; - assert(_PyUnicode_CheckConsistency(v, 1)); - return v; - - overflow: - PyErr_SetString(PyExc_OverflowError, - "bytes object is too large to make repr"); - return NULL; -} - -static PyObject * -bytes_repr(PyObject *op) -{ - return PyBytes_Repr(op, 1); -} - -static PyObject * -bytes_str(PyObject *op) -{ - if (Py_BytesWarningFlag) { - if (PyErr_WarnEx(PyExc_BytesWarning, - "str() on a bytes instance", 1)) - return NULL; - } - return bytes_repr(op); -} - -static Py_ssize_t -bytes_length(PyBytesObject *a) -{ - return Py_SIZE(a); -} - -/* This is also used by PyBytes_Concat() */ -static PyObject * -bytes_concat(PyObject *a, PyObject *b) -{ - Py_buffer va, vb; - PyObject *result = NULL; - - va.len = -1; - vb.len = -1; - if (PyObject_GetBuffer(a, &va, PyBUF_SIMPLE) != 0 || - PyObject_GetBuffer(b, &vb, PyBUF_SIMPLE) != 0) { - PyErr_Format(PyExc_TypeError, "can't concat %.100s to %.100s", - Py_TYPE(b)->tp_name, Py_TYPE(a)->tp_name); - goto done; - } - - /* Optimize end cases */ - if (va.len == 0 && PyBytes_CheckExact(b)) { - result = b; - Py_INCREF(result); - goto done; - } - if (vb.len == 0 && PyBytes_CheckExact(a)) { - result = a; - Py_INCREF(result); - goto done; - } - - if (va.len > PY_SSIZE_T_MAX - vb.len) { - PyErr_NoMemory(); - goto done; - } - - result = PyBytes_FromStringAndSize(NULL, va.len + vb.len); - if (result != NULL) { - memcpy(PyBytes_AS_STRING(result), va.buf, va.len); - memcpy(PyBytes_AS_STRING(result) + va.len, vb.buf, vb.len); - } - - done: - if (va.len != -1) - PyBuffer_Release(&va); - if (vb.len != -1) - PyBuffer_Release(&vb); - return result; -} - -static PyObject * -bytes_repeat(PyBytesObject *a, Py_ssize_t n) -{ - Py_ssize_t i; - Py_ssize_t j; - Py_ssize_t size; - PyBytesObject *op; - size_t nbytes; - if (n < 0) - n = 0; - /* watch out for overflows: the size can overflow int, - * and the # of bytes needed can overflow size_t - */ - if (n > 0 && Py_SIZE(a) > PY_SSIZE_T_MAX / n) { - PyErr_SetString(PyExc_OverflowError, - "repeated bytes are too long"); - return NULL; - } - size = Py_SIZE(a) * n; - if (size == Py_SIZE(a) && PyBytes_CheckExact(a)) { - Py_INCREF(a); - return (PyObject *)a; - } - nbytes = (size_t)size; - if (nbytes + PyBytesObject_SIZE <= nbytes) { - PyErr_SetString(PyExc_OverflowError, - "repeated bytes are too long"); - return NULL; - } - op = (PyBytesObject *)PyObject_MALLOC(PyBytesObject_SIZE + nbytes); - if (op == NULL) - return PyErr_NoMemory(); - (void)PyObject_INIT_VAR(op, &PyBytes_Type, size); - op->ob_shash = -1; - op->ob_sval[size] = '\0'; - if (Py_SIZE(a) == 1 && n > 0) { - memset(op->ob_sval, a->ob_sval[0] , n); - return (PyObject *) op; - } - i = 0; - if (i < size) { - memcpy(op->ob_sval, a->ob_sval, Py_SIZE(a)); - i = Py_SIZE(a); - } - while (i < size) { - j = (i <= size-i) ? i : size-i; - memcpy(op->ob_sval+i, op->ob_sval, j); - i += j; - } - return (PyObject *) op; -} - -static int -bytes_contains(PyObject *self, PyObject *arg) -{ - return _Py_bytes_contains(PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), arg); -} - -static PyObject * -bytes_item(PyBytesObject *a, Py_ssize_t i) -{ - if (i < 0 || i >= Py_SIZE(a)) { - PyErr_SetString(PyExc_IndexError, "index out of range"); - return NULL; - } - return PyLong_FromLong((unsigned char)a->ob_sval[i]); -} - -static int -bytes_compare_eq(PyBytesObject *a, PyBytesObject *b) -{ - int cmp; - Py_ssize_t len; - - len = Py_SIZE(a); - if (Py_SIZE(b) != len) - return 0; - - if (a->ob_sval[0] != b->ob_sval[0]) - return 0; - - cmp = memcmp(a->ob_sval, b->ob_sval, len); - return (cmp == 0); -} - -static PyObject* -bytes_richcompare(PyBytesObject *a, PyBytesObject *b, int op) -{ - int c; - Py_ssize_t len_a, len_b; - Py_ssize_t min_len; - PyObject *result; - int rc; - - /* Make sure both arguments are strings. */ - if (!(PyBytes_Check(a) && PyBytes_Check(b))) { - if (Py_BytesWarningFlag && (op == Py_EQ || op == Py_NE)) { - rc = PyObject_IsInstance((PyObject*)a, - (PyObject*)&PyUnicode_Type); - if (!rc) - rc = PyObject_IsInstance((PyObject*)b, - (PyObject*)&PyUnicode_Type); - if (rc < 0) - return NULL; - if (rc) { - if (PyErr_WarnEx(PyExc_BytesWarning, - "Comparison between bytes and string", 1)) - return NULL; - } - else { - rc = PyObject_IsInstance((PyObject*)a, - (PyObject*)&PyLong_Type); - if (!rc) - rc = PyObject_IsInstance((PyObject*)b, - (PyObject*)&PyLong_Type); - if (rc < 0) - return NULL; - if (rc) { - if (PyErr_WarnEx(PyExc_BytesWarning, - "Comparison between bytes and int", 1)) - return NULL; - } - } - } - result = Py_NotImplemented; - } - else if (a == b) { - switch (op) { - case Py_EQ: - case Py_LE: - case Py_GE: - /* a string is equal to itself */ - result = Py_True; - break; - case Py_NE: - case Py_LT: - case Py_GT: - result = Py_False; - break; - default: - PyErr_BadArgument(); - return NULL; - } - } - else if (op == Py_EQ || op == Py_NE) { - int eq = bytes_compare_eq(a, b); - eq ^= (op == Py_NE); - result = eq ? Py_True : Py_False; - } - else { - len_a = Py_SIZE(a); - len_b = Py_SIZE(b); - min_len = Py_MIN(len_a, len_b); - if (min_len > 0) { - c = Py_CHARMASK(*a->ob_sval) - Py_CHARMASK(*b->ob_sval); - if (c == 0) - c = memcmp(a->ob_sval, b->ob_sval, min_len); - } - else - c = 0; - if (c == 0) - c = (len_a < len_b) ? -1 : (len_a > len_b) ? 1 : 0; - switch (op) { - case Py_LT: c = c < 0; break; - case Py_LE: c = c <= 0; break; - case Py_GT: c = c > 0; break; - case Py_GE: c = c >= 0; break; - default: - PyErr_BadArgument(); - return NULL; - } - result = c ? Py_True : Py_False; - } - - Py_INCREF(result); - return result; -} - -static Py_hash_t -bytes_hash(PyBytesObject *a) -{ - if (a->ob_shash == -1) { - /* Can't fail */ - a->ob_shash = _Py_HashBytes(a->ob_sval, Py_SIZE(a)); - } - return a->ob_shash; -} - -static PyObject* -bytes_subscript(PyBytesObject* self, PyObject* item) -{ - if (PyIndex_Check(item)) { - Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); - if (i == -1 && PyErr_Occurred()) - return NULL; - if (i < 0) - i += PyBytes_GET_SIZE(self); - if (i < 0 || i >= PyBytes_GET_SIZE(self)) { - PyErr_SetString(PyExc_IndexError, - "index out of range"); - return NULL; - } - return PyLong_FromLong((unsigned char)self->ob_sval[i]); - } - else if (PySlice_Check(item)) { - Py_ssize_t start, stop, step, slicelength, cur, i; - char* source_buf; - char* result_buf; - PyObject* result; - - if (PySlice_Unpack(item, &start, &stop, &step) < 0) { - return NULL; - } - slicelength = PySlice_AdjustIndices(PyBytes_GET_SIZE(self), &start, - &stop, step); - - if (slicelength <= 0) { - return PyBytes_FromStringAndSize("", 0); - } - else if (start == 0 && step == 1 && - slicelength == PyBytes_GET_SIZE(self) && - PyBytes_CheckExact(self)) { - Py_INCREF(self); - return (PyObject *)self; - } - else if (step == 1) { - return PyBytes_FromStringAndSize( - PyBytes_AS_STRING(self) + start, - slicelength); - } - else { - source_buf = PyBytes_AS_STRING(self); - result = PyBytes_FromStringAndSize(NULL, slicelength); - if (result == NULL) - return NULL; - - result_buf = PyBytes_AS_STRING(result); - for (cur = start, i = 0; i < slicelength; - cur += step, i++) { - result_buf[i] = source_buf[cur]; - } - - return result; - } - } - else { - PyErr_Format(PyExc_TypeError, - "byte indices must be integers or slices, not %.200s", - Py_TYPE(item)->tp_name); - return NULL; - } -} - -static int -bytes_buffer_getbuffer(PyBytesObject *self, Py_buffer *view, int flags) -{ - return PyBuffer_FillInfo(view, (PyObject*)self, (void *)self->ob_sval, Py_SIZE(self), - 1, flags); -} - -static PySequenceMethods bytes_as_sequence = { - (lenfunc)bytes_length, /*sq_length*/ - (binaryfunc)bytes_concat, /*sq_concat*/ - (ssizeargfunc)bytes_repeat, /*sq_repeat*/ - (ssizeargfunc)bytes_item, /*sq_item*/ - 0, /*sq_slice*/ - 0, /*sq_ass_item*/ - 0, /*sq_ass_slice*/ - (objobjproc)bytes_contains /*sq_contains*/ -}; - -static PyMappingMethods bytes_as_mapping = { - (lenfunc)bytes_length, - (binaryfunc)bytes_subscript, - 0, -}; - -static PyBufferProcs bytes_as_buffer = { - (getbufferproc)bytes_buffer_getbuffer, - NULL, -}; - - -#define LEFTSTRIP 0 -#define RIGHTSTRIP 1 -#define BOTHSTRIP 2 - -/*[clinic input] -bytes.split - - sep: object = None - The delimiter according which to split the bytes. - None (the default value) means split on ASCII whitespace characters - (space, tab, return, newline, formfeed, vertical tab). - maxsplit: Py_ssize_t = -1 - Maximum number of splits to do. - -1 (the default value) means no limit. - -Return a list of the sections in the bytes, using sep as the delimiter. -[clinic start generated code]*/ - -static PyObject * -bytes_split_impl(PyBytesObject *self, PyObject *sep, Py_ssize_t maxsplit) -/*[clinic end generated code: output=52126b5844c1d8ef input=8b809b39074abbfa]*/ -{ - Py_ssize_t len = PyBytes_GET_SIZE(self), n; - const char *s = PyBytes_AS_STRING(self), *sub; - Py_buffer vsub; - PyObject *list; - - if (maxsplit < 0) - maxsplit = PY_SSIZE_T_MAX; - if (sep == Py_None) - return stringlib_split_whitespace((PyObject*) self, s, len, maxsplit); - if (PyObject_GetBuffer(sep, &vsub, PyBUF_SIMPLE) != 0) - return NULL; - sub = vsub.buf; - n = vsub.len; - - list = stringlib_split((PyObject*) self, s, len, sub, n, maxsplit); - PyBuffer_Release(&vsub); - return list; -} - -/*[clinic input] -bytes.partition - - sep: Py_buffer - / - -Partition the bytes into three parts using the given separator. - -This will search for the separator sep in the bytes. If the separator is found, -returns a 3-tuple containing the part before the separator, the separator -itself, and the part after it. - -If the separator is not found, returns a 3-tuple containing the original bytes -object and two empty bytes objects. -[clinic start generated code]*/ - -static PyObject * -bytes_partition_impl(PyBytesObject *self, Py_buffer *sep) -/*[clinic end generated code: output=f532b392a17ff695 input=61cca95519406099]*/ -{ - return stringlib_partition( - (PyObject*) self, - PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), - sep->obj, (const char *)sep->buf, sep->len - ); -} - -/*[clinic input] -bytes.rpartition - - sep: Py_buffer - / - -Partition the bytes into three parts using the given separator. - -This will search for the separator sep in the bytes, starting at the end. If -the separator is found, returns a 3-tuple containing the part before the -separator, the separator itself, and the part after it. - -If the separator is not found, returns a 3-tuple containing two empty bytes -objects and the original bytes object. -[clinic start generated code]*/ - -static PyObject * -bytes_rpartition_impl(PyBytesObject *self, Py_buffer *sep) -/*[clinic end generated code: output=191b114cbb028e50 input=d78db010c8cfdbe1]*/ -{ - return stringlib_rpartition( - (PyObject*) self, - PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), - sep->obj, (const char *)sep->buf, sep->len - ); -} - -/*[clinic input] -bytes.rsplit = bytes.split - -Return a list of the sections in the bytes, using sep as the delimiter. - -Splitting is done starting at the end of the bytes and working to the front. -[clinic start generated code]*/ - -static PyObject * -bytes_rsplit_impl(PyBytesObject *self, PyObject *sep, Py_ssize_t maxsplit) -/*[clinic end generated code: output=ba698d9ea01e1c8f input=0f86c9f28f7d7b7b]*/ -{ - Py_ssize_t len = PyBytes_GET_SIZE(self), n; - const char *s = PyBytes_AS_STRING(self), *sub; - Py_buffer vsub; - PyObject *list; - - if (maxsplit < 0) - maxsplit = PY_SSIZE_T_MAX; - if (sep == Py_None) - return stringlib_rsplit_whitespace((PyObject*) self, s, len, maxsplit); - if (PyObject_GetBuffer(sep, &vsub, PyBUF_SIMPLE) != 0) - return NULL; - sub = vsub.buf; - n = vsub.len; - - list = stringlib_rsplit((PyObject*) self, s, len, sub, n, maxsplit); - PyBuffer_Release(&vsub); - return list; -} - - -/*[clinic input] -bytes.join - - iterable_of_bytes: object - / - -Concatenate any number of bytes objects. - -The bytes whose method is called is inserted in between each pair. - -The result is returned as a new bytes object. - -Example: b'.'.join([b'ab', b'pq', b'rs']) -> b'ab.pq.rs'. -[clinic start generated code]*/ - -static PyObject * -bytes_join(PyBytesObject *self, PyObject *iterable_of_bytes) -/*[clinic end generated code: output=a046f379f626f6f8 input=7fe377b95bd549d2]*/ -{ - return stringlib_bytes_join((PyObject*)self, iterable_of_bytes); -} - -PyObject * -_PyBytes_Join(PyObject *sep, PyObject *x) -{ - assert(sep != NULL && PyBytes_Check(sep)); - assert(x != NULL); - return bytes_join((PyBytesObject*)sep, x); -} - -static PyObject * -bytes_find(PyBytesObject *self, PyObject *args) -{ - return _Py_bytes_find(PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), args); -} - -static PyObject * -bytes_index(PyBytesObject *self, PyObject *args) -{ - return _Py_bytes_index(PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), args); -} - - -static PyObject * -bytes_rfind(PyBytesObject *self, PyObject *args) -{ - return _Py_bytes_rfind(PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), args); -} - - -static PyObject * -bytes_rindex(PyBytesObject *self, PyObject *args) -{ - return _Py_bytes_rindex(PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), args); -} - - -Py_LOCAL_INLINE(PyObject *) -do_xstrip(PyBytesObject *self, int striptype, PyObject *sepobj) -{ - Py_buffer vsep; - char *s = PyBytes_AS_STRING(self); - Py_ssize_t len = PyBytes_GET_SIZE(self); - char *sep; - Py_ssize_t seplen; - Py_ssize_t i, j; - - if (PyObject_GetBuffer(sepobj, &vsep, PyBUF_SIMPLE) != 0) - return NULL; - sep = vsep.buf; - seplen = vsep.len; - - i = 0; - if (striptype != RIGHTSTRIP) { - while (i < len && memchr(sep, Py_CHARMASK(s[i]), seplen)) { - i++; - } - } - - j = len; - if (striptype != LEFTSTRIP) { - do { - j--; - } while (j >= i && memchr(sep, Py_CHARMASK(s[j]), seplen)); - j++; - } - - PyBuffer_Release(&vsep); - - if (i == 0 && j == len && PyBytes_CheckExact(self)) { - Py_INCREF(self); - return (PyObject*)self; - } - else - return PyBytes_FromStringAndSize(s+i, j-i); -} - - -Py_LOCAL_INLINE(PyObject *) -do_strip(PyBytesObject *self, int striptype) -{ - char *s = PyBytes_AS_STRING(self); - Py_ssize_t len = PyBytes_GET_SIZE(self), i, j; - - i = 0; - if (striptype != RIGHTSTRIP) { - while (i < len && Py_ISSPACE(s[i])) { - i++; - } - } - - j = len; - if (striptype != LEFTSTRIP) { - do { - j--; - } while (j >= i && Py_ISSPACE(s[j])); - j++; - } - - if (i == 0 && j == len && PyBytes_CheckExact(self)) { - Py_INCREF(self); - return (PyObject*)self; - } - else - return PyBytes_FromStringAndSize(s+i, j-i); -} - - -Py_LOCAL_INLINE(PyObject *) -do_argstrip(PyBytesObject *self, int striptype, PyObject *bytes) -{ - if (bytes != NULL && bytes != Py_None) { - return do_xstrip(self, striptype, bytes); - } - return do_strip(self, striptype); -} - -/*[clinic input] -bytes.strip - - bytes: object = None - / - -Strip leading and trailing bytes contained in the argument. - -If the argument is omitted or None, strip leading and trailing ASCII whitespace. -[clinic start generated code]*/ - -static PyObject * -bytes_strip_impl(PyBytesObject *self, PyObject *bytes) -/*[clinic end generated code: output=c7c228d3bd104a1b input=8a354640e4e0b3ef]*/ -{ - return do_argstrip(self, BOTHSTRIP, bytes); -} - -/*[clinic input] -bytes.lstrip - - bytes: object = None - / - -Strip leading bytes contained in the argument. - -If the argument is omitted or None, strip leading ASCII whitespace. -[clinic start generated code]*/ - -static PyObject * -bytes_lstrip_impl(PyBytesObject *self, PyObject *bytes) -/*[clinic end generated code: output=28602e586f524e82 input=9baff4398c3f6857]*/ -{ - return do_argstrip(self, LEFTSTRIP, bytes); -} - -/*[clinic input] -bytes.rstrip - - bytes: object = None - / - -Strip trailing bytes contained in the argument. - -If the argument is omitted or None, strip trailing ASCII whitespace. -[clinic start generated code]*/ - -static PyObject * -bytes_rstrip_impl(PyBytesObject *self, PyObject *bytes) -/*[clinic end generated code: output=547e3815c95447da input=b78af445c727e32b]*/ -{ - return do_argstrip(self, RIGHTSTRIP, bytes); -} - - -static PyObject * -bytes_count(PyBytesObject *self, PyObject *args) -{ - return _Py_bytes_count(PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), args); -} - - -/*[clinic input] -bytes.translate - - table: object - Translation table, which must be a bytes object of length 256. - / - delete as deletechars: object(c_default="NULL") = b'' - -Return a copy with each character mapped by the given translation table. - -All characters occurring in the optional argument delete are removed. -The remaining characters are mapped through the given translation table. -[clinic start generated code]*/ - -static PyObject * -bytes_translate_impl(PyBytesObject *self, PyObject *table, - PyObject *deletechars) -/*[clinic end generated code: output=43be3437f1956211 input=0ecdf159f654233c]*/ -{ - char *input, *output; - Py_buffer table_view = {NULL, NULL}; - Py_buffer del_table_view = {NULL, NULL}; - const char *table_chars; - Py_ssize_t i, c, changed = 0; - PyObject *input_obj = (PyObject*)self; - const char *output_start, *del_table_chars=NULL; - Py_ssize_t inlen, tablen, dellen = 0; - PyObject *result; - int trans_table[256]; - - if (PyBytes_Check(table)) { - table_chars = PyBytes_AS_STRING(table); - tablen = PyBytes_GET_SIZE(table); - } - else if (table == Py_None) { - table_chars = NULL; - tablen = 256; - } - else { - if (PyObject_GetBuffer(table, &table_view, PyBUF_SIMPLE) != 0) - return NULL; - table_chars = table_view.buf; - tablen = table_view.len; - } - - if (tablen != 256) { - PyErr_SetString(PyExc_ValueError, - "translation table must be 256 characters long"); - PyBuffer_Release(&table_view); - return NULL; - } - - if (deletechars != NULL) { - if (PyBytes_Check(deletechars)) { - del_table_chars = PyBytes_AS_STRING(deletechars); - dellen = PyBytes_GET_SIZE(deletechars); - } - else { - if (PyObject_GetBuffer(deletechars, &del_table_view, PyBUF_SIMPLE) != 0) { - PyBuffer_Release(&table_view); - return NULL; - } - del_table_chars = del_table_view.buf; - dellen = del_table_view.len; - } - } - else { - del_table_chars = NULL; - dellen = 0; - } - - inlen = PyBytes_GET_SIZE(input_obj); - result = PyBytes_FromStringAndSize((char *)NULL, inlen); - if (result == NULL) { - PyBuffer_Release(&del_table_view); - PyBuffer_Release(&table_view); - return NULL; - } - output_start = output = PyBytes_AS_STRING(result); - input = PyBytes_AS_STRING(input_obj); - - if (dellen == 0 && table_chars != NULL) { - /* If no deletions are required, use faster code */ - for (i = inlen; --i >= 0; ) { - c = Py_CHARMASK(*input++); - if (Py_CHARMASK((*output++ = table_chars[c])) != c) - changed = 1; - } - if (!changed && PyBytes_CheckExact(input_obj)) { - Py_INCREF(input_obj); - Py_DECREF(result); - result = input_obj; - } - PyBuffer_Release(&del_table_view); - PyBuffer_Release(&table_view); - return result; - } - - if (table_chars == NULL) { - for (i = 0; i < 256; i++) - trans_table[i] = Py_CHARMASK(i); - } else { - for (i = 0; i < 256; i++) - trans_table[i] = Py_CHARMASK(table_chars[i]); - } - PyBuffer_Release(&table_view); - - for (i = 0; i < dellen; i++) - trans_table[(int) Py_CHARMASK(del_table_chars[i])] = -1; - PyBuffer_Release(&del_table_view); - - for (i = inlen; --i >= 0; ) { - c = Py_CHARMASK(*input++); - if (trans_table[c] != -1) - if (Py_CHARMASK(*output++ = (char)trans_table[c]) == c) - continue; - changed = 1; - } - if (!changed && PyBytes_CheckExact(input_obj)) { - Py_DECREF(result); - Py_INCREF(input_obj); - return input_obj; - } - /* Fix the size of the resulting string */ - if (inlen > 0) - _PyBytes_Resize(&result, output - output_start); - return result; -} - - -/*[clinic input] - - at staticmethod -bytes.maketrans - - frm: Py_buffer - to: Py_buffer - / - -Return a translation table useable for the bytes or bytearray translate method. - -The returned table will be one where each byte in frm is mapped to the byte at -the same position in to. - -The bytes objects frm and to must be of the same length. -[clinic start generated code]*/ - -static PyObject * -bytes_maketrans_impl(Py_buffer *frm, Py_buffer *to) -/*[clinic end generated code: output=a36f6399d4b77f6f input=de7a8fc5632bb8f1]*/ -{ - return _Py_bytes_maketrans(frm, to); -} - - -/*[clinic input] -bytes.replace - - old: Py_buffer - new: Py_buffer - count: Py_ssize_t = -1 - Maximum number of occurrences to replace. - -1 (the default value) means replace all occurrences. - / - -Return a copy with all occurrences of substring old replaced by new. - -If the optional argument count is given, only the first count occurrences are -replaced. -[clinic start generated code]*/ - -static PyObject * -bytes_replace_impl(PyBytesObject *self, Py_buffer *old, Py_buffer *new, - Py_ssize_t count) -/*[clinic end generated code: output=994fa588b6b9c104 input=b2fbbf0bf04de8e5]*/ -{ - return stringlib_replace((PyObject *)self, - (const char *)old->buf, old->len, - (const char *)new->buf, new->len, count); -} - -/** End DALKE **/ - - -static PyObject * -bytes_startswith(PyBytesObject *self, PyObject *args) -{ - return _Py_bytes_startswith(PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), args); -} - -static PyObject * -bytes_endswith(PyBytesObject *self, PyObject *args) -{ - return _Py_bytes_endswith(PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), args); -} - - -/*[clinic input] -bytes.decode - - encoding: str(c_default="NULL") = 'utf-8' - The encoding with which to decode the bytes. - errors: str(c_default="NULL") = 'strict' - The error handling scheme to use for the handling of decoding errors. - The default is 'strict' meaning that decoding errors raise a - UnicodeDecodeError. Other possible values are 'ignore' and 'replace' - as well as any other name registered with codecs.register_error that - can handle UnicodeDecodeErrors. - -Decode the bytes using the codec registered for encoding. -[clinic start generated code]*/ - -static PyObject * -bytes_decode_impl(PyBytesObject *self, const char *encoding, - const char *errors) -/*[clinic end generated code: output=5649a53dde27b314 input=958174769d2a40ca]*/ -{ - return PyUnicode_FromEncodedObject((PyObject*)self, encoding, errors); -} - - -/*[clinic input] -bytes.splitlines - - keepends: bool(accept={int}) = False - -Return a list of the lines in the bytes, breaking at line boundaries. - -Line breaks are not included in the resulting list unless keepends is given and -true. -[clinic start generated code]*/ - -static PyObject * -bytes_splitlines_impl(PyBytesObject *self, int keepends) -/*[clinic end generated code: output=3484149a5d880ffb input=a8b32eb01ff5a5ed]*/ -{ - return stringlib_splitlines( - (PyObject*) self, PyBytes_AS_STRING(self), - PyBytes_GET_SIZE(self), keepends - ); -} - -/*[clinic input] - at classmethod -bytes.fromhex - - string: unicode - / - -Create a bytes object from a string of hexadecimal numbers. - -Spaces between two numbers are accepted. -Example: bytes.fromhex('B9 01EF') -> b'\\xb9\\x01\\xef'. -[clinic start generated code]*/ - -static PyObject * -bytes_fromhex_impl(PyTypeObject *type, PyObject *string) -/*[clinic end generated code: output=0973acc63661bb2e input=bf4d1c361670acd3]*/ -{ - PyObject *result = _PyBytes_FromHex(string, 0); - if (type != &PyBytes_Type && result != NULL) { - Py_SETREF(result, PyObject_CallFunctionObjArgs((PyObject *)type, - result, NULL)); - } - return result; -} - -PyObject* -_PyBytes_FromHex(PyObject *string, int use_bytearray) -{ - char *buf; - Py_ssize_t hexlen, invalid_char; - unsigned int top, bot; - Py_UCS1 *str, *end; - _PyBytesWriter writer; - - _PyBytesWriter_Init(&writer); - writer.use_bytearray = use_bytearray; - - assert(PyUnicode_Check(string)); - if (PyUnicode_READY(string)) - return NULL; - hexlen = PyUnicode_GET_LENGTH(string); - - if (!PyUnicode_IS_ASCII(string)) { - void *data = PyUnicode_DATA(string); - unsigned int kind = PyUnicode_KIND(string); - Py_ssize_t i; - - /* search for the first non-ASCII character */ - for (i = 0; i < hexlen; i++) { - if (PyUnicode_READ(kind, data, i) >= 128) - break; - } - invalid_char = i; - goto error; - } - - assert(PyUnicode_KIND(string) == PyUnicode_1BYTE_KIND); - str = PyUnicode_1BYTE_DATA(string); - - /* This overestimates if there are spaces */ - buf = _PyBytesWriter_Alloc(&writer, hexlen / 2); - if (buf == NULL) - return NULL; - - end = str + hexlen; - while (str < end) { - /* skip over spaces in the input */ - if (Py_ISSPACE(*str)) { - do { - str++; - } while (Py_ISSPACE(*str)); - if (str >= end) - break; - } - - top = _PyLong_DigitValue[*str]; - if (top >= 16) { - invalid_char = str - PyUnicode_1BYTE_DATA(string); - goto error; - } - str++; - - bot = _PyLong_DigitValue[*str]; - if (bot >= 16) { - invalid_char = str - PyUnicode_1BYTE_DATA(string); - goto error; - } - str++; - - *buf++ = (unsigned char)((top << 4) + bot); - } - - return _PyBytesWriter_Finish(&writer, buf); - - error: - PyErr_Format(PyExc_ValueError, - "non-hexadecimal number found in " - "fromhex() arg at position %zd", invalid_char); - _PyBytesWriter_Dealloc(&writer); - return NULL; -} - -PyDoc_STRVAR(hex__doc__, -"B.hex() -> string\n\ -\n\ -Create a string of hexadecimal numbers from a bytes object.\n\ -Example: b'\\xb9\\x01\\xef'.hex() -> 'b901ef'."); - -static PyObject * -bytes_hex(PyBytesObject *self) -{ - char* argbuf = PyBytes_AS_STRING(self); - Py_ssize_t arglen = PyBytes_GET_SIZE(self); - return _Py_strhex(argbuf, arglen); -} - -static PyObject * -bytes_getnewargs(PyBytesObject *v) -{ - return Py_BuildValue("(y#)", v->ob_sval, Py_SIZE(v)); -} - - -static PyMethodDef -bytes_methods[] = { - {"__getnewargs__", (PyCFunction)bytes_getnewargs, METH_NOARGS}, - {"capitalize", (PyCFunction)stringlib_capitalize, METH_NOARGS, - _Py_capitalize__doc__}, - {"center", (PyCFunction)stringlib_center, METH_VARARGS, - _Py_center__doc__}, - {"count", (PyCFunction)bytes_count, METH_VARARGS, - _Py_count__doc__}, - BYTES_DECODE_METHODDEF - {"endswith", (PyCFunction)bytes_endswith, METH_VARARGS, - _Py_endswith__doc__}, - {"expandtabs", (PyCFunction)stringlib_expandtabs, METH_VARARGS | METH_KEYWORDS, - _Py_expandtabs__doc__}, - {"find", (PyCFunction)bytes_find, METH_VARARGS, - _Py_find__doc__}, - BYTES_FROMHEX_METHODDEF - {"hex", (PyCFunction)bytes_hex, METH_NOARGS, hex__doc__}, - {"index", (PyCFunction)bytes_index, METH_VARARGS, _Py_index__doc__}, - {"isalnum", (PyCFunction)stringlib_isalnum, METH_NOARGS, - _Py_isalnum__doc__}, - {"isalpha", (PyCFunction)stringlib_isalpha, METH_NOARGS, - _Py_isalpha__doc__}, - {"isdigit", (PyCFunction)stringlib_isdigit, METH_NOARGS, - _Py_isdigit__doc__}, - {"islower", (PyCFunction)stringlib_islower, METH_NOARGS, - _Py_islower__doc__}, - {"isspace", (PyCFunction)stringlib_isspace, METH_NOARGS, - _Py_isspace__doc__}, - {"istitle", (PyCFunction)stringlib_istitle, METH_NOARGS, - _Py_istitle__doc__}, - {"isupper", (PyCFunction)stringlib_isupper, METH_NOARGS, - _Py_isupper__doc__}, - BYTES_JOIN_METHODDEF - {"ljust", (PyCFunction)stringlib_ljust, METH_VARARGS, _Py_ljust__doc__}, - {"lower", (PyCFunction)stringlib_lower, METH_NOARGS, _Py_lower__doc__}, - BYTES_LSTRIP_METHODDEF - BYTES_MAKETRANS_METHODDEF - BYTES_PARTITION_METHODDEF - BYTES_REPLACE_METHODDEF - {"rfind", (PyCFunction)bytes_rfind, METH_VARARGS, _Py_rfind__doc__}, - {"rindex", (PyCFunction)bytes_rindex, METH_VARARGS, _Py_rindex__doc__}, - {"rjust", (PyCFunction)stringlib_rjust, METH_VARARGS, _Py_rjust__doc__}, - BYTES_RPARTITION_METHODDEF - BYTES_RSPLIT_METHODDEF - BYTES_RSTRIP_METHODDEF - BYTES_SPLIT_METHODDEF - BYTES_SPLITLINES_METHODDEF - {"startswith", (PyCFunction)bytes_startswith, METH_VARARGS, - _Py_startswith__doc__}, - BYTES_STRIP_METHODDEF - {"swapcase", (PyCFunction)stringlib_swapcase, METH_NOARGS, - _Py_swapcase__doc__}, - {"title", (PyCFunction)stringlib_title, METH_NOARGS, _Py_title__doc__}, - BYTES_TRANSLATE_METHODDEF - {"upper", (PyCFunction)stringlib_upper, METH_NOARGS, _Py_upper__doc__}, - {"zfill", (PyCFunction)stringlib_zfill, METH_VARARGS, _Py_zfill__doc__}, - {NULL, NULL} /* sentinel */ -}; - -static PyObject * -bytes_mod(PyObject *self, PyObject *arg) -{ - if (!PyBytes_Check(self)) { - Py_RETURN_NOTIMPLEMENTED; - } - return _PyBytes_FormatEx(PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self), - arg, 0); -} - -static PyNumberMethods bytes_as_number = { - 0, /*nb_add*/ - 0, /*nb_subtract*/ - 0, /*nb_multiply*/ - bytes_mod, /*nb_remainder*/ -}; - -static PyObject * -bytes_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds); - -static PyObject * -bytes_new(PyTypeObject *type, PyObject *args, PyObject *kwds) -{ - PyObject *x = NULL; - const char *encoding = NULL; - const char *errors = NULL; - PyObject *new = NULL; - PyObject *func; - Py_ssize_t size; - static char *kwlist[] = {"source", "encoding", "errors", 0}; - _Py_IDENTIFIER(__bytes__); - - if (type != &PyBytes_Type) - return bytes_subtype_new(type, args, kwds); - if (!PyArg_ParseTupleAndKeywords(args, kwds, "|Oss:bytes", kwlist, &x, - &encoding, &errors)) - return NULL; - if (x == NULL) { - if (encoding != NULL || errors != NULL) { - PyErr_SetString(PyExc_TypeError, - "encoding or errors without sequence " - "argument"); - return NULL; - } - return PyBytes_FromStringAndSize(NULL, 0); - } - - if (encoding != NULL) { - /* Encode via the codec registry */ - if (!PyUnicode_Check(x)) { - PyErr_SetString(PyExc_TypeError, - "encoding without a string argument"); - return NULL; - } - new = PyUnicode_AsEncodedString(x, encoding, errors); - if (new == NULL) - return NULL; - assert(PyBytes_Check(new)); - return new; - } - - if (errors != NULL) { - PyErr_SetString(PyExc_TypeError, - PyUnicode_Check(x) ? - "string argument without an encoding" : - "errors without a string argument"); - return NULL; - } - - /* We'd like to call PyObject_Bytes here, but we need to check for an - integer argument before deferring to PyBytes_FromObject, something - PyObject_Bytes doesn't do. */ - func = _PyObject_LookupSpecial(x, &PyId___bytes__); - if (func != NULL) { - new = _PyObject_CallNoArg(func); - Py_DECREF(func); - if (new == NULL) - return NULL; - if (!PyBytes_Check(new)) { - PyErr_Format(PyExc_TypeError, - "__bytes__ returned non-bytes (type %.200s)", - Py_TYPE(new)->tp_name); - Py_DECREF(new); - return NULL; - } - return new; - } - else if (PyErr_Occurred()) - return NULL; - - if (PyUnicode_Check(x)) { - PyErr_SetString(PyExc_TypeError, - "string argument without an encoding"); - return NULL; - } - /* Is it an integer? */ - if (PyIndex_Check(x)) { - size = PyNumber_AsSsize_t(x, PyExc_OverflowError); - if (size == -1 && PyErr_Occurred()) { - if (PyErr_ExceptionMatches(PyExc_OverflowError)) - return NULL; - PyErr_Clear(); /* fall through */ - } - else { - if (size < 0) { - PyErr_SetString(PyExc_ValueError, "negative count"); - return NULL; - } - new = _PyBytes_FromSize(size, 1); - if (new == NULL) - return NULL; - return new; - } - } - - return PyBytes_FromObject(x); -} - -static PyObject* -_PyBytes_FromBuffer(PyObject *x) -{ - PyObject *new; - Py_buffer view; - - if (PyObject_GetBuffer(x, &view, PyBUF_FULL_RO) < 0) - return NULL; - - new = PyBytes_FromStringAndSize(NULL, view.len); - if (!new) - goto fail; - if (PyBuffer_ToContiguous(((PyBytesObject *)new)->ob_sval, - &view, view.len, 'C') < 0) - goto fail; - PyBuffer_Release(&view); - return new; - -fail: - Py_XDECREF(new); - PyBuffer_Release(&view); - return NULL; -} - -#define _PyBytes_FROM_LIST_BODY(x, GET_ITEM) \ - do { \ - PyObject *bytes; \ - Py_ssize_t i; \ - Py_ssize_t value; \ - char *str; \ - PyObject *item; \ - \ - bytes = PyBytes_FromStringAndSize(NULL, Py_SIZE(x)); \ - if (bytes == NULL) \ - return NULL; \ - str = ((PyBytesObject *)bytes)->ob_sval; \ - \ - for (i = 0; i < Py_SIZE(x); i++) { \ - item = GET_ITEM((x), i); \ - value = PyNumber_AsSsize_t(item, NULL); \ - if (value == -1 && PyErr_Occurred()) \ - goto error; \ - \ - if (value < 0 || value >= 256) { \ - PyErr_SetString(PyExc_ValueError, \ - "bytes must be in range(0, 256)"); \ - goto error; \ - } \ - *str++ = (char) value; \ - } \ - return bytes; \ - \ - error: \ - Py_DECREF(bytes); \ - return NULL; \ - } while (0) - -static PyObject* -_PyBytes_FromList(PyObject *x) -{ - _PyBytes_FROM_LIST_BODY(x, PyList_GET_ITEM); -} - -static PyObject* -_PyBytes_FromTuple(PyObject *x) -{ - _PyBytes_FROM_LIST_BODY(x, PyTuple_GET_ITEM); -} - -static PyObject * -_PyBytes_FromIterator(PyObject *it, PyObject *x) -{ - char *str; - Py_ssize_t i, size; - _PyBytesWriter writer; - - /* For iterator version, create a string object and resize as needed */ - size = PyObject_LengthHint(x, 64); - if (size == -1 && PyErr_Occurred()) - return NULL; - - _PyBytesWriter_Init(&writer); - str = _PyBytesWriter_Alloc(&writer, size); - if (str == NULL) - return NULL; - writer.overallocate = 1; - size = writer.allocated; - - /* Run the iterator to exhaustion */ - for (i = 0; ; i++) { - PyObject *item; - Py_ssize_t value; - - /* Get the next item */ - item = PyIter_Next(it); - if (item == NULL) { - if (PyErr_Occurred()) - goto error; - break; - } - - /* Interpret it as an int (__index__) */ - value = PyNumber_AsSsize_t(item, NULL); - Py_DECREF(item); - if (value == -1 && PyErr_Occurred()) - goto error; - - /* Range check */ - if (value < 0 || value >= 256) { - PyErr_SetString(PyExc_ValueError, - "bytes must be in range(0, 256)"); - goto error; - } - - /* Append the byte */ - if (i >= size) { - str = _PyBytesWriter_Resize(&writer, str, size+1); - if (str == NULL) - return NULL; - size = writer.allocated; - } - *str++ = (char) value; - } - - return _PyBytesWriter_Finish(&writer, str); - - error: - _PyBytesWriter_Dealloc(&writer); - return NULL; -} - -PyObject * -PyBytes_FromObject(PyObject *x) -{ - PyObject *it, *result; - - if (x == NULL) { - PyErr_BadInternalCall(); - return NULL; - } - - if (PyBytes_CheckExact(x)) { - Py_INCREF(x); - return x; - } - - /* Use the modern buffer interface */ - if (PyObject_CheckBuffer(x)) - return _PyBytes_FromBuffer(x); - - if (PyList_CheckExact(x)) - return _PyBytes_FromList(x); - - if (PyTuple_CheckExact(x)) - return _PyBytes_FromTuple(x); - - if (!PyUnicode_Check(x)) { - it = PyObject_GetIter(x); - if (it != NULL) { - result = _PyBytes_FromIterator(it, x); - Py_DECREF(it); - return result; - } - } - - PyErr_Format(PyExc_TypeError, - "cannot convert '%.200s' object to bytes", - x->ob_type->tp_name); - return NULL; -} - -static PyObject * -bytes_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds) -{ - PyObject *tmp, *pnew; - Py_ssize_t n; - - assert(PyType_IsSubtype(type, &PyBytes_Type)); - tmp = bytes_new(&PyBytes_Type, args, kwds); - if (tmp == NULL) - return NULL; - assert(PyBytes_Check(tmp)); - n = PyBytes_GET_SIZE(tmp); - pnew = type->tp_alloc(type, n); - if (pnew != NULL) { - memcpy(PyBytes_AS_STRING(pnew), - PyBytes_AS_STRING(tmp), n+1); - ((PyBytesObject *)pnew)->ob_shash = - ((PyBytesObject *)tmp)->ob_shash; - } - Py_DECREF(tmp); - return pnew; -} - -PyDoc_STRVAR(bytes_doc, -"bytes(iterable_of_ints) -> bytes\n\ -bytes(string, encoding[, errors]) -> bytes\n\ -bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer\n\ -bytes(int) -> bytes object of size given by the parameter initialized with null bytes\n\ -bytes() -> empty bytes object\n\ -\n\ -Construct an immutable array of bytes from:\n\ - - an iterable yielding integers in range(256)\n\ - - a text string encoded using the specified encoding\n\ - - any object implementing the buffer API.\n\ - - an integer"); - -static PyObject *bytes_iter(PyObject *seq); - -PyTypeObject PyBytes_Type = { - PyVarObject_HEAD_INIT(&PyType_Type, 0) - "bytes", - PyBytesObject_SIZE, - sizeof(char), - bytes_dealloc, /* tp_dealloc */ - 0, /* tp_print */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_reserved */ - (reprfunc)bytes_repr, /* tp_repr */ - &bytes_as_number, /* tp_as_number */ - &bytes_as_sequence, /* tp_as_sequence */ - &bytes_as_mapping, /* tp_as_mapping */ - (hashfunc)bytes_hash, /* tp_hash */ - 0, /* tp_call */ - bytes_str, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - 0, /* tp_setattro */ - &bytes_as_buffer, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | - Py_TPFLAGS_BYTES_SUBCLASS, /* tp_flags */ - bytes_doc, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - (richcmpfunc)bytes_richcompare, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - bytes_iter, /* tp_iter */ - 0, /* tp_iternext */ - bytes_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - &PyBaseObject_Type, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - bytes_new, /* tp_new */ - PyObject_Del, /* tp_free */ -}; - -void -PyBytes_Concat(PyObject **pv, PyObject *w) -{ - assert(pv != NULL); - if (*pv == NULL) - return; - if (w == NULL) { - Py_CLEAR(*pv); - return; - } - - if (Py_REFCNT(*pv) == 1 && PyBytes_CheckExact(*pv)) { - /* Only one reference, so we can resize in place */ - Py_ssize_t oldsize; - Py_buffer wb; - - wb.len = -1; - if (PyObject_GetBuffer(w, &wb, PyBUF_SIMPLE) != 0) { - PyErr_Format(PyExc_TypeError, "can't concat %.100s to %.100s", - Py_TYPE(w)->tp_name, Py_TYPE(*pv)->tp_name); - Py_CLEAR(*pv); - return; - } - - oldsize = PyBytes_GET_SIZE(*pv); - if (oldsize > PY_SSIZE_T_MAX - wb.len) { - PyErr_NoMemory(); - goto error; - } - if (_PyBytes_Resize(pv, oldsize + wb.len) < 0) - goto error; - - memcpy(PyBytes_AS_STRING(*pv) + oldsize, wb.buf, wb.len); - PyBuffer_Release(&wb); - return; - - error: - PyBuffer_Release(&wb); - Py_CLEAR(*pv); - return; - } - - else { - /* Multiple references, need to create new object */ - PyObject *v; - v = bytes_concat(*pv, w); - Py_SETREF(*pv, v); - } -} - -void -PyBytes_ConcatAndDel(PyObject **pv, PyObject *w) -{ - PyBytes_Concat(pv, w); - Py_XDECREF(w); -} - - -/* The following function breaks the notion that bytes are immutable: - it changes the size of a bytes object. We get away with this only if there - is only one module referencing the object. You can also think of it - as creating a new bytes object and destroying the old one, only - more efficiently. In any case, don't use this if the bytes object may - already be known to some other part of the code... - Note that if there's not enough memory to resize the bytes object, the - original bytes object at *pv is deallocated, *pv is set to NULL, an "out of - memory" exception is set, and -1 is returned. Else (on success) 0 is - returned, and the value in *pv may or may not be the same as on input. - As always, an extra byte is allocated for a trailing \0 byte (newsize - does *not* include that), and a trailing \0 byte is stored. -*/ - -int -_PyBytes_Resize(PyObject **pv, Py_ssize_t newsize) -{ - PyObject *v; - PyBytesObject *sv; - v = *pv; - if (!PyBytes_Check(v) || newsize < 0) { - goto error; - } - if (Py_SIZE(v) == newsize) { - /* return early if newsize equals to v->ob_size */ - return 0; - } - if (Py_REFCNT(v) != 1) { - goto error; - } - /* XXX UNREF/NEWREF interface should be more symmetrical */ - _Py_DEC_REFTOTAL; - _Py_ForgetReference(v); - *pv = (PyObject *) - PyObject_REALLOC(v, PyBytesObject_SIZE + newsize); - if (*pv == NULL) { - PyObject_Del(v); - PyErr_NoMemory(); - return -1; - } - _Py_NewReference(*pv); - sv = (PyBytesObject *) *pv; - Py_SIZE(sv) = newsize; - sv->ob_sval[newsize] = '\0'; - sv->ob_shash = -1; /* invalidate cached hash value */ - return 0; -error: - *pv = 0; - Py_DECREF(v); - PyErr_BadInternalCall(); - return -1; -} - -void -PyBytes_Fini(void) -{ - int i; - for (i = 0; i < UCHAR_MAX + 1; i++) - Py_CLEAR(characters[i]); - Py_CLEAR(nullstring); -} - -/*********************** Bytes Iterator ****************************/ - -typedef struct { - PyObject_HEAD - Py_ssize_t it_index; - PyBytesObject *it_seq; /* Set to NULL when iterator is exhausted */ -} striterobject; - -static void -striter_dealloc(striterobject *it) -{ - _PyObject_GC_UNTRACK(it); - Py_XDECREF(it->it_seq); - PyObject_GC_Del(it); -} - -static int -striter_traverse(striterobject *it, visitproc visit, void *arg) -{ - Py_VISIT(it->it_seq); - return 0; -} - -static PyObject * -striter_next(striterobject *it) -{ - PyBytesObject *seq; - PyObject *item; - - assert(it != NULL); - seq = it->it_seq; - if (seq == NULL) - return NULL; - assert(PyBytes_Check(seq)); - - if (it->it_index < PyBytes_GET_SIZE(seq)) { - item = PyLong_FromLong( - (unsigned char)seq->ob_sval[it->it_index]); - if (item != NULL) - ++it->it_index; - return item; - } - - it->it_seq = NULL; - Py_DECREF(seq); - return NULL; -} - -static PyObject * -striter_len(striterobject *it) -{ - Py_ssize_t len = 0; - if (it->it_seq) - len = PyBytes_GET_SIZE(it->it_seq) - it->it_index; - return PyLong_FromSsize_t(len); -} - -PyDoc_STRVAR(length_hint_doc, - "Private method returning an estimate of len(list(it))."); - -static PyObject * -striter_reduce(striterobject *it) -{ - if (it->it_seq != NULL) { - return Py_BuildValue("N(O)n", _PyObject_GetBuiltin("iter"), - it->it_seq, it->it_index); - } else { - return Py_BuildValue("N(())", _PyObject_GetBuiltin("iter")); - } -} - -PyDoc_STRVAR(reduce_doc, "Return state information for pickling."); - -static PyObject * -striter_setstate(striterobject *it, PyObject *state) -{ - Py_ssize_t index = PyLong_AsSsize_t(state); - if (index == -1 && PyErr_Occurred()) - return NULL; - if (it->it_seq != NULL) { - if (index < 0) - index = 0; - else if (index > PyBytes_GET_SIZE(it->it_seq)) - index = PyBytes_GET_SIZE(it->it_seq); /* iterator exhausted */ - it->it_index = index; - } - Py_RETURN_NONE; -} - -PyDoc_STRVAR(setstate_doc, "Set state information for unpickling."); - -static PyMethodDef striter_methods[] = { - {"__length_hint__", (PyCFunction)striter_len, METH_NOARGS, - length_hint_doc}, - {"__reduce__", (PyCFunction)striter_reduce, METH_NOARGS, - reduce_doc}, - {"__setstate__", (PyCFunction)striter_setstate, METH_O, - setstate_doc}, - {NULL, NULL} /* sentinel */ -}; - -PyTypeObject PyBytesIter_Type = { - PyVarObject_HEAD_INIT(&PyType_Type, 0) - "bytes_iterator", /* tp_name */ - sizeof(striterobject), /* tp_basicsize */ - 0, /* tp_itemsize */ - /* methods */ - (destructor)striter_dealloc, /* tp_dealloc */ - 0, /* tp_print */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_reserved */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */ - 0, /* tp_doc */ - (traverseproc)striter_traverse, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - PyObject_SelfIter, /* tp_iter */ - (iternextfunc)striter_next, /* tp_iternext */ - striter_methods, /* tp_methods */ - 0, -}; - -static PyObject * -bytes_iter(PyObject *seq) -{ - striterobject *it; - - if (!PyBytes_Check(seq)) { - PyErr_BadInternalCall(); - return NULL; - } - it = PyObject_GC_New(striterobject, &PyBytesIter_Type); - if (it == NULL) - return NULL; - it->it_index = 0; - Py_INCREF(seq); - it->it_seq = (PyBytesObject *)seq; - _PyObject_GC_TRACK(it); - return (PyObject *)it; -} - - -/* _PyBytesWriter API */ - -#ifdef MS_WINDOWS - /* On Windows, overallocate by 50% is the best factor */ -# define OVERALLOCATE_FACTOR 2 -#else - /* On Linux, overallocate by 25% is the best factor */ -# define OVERALLOCATE_FACTOR 4 -#endif - -void -_PyBytesWriter_Init(_PyBytesWriter *writer) -{ - /* Set all attributes before small_buffer to 0 */ - memset(writer, 0, offsetof(_PyBytesWriter, small_buffer)); -#ifdef Py_DEBUG - memset(writer->small_buffer, 0xCB, sizeof(writer->small_buffer)); -#endif -} - -void -_PyBytesWriter_Dealloc(_PyBytesWriter *writer) -{ - Py_CLEAR(writer->buffer); -} - -Py_LOCAL_INLINE(char*) -_PyBytesWriter_AsString(_PyBytesWriter *writer) -{ - if (writer->use_small_buffer) { - assert(writer->buffer == NULL); - return writer->small_buffer; - } - else if (writer->use_bytearray) { - assert(writer->buffer != NULL); - return PyByteArray_AS_STRING(writer->buffer); - } - else { - assert(writer->buffer != NULL); - return PyBytes_AS_STRING(writer->buffer); - } -} - -Py_LOCAL_INLINE(Py_ssize_t) -_PyBytesWriter_GetSize(_PyBytesWriter *writer, char *str) -{ - char *start = _PyBytesWriter_AsString(writer); - assert(str != NULL); - assert(str >= start); - assert(str - start <= writer->allocated); - return str - start; -} - -Py_LOCAL_INLINE(void) -_PyBytesWriter_CheckConsistency(_PyBytesWriter *writer, char *str) -{ -#ifdef Py_DEBUG - char *start, *end; - - if (writer->use_small_buffer) { - assert(writer->buffer == NULL); - } - else { - assert(writer->buffer != NULL); - if (writer->use_bytearray) - assert(PyByteArray_CheckExact(writer->buffer)); - else - assert(PyBytes_CheckExact(writer->buffer)); - assert(Py_REFCNT(writer->buffer) == 1); - } - - if (writer->use_bytearray) { - /* bytearray has its own overallocation algorithm, - writer overallocation must be disabled */ - assert(!writer->overallocate); - } - - assert(0 <= writer->allocated); - assert(0 <= writer->min_size && writer->min_size <= writer->allocated); - /* the last byte must always be null */ - start = _PyBytesWriter_AsString(writer); - assert(start[writer->allocated] == 0); - - end = start + writer->allocated; - assert(str != NULL); - assert(start <= str && str <= end); -#endif -} - -void* -_PyBytesWriter_Resize(_PyBytesWriter *writer, void *str, Py_ssize_t size) -{ - Py_ssize_t allocated, pos; - - _PyBytesWriter_CheckConsistency(writer, str); - assert(writer->allocated < size); - - allocated = size; - if (writer->overallocate - && allocated <= (PY_SSIZE_T_MAX - allocated / OVERALLOCATE_FACTOR)) { - /* overallocate to limit the number of realloc() */ - allocated += allocated / OVERALLOCATE_FACTOR; - } - - pos = _PyBytesWriter_GetSize(writer, str); - if (!writer->use_small_buffer) { - if (writer->use_bytearray) { - if (PyByteArray_Resize(writer->buffer, allocated)) - goto error; - /* writer->allocated can be smaller than writer->buffer->ob_alloc, - but we cannot use ob_alloc because bytes may need to be moved - to use the whole buffer. bytearray uses an internal optimization - to avoid moving or copying bytes when bytes are removed at the - beginning (ex: del bytearray[:1]). */ - } - else { - if (_PyBytes_Resize(&writer->buffer, allocated)) - goto error; - } - } - else { - /* convert from stack buffer to bytes object buffer */ - assert(writer->buffer == NULL); - - if (writer->use_bytearray) - writer->buffer = PyByteArray_FromStringAndSize(NULL, allocated); - else - writer->buffer = PyBytes_FromStringAndSize(NULL, allocated); - if (writer->buffer == NULL) - goto error; - - if (pos != 0) { - char *dest; - if (writer->use_bytearray) - dest = PyByteArray_AS_STRING(writer->buffer); - else - dest = PyBytes_AS_STRING(writer->buffer); - memcpy(dest, - writer->small_buffer, - pos); - } - - writer->use_small_buffer = 0; -#ifdef Py_DEBUG - memset(writer->small_buffer, 0xDB, sizeof(writer->small_buffer)); -#endif - } - writer->allocated = allocated; - - str = _PyBytesWriter_AsString(writer) + pos; - _PyBytesWriter_CheckConsistency(writer, str); - return str; - -error: - _PyBytesWriter_Dealloc(writer); - return NULL; -} - -void* -_PyBytesWriter_Prepare(_PyBytesWriter *writer, void *str, Py_ssize_t size) -{ - Py_ssize_t new_min_size; - - _PyBytesWriter_CheckConsistency(writer, str); - assert(size >= 0); - - if (size == 0) { - /* nothing to do */ - return str; - } - - if (writer->min_size > PY_SSIZE_T_MAX - size) { - PyErr_NoMemory(); - _PyBytesWriter_Dealloc(writer); - return NULL; - } - new_min_size = writer->min_size + size; - - if (new_min_size > writer->allocated) - str = _PyBytesWriter_Resize(writer, str, new_min_size); - - writer->min_size = new_min_size; - return str; -} - -/* Allocate the buffer to write size bytes. - Return the pointer to the beginning of buffer data. - Raise an exception and return NULL on error. */ -void* -_PyBytesWriter_Alloc(_PyBytesWriter *writer, Py_ssize_t size) -{ - /* ensure that _PyBytesWriter_Alloc() is only called once */ - assert(writer->min_size == 0 && writer->buffer == NULL); - assert(size >= 0); - - writer->use_small_buffer = 1; -#ifdef Py_DEBUG - writer->allocated = sizeof(writer->small_buffer) - 1; - /* In debug mode, don't use the full small buffer because it is less - efficient than bytes and bytearray objects to detect buffer underflow - and buffer overflow. Use 10 bytes of the small buffer to test also - code using the smaller buffer in debug mode. - - Don't modify the _PyBytesWriter structure (use a shorter small buffer) - in debug mode to also be able to detect stack overflow when running - tests in debug mode. The _PyBytesWriter is large (more than 512 bytes), - if Py_EnterRecursiveCall() is not used in deep C callback, we may hit a - stack overflow. */ - writer->allocated = Py_MIN(writer->allocated, 10); - /* _PyBytesWriter_CheckConsistency() requires the last byte to be 0, - to detect buffer overflow */ - writer->small_buffer[writer->allocated] = 0; -#else - writer->allocated = sizeof(writer->small_buffer); -#endif - return _PyBytesWriter_Prepare(writer, writer->small_buffer, size); -} - -PyObject * -_PyBytesWriter_Finish(_PyBytesWriter *writer, void *str) -{ - Py_ssize_t size; - PyObject *result; - - _PyBytesWriter_CheckConsistency(writer, str); - - size = _PyBytesWriter_GetSize(writer, str); - if (size == 0 && !writer->use_bytearray) { - Py_CLEAR(writer->buffer); - /* Get the empty byte string singleton */ - result = PyBytes_FromStringAndSize(NULL, 0); - } - else if (writer->use_small_buffer) { - if (writer->use_bytearray) { - result = PyByteArray_FromStringAndSize(writer->small_buffer, size); - } - else { - result = PyBytes_FromStringAndSize(writer->small_buffer, size); - } - } - else { - result = writer->buffer; - writer->buffer = NULL; - - if (size != writer->allocated) { - if (writer->use_bytearray) { - if (PyByteArray_Resize(result, size)) { - Py_DECREF(result); - return NULL; - } - } - else { - if (_PyBytes_Resize(&result, size)) { - assert(result == NULL); - return NULL; - } - } - } - } - return result; -} - -void* -_PyBytesWriter_WriteBytes(_PyBytesWriter *writer, void *ptr, - const void *bytes, Py_ssize_t size) -{ - char *str = (char *)ptr; - - str = _PyBytesWriter_Prepare(writer, str, size); - if (str == NULL) - return NULL; - - memcpy(str, bytes, size); - str += size; - - return str; -} From webhook-mailer at python.org Thu Sep 13 19:35:59 2018 From: webhook-mailer at python.org (Andrew Svetlov) Date: Thu, 13 Sep 2018 23:35:59 -0000 Subject: [Python-checkins] Fix test_asyncio for AIX - do not call transport.get_extra_info('sockname') (#8907) Message-ID: https://github.com/python/cpython/commit/413118ebf3162418639a5c4af14b02d26571a02c commit: 413118ebf3162418639a5c4af14b02d26571a02c branch: master author: Michael Felt committer: Andrew Svetlov date: 2018-09-13T16:35:56-07:00 summary: Fix test_asyncio for AIX - do not call transport.get_extra_info('sockname') (#8907) files: A Misc/NEWS.d/next/Tests/2018-08-24-20-23-15.bpo-34490.vb2cx4.rst M Lib/test/test_asyncio/test_events.py diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py index 01f616ab5b19..708fb3273ddf 100644 --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -41,9 +41,11 @@ def tearDownModule(): asyncio.set_event_loop_policy(None) -def osx_tiger(): +def broken_unix_getsockname(): """Return True if the platform is Mac OS 10.4 or older.""" - if sys.platform != 'darwin': + if sys.platform.startswith("aix"): + return True + elif sys.platform != 'darwin': return False version = platform.mac_ver()[0] version = tuple(map(int, version.split('.'))) @@ -617,7 +619,7 @@ def test_create_connection(self): def test_create_unix_connection(self): # Issue #20682: On Mac OS X Tiger, getsockname() returns a # zero-length address for UNIX socket. - check_sockname = not osx_tiger() + check_sockname = not broken_unix_getsockname() with test_utils.run_test_unix_server() as httpd: conn_fut = self.loop.create_unix_connection( @@ -748,7 +750,7 @@ def test_create_ssl_connection(self): def test_create_ssl_unix_connection(self): # Issue #20682: On Mac OS X Tiger, getsockname() returns a # zero-length address for UNIX socket. - check_sockname = not osx_tiger() + check_sockname = not broken_unix_getsockname() with test_utils.run_test_unix_server(use_ssl=True) as httpd: create_connection = functools.partial( diff --git a/Misc/NEWS.d/next/Tests/2018-08-24-20-23-15.bpo-34490.vb2cx4.rst b/Misc/NEWS.d/next/Tests/2018-08-24-20-23-15.bpo-34490.vb2cx4.rst new file mode 100644 index 000000000000..c778f94b5e8d --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-08-24-20-23-15.bpo-34490.vb2cx4.rst @@ -0,0 +1,2 @@ +On AIX with AF_UNIX family sockets getsockname() does not provide 'sockname', +so skip calls to transport.get_extra_info('sockname') From webhook-mailer at python.org Thu Sep 13 19:53:54 2018 From: webhook-mailer at python.org (Andrew Svetlov) Date: Thu, 13 Sep 2018 23:53:54 -0000 Subject: [Python-checkins] bpo-34666: Implement stream.awrite() and stream.aclose() (GH-9274) Message-ID: https://github.com/python/cpython/commit/11194c877c902a6c3b769d85be887c2272e0a541 commit: 11194c877c902a6c3b769d85be887c2272e0a541 branch: master author: Andrew Svetlov committer: GitHub date: 2018-09-13T16:53:49-07:00 summary: bpo-34666: Implement stream.awrite() and stream.aclose() (GH-9274) files: A Misc/NEWS.d/next/Library/2018-09-13-11-49-52.bpo-34666.3uLtWv.rst M Doc/library/asyncio-stream.rst M Lib/asyncio/streams.py M Lib/test/test_asyncio/test_streams.py diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 0cfecda91e99..80b76253d065 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -20,13 +20,13 @@ streams:: '127.0.0.1', 8888) print(f'Send: {message!r}') - writer.write(message.encode()) + await writer.awrite(message.encode()) data = await reader.read(100) print(f'Received: {data.decode()!r}') print('Close the connection') - writer.close() + await writer.aclose() asyncio.run(tcp_echo_client('Hello World!')) @@ -229,14 +229,57 @@ StreamWriter directly; use :func:`open_connection` and :func:`start_server` instead. + .. coroutinemethod:: awrite(data) + + Write *data* to the stream. + + The method respects control-flow, execution is paused if write + buffer reaches high-water limit. + + .. versionadded:: 3.8 + + .. coroutinemethod:: aclose() + + Close the stream. + + Wait for finishing all closing actions, e.g. SSL shutdown for + secure sockets. + + .. versionadded:: 3.8 + + .. method:: can_write_eof() + + Return *True* if the underlying transport supports + the :meth:`write_eof` method, *False* otherwise. + + .. method:: write_eof() + + Close the write end of the stream after the buffered write + data is flushed. + + .. attribute:: transport + + Return the underlying asyncio transport. + + .. method:: get_extra_info(name, default=None) + + Access optional transport information; see + :meth:`BaseTransport.get_extra_info` for details. + .. method:: write(data) Write *data* to the stream. + This method doesn't apply control-flow. The call should be + followed by :meth:`drain`. + .. method:: writelines(data) Write a list (or any iterable) of bytes to the stream. + This method doesn't apply control-flow. The call should be + followed by :meth:`drain`. + .. coroutinemethod:: drain() Wait until it is appropriate to resume writing to the stream. @@ -272,25 +315,6 @@ StreamWriter .. versionadded:: 3.7 - .. method:: can_write_eof() - - Return *True* if the underlying transport supports - the :meth:`write_eof` method, *False* otherwise. - - .. method:: write_eof() - - Close the write end of the stream after the buffered write - data is flushed. - - .. attribute:: transport - - Return the underlying asyncio transport. - - .. method:: get_extra_info(name, default=None) - - Access optional transport information; see - :meth:`BaseTransport.get_extra_info` for details. - Examples ======== diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py index e7fb22ee5d1a..0afc66a473d4 100644 --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -348,7 +348,7 @@ def close(self): # a reader can be garbage collected # after connection closing self._protocol._untrack_reader() - return self._transport.close() + self._transport.close() def is_closing(self): return self._transport.is_closing() @@ -381,6 +381,14 @@ def get_extra_info(self, name, default=None): await sleep(0, loop=self._loop) await self._protocol._drain_helper() + async def aclose(self): + self.close() + await self.wait_closed() + + async def awrite(self, data): + self.write(data) + await self.drain() + class StreamReader: diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py index 67ac9d91a0b1..d8e371510dea 100644 --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -964,6 +964,28 @@ def test_del_stream_before_connection_made(self): 'call "stream.close()" explicitly.', messages[0]['message']) + def test_async_writer_api(self): + messages = [] + self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx)) + + with test_utils.run_test_server() as httpd: + rd, wr = self.loop.run_until_complete( + asyncio.open_connection(*httpd.address, + loop=self.loop)) + + f = wr.awrite(b'GET / HTTP/1.0\r\n\r\n') + self.loop.run_until_complete(f) + f = rd.readline() + data = self.loop.run_until_complete(f) + self.assertEqual(data, b'HTTP/1.0 200 OK\r\n') + f = rd.read() + data = self.loop.run_until_complete(f) + self.assertTrue(data.endswith(b'\r\n\r\nTest message')) + f = wr.aclose() + self.loop.run_until_complete(f) + + self.assertEqual(messages, []) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS.d/next/Library/2018-09-13-11-49-52.bpo-34666.3uLtWv.rst b/Misc/NEWS.d/next/Library/2018-09-13-11-49-52.bpo-34666.3uLtWv.rst new file mode 100644 index 000000000000..be82cfed7f13 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-13-11-49-52.bpo-34666.3uLtWv.rst @@ -0,0 +1,3 @@ +Implement ``asyncio.StreamWriter.awrite`` and +``asyncio.StreamWriter.aclose()`` coroutines. Methods are needed for +providing a consistent stream API with control flow switched on by default. From webhook-mailer at python.org Thu Sep 13 20:03:40 2018 From: webhook-mailer at python.org (Andrew Svetlov) Date: Fri, 14 Sep 2018 00:03:40 -0000 Subject: [Python-checkins] [3.7] bpo-34490: Fix test_asyncio for AIX - do not call transport.get_extra_info('sockname') (GH-8907) (#9286) Message-ID: https://github.com/python/cpython/commit/d0491cd7fd8a22093ab1ae54eee51fb0e7805c0d commit: d0491cd7fd8a22093ab1ae54eee51fb0e7805c0d branch: 3.7 author: Andrew Svetlov committer: GitHub date: 2018-09-13T17:03:35-07:00 summary: [3.7] bpo-34490: Fix test_asyncio for AIX - do not call transport.get_extra_info('sockname') (GH-8907) (#9286) * [3.7] Fix test_asyncio for AIX - do not call transport.get_extra_info('sockname') (GH-8907). (cherry picked from commit 413118ebf3162418639a5c4af14b02d26571a02c) Co-authored-by: Michael Felt * Update test_events.py files: A Misc/NEWS.d/next/Tests/2018-08-24-20-23-15.bpo-34490.vb2cx4.rst M Lib/test/test_asyncio/test_events.py diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py index 01ed47b36495..66b060349007 100644 --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -37,9 +37,11 @@ from test import support -def osx_tiger(): +def broken_unix_getsockname(): """Return True if the platform is Mac OS 10.4 or older.""" - if sys.platform != 'darwin': + if sys.platform.startswith("aix"): + return True + elif sys.platform != 'darwin': return False version = platform.mac_ver()[0] version = tuple(map(int, version.split('.'))) @@ -613,7 +615,7 @@ def test_create_connection(self): def test_create_unix_connection(self): # Issue #20682: On Mac OS X Tiger, getsockname() returns a # zero-length address for UNIX socket. - check_sockname = not osx_tiger() + check_sockname = not broken_unix_getsockname() with test_utils.run_test_unix_server() as httpd: conn_fut = self.loop.create_unix_connection( @@ -744,7 +746,7 @@ def test_create_ssl_connection(self): def test_create_ssl_unix_connection(self): # Issue #20682: On Mac OS X Tiger, getsockname() returns a # zero-length address for UNIX socket. - check_sockname = not osx_tiger() + check_sockname = not broken_unix_getsockname() with test_utils.run_test_unix_server(use_ssl=True) as httpd: create_connection = functools.partial( diff --git a/Misc/NEWS.d/next/Tests/2018-08-24-20-23-15.bpo-34490.vb2cx4.rst b/Misc/NEWS.d/next/Tests/2018-08-24-20-23-15.bpo-34490.vb2cx4.rst new file mode 100644 index 000000000000..c778f94b5e8d --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-08-24-20-23-15.bpo-34490.vb2cx4.rst @@ -0,0 +1,2 @@ +On AIX with AF_UNIX family sockets getsockname() does not provide 'sockname', +so skip calls to transport.get_extra_info('sockname') From webhook-mailer at python.org Thu Sep 13 21:28:24 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 14 Sep 2018 01:28:24 -0000 Subject: [Python-checkins] bpo-33649: Polish asyncio subprocess and sync docs (GH-9285) Message-ID: https://github.com/python/cpython/commit/4e824e96491f33c8a8462aa4970c55942064ae76 commit: 4e824e96491f33c8a8462aa4970c55942064ae76 branch: master author: Carol Willing committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-13T18:28:19-07:00 summary: bpo-33649: Polish asyncio subprocess and sync docs (GH-9285) Second pass for asyncio subprocess and sync docs. https://bugs.python.org/issue33649 files: M Doc/library/asyncio-subprocess.rst M Doc/library/asyncio-sync.rst diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index b05c236bb0c8..92c081b70bfc 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -62,8 +62,7 @@ Creating Subprocesses The *limit* argument sets the buffer limit for :class:`StreamReader` wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` - (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* - arguments). + (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). Return a :class:`~asyncio.subprocess.Process` instance. @@ -78,15 +77,14 @@ Creating Subprocesses The *limit* argument sets the buffer limit for :class:`StreamReader` wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` - (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* - arguments). + (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). Return a :class:`~asyncio.subprocess.Process` instance. See the documentation of :meth:`loop.subprocess_shell` for other parameters. -.. note:: +.. important:: It is the application's responsibility to ensure that all whitespace and metacharacters are quoted appropriately to avoid `shell injection @@ -98,7 +96,8 @@ Creating Subprocesses .. note:: The default event loop that asyncio is pre-configured - to use on **Windows** does not support subprocesses. + to use on **Windows** does not support subprocesses. Subprocesses are + available for Windows if the :class:`ProactorEventLoop` is used. See :ref:`Subprocess Support on Windows ` for details. @@ -206,7 +205,7 @@ communicate with them. exception is ignored. This condition occurs when the process exits before all data are written into *stdin*. - If its desired to send data to the process' *stdin*, + If it is desired to send data to the process' *stdin*, the process needs to be created with ``stdin=PIPE``. Similarly, to get anything other than ``None`` in the result tuple, the process has to be created with ``stdout=PIPE`` and/or @@ -265,8 +264,8 @@ communicate with them. Use the :meth:`communicate` method rather than :attr:`process.stdin.write() `, :attr:`await process.stdout.read() ` or - :attr:`await process.stderr.read ` - to avoid deadlocks due to streams pausing reading or writing + :attr:`await process.stderr.read `. + This avoids deadlocks due to streams pausing reading or writing and blocking the child process. .. attribute:: pid diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index 8e01ca944083..8da5aa81ab9e 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -13,11 +13,11 @@ those of the :mod:`threading` module with two important caveats: be used for OS threads synchronization (use :mod:`threading` for that); -* methods of synchronization objects do not accept the *timeout* +* methods of synchronization primitives do not accept the *timeout* argument; use the :func:`asyncio.wait_for` function to perform operations with timeouts. -asyncio has the following basic primitives: +asyncio has the following basic sychronization primitives: * :class:`Lock` * :class:`Event` @@ -72,7 +72,7 @@ Lock When the lock is *locked*, reset it to *unlocked* and return. - If the lock is *unlocked* a :exc:`RuntimeError` is raised. + If the lock is *unlocked*, a :exc:`RuntimeError` is raised. .. method:: locked() @@ -97,7 +97,7 @@ Event Example:: async def waiter(event): - print('waiting ...') + print('waiting for it ...') await event.wait() print('... got it!') From webhook-mailer at python.org Fri Sep 14 00:17:43 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Fri, 14 Sep 2018 04:17:43 -0000 Subject: [Python-checkins] Remove wording that could be deemed to be perjorative (GH-9287) Message-ID: https://github.com/python/cpython/commit/1401018da127c668a607bc63a14e5b6c3156f916 commit: 1401018da127c668a607bc63a14e5b6c3156f916 branch: master author: Raymond Hettinger committer: GitHub date: 2018-09-13T21:17:40-07:00 summary: Remove wording that could be deemed to be perjorative (GH-9287) files: M Lib/_pyio.py M Modules/_ctypes/libffi_msvc/README M Modules/_ctypes/libffi_osx/README M Tools/clinic/clinic.py diff --git a/Lib/_pyio.py b/Lib/_pyio.py index f0d4f4ed27a2..01ef5b7b0bb5 100644 --- a/Lib/_pyio.py +++ b/Lib/_pyio.py @@ -2283,7 +2283,7 @@ def tell(self): # current pos. # Rationale: calling decoder.decode() has a large overhead # regardless of chunk size; we want the number of such calls to - # be O(1) in most situations (common decoders, non-crazy input). + # be O(1) in most situations (common decoders, sensible input). # Actually, it will be exactly 1 for fixed-size codecs (all # 8-bit codecs, also UTF-16 and UTF-32). skip_bytes = int(self._b2cratio * chars_to_skip) diff --git a/Modules/_ctypes/libffi_msvc/README b/Modules/_ctypes/libffi_msvc/README index 1fc27470d0a3..69e46cbf8a42 100644 --- a/Modules/_ctypes/libffi_msvc/README +++ b/Modules/_ctypes/libffi_msvc/README @@ -372,8 +372,8 @@ single-precision anyway. This causes one test to fail (the `many arguments' test). -What's With The Crazy Comments? -=============================== +What's With The Cryptic Comments? +================================= You might notice a number of cryptic comments in the code, delimited by /*@ and @*/. These are annotations read by the program LCLint, a diff --git a/Modules/_ctypes/libffi_osx/README b/Modules/_ctypes/libffi_osx/README index 1fc27470d0a3..69e46cbf8a42 100644 --- a/Modules/_ctypes/libffi_osx/README +++ b/Modules/_ctypes/libffi_osx/README @@ -372,8 +372,8 @@ single-precision anyway. This causes one test to fail (the `many arguments' test). -What's With The Crazy Comments? -=============================== +What's With The Cryptic Comments? +================================= You might notice a number of cryptic comments in the code, delimited by /*@ and @*/. These are annotations read by the program LCLint, a diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 653afbea5fc9..a6a43d1361d0 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2787,7 +2787,7 @@ def cleanup(self): # # This is the fourth or fifth rewrite of registering all the -# crazy string converter format units. Previous approaches hid +# string converter format units. Previous approaches hid # bugs--generally mismatches between the semantics of the format # unit and the arguments necessary to represent those semantics # properly. Hopefully with this approach we'll get it 100% right. From webhook-mailer at python.org Fri Sep 14 00:50:24 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 14 Sep 2018 04:50:24 -0000 Subject: [Python-checkins] bpo-34552: Clarify built-in types comparisons (GH-9035) Message-ID: https://github.com/python/cpython/commit/1aeba7458d2aaf8a03b5d443179d122ceb2ccece commit: 1aeba7458d2aaf8a03b5d443179d122ceb2ccece branch: master author: Windson yang committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-13T21:50:18-07:00 summary: bpo-34552: Clarify built-in types comparisons (GH-9035) Some updates to ancient text about comparisons; fixes bp-34552. files: A Misc/NEWS.d/next/Documentation/2018-09-12-10-18-04.bpo-34552.p9PoYv.rst M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index f7f59cd3a6bb..fd59a5170dae 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -20,10 +20,10 @@ rearrange their members in place, and don't return a specific item, never return the collection instance itself but ``None``. Some operations are supported by several object types; in particular, -practically all objects can be compared, tested for truth value, and converted -to a string (with the :func:`repr` function or the slightly different -:func:`str` function). The latter function is implicitly used when an object is -written by the :func:`print` function. +practically all objects can be compared for equality, tested for truth +value, and converted to a string (with the :func:`repr` function or the +slightly different :func:`str` function). The latter function is implicitly +used when an object is written by the :func:`print` function. .. _truth: @@ -164,12 +164,10 @@ This table summarizes the comparison operations: pair: objects; comparing Objects of different types, except different numeric types, never compare equal. -Furthermore, some types (for example, function objects) support only a degenerate -notion of comparison where any two objects of that type are unequal. The ``<``, -``<=``, ``>`` and ``>=`` operators will raise a :exc:`TypeError` exception when -comparing a complex number with another built-in numeric type, when the objects -are of different types that cannot be compared, or in other cases where there is -no defined ordering. +The ``==`` operator is always defined but for some object types (for example, +class objects) is equivalent to :keyword:`is`. The ``<``, ``<=``, ``>`` and ``>=`` +operators are only defined where they make sense; for example, they raise a +:exc:`TypeError` exception when one of the arguments is a complex number. .. index:: single: __eq__() (instance method) diff --git a/Misc/NEWS.d/next/Documentation/2018-09-12-10-18-04.bpo-34552.p9PoYv.rst b/Misc/NEWS.d/next/Documentation/2018-09-12-10-18-04.bpo-34552.p9PoYv.rst new file mode 100644 index 000000000000..9e7605bc6d68 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-09-12-10-18-04.bpo-34552.p9PoYv.rst @@ -0,0 +1,2 @@ +Make clear that ``==`` operator sometimes is equivalent to `is`. The ``<``, +``<=``, ``>`` and ``>=`` operators are only defined where they make sense. From webhook-mailer at python.org Fri Sep 14 00:57:34 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 04:57:34 -0000 Subject: [Python-checkins] bpo-34674: Assume unistd.h exists on Unix. (GH-9290) Message-ID: https://github.com/python/cpython/commit/ea13740a37347d68d096b11b87c9167917ccfc22 commit: ea13740a37347d68d096b11b87c9167917ccfc22 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-13T21:57:31-07:00 summary: bpo-34674: Assume unistd.h exists on Unix. (GH-9290) files: M Include/Python.h M Modules/posixmodule.c M Modules/resource.c M configure M configure.ac diff --git a/Include/Python.h b/Include/Python.h index bf1ca6cfe98e..1bac5bd54287 100644 --- a/Include/Python.h +++ b/Include/Python.h @@ -32,7 +32,7 @@ #include #endif #include -#ifdef HAVE_UNISTD_H +#ifndef MS_WINDOWS #include #endif #ifdef HAVE_CRYPT_H diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 7c02351a4661..2fddd9587f76 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -213,41 +213,6 @@ module os extern char *ctermid_r(char *); #endif -#ifndef HAVE_UNISTD_H -#if ( defined(__WATCOMC__) || defined(_MSC_VER) ) && !defined(__QNX__) -extern int mkdir(const char *); -#else -extern int mkdir(const char *, mode_t); -#endif -#if defined(__IBMC__) || defined(__IBMCPP__) -extern int chdir(char *); -extern int rmdir(char *); -#else -extern int chdir(const char *); -extern int rmdir(const char *); -#endif -extern int chmod(const char *, mode_t); -/*#ifdef HAVE_FCHMOD -extern int fchmod(int, mode_t); -#endif*/ -/*#ifdef HAVE_LCHMOD -extern int lchmod(const char *, mode_t); -#endif*/ -extern int chown(const char *, uid_t, gid_t); -extern char *getcwd(char *, int); -extern char *strerror(int); -extern int link(const char *, const char *); -extern int rename(const char *, const char *); -extern int stat(const char *, struct stat *); -extern int unlink(const char *); -#ifdef HAVE_SYMLINK -extern int symlink(const char *, const char *); -#endif /* HAVE_SYMLINK */ -#ifdef HAVE_LSTAT -extern int lstat(const char *, struct stat *); -#endif /* HAVE_LSTAT */ -#endif /* !HAVE_UNISTD_H */ - #endif /* !_MSC_VER */ #ifdef HAVE_POSIX_SPAWN diff --git a/Modules/resource.c b/Modules/resource.c index e59280f249df..87c72e740989 100644 --- a/Modules/resource.c +++ b/Modules/resource.c @@ -4,10 +4,7 @@ #include #include #include -/* for sysconf */ -#if defined(HAVE_UNISTD_H) #include -#endif /* On some systems, these aren't in any header file. On others they are, with inconsistent prototypes. diff --git a/configure b/configure index ddfd1c4434c2..22fc02153d14 100755 --- a/configure +++ b/configure @@ -7725,7 +7725,7 @@ for ac_header in asm/types.h crypt.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \ sched.h shadow.h signal.h stropts.h termios.h \ -unistd.h utime.h \ +utime.h \ poll.h sys/devpoll.h sys/epoll.h sys/poll.h \ sys/audioio.h sys/xattr.h sys/bsdtty.h sys/event.h sys/file.h sys/ioctl.h \ sys/kern_control.h sys/loadavg.h sys/lock.h sys/mkdev.h sys/modem.h \ diff --git a/configure.ac b/configure.ac index cc1bba4b384d..03953622ea7d 100644 --- a/configure.ac +++ b/configure.ac @@ -2066,7 +2066,7 @@ AC_CHECK_HEADERS(asm/types.h crypt.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \ sched.h shadow.h signal.h stropts.h termios.h \ -unistd.h utime.h \ +utime.h \ poll.h sys/devpoll.h sys/epoll.h sys/poll.h \ sys/audioio.h sys/xattr.h sys/bsdtty.h sys/event.h sys/file.h sys/ioctl.h \ sys/kern_control.h sys/loadavg.h sys/lock.h sys/mkdev.h sys/modem.h \ From webhook-mailer at python.org Fri Sep 14 01:08:36 2018 From: webhook-mailer at python.org (Gregory P. Smith) Date: Fri, 14 Sep 2018 05:08:36 -0000 Subject: [Python-checkins] bpo-6721: Hold logging locks across fork() (GH-4071) Message-ID: https://github.com/python/cpython/commit/19003841e965bbf56fd06824d6093620c1b66f9e commit: 19003841e965bbf56fd06824d6093620c1b66f9e branch: master author: Gregory P. Smith committer: GitHub date: 2018-09-13T22:08:31-07:00 summary: bpo-6721: Hold logging locks across fork() (GH-4071) bpo-6721: When os.fork() was called while another thread holds a logging lock, the child process may deadlock when it tries to log. This fixes that by acquiring all logging locks before fork and releasing them afterwards. A regression test that fails before this change is included. Within the new unittest itself: There is a small _potential_ due to mixing of fork and a thread in the child process if the parent's thread happened to hold a non-reentrant library call lock (malloc?) when the os.fork() happens. buildbots and time will tell if this actually manifests itself in this test or not. :/ A functionality test that avoids that would be a challenge. An alternate test that isn't trying to produce the deadlock itself but just checking that the release and acquire calls are made would be the next best alternative if so. files: A Misc/NEWS.d/next/Library/2018-09-13-10-09-19.bpo-6721.ZUL_F3.rst M Lib/logging/__init__.py M Lib/test/test_logging.py diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py index a4a950dc4e2c..7aeff45f2666 100644 --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -225,6 +225,55 @@ def _releaseLock(): if _lock: _lock.release() + +# Prevent a held logging lock from blocking a child from logging. + +if not hasattr(os, 'register_at_fork'): # Windows and friends. + def _register_at_fork_acquire_release(instance): + pass # no-op when os.register_at_fork does not exist. +else: # The os.register_at_fork API exists + os.register_at_fork(before=_acquireLock, + after_in_child=_releaseLock, + after_in_parent=_releaseLock) + + # A collection of instances with acquire and release methods (logging.Handler) + # to be called before and after fork. The weakref avoids us keeping discarded + # Handler instances alive forever in case an odd program creates and destroys + # many over its lifetime. + _at_fork_acquire_release_weakset = weakref.WeakSet() + + + def _register_at_fork_acquire_release(instance): + # We put the instance itself in a single WeakSet as we MUST have only + # one atomic weak ref. used by both before and after atfork calls to + # guarantee matched pairs of acquire and release calls. + _at_fork_acquire_release_weakset.add(instance) + + + def _at_fork_weak_calls(method_name): + for instance in _at_fork_acquire_release_weakset: + method = getattr(instance, method_name) + try: + method() + except Exception as err: + # Similar to what PyErr_WriteUnraisable does. + print("Ignoring exception from logging atfork", instance, + method_name, "method:", err, file=sys.stderr) + + + def _before_at_fork_weak_calls(): + _at_fork_weak_calls('acquire') + + + def _after_at_fork_weak_calls(): + _at_fork_weak_calls('release') + + + os.register_at_fork(before=_before_at_fork_weak_calls, + after_in_child=_after_at_fork_weak_calls, + after_in_parent=_after_at_fork_weak_calls) + + #--------------------------------------------------------------------------- # The logging record #--------------------------------------------------------------------------- @@ -795,6 +844,7 @@ def createLock(self): Acquire a thread lock for serializing access to the underlying I/O. """ self.lock = threading.RLock() + _register_at_fork_acquire_release(self) def acquire(self): """ diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index f7232f67eee7..b9dad64a5949 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -35,6 +35,7 @@ import queue import random import re +import signal import socket import struct import sys @@ -666,6 +667,72 @@ def remove_loop(fname, tries): if os.path.exists(fn): os.unlink(fn) + # The implementation relies on os.register_at_fork existing, but we test + # based on os.fork existing because that is what users and this test use. + # This helps ensure that when fork exists (the important concept) that the + # register_at_fork mechanism is also present and used. + @unittest.skipIf(not hasattr(os, 'fork'), 'Test requires os.fork().') + def test_post_fork_child_no_deadlock(self): + """Ensure forked child logging locks are not held; bpo-6721.""" + refed_h = logging.Handler() + refed_h.name = 'because we need at least one for this test' + self.assertGreater(len(logging._handlers), 0) + + locks_held__ready_to_fork = threading.Event() + fork_happened__release_locks_and_end_thread = threading.Event() + + def lock_holder_thread_fn(): + logging._acquireLock() + try: + refed_h.acquire() + try: + # Tell the main thread to do the fork. + locks_held__ready_to_fork.set() + + # If the deadlock bug exists, the fork will happen + # without dealing with the locks we hold, deadlocking + # the child. + + # Wait for a successful fork or an unreasonable amount of + # time before releasing our locks. To avoid a timing based + # test we'd need communication from os.fork() as to when it + # has actually happened. Given this is a regression test + # for a fixed issue, potentially less reliably detecting + # regression via timing is acceptable for simplicity. + # The test will always take at least this long. :( + fork_happened__release_locks_and_end_thread.wait(0.5) + finally: + refed_h.release() + finally: + logging._releaseLock() + + lock_holder_thread = threading.Thread( + target=lock_holder_thread_fn, + name='test_post_fork_child_no_deadlock lock holder') + lock_holder_thread.start() + + locks_held__ready_to_fork.wait() + pid = os.fork() + if pid == 0: # Child. + logging.error(r'Child process did not deadlock. \o/') + os._exit(0) + else: # Parent. + fork_happened__release_locks_and_end_thread.set() + lock_holder_thread.join() + start_time = time.monotonic() + while True: + waited_pid, status = os.waitpid(pid, os.WNOHANG) + if waited_pid == pid: + break # child process exited. + if time.monotonic() - start_time > 7: + break # so long? implies child deadlock. + time.sleep(0.05) + if waited_pid != pid: + os.kill(pid, signal.SIGKILL) + waited_pid, status = os.waitpid(pid, 0) + self.fail("child process deadlocked.") + self.assertEqual(status, 0, msg="child process error") + class BadStream(object): def write(self, data): diff --git a/Misc/NEWS.d/next/Library/2018-09-13-10-09-19.bpo-6721.ZUL_F3.rst b/Misc/NEWS.d/next/Library/2018-09-13-10-09-19.bpo-6721.ZUL_F3.rst new file mode 100644 index 000000000000..073a441db679 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-13-10-09-19.bpo-6721.ZUL_F3.rst @@ -0,0 +1,2 @@ +Acquire the logging module's commonly used internal locks while fork()ing to +avoid deadlocks in the child process. From webhook-mailer at python.org Fri Sep 14 01:45:03 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 05:45:03 -0000 Subject: [Python-checkins] Change the xkcd link in comment over https. (GH-5452) Message-ID: https://github.com/python/cpython/commit/83df50ea5757816c7338d27f21fd18b1e79206f7 commit: 83df50ea5757816c7338d27f21fd18b1e79206f7 branch: master author: ? committer: Benjamin Peterson date: 2018-09-13T22:45:00-07:00 summary: Change the xkcd link in comment over https. (GH-5452) files: M Lib/antigravity.py diff --git a/Lib/antigravity.py b/Lib/antigravity.py index 9b1436803722..c6f174ca6d87 100644 --- a/Lib/antigravity.py +++ b/Lib/antigravity.py @@ -11,7 +11,7 @@ def geohash(latitude, longitude, datedow): 37.857713 -122.544543 ''' - # http://xkcd.com/426/ + # https://xkcd.com/426/ h = hashlib.md5(datedow).hexdigest() p, q = [('%f' % float.fromhex('0.' + x)) for x in (h[:16], h[16:32])] print('%d%s %d%s' % (latitude, p[1:], longitude, q[1:])) From webhook-mailer at python.org Fri Sep 14 02:56:27 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Fri, 14 Sep 2018 06:56:27 -0000 Subject: [Python-checkins] bpo-33073: Adding as_integer_ratio to ints. (GH-8750) Message-ID: https://github.com/python/cpython/commit/5ac704306f4b81ae3f28d8742408d3214b145e8a commit: 5ac704306f4b81ae3f28d8742408d3214b145e8a branch: master author: Lisa Roach committer: Raymond Hettinger date: 2018-09-13T23:56:23-07:00 summary: bpo-33073: Adding as_integer_ratio to ints. (GH-8750) files: A Misc/NEWS.d/next/Core and Builtins/2018-08-12-16-03-58.bpo-33073.XWu1Jh.rst M Doc/library/stdtypes.rst M Doc/whatsnew/3.8.rst M Lib/test/test_doctest.py M Lib/test/test_long.py M Misc/ACKS M Objects/clinic/longobject.c.h M Objects/longobject.c diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index fd59a5170dae..5a133e32fced 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -537,6 +537,14 @@ class`. In addition, it provides a few more methods: .. versionadded:: 3.2 +.. method:: int.as_integer_ratio() + + Return a pair of integers whose ratio is exactly equal to the original + integer and with a positive denominator. The integer ratio of integers + (whole numbers) is always the integer as the numerator and ``1`` as the + denominator. + + .. versionadded:: 3.8 Additional Methods on Float --------------------------- diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index b2475c7df33e..38b8623dddd2 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -91,6 +91,10 @@ Other Language Changes was lifted. (Contributed by Serhiy Storchaka in :issue:`32489`.) +* The ``int`` type now has a new ``as_integer_ratio`` method compatible + with the existing ``float.as_integer_ratio`` method. + (Contributed by Lisa Roach in :issue:`33073`.) + * Added support of ``\N{name}`` escapes in :mod:`regular expressions `. (Contributed by Jonathan Eunice and Serhiy Storchaka in :issue:`30688`.) diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py index 83941c129f44..797bdb847190 100644 --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest.py @@ -665,7 +665,7 @@ def non_Python_modules(): r""" True >>> real_tests = [t for t in tests if len(t.examples) > 0] >>> len(real_tests) # objects that actually have doctests - 8 + 9 >>> for t in real_tests: ... print('{} {}'.format(len(t.examples), t.name)) ... @@ -675,6 +675,7 @@ def non_Python_modules(): r""" 2 builtins.float.hex 1 builtins.hex 1 builtins.int + 3 builtins.int.as_integer_ratio 2 builtins.int.bit_length 1 builtins.oct diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py index 8472889d48ba..7c883baebb41 100644 --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -3,6 +3,7 @@ import sys +import enum import random import math import array @@ -1349,6 +1350,37 @@ def test_shift_bool(self): self.assertEqual(type(value << shift), int) self.assertEqual(type(value >> shift), int) + def test_as_integer_ratio(self): + tests = [10, 0, -10, 1] + for value in tests: + numerator, denominator = value.as_integer_ratio() + self.assertEqual((numerator, denominator), (value, 1)) + self.assertIsInstance(numerator, int) + self.assertIsInstance(denominator, int) + + def test_as_integer_ratio_maxint(self): + x = sys.maxsize + 1 + self.assertEqual(x.as_integer_ratio()[0], x) + + def test_as_integer_ratio_bool(self): + self.assertEqual(True.as_integer_ratio(), (1, 1)) + self.assertEqual(False.as_integer_ratio(), (0, 1)) + self.assertEqual(type(True.as_integer_ratio()[0]), int) + self.assertEqual(type(False.as_integer_ratio()[0]), int) + + def test_as_integer_ratio_int_enum(self): + class Foo(enum.IntEnum): + X = 42 + self.assertEqual(Foo.X.as_integer_ratio(), (42, 1)) + self.assertEqual(type(Foo.X.as_integer_ratio()[0]), int) + + def test_as_integer_ratio_int_flag(self): + class Foo(enum.IntFlag): + R = 1 << 2 + self.assertEqual(Foo.R.as_integer_ratio(), (4, 1)) + self.assertEqual(type(Foo.R.as_integer_ratio()[0]), int) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/ACKS b/Misc/ACKS index 96985358e236..a29ff6020bbf 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1350,6 +1350,7 @@ Juan M. Bello Rivas Mohd Sanad Zaki Rizvi Davide Rizzo Anthony Roach +Lisa Roach Carl Robben Ben Roberts Mark Roberts diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-08-12-16-03-58.bpo-33073.XWu1Jh.rst b/Misc/NEWS.d/next/Core and Builtins/2018-08-12-16-03-58.bpo-33073.XWu1Jh.rst new file mode 100644 index 000000000000..ce9b6129f96e --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-08-12-16-03-58.bpo-33073.XWu1Jh.rst @@ -0,0 +1 @@ +Added as_integer_ratio to ints to make them more interoperable with floats. diff --git a/Objects/clinic/longobject.c.h b/Objects/clinic/longobject.c.h index 14f5515c7a62..0e70fe5d8c44 100644 --- a/Objects/clinic/longobject.c.h +++ b/Objects/clinic/longobject.c.h @@ -118,6 +118,34 @@ int_bit_length(PyObject *self, PyObject *Py_UNUSED(ignored)) return int_bit_length_impl(self); } +PyDoc_STRVAR(int_as_integer_ratio__doc__, +"as_integer_ratio($self, /)\n" +"--\n" +"\n" +"Return integer ratio.\n" +"\n" +"Return a pair of integers, whose ratio is exactly equal to the original int\n" +"and with a positive denominator.\n" +"\n" +">>> (10).as_integer_ratio()\n" +"(10, 1)\n" +">>> (-10).as_integer_ratio()\n" +"(-10, 1)\n" +">>> (0).as_integer_ratio()\n" +"(0, 1)"); + +#define INT_AS_INTEGER_RATIO_METHODDEF \ + {"as_integer_ratio", (PyCFunction)int_as_integer_ratio, METH_NOARGS, int_as_integer_ratio__doc__}, + +static PyObject * +int_as_integer_ratio_impl(PyObject *self); + +static PyObject * +int_as_integer_ratio(PyObject *self, PyObject *Py_UNUSED(ignored)) +{ + return int_as_integer_ratio_impl(self); +} + PyDoc_STRVAR(int_to_bytes__doc__, "to_bytes($self, /, length, byteorder, *, signed=False)\n" "--\n" @@ -211,4 +239,4 @@ int_from_bytes(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs, PyOb exit: return return_value; } -/*[clinic end generated code: output=fd64beb83bd16df3 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=6d5e92d7dc803751 input=a9049054013a1b77]*/ diff --git a/Objects/longobject.c b/Objects/longobject.c index 399d35427099..98ff9a8c265b 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -5260,6 +5260,36 @@ long_is_finite(PyObject *v) } #endif +/*[clinic input] +int.as_integer_ratio + +Return integer ratio. + +Return a pair of integers, whose ratio is exactly equal to the original int +and with a positive denominator. + +>>> (10).as_integer_ratio() +(10, 1) +>>> (-10).as_integer_ratio() +(-10, 1) +>>> (0).as_integer_ratio() +(0, 1) +[clinic start generated code]*/ + +static PyObject * +int_as_integer_ratio_impl(PyObject *self) +/*[clinic end generated code: output=e60803ae1cc8621a input=55ce3058e15de393]*/ +{ + if PyLong_CheckExact(self) { + return PyTuple_Pack(2, self, _PyLong_One); + } else { + PyObject *numerator = _PyLong_Copy(self); + PyObject *ratio_tuple = PyTuple_Pack(2, numerator, _PyLong_One); + Py_DECREF(numerator); + return ratio_tuple; + } +} + /*[clinic input] int.to_bytes @@ -5392,6 +5422,7 @@ static PyMethodDef long_methods[] = { #endif INT_TO_BYTES_METHODDEF INT_FROM_BYTES_METHODDEF + INT_AS_INTEGER_RATIO_METHODDEF {"__trunc__", long_long_meth, METH_NOARGS, "Truncating an Integral returns itself."}, {"__floor__", long_long_meth, METH_NOARGS, From webhook-mailer at python.org Fri Sep 14 03:53:23 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Fri, 14 Sep 2018 07:53:23 -0000 Subject: [Python-checkins] Note that distinct argument patterns can be cached separately (GH-9298) Message-ID: https://github.com/python/cpython/commit/902bcd9a1e2c73c6de5510b771c590b618c4c94e commit: 902bcd9a1e2c73c6de5510b771c590b618c4c94e branch: master author: Raymond Hettinger committer: GitHub date: 2018-09-14T00:53:20-07:00 summary: Note that distinct argument patterns can be cached separately (GH-9298) files: M Doc/library/functools.rst diff --git a/Doc/library/functools.rst b/Doc/library/functools.rst index 1b94f3396005..214d57334c8d 100644 --- a/Doc/library/functools.rst +++ b/Doc/library/functools.rst @@ -85,6 +85,11 @@ The :mod:`functools` module defines the following functions: Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable. + Distinct argument patterns may be considered to be distinct calls with + separate cache entries. For example, `f(a=1, b=2)` and `f(b=2, a=1)` + differ in their keyword argument order and may have two separate cache + entries. + If *maxsize* is set to ``None``, the LRU feature is disabled and the cache can grow without bound. The LRU feature performs best when *maxsize* is a power-of-two. From webhook-mailer at python.org Fri Sep 14 04:00:16 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Fri, 14 Sep 2018 08:00:16 -0000 Subject: [Python-checkins] Fix-up parenthesis, organization, and NULL check (GH-9297) Message-ID: https://github.com/python/cpython/commit/00bc08ec11d99cc68c7d5dc790ad3e609982a9c7 commit: 00bc08ec11d99cc68c7d5dc790ad3e609982a9c7 branch: master author: Raymond Hettinger committer: GitHub date: 2018-09-14T01:00:11-07:00 summary: Fix-up parenthesis, organization, and NULL check (GH-9297) files: M Objects/longobject.c diff --git a/Objects/longobject.c b/Objects/longobject.c index 98ff9a8c265b..102093e19865 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -5280,14 +5280,19 @@ static PyObject * int_as_integer_ratio_impl(PyObject *self) /*[clinic end generated code: output=e60803ae1cc8621a input=55ce3058e15de393]*/ { - if PyLong_CheckExact(self) { + PyObject *numerator; + PyObject *ratio_tuple; + + if (PyLong_CheckExact(self)) { return PyTuple_Pack(2, self, _PyLong_One); - } else { - PyObject *numerator = _PyLong_Copy(self); - PyObject *ratio_tuple = PyTuple_Pack(2, numerator, _PyLong_One); - Py_DECREF(numerator); - return ratio_tuple; } + numerator = _PyLong_Copy(self); + if (numerator == NULL) { + return NULL; + } + ratio_tuple = PyTuple_Pack(2, numerator, _PyLong_One); + Py_DECREF(numerator); + return ratio_tuple; } /*[clinic input] From webhook-mailer at python.org Fri Sep 14 04:13:23 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Fri, 14 Sep 2018 08:13:23 -0000 Subject: [Python-checkins] Note that distinct argument patterns can be cached separately (GH-9298) (GH-9299) Message-ID: https://github.com/python/cpython/commit/a8f189f4571ca16e85d322cb45d1599526dbd9b4 commit: a8f189f4571ca16e85d322cb45d1599526dbd9b4 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2018-09-14T01:13:17-07:00 summary: Note that distinct argument patterns can be cached separately (GH-9298) (GH-9299) (cherry picked from commit 902bcd9a1e2c73c6de5510b771c590b618c4c94e) Co-authored-by: Raymond Hettinger files: M Doc/library/functools.rst diff --git a/Doc/library/functools.rst b/Doc/library/functools.rst index a81e819103ad..3413cd353ca9 100644 --- a/Doc/library/functools.rst +++ b/Doc/library/functools.rst @@ -52,6 +52,11 @@ The :mod:`functools` module defines the following functions: Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable. + Distinct argument patterns may be considered to be distinct calls with + separate cache entries. For example, `f(a=1, b=2)` and `f(b=2, a=1)` + differ in their keyword argument order and may have two separate cache + entries. + If *maxsize* is set to ``None``, the LRU feature is disabled and the cache can grow without bound. The LRU feature performs best when *maxsize* is a power-of-two. From webhook-mailer at python.org Fri Sep 14 04:36:06 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Fri, 14 Sep 2018 08:36:06 -0000 Subject: [Python-checkins] Fix compiler warning with a type cast (GH-9300) Message-ID: https://github.com/python/cpython/commit/73820a60cc3c990abb351540ca27bf7689bce8ac commit: 73820a60cc3c990abb351540ca27bf7689bce8ac branch: master author: Raymond Hettinger committer: GitHub date: 2018-09-14T01:35:59-07:00 summary: Fix compiler warning with a type cast (GH-9300) files: M Objects/longobject.c diff --git a/Objects/longobject.c b/Objects/longobject.c index 102093e19865..afe30bc0532a 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -5286,7 +5286,7 @@ int_as_integer_ratio_impl(PyObject *self) if (PyLong_CheckExact(self)) { return PyTuple_Pack(2, self, _PyLong_One); } - numerator = _PyLong_Copy(self); + numerator = _PyLong_Copy((PyLongObject *) self); if (numerator == NULL) { return NULL; } From solipsis at pitrou.net Fri Sep 14 05:09:08 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 14 Sep 2018 09:09:08 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=4 Message-ID: <20180914090908.1.875E44271BAB83A6@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog5DFVaB', '--timeout', '7200'] From webhook-mailer at python.org Fri Sep 14 11:32:19 2018 From: webhook-mailer at python.org (Eric V. Smith) Date: Fri, 14 Sep 2018 15:32:19 -0000 Subject: [Python-checkins] bpo-34363: dataclasses.asdict() and .astuple() now handle fields which are namedtuples. (GH-9151) Message-ID: https://github.com/python/cpython/commit/9b9d97dd139a799d28ff8bc90d118b1cac190b03 commit: 9b9d97dd139a799d28ff8bc90d118b1cac190b03 branch: master author: Eric V. Smith committer: GitHub date: 2018-09-14T11:32:16-04:00 summary: bpo-34363: dataclasses.asdict() and .astuple() now handle fields which are namedtuples. (GH-9151) files: A Misc/NEWS.d/next/Library/2018-09-10-21-09-34.bpo-34363.YuSb0T.rst M Lib/dataclasses.py M Lib/test/test_dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index a43d07693ac3..28e9f75127b1 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -1026,11 +1026,36 @@ def _asdict_inner(obj, dict_factory): value = _asdict_inner(getattr(obj, f.name), dict_factory) result.append((f.name, value)) return dict_factory(result) + elif isinstance(obj, tuple) and hasattr(obj, '_fields'): + # obj is a namedtuple. Recurse into it, but the returned + # object is another namedtuple of the same type. This is + # similar to how other list- or tuple-derived classes are + # treated (see below), but we just need to create them + # differently because a namedtuple's __init__ needs to be + # called differently (see bpo-34363). + + # I'm not using namedtuple's _asdict() + # method, because: + # - it does not recurse in to the namedtuple fields and + # convert them to dicts (using dict_factory). + # - I don't actually want to return a dict here. The the main + # use case here is json.dumps, and it handles converting + # namedtuples to lists. Admittedly we're losing some + # information here when we produce a json list instead of a + # dict. Note that if we returned dicts here instead of + # namedtuples, we could no longer call asdict() on a data + # structure where a namedtuple was used as a dict key. + + return type(obj)(*[_asdict_inner(v, dict_factory) for v in obj]) elif isinstance(obj, (list, tuple)): + # Assume we can create an object of this type by passing in a + # generator (which is not true for namedtuples, handled + # above). return type(obj)(_asdict_inner(v, dict_factory) for v in obj) elif isinstance(obj, dict): - return type(obj)((_asdict_inner(k, dict_factory), _asdict_inner(v, dict_factory)) - for k, v in obj.items()) + return type(obj)((_asdict_inner(k, dict_factory), + _asdict_inner(v, dict_factory)) + for k, v in obj.items()) else: return copy.deepcopy(obj) @@ -1066,7 +1091,18 @@ def _astuple_inner(obj, tuple_factory): value = _astuple_inner(getattr(obj, f.name), tuple_factory) result.append(value) return tuple_factory(result) + elif isinstance(obj, tuple) and hasattr(obj, '_fields'): + # obj is a namedtuple. Recurse into it, but the returned + # object is another namedtuple of the same type. This is + # similar to how other list- or tuple-derived classes are + # treated (see below), but we just need to create them + # differently because a namedtuple's __init__ needs to be + # called differently (see bpo-34363). + return type(obj)(*[_astuple_inner(v, tuple_factory) for v in obj]) elif isinstance(obj, (list, tuple)): + # Assume we can create an object of this type by passing in a + # generator (which is not true for namedtuples, handled + # above). return type(obj)(_astuple_inner(v, tuple_factory) for v in obj) elif isinstance(obj, dict): return type(obj)((_astuple_inner(k, tuple_factory), _astuple_inner(v, tuple_factory)) diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index 4c93513956a2..6efe785bc328 100755 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -1429,6 +1429,70 @@ class C: self.assertEqual(d, OrderedDict([('x', 42), ('y', 2)])) self.assertIs(type(d), OrderedDict) + def test_helper_asdict_namedtuple(self): + T = namedtuple('T', 'a b c') + @dataclass + class C: + x: str + y: T + c = C('outer', T(1, C('inner', T(11, 12, 13)), 2)) + + d = asdict(c) + self.assertEqual(d, {'x': 'outer', + 'y': T(1, + {'x': 'inner', + 'y': T(11, 12, 13)}, + 2), + } + ) + + # Now with a dict_factory. OrderedDict is convenient, but + # since it compares to dicts, we also need to have separate + # assertIs tests. + d = asdict(c, dict_factory=OrderedDict) + self.assertEqual(d, {'x': 'outer', + 'y': T(1, + {'x': 'inner', + 'y': T(11, 12, 13)}, + 2), + } + ) + + # Make sure that the returned dicts are actuall OrderedDicts. + self.assertIs(type(d), OrderedDict) + self.assertIs(type(d['y'][1]), OrderedDict) + + def test_helper_asdict_namedtuple_key(self): + # Ensure that a field that contains a dict which has a + # namedtuple as a key works with asdict(). + + @dataclass + class C: + f: dict + T = namedtuple('T', 'a') + + c = C({T('an a'): 0}) + + self.assertEqual(asdict(c), {'f': {T(a='an a'): 0}}) + + def test_helper_asdict_namedtuple_derived(self): + class T(namedtuple('Tbase', 'a')): + def my_a(self): + return self.a + + @dataclass + class C: + f: T + + t = T(6) + c = C(t) + + d = asdict(c) + self.assertEqual(d, {'f': T(a=6)}) + # Make sure that t has been copied, not used directly. + self.assertIsNot(d['f'], t) + self.assertEqual(d['f'].my_a(), 6) + def test_helper_astuple(self): # Basic tests for astuple(), it should return a new tuple. @dataclass @@ -1541,6 +1605,21 @@ def nt(lst): self.assertEqual(t, NT(42, 2)) self.assertIs(type(t), NT) + def test_helper_astuple_namedtuple(self): + T = namedtuple('T', 'a b c') + @dataclass + class C: + x: str + y: T + c = C('outer', T(1, C('inner', T(11, 12, 13)), 2)) + + t = astuple(c) + self.assertEqual(t, ('outer', T(1, ('inner', (11, 12, 13)), 2))) + + # Now, using a tuple_factory. list is convenient here. + t = astuple(c, tuple_factory=list) + self.assertEqual(t, ['outer', T(1, ['inner', T(11, 12, 13)], 2)]) + def test_dynamic_class_creation(self): cls_dict = {'__annotations__': {'x': int, 'y': int}, } diff --git a/Misc/NEWS.d/next/Library/2018-09-10-21-09-34.bpo-34363.YuSb0T.rst b/Misc/NEWS.d/next/Library/2018-09-10-21-09-34.bpo-34363.YuSb0T.rst new file mode 100644 index 000000000000..5691efb1e276 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-21-09-34.bpo-34363.YuSb0T.rst @@ -0,0 +1 @@ +dataclasses.asdict() and .astuple() now handle namedtuples correctly. From webhook-mailer at python.org Fri Sep 14 11:59:00 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 15:59:00 -0000 Subject: [Python-checkins] Don't run AC_STRUCT_TIMEZONE twice. (GH-9305) Message-ID: https://github.com/python/cpython/commit/a4414ef20b971e6803309acebfa85b1621ac625e commit: a4414ef20b971e6803309acebfa85b1621ac625e branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-14T08:58:57-07:00 summary: Don't run AC_STRUCT_TIMEZONE twice. (GH-9305) files: M configure M configure.ac diff --git a/configure b/configure index 22fc02153d14..7b0c734b5e25 100755 --- a/configure +++ b/configure @@ -15344,77 +15344,6 @@ $as_echo "#define HAVE_BROKEN_POLL 1" >>confdefs.h fi -# Before we can test tzset, we need to check if struct tm has a tm_zone -# (which is not required by ISO C or UNIX spec) and/or if we support -# tzname[] -ac_fn_c_check_member "$LINENO" "struct tm" "tm_zone" "ac_cv_member_struct_tm_tm_zone" "#include -#include <$ac_cv_struct_tm> - -" -if test "x$ac_cv_member_struct_tm_tm_zone" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_STRUCT_TM_TM_ZONE 1 -_ACEOF - - -fi - -if test "$ac_cv_member_struct_tm_tm_zone" = yes; then - -$as_echo "#define HAVE_TM_ZONE 1" >>confdefs.h - -else - ac_fn_c_check_decl "$LINENO" "tzname" "ac_cv_have_decl_tzname" "#include -" -if test "x$ac_cv_have_decl_tzname" = xyes; then : - ac_have_decl=1 -else - ac_have_decl=0 -fi - -cat >>confdefs.h <<_ACEOF -#define HAVE_DECL_TZNAME $ac_have_decl -_ACEOF - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for tzname" >&5 -$as_echo_n "checking for tzname... " >&6; } -if ${ac_cv_var_tzname+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -#if !HAVE_DECL_TZNAME -extern char *tzname[]; -#endif - -int -main () -{ -return tzname[0][0]; - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_var_tzname=yes -else - ac_cv_var_tzname=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_var_tzname" >&5 -$as_echo "$ac_cv_var_tzname" >&6; } - if test $ac_cv_var_tzname = yes; then - -$as_echo "#define HAVE_TZNAME 1" >>confdefs.h - - fi -fi - - # check tzset(3) exists and works like we expect it to { $as_echo "$as_me:${as_lineno-$LINENO}: checking for working tzset()" >&5 $as_echo_n "checking for working tzset()... " >&6; } diff --git a/configure.ac b/configure.ac index 03953622ea7d..3e5388ab0804 100644 --- a/configure.ac +++ b/configure.ac @@ -4744,11 +4744,6 @@ then [Define if poll() sets errno on invalid file descriptors.]) fi -# Before we can test tzset, we need to check if struct tm has a tm_zone -# (which is not required by ISO C or UNIX spec) and/or if we support -# tzname[] -AC_STRUCT_TIMEZONE - # check tzset(3) exists and works like we expect it to AC_MSG_CHECKING(for working tzset()) AC_CACHE_VAL(ac_cv_working_tzset, [ From webhook-mailer at python.org Fri Sep 14 12:09:08 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 16:09:08 -0000 Subject: [Python-checkins] bpo-34672: Try to pass the C library's own timezone strings back to it. (GH-9288) Message-ID: https://github.com/python/cpython/commit/5633c4f342d957df2ef0d67b9bfb472a0d28a76b commit: 5633c4f342d957df2ef0d67b9bfb472a0d28a76b branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-14T09:09:04-07:00 summary: bpo-34672: Try to pass the C library's own timezone strings back to it. (GH-9288) files: A Misc/NEWS.d/next/Library/2018-09-13-21-04-23.bpo-34672.BYuKKS.rst M Modules/timemodule.c diff --git a/Misc/NEWS.d/next/Library/2018-09-13-21-04-23.bpo-34672.BYuKKS.rst b/Misc/NEWS.d/next/Library/2018-09-13-21-04-23.bpo-34672.BYuKKS.rst new file mode 100644 index 000000000000..59d106b60042 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-13-21-04-23.bpo-34672.BYuKKS.rst @@ -0,0 +1,2 @@ +Add a workaround, so the ``'Z'`` :func:`time.strftime` specifier on the musl +C library can work in some cases. diff --git a/Modules/timemodule.c b/Modules/timemodule.c index dbe2fbaf0796..8118b31c9173 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -523,6 +523,10 @@ time_localtime(PyObject *self, PyObject *args) #endif } +#if defined(__linux__) && !defined(__GLIBC__) +static const char *utc_string = NULL; +#endif + PyDoc_STRVAR(localtime_doc, "localtime([seconds]) -> (tm_year,tm_mon,tm_mday,tm_hour,tm_min,\n\ tm_sec,tm_wday,tm_yday,tm_isdst)\n\ @@ -565,11 +569,32 @@ gettmarg(PyObject *args, struct tm *p, const char *format) if (Py_TYPE(args) == &StructTimeType) { PyObject *item; item = PyTuple_GET_ITEM(args, 9); - p->tm_zone = item == Py_None ? NULL : (char*)PyUnicode_AsUTF8(item); + if (item != Py_None) { + p->tm_zone = PyUnicode_AsUTF8(item); + if (p->tm_zone == NULL) { + return 0; + } +#if defined(__linux__) && !defined(__GLIBC__) + // Make an attempt to return the C library's own timezone strings to + // it. musl refuses to process a tm_zone field unless it produced + // it. See issue #34672. + if (utc_string && strcmp(p->tm_zone, utc_string) == 0) { + p->tm_zone = utc_string; + } + else if (tzname[0] && strcmp(p->tm_zone, tzname[0]) == 0) { + p->tm_zone = tzname[0]; + } + else if (tzname[1] && strcmp(p->tm_zone, tzname[1]) == 0) { + p->tm_zone = tzname[1]; + } +#endif + } item = PyTuple_GET_ITEM(args, 10); - p->tm_gmtoff = item == Py_None ? 0 : PyLong_AsLong(item); - if (PyErr_Occurred()) - return 0; + if (item != Py_None) { + p->tm_gmtoff = PyLong_AsLong(item); + if (PyErr_Occurred()) + return 0; + } } #endif /* HAVE_STRUCT_TM_TM_ZONE */ return 1; @@ -1736,6 +1761,13 @@ PyInit_time(void) PyModule_AddIntConstant(m, "_STRUCT_TM_ITEMS", 11); PyModule_AddObject(m, "struct_time", (PyObject*) &StructTimeType); initialized = 1; + +#if defined(__linux__) && !defined(__GLIBC__) + struct tm tm; + if (gmtime_r(0, &tm) != NULL) + utc_string = tm.tm_zone; +#endif + return m; } From webhook-mailer at python.org Fri Sep 14 12:22:52 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 14 Sep 2018 16:22:52 -0000 Subject: [Python-checkins] Don't run AC_STRUCT_TIMEZONE twice. (GH-9305) Message-ID: https://github.com/python/cpython/commit/c974051d737a25a4c2066d1bb2832286adb12dd9 commit: c974051d737a25a4c2066d1bb2832286adb12dd9 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-14T09:22:48-07:00 summary: Don't run AC_STRUCT_TIMEZONE twice. (GH-9305) (cherry picked from commit a4414ef20b971e6803309acebfa85b1621ac625e) Co-authored-by: Benjamin Peterson files: M configure M configure.ac diff --git a/configure b/configure index ee08da97b430..36e117a75522 100755 --- a/configure +++ b/configure @@ -15510,77 +15510,6 @@ $as_echo "#define HAVE_BROKEN_POLL 1" >>confdefs.h fi -# Before we can test tzset, we need to check if struct tm has a tm_zone -# (which is not required by ISO C or UNIX spec) and/or if we support -# tzname[] -ac_fn_c_check_member "$LINENO" "struct tm" "tm_zone" "ac_cv_member_struct_tm_tm_zone" "#include -#include <$ac_cv_struct_tm> - -" -if test "x$ac_cv_member_struct_tm_tm_zone" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_STRUCT_TM_TM_ZONE 1 -_ACEOF - - -fi - -if test "$ac_cv_member_struct_tm_tm_zone" = yes; then - -$as_echo "#define HAVE_TM_ZONE 1" >>confdefs.h - -else - ac_fn_c_check_decl "$LINENO" "tzname" "ac_cv_have_decl_tzname" "#include -" -if test "x$ac_cv_have_decl_tzname" = xyes; then : - ac_have_decl=1 -else - ac_have_decl=0 -fi - -cat >>confdefs.h <<_ACEOF -#define HAVE_DECL_TZNAME $ac_have_decl -_ACEOF - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for tzname" >&5 -$as_echo_n "checking for tzname... " >&6; } -if ${ac_cv_var_tzname+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -#if !HAVE_DECL_TZNAME -extern char *tzname[]; -#endif - -int -main () -{ -return tzname[0][0]; - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_var_tzname=yes -else - ac_cv_var_tzname=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_var_tzname" >&5 -$as_echo "$ac_cv_var_tzname" >&6; } - if test $ac_cv_var_tzname = yes; then - -$as_echo "#define HAVE_TZNAME 1" >>confdefs.h - - fi -fi - - # check tzset(3) exists and works like we expect it to { $as_echo "$as_me:${as_lineno-$LINENO}: checking for working tzset()" >&5 $as_echo_n "checking for working tzset()... " >&6; } diff --git a/configure.ac b/configure.ac index 0cd1b78cf77f..620b48a6e7b5 100644 --- a/configure.ac +++ b/configure.ac @@ -4866,11 +4866,6 @@ then [Define if poll() sets errno on invalid file descriptors.]) fi -# Before we can test tzset, we need to check if struct tm has a tm_zone -# (which is not required by ISO C or UNIX spec) and/or if we support -# tzname[] -AC_STRUCT_TIMEZONE - # check tzset(3) exists and works like we expect it to AC_MSG_CHECKING(for working tzset()) AC_CACHE_VAL(ac_cv_working_tzset, [ From webhook-mailer at python.org Fri Sep 14 12:27:52 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 16:27:52 -0000 Subject: [Python-checkins] Change the xkcd link in comment over https. (GH-9294) Message-ID: https://github.com/python/cpython/commit/bd844b0336376408d7cc3b7fd6830479cd9b3a91 commit: bd844b0336376408d7cc3b7fd6830479cd9b3a91 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Benjamin Peterson date: 2018-09-14T09:27:48-07:00 summary: Change the xkcd link in comment over https. (GH-9294) (cherry picked from commit 83df50ea5757816c7338d27f21fd18b1e79206f7) Co-authored-by: ? files: M Lib/antigravity.py diff --git a/Lib/antigravity.py b/Lib/antigravity.py index 9b1436803722..c6f174ca6d87 100644 --- a/Lib/antigravity.py +++ b/Lib/antigravity.py @@ -11,7 +11,7 @@ def geohash(latitude, longitude, datedow): 37.857713 -122.544543 ''' - # http://xkcd.com/426/ + # https://xkcd.com/426/ h = hashlib.md5(datedow).hexdigest() p, q = [('%f' % float.fromhex('0.' + x)) for x in (h[:16], h[16:32])] print('%d%s %d%s' % (latitude, p[1:], longitude, q[1:])) From webhook-mailer at python.org Fri Sep 14 12:28:10 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 16:28:10 -0000 Subject: [Python-checkins] Change the xkcd link in comment over https. (GH-9293) Message-ID: https://github.com/python/cpython/commit/bd979b2d2b30ff2bfa44bdbbaba9d7a30900102c commit: bd979b2d2b30ff2bfa44bdbbaba9d7a30900102c branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Benjamin Peterson date: 2018-09-14T09:28:07-07:00 summary: Change the xkcd link in comment over https. (GH-9293) (cherry picked from commit 83df50ea5757816c7338d27f21fd18b1e79206f7) Co-authored-by: ? files: M Lib/antigravity.py diff --git a/Lib/antigravity.py b/Lib/antigravity.py index 9b1436803722..c6f174ca6d87 100644 --- a/Lib/antigravity.py +++ b/Lib/antigravity.py @@ -11,7 +11,7 @@ def geohash(latitude, longitude, datedow): 37.857713 -122.544543 ''' - # http://xkcd.com/426/ + # https://xkcd.com/426/ h = hashlib.md5(datedow).hexdigest() p, q = [('%f' % float.fromhex('0.' + x)) for x in (h[:16], h[16:32])] print('%d%s %d%s' % (latitude, p[1:], longitude, q[1:])) From webhook-mailer at python.org Fri Sep 14 12:28:34 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 14 Sep 2018 16:28:34 -0000 Subject: [Python-checkins] Don't run AC_STRUCT_TIMEZONE twice. (GH-9305) Message-ID: https://github.com/python/cpython/commit/03d8e7c02ddf79ca4a9ef2c9674d4ee2dec7c3c2 commit: 03d8e7c02ddf79ca4a9ef2c9674d4ee2dec7c3c2 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-14T09:28:31-07:00 summary: Don't run AC_STRUCT_TIMEZONE twice. (GH-9305) (cherry picked from commit a4414ef20b971e6803309acebfa85b1621ac625e) Co-authored-by: Benjamin Peterson files: M configure M configure.ac diff --git a/configure b/configure index 1d93d683a6c4..bfa4b83cd6c0 100755 --- a/configure +++ b/configure @@ -15700,77 +15700,6 @@ $as_echo "#define HAVE_BROKEN_POLL 1" >>confdefs.h fi -# Before we can test tzset, we need to check if struct tm has a tm_zone -# (which is not required by ISO C or UNIX spec) and/or if we support -# tzname[] -ac_fn_c_check_member "$LINENO" "struct tm" "tm_zone" "ac_cv_member_struct_tm_tm_zone" "#include -#include <$ac_cv_struct_tm> - -" -if test "x$ac_cv_member_struct_tm_tm_zone" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_STRUCT_TM_TM_ZONE 1 -_ACEOF - - -fi - -if test "$ac_cv_member_struct_tm_tm_zone" = yes; then - -$as_echo "#define HAVE_TM_ZONE 1" >>confdefs.h - -else - ac_fn_c_check_decl "$LINENO" "tzname" "ac_cv_have_decl_tzname" "#include -" -if test "x$ac_cv_have_decl_tzname" = xyes; then : - ac_have_decl=1 -else - ac_have_decl=0 -fi - -cat >>confdefs.h <<_ACEOF -#define HAVE_DECL_TZNAME $ac_have_decl -_ACEOF - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for tzname" >&5 -$as_echo_n "checking for tzname... " >&6; } -if ${ac_cv_var_tzname+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -#if !HAVE_DECL_TZNAME -extern char *tzname[]; -#endif - -int -main () -{ -return tzname[0][0]; - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_var_tzname=yes -else - ac_cv_var_tzname=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_var_tzname" >&5 -$as_echo "$ac_cv_var_tzname" >&6; } - if test $ac_cv_var_tzname = yes; then - -$as_echo "#define HAVE_TZNAME 1" >>confdefs.h - - fi -fi - - # check tzset(3) exists and works like we expect it to { $as_echo "$as_me:${as_lineno-$LINENO}: checking for working tzset()" >&5 $as_echo_n "checking for working tzset()... " >&6; } diff --git a/configure.ac b/configure.ac index 7ec62c2dd7eb..3f2459ab7100 100644 --- a/configure.ac +++ b/configure.ac @@ -4904,11 +4904,6 @@ then [Define if poll() sets errno on invalid file descriptors.]) fi -# Before we can test tzset, we need to check if struct tm has a tm_zone -# (which is not required by ISO C or UNIX spec) and/or if we support -# tzname[] -AC_STRUCT_TIMEZONE - # check tzset(3) exists and works like we expect it to AC_MSG_CHECKING(for working tzset()) AC_CACHE_VAL(ac_cv_working_tzset, [ From webhook-mailer at python.org Fri Sep 14 12:53:15 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 16:53:15 -0000 Subject: [Python-checkins] [2.7] Don't run AC_STRUCT_TIMEZONE twice. (GH-9309) Message-ID: https://github.com/python/cpython/commit/857550311595ebcb142101867666ae72caabacb1 commit: 857550311595ebcb142101867666ae72caabacb1 branch: 2.7 author: Benjamin Peterson committer: GitHub date: 2018-09-14T09:53:09-07:00 summary: [2.7] Don't run AC_STRUCT_TIMEZONE twice. (GH-9309) (cherry picked from commit a4414ef20b971e6803309acebfa85b1621ac625e) files: M configure M configure.ac diff --git a/configure b/configure index 505f8bbc045d..ea8527fea18b 100755 --- a/configure +++ b/configure @@ -14379,77 +14379,6 @@ $as_echo "#define HAVE_BROKEN_POLL 1" >>confdefs.h fi -# Before we can test tzset, we need to check if struct tm has a tm_zone -# (which is not required by ISO C or UNIX spec) and/or if we support -# tzname[] -ac_fn_c_check_member "$LINENO" "struct tm" "tm_zone" "ac_cv_member_struct_tm_tm_zone" "#include -#include <$ac_cv_struct_tm> - -" -if test "x$ac_cv_member_struct_tm_tm_zone" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_STRUCT_TM_TM_ZONE 1 -_ACEOF - - -fi - -if test "$ac_cv_member_struct_tm_tm_zone" = yes; then - -$as_echo "#define HAVE_TM_ZONE 1" >>confdefs.h - -else - ac_fn_c_check_decl "$LINENO" "tzname" "ac_cv_have_decl_tzname" "#include -" -if test "x$ac_cv_have_decl_tzname" = xyes; then : - ac_have_decl=1 -else - ac_have_decl=0 -fi - -cat >>confdefs.h <<_ACEOF -#define HAVE_DECL_TZNAME $ac_have_decl -_ACEOF - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for tzname" >&5 -$as_echo_n "checking for tzname... " >&6; } -if ${ac_cv_var_tzname+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -#if !HAVE_DECL_TZNAME -extern char *tzname[]; -#endif - -int -main () -{ -return tzname[0][0]; - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_var_tzname=yes -else - ac_cv_var_tzname=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_var_tzname" >&5 -$as_echo "$ac_cv_var_tzname" >&6; } - if test $ac_cv_var_tzname = yes; then - -$as_echo "#define HAVE_TZNAME 1" >>confdefs.h - - fi -fi - - # check tzset(3) exists and works like we expect it to { $as_echo "$as_me:${as_lineno-$LINENO}: checking for working tzset()" >&5 $as_echo_n "checking for working tzset()... " >&6; } diff --git a/configure.ac b/configure.ac index 33870961b3a8..5dfb824b93f1 100644 --- a/configure.ac +++ b/configure.ac @@ -4413,11 +4413,6 @@ then [Define if poll() sets errno on invalid file descriptors.]) fi -# Before we can test tzset, we need to check if struct tm has a tm_zone -# (which is not required by ISO C or UNIX spec) and/or if we support -# tzname[] -AC_STRUCT_TIMEZONE - # check tzset(3) exists and works like we expect it to AC_MSG_CHECKING(for working tzset()) AC_CACHE_VAL(ac_cv_working_tzset, [ From webhook-mailer at python.org Fri Sep 14 13:06:58 2018 From: webhook-mailer at python.org (Carol Willing) Date: Fri, 14 Sep 2018 17:06:58 -0000 Subject: [Python-checkins] bpo-33649 Polish asyncio docs on queues, protocols, and subproccesses (#9306) Message-ID: https://github.com/python/cpython/commit/c9d66f0ed4f07b9d184d22abbfdd4c3c8e2702df commit: c9d66f0ed4f07b9d184d22abbfdd4c3c8e2702df branch: master author: Carol Willing committer: GitHub date: 2018-09-14T10:06:55-07:00 summary: bpo-33649 Polish asyncio docs on queues, protocols, and subproccesses (#9306) * small clarification * edits to protocols doc * Edit async queue doc files: M Doc/library/asyncio-protocol.rst M Doc/library/asyncio-queue.rst diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst index 84a275e0d0b6..d7ecb25bcea3 100644 --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -29,7 +29,7 @@ abstraction for a socket (or similar I/O endpoint) while a protocol is an abstraction for an application, from the transport's point of view. -Yet another view is simply that the transport and protocol interfaces +Yet another view is the transport and protocol interfaces together define an abstract interface for using network I/O and interprocess I/O. @@ -109,7 +109,7 @@ Transports Hierarchy Interface representing a bidirectional transport, such as a TCP connection. - The user never instantiates a transport directly; they call a + The user does not instantiate a transport directly; they call a utility function, passing it a protocol factory and other information necessary to create the transport and protocol. @@ -388,7 +388,7 @@ Subprocess Transports .. method:: SubprocessTransport.get_returncode() Return the subprocess return code as an integer or :const:`None` - if it hasn't returned, similarly to the + if it hasn't returned, which is similar to the :attr:`subprocess.Popen.returncode` attribute. .. method:: SubprocessTransport.kill() @@ -427,11 +427,10 @@ asyncio provides a set of abstract base classes that should be used to implement network protocols. Those classes are meant to be used together with :ref:`transports `. -Subclasses of abstract base protocol classes can implement some or -all methods. All those methods are callbacks: they are called by +Subclasses of abstract base protocol classes may implement some or +all methods. All these methods are callbacks: they are called by transports on certain events, for example when some data is received. -Base protocol methods are not supposed to be called by anything but -the corresponding transport. +A base protocol method should be called by the corresponding transport. Base Protocols @@ -531,7 +530,7 @@ accept factories that return streaming protocols. Whether the data is buffered, chunked or reassembled depends on the transport. In general, you shouldn't rely on specific semantics - and instead make your parsing generic and flexible enough. However, + and instead make your parsing generic and flexible. However, data is always received in the correct order. The method can be called an arbitrary number of times during @@ -551,12 +550,12 @@ accept factories that return streaming protocols. This method may return a false value (including ``None``), in which case the transport will close itself. Conversely, if this method returns a - true value, closing the transport is up to the protocol. Since the - default implementation returns ``None``, it implicitly closes the + true value, the protocol used determines whether to close the transport. + Since the default implementation returns ``None``, it implicitly closes the connection. Some transports such as SSL don't support half-closed connections, - in which case returning true from this method will not prevent closing + in which case returning true from this method will result in closing the connection. @@ -581,8 +580,8 @@ Buffered Streaming Protocols Buffered Protocols can be used with any event loop method that supports `Streaming Protocols`_. -The idea of ``BufferedProtocol`` is that it allows to manually allocate -and control the receive buffer. Event loops can then use the buffer +The idea of ``BufferedProtocol`` is that it allows manual allocation +and control of the receive buffer. Event loops can then use the buffer provided by the protocol to avoid unnecessary data copies. This can result in noticeable performance improvement for protocols that receive big amounts of data. Sophisticated protocols implementations @@ -658,10 +657,10 @@ factories passed to the :meth:`loop.create_datagram_endpoint` method. .. note:: On BSD systems (macOS, FreeBSD, etc.) flow control is not supported - for datagram protocols, because send failures caused by - writing too many packets cannot be detected easily. + for datagram protocols, because it is difficult to detect easily send + failures caused by writing too many packets. - The socket always appears 'ready' and excess packets are dropped; an + The socket always appears 'ready' and excess packets are dropped. An :class:`OSError` with ``errno`` set to :const:`errno.ENOBUFS` may or may not be raised; if it is raised, it will be reported to :meth:`DatagramProtocol.error_received` but otherwise ignored. @@ -705,8 +704,8 @@ Examples TCP Echo Server --------------- -TCP echo server using the :meth:`loop.create_server` method, send back -received data and close the connection:: +Create a TCP echo server using the :meth:`loop.create_server` method, send back +received data, and close the connection:: import asyncio @@ -754,8 +753,8 @@ received data and close the connection:: TCP Echo Client --------------- -TCP echo client using the :meth:`loop.create_connection` method, send -data and wait until the connection is closed:: +A TCP echo client using the :meth:`loop.create_connection` method, sends +data, and waits until the connection is closed:: import asyncio @@ -812,8 +811,8 @@ data and wait until the connection is closed:: UDP Echo Server --------------- -UDP echo server using the :meth:`loop.create_datagram_endpoint` -method, send back received data:: +A UDP echo server, using the :meth:`loop.create_datagram_endpoint` +method, sends back received data:: import asyncio @@ -856,8 +855,8 @@ method, send back received data:: UDP Echo Client --------------- -UDP echo client using the :meth:`loop.create_datagram_endpoint` -method, send data and close the transport when we received the answer:: +A UDP echo client, using the :meth:`loop.create_datagram_endpoint` +method, sends data and closes the transport when it receives the answer:: import asyncio @@ -978,7 +977,7 @@ Wait until a socket receives data using the loop.subprocess_exec() and SubprocessProtocol --------------------------------------------- -An example of a subprocess protocol using to get the output of a +An example of a subprocess protocol used to get the output of a subprocess and to wait for the subprocess exit. The subprocess is created by th :meth:`loop.subprocess_exec` method:: diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index 1e4470adee98..a910dc73ab26 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -60,7 +60,7 @@ Queue .. coroutinemethod:: join() - Block until all items in the queue have been gotten and processed. + Block until all items in the queue have been received and processed. The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls From webhook-mailer at python.org Fri Sep 14 13:13:14 2018 From: webhook-mailer at python.org (Petr Viktorin) Date: Fri, 14 Sep 2018 17:13:14 -0000 Subject: [Python-checkins] Fix "Python" casing in a few places (GH-9001) Message-ID: https://github.com/python/cpython/commit/271818fe279df5ab292789f97c3a52c477bd8f13 commit: 271818fe279df5ab292789f97c3a52c477bd8f13 branch: master author: Andr?s Delfino committer: Petr Viktorin date: 2018-09-14T10:13:09-07:00 summary: Fix "Python" casing in a few places (GH-9001) files: M Doc/howto/descriptor.rst M Doc/howto/instrumentation.rst M Doc/library/email.compat32-message.rst M Doc/library/idle.rst M Doc/library/imaplib.rst M Doc/library/inspect.rst M Doc/library/pyclbr.rst M Doc/library/syslog.rst M Doc/library/test.rst M Doc/library/unittest.mock-examples.rst M Doc/library/unittest.mock.rst M Doc/library/zipapp.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 7b00d947f46b..672324b17238 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -11,7 +11,7 @@ Abstract -------- Defines descriptors, summarizes the protocol, and shows how descriptors are -called. Examines a custom descriptor and several built-in python descriptors +called. Examines a custom descriptor and several built-in Python descriptors including functions, properties, static methods, and class methods. Shows how each works by giving a pure Python equivalent and a sample application. @@ -275,7 +275,7 @@ variable name. To support method calls, functions include the :meth:`__get__` method for binding methods during attribute access. This means that all functions are non-data descriptors which return bound methods when they are invoked from an -object. In pure python, it works like this:: +object. In pure Python, it works like this:: class Function(object): . . . diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst index b63c43c81f71..50cde3595034 100644 --- a/Doc/howto/instrumentation.rst +++ b/Doc/howto/instrumentation.rst @@ -369,13 +369,13 @@ available: .. c:function:: python.function.entry(str filename, str funcname, int lineno, frameptr) This probe point indicates that execution of a Python function has begun. - It is only triggered for pure-python (bytecode) functions. + It is only triggered for pure-Python (bytecode) functions. .. c:function:: python.function.return(str filename, str funcname, int lineno, frameptr) This probe point is the converse of :c:func:`python.function.return`, and indicates that execution of a Python function has ended (either via - ``return``, or via an exception). It is only triggered for pure-python + ``return``, or via an exception). It is only triggered for pure-Python (bytecode) functions. diff --git a/Doc/library/email.compat32-message.rst b/Doc/library/email.compat32-message.rst index 6f8e489a7328..2e189dccdb1a 100644 --- a/Doc/library/email.compat32-message.rst +++ b/Doc/library/email.compat32-message.rst @@ -5,7 +5,7 @@ .. module:: email.message :synopsis: The base class representing email messages in a fashion - backward compatible with python3.2 + backward compatible with Python 3.2 The :class:`Message` class is very similar to the diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 58e61931aaf4..0eb1b44f8022 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -303,7 +303,7 @@ Python Docs and open docs.python.org showing the latest Python documentation. Turtle Demo - Run the turtledemo module with example python code and turtle drawings. + Run the turtledemo module with example Python code and turtle drawings. Additional help sources may be added here with the Configure IDLE dialog under the General tab. diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index 2e2c59c9ce00..040dab6e9f7a 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -517,7 +517,7 @@ An :class:`IMAP4` instance has the following methods: create such tags. Although it is an RFC violation and IMAP clients and servers are supposed to be strict, imaplib nonetheless continues to allow such tags to be created for backward compatibility reasons, and as of - python 3.6, handles them if they are sent from the server, since this + Python 3.6, handles them if they are sent from the server, since this improves real-world compatibility. .. method:: IMAP4.subscribe(mailbox) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index e33876b98c1d..523a5f34179b 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -557,7 +557,7 @@ function. >>> sig.parameters['b'].annotation - Accepts a wide range of python callables, from plain functions and classes to + Accepts a wide range of Python callables, from plain functions and classes to :func:`functools.partial` objects. Raises :exc:`ValueError` if no signature can be provided, and diff --git a/Doc/library/pyclbr.rst b/Doc/library/pyclbr.rst index ea34dd0638ca..a70c8df6a7b1 100644 --- a/Doc/library/pyclbr.rst +++ b/Doc/library/pyclbr.rst @@ -11,9 +11,9 @@ -------------- The :mod:`pyclbr` module provides limited information about the -functions, classes, and methods defined in a python-coded module. The +functions, classes, and methods defined in a Python-coded module. The information is sufficient to implement a module browser. The -information is extracted from the python source code rather than by +information is extracted from the Python source code rather than by importing the module, so this module is safe to use with untrusted code. This restriction makes it impossible to use this module with modules not implemented in Python, including all standard and optional extension diff --git a/Doc/library/syslog.rst b/Doc/library/syslog.rst index af3fb9b57f73..7151527ce57a 100644 --- a/Doc/library/syslog.rst +++ b/Doc/library/syslog.rst @@ -48,7 +48,7 @@ The module defines the following functions: .. versionchanged:: 3.2 In previous versions, keyword arguments were not allowed, and *ident* was required. The default for *ident* was dependent on the system libraries, - and often was ``python`` instead of the name of the python program file. + and often was ``python`` instead of the name of the Python program file. .. function:: closelog() diff --git a/Doc/library/test.rst b/Doc/library/test.rst index aeeed0042fce..95d7f54ba442 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -1076,7 +1076,7 @@ The :mod:`test.support` module defines the following functions: Either this method or :func:`bind_port` should be used for any tests where a server socket needs to be bound to a particular port for the duration of the test. - Which one to use depends on whether the calling code is creating a python + Which one to use depends on whether the calling code is creating a Python socket, or if an unused port needs to be provided in a constructor or passed to an external program (i.e. the ``-accept`` argument to openssl's s_server mode). Always prefer :func:`bind_port` over diff --git a/Doc/library/unittest.mock-examples.rst b/Doc/library/unittest.mock-examples.rst index 5bf3d575520e..65dee7c0eb2d 100644 --- a/Doc/library/unittest.mock-examples.rst +++ b/Doc/library/unittest.mock-examples.rst @@ -392,7 +392,7 @@ You can stack up multiple patch decorators using this pattern: >>> MyTest('test_something').test_something() When you nest patch decorators the mocks are passed in to the decorated -function in the same order they applied (the normal *python* order that +function in the same order they applied (the normal *Python* order that decorators are applied). This means from the bottom up, so in the example above the mock for ``test_module.ClassName2`` is passed in first. diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index d1b18d08f797..06009e4a0971 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -98,7 +98,7 @@ mock (or other object) during the test and restored when the test ends: .. note:: When you nest patch decorators the mocks are passed in to the decorated - function in the same order they applied (the normal *python* order that + function in the same order they applied (the normal *Python* order that decorators are applied). This means from the bottom up, so in the example above the mock for ``module.ClassName1`` is passed in first. diff --git a/Doc/library/zipapp.rst b/Doc/library/zipapp.rst index 26b0f19fdeb6..1c45b75955da 100644 --- a/Doc/library/zipapp.rst +++ b/Doc/library/zipapp.rst @@ -1,8 +1,8 @@ -:mod:`zipapp` --- Manage executable python zip archives +:mod:`zipapp` --- Manage executable Python zip archives ======================================================= .. module:: zipapp - :synopsis: Manage executable python zip archives + :synopsis: Manage executable Python zip archives .. versionadded:: 3.5 From webhook-mailer at python.org Fri Sep 14 13:39:16 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 17:39:16 -0000 Subject: [Python-checkins] bpo-34672: Don't pass NULL to gmtime_r. (GH-9312) Message-ID: https://github.com/python/cpython/commit/b93062b7fbc965cd0d522f597ed51eb4e493dfc2 commit: b93062b7fbc965cd0d522f597ed51eb4e493dfc2 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-14T10:39:13-07:00 summary: bpo-34672: Don't pass NULL to gmtime_r. (GH-9312) files: M Modules/timemodule.c diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 8118b31c9173..1a4cff23d65e 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1764,7 +1764,8 @@ PyInit_time(void) #if defined(__linux__) && !defined(__GLIBC__) struct tm tm; - if (gmtime_r(0, &tm) != NULL) + const time_t zero = 0; + if (gmtime_r(&zero, &tm) != NULL) utc_string = tm.tm_zone; #endif From webhook-mailer at python.org Fri Sep 14 13:47:18 2018 From: webhook-mailer at python.org (Eric V. Smith) Date: Fri, 14 Sep 2018 17:47:18 -0000 Subject: [Python-checkins] bpo-34363: dataclasses.asdict() and .astuple() now handle fields which are namedtuples. (GH-9151) (GH-9304) Message-ID: https://github.com/python/cpython/commit/78aa3d8f5204bc856d7b2eb67288cf90c6a30660 commit: 78aa3d8f5204bc856d7b2eb67288cf90c6a30660 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Eric V. Smith date: 2018-09-14T13:47:14-04:00 summary: bpo-34363: dataclasses.asdict() and .astuple() now handle fields which are namedtuples. (GH-9151) (GH-9304) (cherry picked from commit 9b9d97dd139a799d28ff8bc90d118b1cac190b03) Co-authored-by: Eric V. Smith files: A Misc/NEWS.d/next/Library/2018-09-10-21-09-34.bpo-34363.YuSb0T.rst M Lib/dataclasses.py M Lib/test/test_dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index a43d07693ac3..28e9f75127b1 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -1026,11 +1026,36 @@ def _asdict_inner(obj, dict_factory): value = _asdict_inner(getattr(obj, f.name), dict_factory) result.append((f.name, value)) return dict_factory(result) + elif isinstance(obj, tuple) and hasattr(obj, '_fields'): + # obj is a namedtuple. Recurse into it, but the returned + # object is another namedtuple of the same type. This is + # similar to how other list- or tuple-derived classes are + # treated (see below), but we just need to create them + # differently because a namedtuple's __init__ needs to be + # called differently (see bpo-34363). + + # I'm not using namedtuple's _asdict() + # method, because: + # - it does not recurse in to the namedtuple fields and + # convert them to dicts (using dict_factory). + # - I don't actually want to return a dict here. The the main + # use case here is json.dumps, and it handles converting + # namedtuples to lists. Admittedly we're losing some + # information here when we produce a json list instead of a + # dict. Note that if we returned dicts here instead of + # namedtuples, we could no longer call asdict() on a data + # structure where a namedtuple was used as a dict key. + + return type(obj)(*[_asdict_inner(v, dict_factory) for v in obj]) elif isinstance(obj, (list, tuple)): + # Assume we can create an object of this type by passing in a + # generator (which is not true for namedtuples, handled + # above). return type(obj)(_asdict_inner(v, dict_factory) for v in obj) elif isinstance(obj, dict): - return type(obj)((_asdict_inner(k, dict_factory), _asdict_inner(v, dict_factory)) - for k, v in obj.items()) + return type(obj)((_asdict_inner(k, dict_factory), + _asdict_inner(v, dict_factory)) + for k, v in obj.items()) else: return copy.deepcopy(obj) @@ -1066,7 +1091,18 @@ def _astuple_inner(obj, tuple_factory): value = _astuple_inner(getattr(obj, f.name), tuple_factory) result.append(value) return tuple_factory(result) + elif isinstance(obj, tuple) and hasattr(obj, '_fields'): + # obj is a namedtuple. Recurse into it, but the returned + # object is another namedtuple of the same type. This is + # similar to how other list- or tuple-derived classes are + # treated (see below), but we just need to create them + # differently because a namedtuple's __init__ needs to be + # called differently (see bpo-34363). + return type(obj)(*[_astuple_inner(v, tuple_factory) for v in obj]) elif isinstance(obj, (list, tuple)): + # Assume we can create an object of this type by passing in a + # generator (which is not true for namedtuples, handled + # above). return type(obj)(_astuple_inner(v, tuple_factory) for v in obj) elif isinstance(obj, dict): return type(obj)((_astuple_inner(k, tuple_factory), _astuple_inner(v, tuple_factory)) diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index 4c93513956a2..6efe785bc328 100755 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -1429,6 +1429,70 @@ class C: self.assertEqual(d, OrderedDict([('x', 42), ('y', 2)])) self.assertIs(type(d), OrderedDict) + def test_helper_asdict_namedtuple(self): + T = namedtuple('T', 'a b c') + @dataclass + class C: + x: str + y: T + c = C('outer', T(1, C('inner', T(11, 12, 13)), 2)) + + d = asdict(c) + self.assertEqual(d, {'x': 'outer', + 'y': T(1, + {'x': 'inner', + 'y': T(11, 12, 13)}, + 2), + } + ) + + # Now with a dict_factory. OrderedDict is convenient, but + # since it compares to dicts, we also need to have separate + # assertIs tests. + d = asdict(c, dict_factory=OrderedDict) + self.assertEqual(d, {'x': 'outer', + 'y': T(1, + {'x': 'inner', + 'y': T(11, 12, 13)}, + 2), + } + ) + + # Make sure that the returned dicts are actuall OrderedDicts. + self.assertIs(type(d), OrderedDict) + self.assertIs(type(d['y'][1]), OrderedDict) + + def test_helper_asdict_namedtuple_key(self): + # Ensure that a field that contains a dict which has a + # namedtuple as a key works with asdict(). + + @dataclass + class C: + f: dict + T = namedtuple('T', 'a') + + c = C({T('an a'): 0}) + + self.assertEqual(asdict(c), {'f': {T(a='an a'): 0}}) + + def test_helper_asdict_namedtuple_derived(self): + class T(namedtuple('Tbase', 'a')): + def my_a(self): + return self.a + + @dataclass + class C: + f: T + + t = T(6) + c = C(t) + + d = asdict(c) + self.assertEqual(d, {'f': T(a=6)}) + # Make sure that t has been copied, not used directly. + self.assertIsNot(d['f'], t) + self.assertEqual(d['f'].my_a(), 6) + def test_helper_astuple(self): # Basic tests for astuple(), it should return a new tuple. @dataclass @@ -1541,6 +1605,21 @@ def nt(lst): self.assertEqual(t, NT(42, 2)) self.assertIs(type(t), NT) + def test_helper_astuple_namedtuple(self): + T = namedtuple('T', 'a b c') + @dataclass + class C: + x: str + y: T + c = C('outer', T(1, C('inner', T(11, 12, 13)), 2)) + + t = astuple(c) + self.assertEqual(t, ('outer', T(1, ('inner', (11, 12, 13)), 2))) + + # Now, using a tuple_factory. list is convenient here. + t = astuple(c, tuple_factory=list) + self.assertEqual(t, ['outer', T(1, ['inner', T(11, 12, 13)], 2)]) + def test_dynamic_class_creation(self): cls_dict = {'__annotations__': {'x': int, 'y': int}, } diff --git a/Misc/NEWS.d/next/Library/2018-09-10-21-09-34.bpo-34363.YuSb0T.rst b/Misc/NEWS.d/next/Library/2018-09-10-21-09-34.bpo-34363.YuSb0T.rst new file mode 100644 index 000000000000..5691efb1e276 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-10-21-09-34.bpo-34363.YuSb0T.rst @@ -0,0 +1 @@ +dataclasses.asdict() and .astuple() now handle namedtuples correctly. From webhook-mailer at python.org Fri Sep 14 13:48:53 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 17:48:53 -0000 Subject: [Python-checkins] closes bpo-28955: Clarified comparisons between NaN and number in reference documentation (GH-5982) Message-ID: https://github.com/python/cpython/commit/ad8a0004206ba7aec5a8a60fce413da718080db2 commit: ad8a0004206ba7aec5a8a60fce413da718080db2 branch: master author: Tony Flury committer: Benjamin Peterson date: 2018-09-14T10:48:50-07:00 summary: closes bpo-28955: Clarified comparisons between NaN and number in reference documentation (GH-5982) Co-authored-by: Benjamin Peterson files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 33f7575a8fce..f4b16182829d 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1336,12 +1336,11 @@ built-in types. involved, they compare mathematically (algorithmically) correct without loss of precision. - The not-a-number values :const:`float('NaN')` and :const:`Decimal('NaN')` - are special. They are identical to themselves (``x is x`` is true) but - are not equal to themselves (``x == x`` is false). Additionally, - comparing any number to a not-a-number value - will return ``False``. For example, both ``3 < float('NaN')`` and - ``float('NaN') < 3`` will return ``False``. + The not-a-number values ``float('NaN')`` and ``decimal.Decimal('NaN')`` are + special. Any ordered comparison of a number to a not-a-number value is false. + A counter-intuitive implication is that not-a-number values are not equal to + themselves. For example, if ``x = float('NaN')``, ``3 < x``, ``x < 3``, ``x + == x``, ``x != x`` are all false. This behavior is compliant with IEEE 754. * Binary sequences (instances of :class:`bytes` or :class:`bytearray`) can be compared within and across their types. They compare lexicographically using From webhook-mailer at python.org Fri Sep 14 14:05:46 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 14 Sep 2018 18:05:46 -0000 Subject: [Python-checkins] closes bpo-28955: Clarified comparisons between NaN and number in reference documentation (GH-5982) Message-ID: https://github.com/python/cpython/commit/ca2fa2841f8e3ce5213a0e8c0abc1d0fdc7d386b commit: ca2fa2841f8e3ce5213a0e8c0abc1d0fdc7d386b branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-14T11:05:38-07:00 summary: closes bpo-28955: Clarified comparisons between NaN and number in reference documentation (GH-5982) Co-authored-by: Benjamin Peterson (cherry picked from commit ad8a0004206ba7aec5a8a60fce413da718080db2) Co-authored-by: Tony Flury files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index ad00a837b68a..4bacac33ae7b 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1341,12 +1341,11 @@ built-in types. involved, they compare mathematically (algorithmically) correct without loss of precision. - The not-a-number values :const:`float('NaN')` and :const:`Decimal('NaN')` - are special. They are identical to themselves (``x is x`` is true) but - are not equal to themselves (``x == x`` is false). Additionally, - comparing any number to a not-a-number value - will return ``False``. For example, both ``3 < float('NaN')`` and - ``float('NaN') < 3`` will return ``False``. + The not-a-number values ``float('NaN')`` and ``decimal.Decimal('NaN')`` are + special. Any ordered comparison of a number to a not-a-number value is false. + A counter-intuitive implication is that not-a-number values are not equal to + themselves. For example, if ``x = float('NaN')``, ``3 < x``, ``x < 3``, ``x + == x``, ``x != x`` are all false. This behavior is compliant with IEEE 754. * Binary sequences (instances of :class:`bytes` or :class:`bytearray`) can be compared within and across their types. They compare lexicographically using From webhook-mailer at python.org Fri Sep 14 15:15:13 2018 From: webhook-mailer at python.org (Petr Viktorin) Date: Fri, 14 Sep 2018 19:15:13 -0000 Subject: [Python-checkins] Fix "Python" casing in a few places (GH-9001) (GH-9313) Message-ID: https://github.com/python/cpython/commit/b2ecb8b486d85d11ee7b523dfd2c76b2820bbe83 commit: b2ecb8b486d85d11ee7b523dfd2c76b2820bbe83 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Petr Viktorin date: 2018-09-14T12:15:10-07:00 summary: Fix "Python" casing in a few places (GH-9001) (GH-9313) (cherry picked from commit 271818fe279df5ab292789f97c3a52c477bd8f13) Co-authored-by: Andr?s Delfino files: M Doc/howto/descriptor.rst M Doc/howto/instrumentation.rst M Doc/library/email.compat32-message.rst M Doc/library/idle.rst M Doc/library/imaplib.rst M Doc/library/inspect.rst M Doc/library/pyclbr.rst M Doc/library/syslog.rst M Doc/library/test.rst M Doc/library/unittest.mock-examples.rst M Doc/library/unittest.mock.rst M Doc/library/zipapp.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index e5412583a674..51520b720328 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -11,7 +11,7 @@ Abstract -------- Defines descriptors, summarizes the protocol, and shows how descriptors are -called. Examines a custom descriptor and several built-in python descriptors +called. Examines a custom descriptor and several built-in Python descriptors including functions, properties, static methods, and class methods. Shows how each works by giving a pure Python equivalent and a sample application. @@ -275,7 +275,7 @@ variable name. To support method calls, functions include the :meth:`__get__` method for binding methods during attribute access. This means that all functions are non-data descriptors which return bound methods when they are invoked from an -object. In pure python, it works like this:: +object. In pure Python, it works like this:: class Function(object): . . . diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst index b63c43c81f71..50cde3595034 100644 --- a/Doc/howto/instrumentation.rst +++ b/Doc/howto/instrumentation.rst @@ -369,13 +369,13 @@ available: .. c:function:: python.function.entry(str filename, str funcname, int lineno, frameptr) This probe point indicates that execution of a Python function has begun. - It is only triggered for pure-python (bytecode) functions. + It is only triggered for pure-Python (bytecode) functions. .. c:function:: python.function.return(str filename, str funcname, int lineno, frameptr) This probe point is the converse of :c:func:`python.function.return`, and indicates that execution of a Python function has ended (either via - ``return``, or via an exception). It is only triggered for pure-python + ``return``, or via an exception). It is only triggered for pure-Python (bytecode) functions. diff --git a/Doc/library/email.compat32-message.rst b/Doc/library/email.compat32-message.rst index 6f8e489a7328..2e189dccdb1a 100644 --- a/Doc/library/email.compat32-message.rst +++ b/Doc/library/email.compat32-message.rst @@ -5,7 +5,7 @@ .. module:: email.message :synopsis: The base class representing email messages in a fashion - backward compatible with python3.2 + backward compatible with Python 3.2 The :class:`Message` class is very similar to the diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 58e61931aaf4..0eb1b44f8022 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -303,7 +303,7 @@ Python Docs and open docs.python.org showing the latest Python documentation. Turtle Demo - Run the turtledemo module with example python code and turtle drawings. + Run the turtledemo module with example Python code and turtle drawings. Additional help sources may be added here with the Configure IDLE dialog under the General tab. diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index 2e2c59c9ce00..040dab6e9f7a 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -517,7 +517,7 @@ An :class:`IMAP4` instance has the following methods: create such tags. Although it is an RFC violation and IMAP clients and servers are supposed to be strict, imaplib nonetheless continues to allow such tags to be created for backward compatibility reasons, and as of - python 3.6, handles them if they are sent from the server, since this + Python 3.6, handles them if they are sent from the server, since this improves real-world compatibility. .. method:: IMAP4.subscribe(mailbox) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index bc4316fabaee..300b3e76b25f 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -556,7 +556,7 @@ function. >>> sig.parameters['b'].annotation - Accepts a wide range of python callables, from plain functions and classes to + Accepts a wide range of Python callables, from plain functions and classes to :func:`functools.partial` objects. Raises :exc:`ValueError` if no signature can be provided, and diff --git a/Doc/library/pyclbr.rst b/Doc/library/pyclbr.rst index ea34dd0638ca..a70c8df6a7b1 100644 --- a/Doc/library/pyclbr.rst +++ b/Doc/library/pyclbr.rst @@ -11,9 +11,9 @@ -------------- The :mod:`pyclbr` module provides limited information about the -functions, classes, and methods defined in a python-coded module. The +functions, classes, and methods defined in a Python-coded module. The information is sufficient to implement a module browser. The -information is extracted from the python source code rather than by +information is extracted from the Python source code rather than by importing the module, so this module is safe to use with untrusted code. This restriction makes it impossible to use this module with modules not implemented in Python, including all standard and optional extension diff --git a/Doc/library/syslog.rst b/Doc/library/syslog.rst index af3fb9b57f73..7151527ce57a 100644 --- a/Doc/library/syslog.rst +++ b/Doc/library/syslog.rst @@ -48,7 +48,7 @@ The module defines the following functions: .. versionchanged:: 3.2 In previous versions, keyword arguments were not allowed, and *ident* was required. The default for *ident* was dependent on the system libraries, - and often was ``python`` instead of the name of the python program file. + and often was ``python`` instead of the name of the Python program file. .. function:: closelog() diff --git a/Doc/library/test.rst b/Doc/library/test.rst index aeeed0042fce..95d7f54ba442 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -1076,7 +1076,7 @@ The :mod:`test.support` module defines the following functions: Either this method or :func:`bind_port` should be used for any tests where a server socket needs to be bound to a particular port for the duration of the test. - Which one to use depends on whether the calling code is creating a python + Which one to use depends on whether the calling code is creating a Python socket, or if an unused port needs to be provided in a constructor or passed to an external program (i.e. the ``-accept`` argument to openssl's s_server mode). Always prefer :func:`bind_port` over diff --git a/Doc/library/unittest.mock-examples.rst b/Doc/library/unittest.mock-examples.rst index 5bf3d575520e..65dee7c0eb2d 100644 --- a/Doc/library/unittest.mock-examples.rst +++ b/Doc/library/unittest.mock-examples.rst @@ -392,7 +392,7 @@ You can stack up multiple patch decorators using this pattern: >>> MyTest('test_something').test_something() When you nest patch decorators the mocks are passed in to the decorated -function in the same order they applied (the normal *python* order that +function in the same order they applied (the normal *Python* order that decorators are applied). This means from the bottom up, so in the example above the mock for ``test_module.ClassName2`` is passed in first. diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 396853cc6125..fb325ed78314 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -98,7 +98,7 @@ mock (or other object) during the test and restored when the test ends: .. note:: When you nest patch decorators the mocks are passed in to the decorated - function in the same order they applied (the normal *python* order that + function in the same order they applied (the normal *Python* order that decorators are applied). This means from the bottom up, so in the example above the mock for ``module.ClassName1`` is passed in first. diff --git a/Doc/library/zipapp.rst b/Doc/library/zipapp.rst index 26b0f19fdeb6..1c45b75955da 100644 --- a/Doc/library/zipapp.rst +++ b/Doc/library/zipapp.rst @@ -1,8 +1,8 @@ -:mod:`zipapp` --- Manage executable python zip archives +:mod:`zipapp` --- Manage executable Python zip archives ======================================================= .. module:: zipapp - :synopsis: Manage executable python zip archives + :synopsis: Manage executable Python zip archives .. versionadded:: 3.5 From webhook-mailer at python.org Fri Sep 14 16:32:12 2018 From: webhook-mailer at python.org (Carol Willing) Date: Fri, 14 Sep 2018 20:32:12 -0000 Subject: [Python-checkins] bpo-33649: Refresh Tasks and Futures pages (#9314) Message-ID: https://github.com/python/cpython/commit/3faaa8857a42a36383bb18425444e597fc876797 commit: 3faaa8857a42a36383bb18425444e597fc876797 branch: master author: Yury Selivanov committer: Carol Willing date: 2018-09-14T13:32:07-07:00 summary: bpo-33649: Refresh Tasks and Futures pages (#9314) * bpo-33649: Refresh Tasks and Futures pages * Fixes * Fix markup files: A Doc/library/asyncio-future.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-protocol.rst M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index baa5234d3e51..e1b47d2588cd 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1429,7 +1429,7 @@ event loop:: .. seealso:: - A similar :ref:`Hello World ` + A similar :ref:`Hello World ` example created with a coroutine and the :func:`run` function. diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst new file mode 100644 index 000000000000..2c41c1c2165c --- /dev/null +++ b/Doc/library/asyncio-future.rst @@ -0,0 +1,240 @@ +.. currentmodule:: asyncio + + +======= +Futures +======= + +*Future* objects are used to bridge low-level callback-based code +with high-level async/await code. + + +Future Functions +================ + +.. function:: isfuture(obj) + + Return ``True`` if *obj* is either of: + + * an instance of :class:`asyncio.Future`, + * an instance of :class:`asyncio.Task`, + * a Future-like object with a ``_asyncio_future_blocking`` + attribute. + + .. versionadded:: 3.5 + + +.. function:: ensure_future(obj, \*, loop=None) + + Return: + + * *obj* argument as is, if *obj* is a :class:`Future`, + a :class:`Task`, or a Future-like object (:func:`isfuture` + is used for the test.) + + * a :class:`Task` object wrapping *obj*, if *obj* is a + coroutine (:func:`iscoroutine` is used for the test.) + + * a :class:`Task` object that would await on *obj*, if *obj* is an + awaitable (:func:`inspect.isawaitable` is used for the test.) + + If *obj* is neither of the above a :exc:`TypeError` is raised. + + .. important:: + + See also the :func:`create_task` function which is the + preferred way for creating new Tasks. + + .. versionchanged:: 3.5.1 + The function accepts any :term:`awaitable` object. + + +.. function:: wrap_future(future, \*, loop=None) + + Wrap a :class:`concurrent.futures.Future` object in a + :class:`asyncio.Future` object. + + +Future Object +============= + +.. class:: Future(\*, loop=None) + + A Future represents an eventual result of an asynchronous + operation. Not thread-safe. + + Future is an :term:`awaitable` object. Coroutines can await on + Future objects until they either have a result or an exception + set, or until they are cancelled. + + Typically Futures are used to enable low-level + callback-based code (e.g. in protocols implemented using asyncio + :ref:`transports `) + to interoperate with high-level async/await code. + + The rule of thumb is to never expose Future objects in user-facing + APIs, and the recommended way to create a Future object is to call + :meth:`loop.create_future`. This way alternative event loop + implementations can inject their own optimized implementations + of a Future object. + + .. versionchanged:: 3.7 + Added support for the :mod:`contextvars` module. + + .. method:: result() + + Return the result of the Future. + + If the Future is *done* and has a result set by the + :meth:`set_result` method, the result value is returned. + + If the Future is *done* and has an exception set by the + :meth:`set_exception` method, this method raises the exception. + + If the Future has been *cancelled*, this method raises + a :exc:`CancelledError` exception. + + If the Future's result isn't yet available, this method raises + a :exc:`InvalidStateError` exception. + + .. method:: set_result(result) + + Mark the Future as *done* and set its result. + + Raises a :exc:`InvalidStateError` error if the Future is + already *done*. + + .. method:: set_exception(exception) + + Mark the Future as *done* and set an exception. + + Raises a :exc:`InvalidStateError` error if the Future is + already *done*. + + .. method:: done() + + Return ``True`` if the Future is *done*. + + A Future is *done* if it was *cancelled* or if it has a result + or an exception set with :meth:`set_result` or + :meth:`set_exception` calls. + + .. method:: add_done_callback(callback, *, context=None) + + Add a callback to be run when the Future is *done*. + + The *callback* is called with the Future object as its only + argument. + + If the Future is already *done* when this method is called, + the callback is scheduled with :meth:`loop.call_soon`. + + An optional keyword-only *context* argument allows specifying a + custom :class:`contextvars.Context` for the *callback* to run in. + The current context is used when no *context* is provided. + + :func:`functools.partial` can be used to pass parameters + to the callback, e.g.:: + + # Call 'print("Future:", fut)' when "fut" is done. + fut.add_done_callback( + functools.partial(print, "Future:")) + + .. versionchanged:: 3.7 + The *context* keyword-only parameter was added. + See :pep:`567` for more details. + + .. method:: remove_done_callback(callback) + + Remove *callback* from the callbacks list. + + Returns the number of callbacks removed, which is typically 1, + unless a callback was added more than once. + + .. method:: cancel() + + Cancel the Future and schedule callbacks. + + If the Future is already *done* or *cancelled*, return ``False``. + Otherwise, change the Future's state to *cancelled*, + schedule the callbacks, and return ``True``. + + .. method:: exception() + + Return the exception that was set on this Future. + + The exception (or ``None`` if no exception was set) is + returned only if the Future is *done*. + + If the Future has been *cancelled*, this method raises a + :exc:`CancelledError` exception. + + If the Future isn't *done* yet, this method raises an + :exc:`InvalidStateError` exception. + + .. method:: get_loop() + + Return the event loop the Future object is bound to. + + .. versionadded:: 3.7 + + .. method:: cancelled() + + Return ``True`` if the Future was *cancelled*. + + +This example creates a Future object, creates and schedules an +asynchronous Task to set result for the Future, and waits until +the Future has a result:: + + async def set_after(fut, delay, value): + # Sleep for *delay* seconds. + await asyncio.sleep(delay) + + # Set *value* as a result of *fut* Future. + fut.set_result(value) + + async def main(): + # Get the current event loop. + loop = asyncio.get_running_loop() + + # Create a new Future object. + fut = loop.create_future() + + # Run "set_after()" coroutine in a parallel Task. + # We are using the low-level "loop.create_task()" API here because + # we already have a reference to the event loop at hand. + # Otherwise we could have just used "asyncio.create_task()". + loop.create_task( + set_after(fut, 1, '... world')) + + print('hello ...') + + # Wait until *fut* has a result (1 second) and print it. + print(await fut) + + asyncio.run(main()) + + +.. important:: + + The Future object was designed to mimic + :class:`concurrent.futures.Future`. Key differences include: + + - unlike asyncio Futures, :class:`concurrent.futures.Future` + instances cannot be awaited. + + - :meth:`asyncio.Future.result` and :meth:`asyncio.Future.exception` + do not accept the *timeout* argument. + + - :meth:`asyncio.Future.result` and :meth:`asyncio.Future.exception` + raise an :exc:`InvalidStateError` exception when the Future is not + *done*. + + - Callbacks registered with :meth:`asyncio.Future.add_done_callback` + are not called immediately. They are scheduled with + :meth:`loop.call_soon` instead. + + - asyncio Future is not compatible with the + :func:`concurrent.futures.wait` and + :func:`concurrent.futures.as_completed` functions. diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst index d7ecb25bcea3..e4aed647c338 100644 --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -1,6 +1,9 @@ .. currentmodule:: asyncio +.. _asyncio-transports-protocols: + + ======================== Transports and Protocols ======================== @@ -393,11 +396,13 @@ Subprocess Transports .. method:: SubprocessTransport.kill() - Kill the subprocess, as in :meth:`subprocess.Popen.kill`. + Kill the subprocess. On POSIX systems, the function sends SIGKILL to the subprocess. On Windows, this method is an alias for :meth:`terminate`. + See also :meth:`subprocess.Popen.kill`. + .. method:: SubprocessTransport.send_signal(signal) Send the *signal* number to the subprocess, as in @@ -405,17 +410,20 @@ Subprocess Transports .. method:: SubprocessTransport.terminate() - Ask the subprocess to stop, as in :meth:`subprocess.Popen.terminate`. + Stop the subprocess. On POSIX systems, this method sends SIGTERM to the subprocess. On Windows, the Windows API function TerminateProcess() is called to stop the subprocess. + See also :meth:`subprocess.Popen.terminate`. + .. method:: SubprocessTransport.close() - Kill the subprocess by calling the :meth:`kill` method - if the subprocess hasn't returned yet, and close transports of all - pipes (*stdin*, *stdout* and *stderr*). + Kill the subprocess by calling the :meth:`kill` method. + + If the subprocess hasn't returned yet, and close transports of + *stdin*, *stdout*, and *stderr* pipes. .. _asyncio-protocol: diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 9f4433ddc339..7e09b166ebc9 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -1,96 +1,118 @@ .. currentmodule:: asyncio -Tasks and coroutines + +==================== +Coroutines and Tasks ==================== -**Source code:** :source:`Lib/asyncio/tasks.py` +This section outlines high-level asyncio APIs to work with coroutines +and Tasks. + +.. contents:: + :depth: 1 + :local: -**Source code:** :source:`Lib/asyncio/coroutines.py` .. _coroutine: Coroutines ----------- +========== -Coroutines used with :mod:`asyncio` may be implemented using the -:keyword:`async def` statement, or by using :term:`generators `. -The :keyword:`async def` type of coroutine was added in Python 3.5, and -is recommended if there is no need to support older Python versions. +Coroutines declared with async/await syntax is the preferred way of +writing asyncio applications. For example, the following snippet +of code prints "hello", waits 1 second, and prints "world":: -Generator-based coroutines should be decorated with :func:`@asyncio.coroutine -`, although this is not strictly enforced. -The decorator enables compatibility with :keyword:`async def` coroutines, -and also serves as documentation. Generator-based -coroutines use the ``yield from`` syntax introduced in :pep:`380`, -instead of the original ``yield`` syntax. + >>> import asyncio -The word "coroutine", like the word "generator", is used for two -different (though related) concepts: + >>> async def main(): + ... print('hello') + ... await asyncio.sleep(1) + ... print('world') -- The function that defines a coroutine - (a function definition using :keyword:`async def` or - decorated with ``@asyncio.coroutine``). If disambiguation is needed - we will call this a *coroutine function* (:func:`iscoroutinefunction` - returns ``True``). + >>> asyncio.run(main()) + hello + world -- The object obtained by calling a coroutine function. This object - represents a computation or an I/O operation (usually a combination) - that will complete eventually. If disambiguation is needed we will - call it a *coroutine object* (:func:`iscoroutine` returns ``True``). +Note that simply calling a coroutine will not schedule it to +be executed:: -Things a coroutine can do: + >>> main() + -- ``result = await future`` or ``result = yield from future`` -- - suspends the coroutine until the - future is done, then returns the future's result, or raises an - exception, which will be propagated. (If the future is cancelled, - it will raise a ``CancelledError`` exception.) Note that tasks are - futures, and everything said about futures also applies to tasks. +To actually run a coroutine asyncio provides three main mechanisms: -- ``result = await coroutine`` or ``result = yield from coroutine`` -- - wait for another coroutine to - produce a result (or raise an exception, which will be propagated). - The ``coroutine`` expression must be a *call* to another coroutine. +* By using the :func:`asyncio.run` function to run the top-level + entry point "main()" function (see the above example.) -- ``return expression`` -- produce a result to the coroutine that is - waiting for this one using :keyword:`await` or ``yield from``. +* By awaiting on a coroutine. The following snippet of code will + print "hello" after waiting for 1 second, and then print "world" + after waiting for *another* 2 seconds:: -- ``raise exception`` -- raise an exception in the coroutine that is - waiting for this one using :keyword:`await` or ``yield from``. + import asyncio + import time -Calling a coroutine does not start its code running -- -the coroutine object returned by the call doesn't do anything until you -schedule its execution. There are two basic ways to start it running: -call ``await coroutine`` or ``yield from coroutine`` from another coroutine -(assuming the other coroutine is already running!), or schedule its execution -using the :func:`ensure_future` function or the :meth:`loop.create_task` -method. + async def say_after(delay, what): + await asyncio.sleep(delay) + print(what) + async def main(): + print('started at', time.strftime('%X')) -Coroutines (and tasks) can only run when the event loop is running. + await say_after(1, 'hello') + await say_after(2, 'world') -.. decorator:: coroutine + print('finished at', time.strftime('%X')) - Decorator to mark generator-based coroutines. This enables - the generator use :keyword:`!yield from` to call :keyword:`async - def` coroutines, and also enables the generator to be called by - :keyword:`async def` coroutines, for instance using an - :keyword:`await` expression. + asyncio.run(main()) - There is no need to decorate :keyword:`async def` coroutines themselves. + Expected output:: - If the generator is not yielded from before it is destroyed, an error - message is logged. See :ref:`Detect coroutines never scheduled - `. + started at 17:13:52 + hello + world + finished at 17:13:55 -.. note:: +* By using the :func:`asyncio.create_task` function to run coroutines + concurrently as asyncio :class:`Tasks `. + + Let's modify the above example and run two "set_after" coroutines + *concurrently*:: + + async def main(): + task1 = asyncio.create_task( + say_after(1, 'hello')) + + task2 = asyncio.create_task( + say_after(2, 'world')) + + print('started at', time.strftime('%X')) - In this documentation, some methods are documented as coroutines, - even if they are plain Python functions returning a :class:`Future`. - This is intentional to have a freedom of tweaking the implementation - of these functions in the future. If such a function is needed to be - used in a callback-style code, wrap its result with :func:`ensure_future`. + # Wait until both tasks are completed (should take + # around 2 seconds.) + await task1 + await task2 + print('finished at', time.strftime('%X')) + + Note that expected output now shows that the snippet runs + 1 second faster than before:: + + started at 17:14:32 + hello + world + finished at 17:14:34 + +Note that in this documentation the term "coroutine" can be used for +two closely related concepts: + +* a *coroutine function*: an :keyword:`async def` function; + +* a *coroutine object*: object returned by calling a + *coroutine function*. + + +Running an asyncio Program +========================== .. function:: run(coro, \*, debug=False) @@ -101,7 +123,7 @@ Coroutines (and tasks) can only run when the event loop is running. This function cannot be called when another asyncio event loop is running in the same thread. - If debug is True, the event loop will be run in debug mode. + If *debug* is ``True``, the event loop will be run in debug mode. This function always creates a new event loop and closes it at the end. It should be used as a main entry point for asyncio @@ -112,34 +134,41 @@ Coroutines (and tasks) can only run when the event loop is running. on a :term:`provisional basis `. -.. _asyncio-hello-world-coroutine: +Creating Tasks +============== -Example: Hello World coroutine -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. function:: create_task(coro, \*, name=None) -Example of coroutine displaying ``"Hello World"``:: + Wrap a :ref:`coroutine ` *coro* into a task and schedule + its execution. Return the task object. - import asyncio + If *name* is not ``None``, it is set as the name of the task using + :meth:`Task.set_name`. + + The task is executed in :func:`get_running_loop` context, + :exc:`RuntimeError` is raised if there is no running loop in + current thread. + + .. versionadded:: 3.7 - async def hello_world(): - print("Hello World!") + .. versionchanged:: 3.8 + Added the ``name`` parameter. - asyncio.run(hello_world()) -.. seealso:: +Sleeping +======== - The :ref:`Hello World with call_soon() ` - example uses the :meth:`loop.call_soon` method to schedule a - callback. +.. coroutinefunction:: sleep(delay, result=None, \*, loop=None) + Block for *delay* seconds. -.. _asyncio-date-coroutine: + If *result* is provided, it is returned to the caller + when the coroutine completes. -Example: Coroutine displaying the current date -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + .. _asyncio-date-coroutine: -Example of coroutine displaying the current date every second during 5 seconds -using the :meth:`sleep` function:: + Example of coroutine displaying the current date every second + during 5 seconds:: import asyncio import datetime @@ -155,418 +184,279 @@ using the :meth:`sleep` function:: asyncio.run(display_date()) -.. seealso:: - - The :ref:`display the current date with call_later() - ` example uses a callback with the - :meth:`loop.call_later` method. - - -Example: Chain coroutines -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Example chaining coroutines:: - - import asyncio - - async def compute(x, y): - print("Compute %s + %s ..." % (x, y)) - await asyncio.sleep(1.0) - return x + y - - async def print_sum(x, y): - result = await compute(x, y) - print("%s + %s = %s" % (x, y, result)) - - loop = asyncio.get_event_loop() - loop.run_until_complete(print_sum(1, 2)) - loop.close() - -``compute()`` is chained to ``print_sum()``: ``print_sum()`` coroutine waits -until ``compute()`` is completed before returning its result. - -Sequence diagram of the example: - -.. image:: tulip_coro.png - :align: center - -The "Task" is created by the :meth:`loop.run_until_complete` method -when it gets a coroutine object instead of a task. - -The diagram shows the control flow, it does not describe exactly how things -work internally. For example, the sleep coroutine creates an internal future -which uses :meth:`loop.call_later` to wake up the task in 1 second. - - -Future ------- - -.. class:: Future(\*, loop=None) - - This class is *almost* compatible with :class:`concurrent.futures.Future`. - - Differences: - - - :meth:`result` and :meth:`exception` do not take a timeout argument and - raise an exception when the future isn't done yet. - - - Callbacks registered with :meth:`add_done_callback` are always called - via the event loop's :meth:`loop.call_soon`. - - - This class is not compatible with the :func:`~concurrent.futures.wait` and - :func:`~concurrent.futures.as_completed` functions in the - :mod:`concurrent.futures` package. - - This class is :ref:`not thread safe `. - - .. method:: cancel() - - Cancel the future and schedule callbacks. - - If the future is already done or cancelled, return ``False``. Otherwise, - change the future's state to cancelled, schedule the callbacks and return - ``True``. - - .. method:: cancelled() - - Return ``True`` if the future was cancelled. - - .. method:: done() - - Return ``True`` if the future is done. - - Done means either that a result / exception are available, or that the - future was cancelled. - .. method:: result() +Running Tasks Concurrently +========================== - Return the result this future represents. +.. coroutinefunction:: gather(\*fs, loop=None, return_exceptions=False) - If the future has been cancelled, raises :exc:`CancelledError`. If the - future's result isn't yet available, raises :exc:`InvalidStateError`. If - the future is done and has an exception set, this exception is raised. + Return a Future aggregating results from the given coroutine objects, + Tasks, or Futures. - .. method:: exception() + If all Tasks/Futures are completed successfully, the result is an + aggregate list of returned values. The result values are in the + order of the original *fs* sequence. - Return the exception that was set on this future. + All coroutines in the *fs* list are automatically + wrapped in :class:`Tasks `. - The exception (or ``None`` if no exception was set) is returned only if - the future is done. If the future has been cancelled, raises - :exc:`CancelledError`. If the future isn't done yet, raises - :exc:`InvalidStateError`. + If *return_exceptions* is ``True``, exceptions in the Tasks/Futures + are treated the same as successful results, and gathered in the + result list. Otherwise, the first raised exception is immediately + propagated to the returned Future. - .. method:: add_done_callback(callback, *, context=None) + If the outer Future is *cancelled*, all submitted Tasks/Futures + (that have not completed yet) are also *cancelled*. - Add a callback to be run when the future becomes done. + If any child is *cancelled*, it is treated as if it raised + :exc:`CancelledError` -- the outer Future is **not** cancelled in + this case. This is to prevent the cancellation of one submitted + Task/Future to cause other Tasks/Futures to be cancelled. - The *callback* is called with a single argument - the future object. If the - future is already done when this is called, the callback is scheduled - with :meth:`loop.call_soon`. + All futures must share the same event loop. - An optional keyword-only *context* argument allows specifying a custom - :class:`contextvars.Context` for the *callback* to run in. The current - context is used when no *context* is provided. - - :ref:`Use functools.partial to pass parameters to the callback - `. For example, - ``fut.add_done_callback(functools.partial(print, "Future:", - flush=True))`` will call ``print("Future:", fut, flush=True)``. - - .. versionchanged:: 3.7 - The *context* keyword-only parameter was added. See :pep:`567` - for more details. - - .. method:: remove_done_callback(fn) - - Remove all instances of a callback from the "call when done" list. - - Returns the number of callbacks removed. - - .. method:: set_result(result) - - Mark the future done and set its result. - - If the future is already done when this method is called, raises - :exc:`InvalidStateError`. - - .. method:: set_exception(exception) - - Mark the future done and set an exception. - - If the future is already done when this method is called, raises - :exc:`InvalidStateError`. - - .. method:: get_loop() - - Return the event loop the future object is bound to. + .. versionchanged:: 3.7 + If the *gather* itself is cancelled, the cancellation is + propagated regardless of *return_exceptions*. - .. versionadded:: 3.7 + Example:: + import asyncio -Example: Future with run_until_complete() -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + async def factorial(name, number): + f = 1 + for i in range(2, number + 1): + print(f"Task {name}: Compute factorial({i})...") + await asyncio.sleep(1) + f *= i + print(f"Task {name}: factorial({number}) = {f}") -Example combining a :class:`Future` and a :ref:`coroutine function -`:: + async def main(): + await asyncio.gather( + factorial("A", 2), + factorial("B", 3), + factorial("C", 4), + )) - import asyncio + asyncio.run(main()) - async def slow_operation(future): - await asyncio.sleep(1) - future.set_result('Future is done!') + # Expected output: + # + # Task A: Compute factorial(2)... + # Task B: Compute factorial(2)... + # Task C: Compute factorial(2)... + # Task A: factorial(2) = 2 + # Task B: Compute factorial(3)... + # Task C: Compute factorial(3)... + # Task B: factorial(3) = 6 + # Task C: Compute factorial(4)... + # Task C: factorial(4) = 24 - loop = asyncio.get_event_loop() - future = asyncio.Future() - asyncio.ensure_future(slow_operation(future)) - loop.run_until_complete(future) - print(future.result()) - loop.close() -The coroutine function is responsible for the computation (which takes 1 second) -and it stores the result into the future. The -:meth:`loop.run_until_complete` method waits for the completion of -the future. +Shielding Tasks From Cancellation +================================= -.. note:: - The :meth:`loop.run_until_complete` method uses internally the - :meth:`~Future.add_done_callback` method to be notified when the future is - done. +.. coroutinefunction:: shield(fut, \*, loop=None) + Wait for a Future/Task while protecting it from being cancelled. -Example: Future with run_forever() -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + *fut* can be a coroutine, a Task, or a Future-like object. If + *fut* is a coroutine it is automatically wrapped in a + :class:`Task`. -The previous example can be written differently using the -:meth:`Future.add_done_callback` method to describe explicitly the control -flow:: + The statement:: - import asyncio + res = await shield(something()) - async def slow_operation(future): - await asyncio.sleep(1) - future.set_result('Future is done!') + is equivalent to the statement:: - def got_result(future): - print(future.result()) - loop.stop() + res = await something() - loop = asyncio.get_event_loop() - future = asyncio.Future() - asyncio.ensure_future(slow_operation(future)) - future.add_done_callback(got_result) - try: - loop.run_forever() - finally: - loop.close() + *except* that if the coroutine containing it is cancelled, the + Task running in ``something()`` is not cancelled. From the point + of view of ``something()``, the cancellation did not happen. + Although its caller is still cancelled, so the "await" expression + still raises :exc:`CancelledError`. -In this example, the future is used to link ``slow_operation()`` to -``got_result()``: when ``slow_operation()`` is done, ``got_result()`` is called -with the result. + If ``something()`` is cancelled by other means (i.e. from within + itself) that would also cancel ``shield()``. + If it is desired to completely ignore cancellation (not recommended) + the ``shield()`` function should be combined with a try/except + clause, as follows:: -Task ----- + try: + res = await shield(something()) + except CancelledError: + res = None -.. function:: create_task(coro, \*, name=None) - Wrap a :ref:`coroutine ` *coro* into a task and schedule - its execution. Return the task object. +Timeouts +======== - If *name* is not ``None``, it is set as the name of the task using - :meth:`Task.set_name`. +.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) - The task is executed in :func:`get_running_loop` context, - :exc:`RuntimeError` is raised if there is no running loop in - current thread. + Wait for the coroutine, Task, or Future to complete with timeout. - .. versionadded:: 3.7 + *fut* can be a coroutine, a Task, or a Future-like object. If + *fut* is a coroutine it is automatically wrapped in a + :class:`Task`. - .. versionchanged:: 3.8 - Added the ``name`` parameter. + *timeout* can either be ``None`` or a float or int number of seconds + to wait for. If *timeout* is ``None``, block until the future + completes. -.. class:: Task(coro, \*, loop=None, name=None) + If a timeout occurs, it cancels the task and raises + :exc:`asyncio.TimeoutError`. - A unit for concurrent running of :ref:`coroutines `, - subclass of :class:`Future`. + To avoid the task cancellation, wrap it in :func:`shield`. - A task is responsible for executing a coroutine object in an event loop. If - the wrapped coroutine yields from a future, the task suspends the execution - of the wrapped coroutine and waits for the completion of the future. When - the future is done, the execution of the wrapped coroutine restarts with the - result or the exception of the future. + The function will wait until the future is actually cancelled, + so the total wait time may exceed the *timeout*. - Event loops use cooperative scheduling: an event loop only runs one task at - a time. Other tasks may run in parallel if other event loops are - running in different threads. While a task waits for the completion of a - future, the event loop executes a new task. + If the wait is cancelled, the future *fut* is also cancelled. - The cancellation of a task is different from the cancellation of a - future. Calling :meth:`cancel` will throw a - :exc:`~concurrent.futures.CancelledError` to the wrapped - coroutine. :meth:`~Future.cancelled` only returns ``True`` if the - wrapped coroutine did not catch the - :exc:`~concurrent.futures.CancelledError` exception, or raised a - :exc:`~concurrent.futures.CancelledError` exception. + Example:: - If a pending task is destroyed, the execution of its wrapped :ref:`coroutine - ` did not complete. It is probably a bug and a warning is - logged: see :ref:`Pending task destroyed `. + async def eternity(): + # Sleep for one hour + await asyncio.sleep(3600) + print('yay!') - Don't directly create :class:`Task` instances: use the :func:`create_task` - function or the :meth:`loop.create_task` method. + async def main(): + # Wait for at most 1 second + try: + await asyncio.wait_for(eternity(), timeout=1.0) + except asyncio.TimeoutError: + print('timeout!') - Tasks support the :mod:`contextvars` module. When a Task - is created it copies the current context and later runs its coroutine - in the copied context. See :pep:`567` for more details. + asyncio.run(main()) - This class is :ref:`not thread safe `. + # Expected output: + # + # timeout! .. versionchanged:: 3.7 - Added support for the :mod:`contextvars` module. - - .. versionchanged:: 3.8 - Added the ``name`` parameter. + When *fut* is cancelled due to a timeout, ``wait_for`` waits + for *fut* to be cancelled. Previously, it raised + :exc:`asyncio.TimeoutError` immediately. - .. classmethod:: all_tasks(loop=None) - Return a set of all tasks for an event loop. +Waiting Primitives +================== - By default all tasks for the current event loop are returned. - If *loop* is ``None``, :func:`get_event_loop` function - is used to get the current loop. - - .. classmethod:: current_task(loop=None) - - Return the currently running task in an event loop or ``None``. - - By default the current task for the current event loop is returned. - - ``None`` is returned when called not in the context of a :class:`Task`. - - .. method:: cancel() - - Request that this task cancel itself. - - This arranges for a :exc:`~concurrent.futures.CancelledError` to be - thrown into the wrapped coroutine on the next cycle through the event - loop. The coroutine then has a chance to clean up or even deny the - request using try/except/finally. +.. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\ + return_when=ALL_COMPLETED) - Unlike :meth:`Future.cancel`, this does not guarantee that the task - will be cancelled: the exception might be caught and acted upon, delaying - cancellation of the task or preventing cancellation completely. The task - may also return a value or raise a different exception. + Wait for a set of Futures to complete. - Immediately after this method is called, :meth:`~Future.cancelled` will - not return ``True`` (unless the task was already cancelled). A task will - be marked as cancelled when the wrapped coroutine terminates with a - :exc:`~concurrent.futures.CancelledError` exception (even if - :meth:`cancel` was not called). + *fs* is a list of coroutines, Futures, and/or Tasks. Coroutines + are automatically wrapped in :class:`Tasks `. - .. method:: get_stack(\*, limit=None) + Returns two sets of Tasks/Futures: ``(done, pending)``. - Return the list of stack frames for this task's coroutine. + *timeout* (a float or int), if specified, can be used to control + the maximum number of seconds to wait before returning. - If the coroutine is not done, this returns the stack where it is - suspended. If the coroutine has completed successfully or was - cancelled, this returns an empty list. If the coroutine was - terminated by an exception, this returns the list of traceback - frames. + Note that this function does not raise :exc:`asyncio.TimeoutError`. + Futures or Tasks that aren't done when the timeout occurs are just + returned in the second set. - The frames are always ordered from oldest to newest. + *return_when* indicates when this function should return. It must + be one of the following constants: - The optional limit gives the maximum number of frames to return; by - default all available frames are returned. Its meaning differs depending - on whether a stack or a traceback is returned: the newest frames of a - stack are returned, but the oldest frames of a traceback are returned. - (This matches the behavior of the traceback module.) + .. tabularcolumns:: |l|L| - For reasons beyond our control, only one stack frame is returned for a - suspended coroutine. + +-----------------------------+----------------------------------------+ + | Constant | Description | + +=============================+========================================+ + | :const:`FIRST_COMPLETED` | The function will return when any | + | | future finishes or is cancelled. | + +-----------------------------+----------------------------------------+ + | :const:`FIRST_EXCEPTION` | The function will return when any | + | | future finishes by raising an | + | | exception. If no future raises an | + | | exception then it is equivalent to | + | | :const:`ALL_COMPLETED`. | + +-----------------------------+----------------------------------------+ + | :const:`ALL_COMPLETED` | The function will return when all | + | | futures finish or are cancelled. | + +-----------------------------+----------------------------------------+ - .. method:: print_stack(\*, limit=None, file=None) + Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the + futures when a timeout occurs. - Print the stack or traceback for this task's coroutine. + Usage:: - This produces output similar to that of the traceback module, for the - frames retrieved by get_stack(). The limit argument is passed to - get_stack(). The file argument is an I/O stream to which the output - is written; by default output is written to sys.stderr. + done, pending = await asyncio.wait(fs) - .. method:: get_name() - Return the name of the task. +.. function:: as_completed(fs, \*, loop=None, timeout=None) - If no name has been explicitly assigned to the task, the default - ``Task`` implementation generates a default name during instantiation. + Return an iterator which values, when waited for, are + :class:`Future` instances. - .. versionadded:: 3.8 + Raises :exc:`asyncio.TimeoutError` if the timeout occurs before + all Futures are done. - .. method:: set_name(value) + Example:: - Set the name of the task. + for f in as_completed(fs): + result = await f + # ... - The *value* argument can be any object, which is then converted to a - string. - In the default ``Task`` implementation, the name will be visible in the - :func:`repr` output of a task object. +Scheduling From Other Threads +============================= - .. versionadded:: 3.8 +.. function:: run_coroutine_threadsafe(coro, loop) + Submit a coroutine to the given event loop. Thread-safe. -Example: Parallel execution of tasks -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Return a :class:`concurrent.futures.Future` to access the result. -Example executing 3 tasks (A, B, C) in parallel:: + This function is meant to be called from a different OS thread + than the one where the event loop is running. Example:: - import asyncio + # Create a coroutine + coro = asyncio.sleep(1, result=3) - async def factorial(name, number): - f = 1 - for i in range(2, number+1): - print("Task %s: Compute factorial(%s)..." % (name, i)) - await asyncio.sleep(1) - f *= i - print("Task %s: factorial(%s) = %s" % (name, number, f)) + # Submit the coroutine to a given loop + future = asyncio.run_coroutine_threadsafe(coro, loop) - loop = asyncio.get_event_loop() - loop.run_until_complete(asyncio.gather( - factorial("A", 2), - factorial("B", 3), - factorial("C", 4), - )) - loop.close() + # Wait for the result with an optional timeout argument + assert future.result(timeout) == 3 -Output:: + If an exception is raised in the coroutine, the returned Future + will be notified. It can also be used to cancel the task in + the event loop:: - Task A: Compute factorial(2)... - Task B: Compute factorial(2)... - Task C: Compute factorial(2)... - Task A: factorial(2) = 2 - Task B: Compute factorial(3)... - Task C: Compute factorial(3)... - Task B: factorial(3) = 6 - Task C: Compute factorial(4)... - Task C: factorial(4) = 24 + try: + result = future.result(timeout) + except asyncio.TimeoutError: + print('The coroutine took too long, cancelling the task...') + future.cancel() + except Exception as exc: + print('The coroutine raised an exception: {!r}'.format(exc)) + else: + print('The coroutine returned: {!r}'.format(result)) -A task is automatically scheduled for execution when it is created. The event -loop stops when all tasks are done. + See the :ref:`concurrency and multithreading ` + section of the documentation. + Unlike other asyncio functions this functions requires the *loop* + argument to be passed explicitly. -Task functions --------------- + .. versionadded:: 3.5.1 -.. note:: - In the functions below, the optional *loop* argument allows explicitly setting - the event loop object used by the underlying task or coroutine. If it's - not provided, the default event loop is used. +Introspection +============= .. function:: current_task(loop=None) - Return the current running :class:`Task` instance or ``None``, if + Return the currently running :class:`Task` instance, or ``None`` if no task is running. If *loop* is ``None`` :func:`get_running_loop` is used to get @@ -577,246 +467,259 @@ Task functions .. function:: all_tasks(loop=None) - Return a set of :class:`Task` objects created for the loop. + Return a set of not yet finished :class:`Task` objects run by + the loop. If *loop* is ``None``, :func:`get_running_loop` is used for getting - current loop (contrary to the deprecated :meth:`Task.all_tasks` method - that uses :func:`get_event_loop`.) + current loop. .. versionadded:: 3.7 -.. function:: as_completed(fs, \*, loop=None, timeout=None) +Task Object +=========== - Return an iterator whose values, when waited for, are :class:`Future` - instances. +.. class:: Task(coro, \*, loop=None, name=None) - Raises :exc:`asyncio.TimeoutError` if the timeout occurs before all Futures - are done. + A :class:`Future`-like object that wraps a Python + :ref:`coroutine `. Not thread-safe. - Example:: + Tasks are used to run coroutines in event loops. + If a coroutine awaits on a Future, the Task suspends + the execution of the coroutine and waits for the completion + of the Future. When the Future is *done*, the execution of + the wrapped coroutine resumes. - for f in as_completed(fs): - result = await f # The 'await' may raise - # Use result + Event loops use cooperative scheduling: an event loop runs + one Task at a time. While a Task awaits for the completion of a + Future, the event loop runs other Tasks, callbacks, or performs + IO operations. - .. note:: + Use the high-level :func:`asyncio.create_task` function to create + Tasks, or low-level :meth:`loop.create_task` or + :func:`ensure_future` functions. Manually instantiating Task + objects is discouraged. - The futures ``f`` are not necessarily members of fs. + To cancel a running Task use the :meth:`cancel` method. Calling it + will cause the Task to throw a :exc:`CancelledError` exception into + the wrapped coroutine. If a coroutine is awaiting on a Future + object during cancellation, the Future object will be cancelled. -.. function:: ensure_future(coro_or_future, \*, loop=None) + :meth:`cancelled` can be used to check if the Task was cancelled. + The method returns ``True`` if the wrapped coroutine did not + suppress the :exc:`CancelledError` exception and was actually + cancelled. - Schedule the execution of a :ref:`coroutine object `: wrap it in - a future. Return a :class:`Task` object. + :class:`asyncio.Task` inherits from :class:`Future` all of its + APIs except :meth:`Future.set_result` and + :meth:`Future.set_exception`. - If the argument is a :class:`Future`, it is returned directly. + Tasks support the :mod:`contextvars` module. When a Task + is created it copies the current context and later runs its + coroutine in the copied context. - .. versionadded:: 3.4.4 + .. versionchanged:: 3.7 + Added support for the :mod:`contextvars` module. - .. versionchanged:: 3.5.1 - The function accepts any :term:`awaitable` object. + .. versionchanged:: 3.8 + Added the ``name`` parameter. - .. note:: + .. method:: cancel() - :func:`create_task` (added in Python 3.7) is the preferable way - for spawning new tasks. + Request the Task to be cancelled. - .. seealso:: + This arranges for a :exc:`CancelledError` exception to be thrown + into the wrapped coroutine on the next cycle of the event loop. - The :func:`create_task` function and - :meth:`loop.create_task` method. + The coroutine then has a chance to clean up or even deny the + request by suppressing the exception with a :keyword:`try` ... + ... ``except CancelledError`` ... :keyword:`finally` block. + Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does + not guarantee that the Task will be cancelled, although + suppressing cancellation completely is not common and is actively + discouraged. -.. function:: wrap_future(future, \*, loop=None) + The following example illustrates how coroutines can intercept + the cancellation request:: - Wrap a :class:`concurrent.futures.Future` object in a :class:`Future` - object. + async def cancel_me(): + print('cancel_me(): before sleep') -.. function:: gather(\*coros_or_futures, loop=None, return_exceptions=False) + try: + # Wait for 1 hour + await asyncio.sleep(3600) + except asyncio.CancelledError: + print('cancel_me(): cancel sleep') + raise + finally: + print('cancel_me(): after sleep') - Return a future aggregating results from the given coroutine objects or - futures. + async def main(): + # Create a "cancel_me" Task + task = asyncio.create_task(cancel_me()) - All futures must share the same event loop. If all the tasks are done - successfully, the returned future's result is the list of results (in the - order of the original sequence, not necessarily the order of results - arrival). If *return_exceptions* is true, exceptions in the tasks are - treated the same as successful results, and gathered in the result list; - otherwise, the first raised exception will be immediately propagated to the - returned future. + # Wait for 1 second + await asyncio.sleep(1) - Cancellation: if the outer Future is cancelled, all children (that have not - completed yet) are also cancelled. If any child is cancelled, this is - treated as if it raised :exc:`~concurrent.futures.CancelledError` -- the - outer Future is *not* cancelled in this case. (This is to prevent the - cancellation of one child to cause other children to be cancelled.) + task.cancel() + try: + await task + except asyncio.CancelledError: + print("main(): cancel_me is cancelled now") - .. versionchanged:: 3.7.0 - If the *gather* itself is cancelled, the cancellation is propagated - regardless of *return_exceptions*. + asyncio.run(main()) -.. function:: iscoroutine(obj) + # Expected output: + # + # cancel_me(): before sleep + # cancel_me(): cancel sleep + # cancel_me(): after sleep + # main(): cancel_me is cancelled now - Return ``True`` if *obj* is a :ref:`coroutine object `, - which may be based on a generator or an :keyword:`async def` coroutine. + .. method:: cancelled() -.. function:: iscoroutinefunction(func) + Return ``True`` if the Task is *cancelled*. - Return ``True`` if *func* is determined to be a :ref:`coroutine function - `, which may be a decorated generator function or an - :keyword:`async def` function. + The Task is *cancelled* when the cancellation was requested with + :meth:`cancel` and the wrapped coroutine propagated the + :exc:`CancelledError` exception thrown into it. -.. function:: run_coroutine_threadsafe(coro, loop) + .. method:: done() - Submit a :ref:`coroutine object ` to a given event loop. + Return ``True`` if the Task is *done*. - Return a :class:`concurrent.futures.Future` to access the result. + A Task is *done* when the wrapped coroutine either returned + a value, raised an exception, or the Task was cancelled. - This function is meant to be called from a different thread than the one - where the event loop is running. Usage:: + .. method:: get_stack(\*, limit=None) - # Create a coroutine - coro = asyncio.sleep(1, result=3) - # Submit the coroutine to a given loop - future = asyncio.run_coroutine_threadsafe(coro, loop) - # Wait for the result with an optional timeout argument - assert future.result(timeout) == 3 + Return the list of stack frames for this Task. - If an exception is raised in the coroutine, the returned future will be - notified. It can also be used to cancel the task in the event loop:: + If the wrapped coroutine is not done, this returns the stack + where it is suspended. If the coroutine has completed + successfully or was cancelled, this returns an empty list. + If the coroutine was terminated by an exception, this returns + the list of traceback frames. - try: - result = future.result(timeout) - except asyncio.TimeoutError: - print('The coroutine took too long, cancelling the task...') - future.cancel() - except Exception as exc: - print('The coroutine raised an exception: {!r}'.format(exc)) - else: - print('The coroutine returned: {!r}'.format(result)) + The frames are always ordered from oldest to newest. - See the :ref:`concurrency and multithreading ` - section of the documentation. + Only one stack frame is returned for a suspended coroutine. - .. note:: + The optional *limit* argument sets the maximum number of frames + to return; by default all available frames are returned. + The ordering of the returned list differs depending on whether + a stack or a traceback is returned: the newest frames of a + stack are returned, but the oldest frames of a traceback are + returned. (This matches the behavior of the traceback module.) - Unlike other functions from the module, - :func:`run_coroutine_threadsafe` requires the *loop* argument to - be passed explicitly. + .. method:: print_stack(\*, limit=None, file=None) - .. versionadded:: 3.5.1 + Print the stack or traceback for this Task. -.. coroutinefunction:: sleep(delay, result=None, \*, loop=None) + This produces output similar to that of the traceback module + for the frames retrieved by :meth:`get_stack`. - Create a :ref:`coroutine ` that completes after a given - time (in seconds). If *result* is provided, it is produced to the caller - when the coroutine completes. + The *limit* argument is passed to :meth:`get_stack` directly. - The resolution of the sleep depends on the :ref:`granularity of the event - loop `. + The *file* argument is an I/O stream to which the output + is written; by default output is written to :data:`sys.stderr`. - This function is a :ref:`coroutine `. + .. method:: get_name() -.. coroutinefunction:: shield(arg, \*, loop=None) + Return the name of the Task. - Wait for a future, shielding it from cancellation. + If no name has been explicitly assigned to the Task, the default + asyncio Task implementation generates a default name during + instantiation. - The statement:: + .. versionadded:: 3.8 - res = await shield(something()) + .. method:: set_name(value) - is exactly equivalent to the statement:: + Set the name of the Task. - res = await something() + The *value* argument can be any object, which is then + converted to a string. - *except* that if the coroutine containing it is cancelled, the task running - in ``something()`` is not cancelled. From the point of view of - ``something()``, the cancellation did not happen. But its caller is still - cancelled, so the yield-from expression still raises - :exc:`~concurrent.futures.CancelledError`. Note: If ``something()`` is - cancelled by other means this will still cancel ``shield()``. + In the default Task implementation, the name will be visible + in the :func:`repr` output of a task object. - If you want to completely ignore cancellation (not recommended) you can - combine ``shield()`` with a try/except clause, as follows:: + .. versionadded:: 3.8 - try: - res = await shield(something()) - except CancelledError: - res = None + .. classmethod:: all_tasks(loop=None) + Return a set of all tasks for an event loop. -.. coroutinefunction:: wait(futures, \*, loop=None, timeout=None,\ - return_when=ALL_COMPLETED) + By default all tasks for the current event loop are returned. + If *loop* is ``None``, the :func:`get_event_loop` function + is used to get the current loop. - Wait for the Futures and coroutine objects given by the sequence *futures* - to complete. Coroutines will be wrapped in Tasks. Returns two sets of - :class:`Future`: (done, pending). + This function is **deprecated** and scheduled for removal in + Python 3.9. Use the :func:`all_tasks` function instead. - The sequence *futures* must not be empty. + .. classmethod:: current_task(loop=None) - *timeout* can be used to control the maximum number of seconds to wait before - returning. *timeout* can be an int or float. If *timeout* is not specified - or ``None``, there is no limit to the wait time. + Return the currently running task or ``None``. - *return_when* indicates when this function should return. It must be one of - the following constants of the :mod:`concurrent.futures` module: + If *loop* is ``None``, the :func:`get_event_loop` function + is used to get the current loop. - .. tabularcolumns:: |l|L| + This function is **deprecated** and scheduled for removal in + Python 3.9. Use the :func:`current_task` function instead. - +-----------------------------+----------------------------------------+ - | Constant | Description | - +=============================+========================================+ - | :const:`FIRST_COMPLETED` | The function will return when any | - | | future finishes or is cancelled. | - +-----------------------------+----------------------------------------+ - | :const:`FIRST_EXCEPTION` | The function will return when any | - | | future finishes by raising an | - | | exception. If no future raises an | - | | exception then it is equivalent to | - | | :const:`ALL_COMPLETED`. | - +-----------------------------+----------------------------------------+ - | :const:`ALL_COMPLETED` | The function will return when all | - | | futures finish or are cancelled. | - +-----------------------------+----------------------------------------+ - Unlike :func:`~asyncio.wait_for`, ``wait()`` will not cancel the futures - when a timeout occurs. +.. _asyncio_generator_based_coro: - This function is a :ref:`coroutine `. +Generator-based Coroutines +========================== - Usage:: +.. note:: - done, pending = await asyncio.wait(fs) + Support for generator-based coroutines is **deprecated** and + scheduled for removal in Python 4.0. - .. note:: +Generator-based coroutines predate async/await syntax. They are +Python generators that use ``yield from`` expression is to await +on Futures and other coroutines. - This does not raise :exc:`asyncio.TimeoutError`! Futures that aren't done - when the timeout occurs are returned in the second set. +Generator-based coroutines should be decorated with +:func:`@asyncio.coroutine `, although this is not +enforced. -.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) +.. decorator:: coroutine - Wait for the single :class:`Future` or :ref:`coroutine object ` - to complete with timeout. If *timeout* is ``None``, block until the future - completes. + Decorator to mark generator-based coroutines. - Coroutine will be wrapped in :class:`Task`. + This decorator enables legacy generator-based coroutines to be + compatible with async/await code:: - Returns result of the Future or coroutine. When a timeout occurs, it - cancels the task and raises :exc:`asyncio.TimeoutError`. To avoid the task - cancellation, wrap it in :func:`shield`. The function will wait until - the future is actually cancelled, so the total wait time may exceed - the *timeout*. + @asyncio.coroutine + def old_style_coroutine(): + yield from asyncio.sleep(1) - If the wait is cancelled, the future *fut* is also cancelled. + async def main(): + await old_style_coroutine() + + This decorator is **deprecated** and is scheduled for removal in + Python 4.0. - This function is a :ref:`coroutine `, usage:: + This decorator should not be used for :keyword:`async def` + coroutines. - result = await asyncio.wait_for(fut, 60.0) +.. function:: iscoroutine(obj) - .. versionchanged:: 3.4.3 - If the wait is cancelled, the future *fut* is now also cancelled. + Return ``True`` if *obj* is a :ref:`coroutine object `. - .. versionchanged:: 3.7 - When *fut* is cancelled due to a timeout, ``wait_for`` now waits - for *fut* to be cancelled. Previously, - it raised :exc:`~asyncio.TimeoutError` immediately. + This method is different from :func:`inspect.iscoroutine` because + it returns ``True`` for generator-based coroutines decorated with + :func:`@coroutine `. + +.. function:: iscoroutinefunction(func) + + Return ``True`` if *func* is a :ref:`coroutine function + `. + + This method is different from :func:`inspect.iscoroutinefunction` + because it returns ``True`` for generator-based coroutine functions + decorated with :func:`@coroutine `. diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 0d58a94c8b9b..2c8ccbb390f0 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -1,15 +1,13 @@ -:mod:`asyncio` --- Asynchronous I/O, event loop, coroutines and tasks -===================================================================== +:mod:`asyncio` --- Asynchronous I/O +=================================== .. module:: asyncio - :synopsis: Asynchronous I/O, event loop, coroutines and tasks. - -.. versionadded:: 3.4 - -**Source code:** :source:`Lib/asyncio/` + :synopsis: Asynchronous I/O. -------------- +.. TODO: rewrite the introduction section + This module provides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives. @@ -44,25 +42,40 @@ programming: see the :ref:`Develop with asyncio ` page which lists common traps and explains how to avoid them. :ref:`Enable the debug mode ` during development to detect common issues. -Table of contents: +High-level APIs: .. toctree:: - :maxdepth: 3 + :maxdepth: 1 - asyncio-eventloop.rst - asyncio-policy.rst - asyncio-platforms.rst asyncio-task.rst - asyncio-protocol.rst asyncio-stream.rst - asyncio-subprocess.rst asyncio-sync.rst + asyncio-subprocess.rst asyncio-queue.rst - asyncio-dev.rst asyncio-exceptions.rst -.. seealso:: +Low-level APIs: + +.. toctree:: + :maxdepth: 1 - The :mod:`asyncio` module was designed in :PEP:`3156`. For a - motivational primer on transports and protocols, see :PEP:`3153`. + asyncio-eventloop.rst + asyncio-future.rst + asyncio-protocol.rst + asyncio-policy.rst + asyncio-platforms.rst + +Guides and Tutorials: + +.. toctree:: + :maxdepth: 1 + + asyncio-dev.rst + + +.. seealso:: + The :mod:`asyncio` module was proposed in :PEP:`3156`. + Since the acceptance of the PEP many new APIs were added and many + original APIs were altered. The PEP should be treated as a + historical document. From webhook-mailer at python.org Fri Sep 14 17:17:31 2018 From: webhook-mailer at python.org (Eric Snow) Date: Fri, 14 Sep 2018 21:17:31 -0000 Subject: [Python-checkins] bpo-34651: Only allow the main interpreter to fork. (gh-9279) Message-ID: https://github.com/python/cpython/commit/5903296045b586b9cd1fce0b1e02caf896028d1d commit: 5903296045b586b9cd1fce0b1e02caf896028d1d branch: master author: Eric Snow committer: GitHub date: 2018-09-14T14:17:20-07:00 summary: bpo-34651: Only allow the main interpreter to fork. (gh-9279) When os.fork() is called (on platforms that support it) all threads but the current one are destroyed in the child process. Consequently we must ensure that all but the associated interpreter are likewise destroyed. The main interpreter is critical for runtime operation, so we must ensure that fork only happens in the main interpreter. https://bugs.python.org/issue34651 files: A Misc/NEWS.d/next/Core and Builtins/2018-09-13-12-21-08.bpo-34651.v-bUeV.rst M Include/internal/pystate.h M Lib/test/test__xxsubinterpreters.py M Modules/_posixsubprocess.c M Modules/posixmodule.c M Python/pystate.c diff --git a/Include/internal/pystate.h b/Include/internal/pystate.h index ef83af59323e..c93dda28954a 100644 --- a/Include/internal/pystate.h +++ b/Include/internal/pystate.h @@ -218,6 +218,7 @@ PyAPI_FUNC(_PyInitError) _PyRuntime_Initialize(void); /* Other */ PyAPI_FUNC(_PyInitError) _PyInterpreterState_Enable(_PyRuntimeState *); +PyAPI_FUNC(void) _PyInterpreterState_DeleteExceptMain(void); #ifdef __cplusplus } diff --git a/Lib/test/test__xxsubinterpreters.py b/Lib/test/test__xxsubinterpreters.py index ac76cc198ad7..26032d6c85f2 100644 --- a/Lib/test/test__xxsubinterpreters.py +++ b/Lib/test/test__xxsubinterpreters.py @@ -824,23 +824,12 @@ def test_fork(self): expected = 'spam spam spam spam spam' script = dedent(f""" - # (inspired by Lib/test/test_fork.py) import os - pid = os.fork() - if pid == 0: # child + try: + os.fork() + except RuntimeError: with open('{file.name}', 'w') as out: out.write('{expected}') - # Kill the unittest runner in the child process. - os._exit(1) - else: - SHORT_SLEEP = 0.1 - import time - for _ in range(10): - spid, status = os.waitpid(pid, os.WNOHANG) - if spid == pid: - break - time.sleep(SHORT_SLEEP) - assert(spid == pid) """) interpreters.run_string(self.id, script) diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-13-12-21-08.bpo-34651.v-bUeV.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-13-12-21-08.bpo-34651.v-bUeV.rst new file mode 100644 index 000000000000..a3f0132fc1d6 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-13-12-21-08.bpo-34651.v-bUeV.rst @@ -0,0 +1,3 @@ +Only allow the main interpreter to fork. The avoids the possibility of +affecting the main interprerter, which is critical to operation of the +runtime. diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index dd69b9eb1ff6..9661e38540ed 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -576,6 +576,11 @@ subprocess_fork_exec(PyObject* self, PyObject *args) &restore_signals, &call_setsid, &preexec_fn)) return NULL; + if (_PyInterpreterState_Get() != PyInterpreterState_Main()) { + PyErr_SetString(PyExc_RuntimeError, "fork not supported for subinterpreters"); + return NULL; + } + if (close_fds && errpipe_write < 3) { /* precondition */ PyErr_SetString(PyExc_ValueError, "errpipe_write must be >= 3"); return NULL; diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 2fddd9587f76..0ac0042f124d 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -32,6 +32,7 @@ #else #include "winreparse.h" #endif +#include "internal/pystate.h" /* On android API level 21, 'AT_EACCESS' is not declared although * HAVE_FACCESSAT is defined. */ @@ -420,6 +421,7 @@ void PyOS_AfterFork_Child(void) { _PyGILState_Reinit(); + _PyInterpreterState_DeleteExceptMain(); PyEval_ReInitThreads(); _PyImport_ReInitLock(); _PySignal_AfterFork(); @@ -5790,6 +5792,10 @@ os_fork1_impl(PyObject *module) { pid_t pid; + if (_PyInterpreterState_Get() != PyInterpreterState_Main()) { + PyErr_SetString(PyExc_RuntimeError, "fork not supported for subinterpreters"); + return NULL; + } PyOS_BeforeFork(); pid = fork1(); if (pid == 0) { @@ -5821,6 +5827,10 @@ os_fork_impl(PyObject *module) { pid_t pid; + if (_PyInterpreterState_Get() != PyInterpreterState_Main()) { + PyErr_SetString(PyExc_RuntimeError, "fork not supported for subinterpreters"); + return NULL; + } PyOS_BeforeFork(); pid = fork(); if (pid == 0) { @@ -6416,6 +6426,10 @@ os_forkpty_impl(PyObject *module) int master_fd = -1; pid_t pid; + if (_PyInterpreterState_Get() != PyInterpreterState_Main()) { + PyErr_SetString(PyExc_RuntimeError, "fork not supported for subinterpreters"); + return NULL; + } PyOS_BeforeFork(); pid = forkpty(&master_fd, NULL, NULL, NULL); if (pid == 0) { diff --git a/Python/pystate.c b/Python/pystate.c index 7d63f4febb93..7b3d3d4c9032 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -268,6 +268,44 @@ PyInterpreterState_Delete(PyInterpreterState *interp) } +/* + * Delete all interpreter states except the main interpreter. If there + * is a current interpreter state, it *must* be the main interpreter. + */ +void +_PyInterpreterState_DeleteExceptMain() +{ + PyThreadState *tstate = PyThreadState_Swap(NULL); + if (tstate != NULL && tstate->interp != _PyRuntime.interpreters.main) { + Py_FatalError("PyInterpreterState_DeleteExceptMain: not main interpreter"); + } + + HEAD_LOCK(); + PyInterpreterState *interp = _PyRuntime.interpreters.head; + _PyRuntime.interpreters.head = NULL; + for (; interp != NULL; interp = interp->next) { + if (interp == _PyRuntime.interpreters.main) { + _PyRuntime.interpreters.main->next = NULL; + _PyRuntime.interpreters.head = interp; + continue; + } + + PyInterpreterState_Clear(interp); // XXX must activate? + zapthreads(interp); + if (interp->id_mutex != NULL) { + PyThread_free_lock(interp->id_mutex); + } + PyMem_RawFree(interp); + } + HEAD_UNLOCK(); + + if (_PyRuntime.interpreters.head == NULL) { + Py_FatalError("PyInterpreterState_DeleteExceptMain: missing main"); + } + PyThreadState_Swap(tstate); +} + + PyInterpreterState * _PyInterpreterState_Get(void) { From webhook-mailer at python.org Fri Sep 14 17:30:07 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Fri, 14 Sep 2018 21:30:07 -0000 Subject: [Python-checkins] bpo-32933: Implement __iter__ method on mock_open() (GH-5974) Message-ID: https://github.com/python/cpython/commit/c83c375ed907bdd54361aa36ce76130360f323a4 commit: c83c375ed907bdd54361aa36ce76130360f323a4 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Berker Peksag date: 2018-09-15T00:30:04+03:00 summary: bpo-32933: Implement __iter__ method on mock_open() (GH-5974) (cherry picked from commit 2087023fdec2c89070bd14f384a3c308c548a94a) Co-authored-by: Tony Flury files: A Misc/NEWS.d/next/Library/2018-04-30-22-43-31.bpo-32933.M3iI_y.rst M Doc/library/unittest.mock.rst M Lib/unittest/mock.py M Lib/unittest/test/testmock/testmock.py M Lib/unittest/test/testmock/testwith.py diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index fb325ed78314..ef1b2e35719d 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -2095,6 +2095,10 @@ mock_open .. versionchanged:: 3.5 *read_data* is now reset on each call to the *mock*. + .. versionchanged:: 3.7.1 + Added :meth:`__iter__` to implementation so that iteration (such as in for + loops) correctly consumes *read_data*. + Using :func:`open` as a context manager is a great way to ensure your file handles are closed properly and is becoming common:: diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py index 382696d6c7e9..cfc0d76ee3f7 100644 --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -2356,14 +2356,16 @@ def _read_side_effect(*args, **kwargs): return type(read_data)().join(_state[0]) def _readline_side_effect(): + yield from _iter_side_effect() + while True: + yield type(read_data)() + + def _iter_side_effect(): if handle.readline.return_value is not None: while True: yield handle.readline.return_value for line in _state[0]: yield line - while True: - yield type(read_data)() - global file_spec if file_spec is None: @@ -2387,6 +2389,7 @@ def _readline_side_effect(): _state[1] = _readline_side_effect() handle.readline.side_effect = _state[1] handle.readlines.side_effect = _readlines_side_effect + handle.__iter__.side_effect = _iter_side_effect def reset_data(*args, **kwargs): _state[0] = _iterate_read_data(read_data) diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py index b64c8663d212..c7bfa277b511 100644 --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -1450,6 +1450,16 @@ def test_mock_open_reuse_issue_21750(self): f2_data = f2.read() self.assertEqual(f1_data, f2_data) + def test_mock_open_dunder_iter_issue(self): + # Test dunder_iter method generates the expected result and + # consumes the iterator. + mocked_open = mock.mock_open(read_data='Remarkable\nNorwegian Blue') + f1 = mocked_open('a-name') + lines = [line for line in f1] + self.assertEqual(lines[0], 'Remarkable\n') + self.assertEqual(lines[1], 'Norwegian Blue') + self.assertEqual(list(f1), []) + def test_mock_open_write(self): # Test exception in file writing write() mock_namedtemp = mock.mock_open(mock.MagicMock(name='JLV')) diff --git a/Lib/unittest/test/testmock/testwith.py b/Lib/unittest/test/testmock/testwith.py index a7bee7300301..43b36a119952 100644 --- a/Lib/unittest/test/testmock/testwith.py +++ b/Lib/unittest/test/testmock/testwith.py @@ -188,6 +188,7 @@ def test_read_data(self): def test_readline_data(self): # Check that readline will return all the lines from the fake file + # And that once fully consumed, readline will return an empty string. mock = mock_open(read_data='foo\nbar\nbaz\n') with patch('%s.open' % __name__, mock, create=True): h = open('bar') @@ -197,6 +198,7 @@ def test_readline_data(self): self.assertEqual(line1, 'foo\n') self.assertEqual(line2, 'bar\n') self.assertEqual(line3, 'baz\n') + self.assertEqual(h.readline(), '') # Check that we properly emulate a file that doesn't end in a newline mock = mock_open(read_data='foo') @@ -204,6 +206,19 @@ def test_readline_data(self): h = open('bar') result = h.readline() self.assertEqual(result, 'foo') + self.assertEqual(h.readline(), '') + + + def test_dunder_iter_data(self): + # Check that dunder_iter will return all the lines from the fake file. + mock = mock_open(read_data='foo\nbar\nbaz\n') + with patch('%s.open' % __name__, mock, create=True): + h = open('bar') + lines = [l for l in h] + self.assertEqual(lines[0], 'foo\n') + self.assertEqual(lines[1], 'bar\n') + self.assertEqual(lines[2], 'baz\n') + self.assertEqual(h.readline(), '') def test_readlines_data(self): diff --git a/Misc/NEWS.d/next/Library/2018-04-30-22-43-31.bpo-32933.M3iI_y.rst b/Misc/NEWS.d/next/Library/2018-04-30-22-43-31.bpo-32933.M3iI_y.rst new file mode 100644 index 000000000000..4de7a8f927d5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-04-30-22-43-31.bpo-32933.M3iI_y.rst @@ -0,0 +1,2 @@ +:func:`unittest.mock.mock_open` now supports iteration over the file +contents. Patch by Tony Flury. From webhook-mailer at python.org Fri Sep 14 17:37:51 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Fri, 14 Sep 2018 21:37:51 -0000 Subject: [Python-checkins] Grammar fix (GH-9318) Message-ID: https://github.com/python/cpython/commit/5acccfaf680ae737bcd8b09f3a85719e3e904ef2 commit: 5acccfaf680ae737bcd8b09f3a85719e3e904ef2 branch: master author: Grant committer: Yury Selivanov date: 2018-09-14T14:37:48-07:00 summary: Grammar fix (GH-9318) files: M Doc/library/asyncio-dev.rst diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index e4986ba5eb26..87abe92c2944 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -407,7 +407,7 @@ traceback where the task was created. Example of log in debug mode: Close transports and event loops -------------------------------- -When a transport is no more needed, call its ``close()`` method to release +When a transport is no longer needed, call its ``close()`` method to release resources. Event loops must also be closed explicitly. If a transport or an event loop is not closed explicitly, a From webhook-mailer at python.org Fri Sep 14 17:57:42 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Fri, 14 Sep 2018 21:57:42 -0000 Subject: [Python-checkins] bpo-33649: Refresh asyncio docs landing page (GH-9322) Message-ID: https://github.com/python/cpython/commit/6c7316439d966cdbc76ffcc005e5d9c563064ba2 commit: 6c7316439d966cdbc76ffcc005e5d9c563064ba2 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-14T14:57:39-07:00 summary: bpo-33649: Refresh asyncio docs landing page (GH-9322) files: M Doc/library/asyncio-future.rst M Doc/library/asyncio-queue.rst M Doc/library/asyncio.rst diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index 2c41c1c2165c..ff04339d8580 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -1,6 +1,8 @@ .. currentmodule:: asyncio +.. _asyncio-futures: + ======= Futures ======= diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index a910dc73ab26..fcad751fe19d 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -1,5 +1,6 @@ .. currentmodule:: asyncio +.. _asyncio-queues: ====== Queues diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 2c8ccbb390f0..c1b04086c35f 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -6,43 +6,46 @@ -------------- -.. TODO: rewrite the introduction section +asyncio is a library to write **concurrent** code using +**async/await** syntax. -This module provides infrastructure for writing single-threaded concurrent -code using coroutines, multiplexing I/O access over sockets and other -resources, running network clients and servers, and other related primitives. -Here is a more detailed list of the package contents: +asyncio is used as a foundation for multiple Python asynchronous +frameworks that provide high-performance network and web-servers, +database connection libraries, distributed task queues, etc. -* a pluggable :ref:`event loop ` with various system-specific - implementations; +asyncio is often a perfect fit for IO-bound and high-level +**structured** network code. -* :ref:`transport ` and :ref:`protocol ` abstractions - (similar to those in `Twisted `_); +asyncio provides a set of **high-level** APIs to: -* concrete support for TCP, UDP, SSL, subprocess pipes, delayed calls, and - others (some may be system-dependent); +* :ref:`run Python coroutines ` concurrently and + have full control over their execution; -* a :class:`Future` class that mimics the one in the :mod:`concurrent.futures` - module, but adapted for use with the event loop; +* perform :ref:`network IO and IPC `; -* coroutines and tasks based on ``yield from`` (:PEP:`380`), to help write - concurrent code in a sequential fashion; +* control :ref:`subprocesses `; -* cancellation support for :class:`Future`\s and coroutines; +* distribute tasks via :ref:`queues `; -* :ref:`synchronization primitives ` for use between coroutines in - a single thread, mimicking those in the :mod:`threading` module; +* :ref:`synchronize ` concurrent code; -* an interface for passing work off to a threadpool, for times when - you absolutely, positively have to use a library that makes blocking - I/O calls. +as well as **low-level** APIs for *library and framework developers* to: -Asynchronous programming is more complex than classical "sequential" -programming: see the :ref:`Develop with asyncio ` page which lists -common traps and explains how to avoid them. :ref:`Enable the debug mode -` during development to detect common issues. +* create and manage :ref:`event loops `, which + provide asynchronous APIs for networking, subprocesses, OS signals, + etc; -High-level APIs: +* implement efficient protocols using + :ref:`transports `; + +* :ref:`bridge ` callback-based libraries and code + with async/await syntax. + + +Contents +-------- + +.. rubric:: High-level APIs .. toctree:: :maxdepth: 1 @@ -54,7 +57,7 @@ High-level APIs: asyncio-queue.rst asyncio-exceptions.rst -Low-level APIs: +.. rubric:: Low-level APIs .. toctree:: :maxdepth: 1 @@ -65,7 +68,7 @@ Low-level APIs: asyncio-policy.rst asyncio-platforms.rst -Guides and Tutorials: +.. rubric:: Guides and Tutorials .. toctree:: :maxdepth: 1 From webhook-mailer at python.org Fri Sep 14 18:11:27 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Fri, 14 Sep 2018 22:11:27 -0000 Subject: [Python-checkins] bpo-33649: Add high-level APIs cheat-sheet (GH-9319) Message-ID: https://github.com/python/cpython/commit/7372c3bbefb4763dbd1b6d66f7971bef28c0f056 commit: 7372c3bbefb4763dbd1b6d66f7971bef28c0f056 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-14T15:11:24-07:00 summary: bpo-33649: Add high-level APIs cheat-sheet (GH-9319) files: A Doc/library/asyncio-api-index.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-exceptions.rst M Doc/library/asyncio-queue.rst M Doc/library/asyncio-stream.rst M Doc/library/asyncio-subprocess.rst M Doc/library/asyncio-sync.rst M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst diff --git a/Doc/library/asyncio-api-index.rst b/Doc/library/asyncio-api-index.rst new file mode 100644 index 000000000000..a4d0e1ba32e1 --- /dev/null +++ b/Doc/library/asyncio-api-index.rst @@ -0,0 +1,206 @@ +.. currentmodule:: asyncio + + +===================== +High-level APIs Index +===================== + +This page lists all high-level async/await enabled asyncio APIs. + + +Tasks +===== + +Utilities to run asyncio programs, create Tasks, and +await on multiple things with timeouts. + +.. list-table:: + :widths: 30 70 + + * - :func:`run` + - Create event loop, run a coroutine, close the loop. + + * - :func:`create_task` + - Start an asyncio Task. + + * - ``await`` :func:`sleep` + - Sleep for a number of seconds. + + * - ``await`` :func:`gather` + - Schedule and wait for things concurrently. + + * - ``await`` :func:`wait_for` + - Run with a timeout. + + * - ``await`` :func:`shield` + - Shield from cancellation. + + * - ``await`` :func:`wait` + - Monitor for completeness. + + * - :func:`current_task` + - Return the current Task. + + * - :func:`all_tasks` + - Return all tasks for an event loop. + + * - :class:`Task` + - Task object. + + +.. rubric:: Examples + +* :ref:`Using asyncio.gather() to run things in parallel + `. + +* :ref:`Using asyncio.wait_for() to enforce a timeout + `. + +* :ref:`Cancellation `. + +* :ref:`Using asyncio.sleep() `. + +* See also the main :ref:`Tasks documentation page `. + + +Queues +====== + +Queues should be used to distribute work amongst multiple asyncio Tasks, +implement connection pools, and pub/sub patterns. + + +.. list-table:: + :widths: 30 70 + + * - :class:`Queue` + - A FIFO queue. + + * - :class:`PriorityQueue` + - A priority queue. + + * - :class:`LifoQueue` + - A LIFO queue. + + +.. rubric:: Examples + +* :ref:`Using asyncio.Queue to distribute workload between several + Tasks `. + +* See also the :ref:`Queues documentation page `. + + +Subprocesses +============ + +Utilities to spawn subprocesses and run shell commands. + +.. list-table:: + :widths: 30 70 + + * - ``await`` :func:`create_subprocess_exec` + - Create a subprocess. + + * - ``await`` :func:`create_subprocess_shell` + - Run a shell command. + + +.. rubric:: Examples + +* :ref:`Executing a shell command `. + +* See also the :ref:`subprocess APIs ` + documentation. + + +Streams +======= + +High-level APIs to work with network IO. + +.. list-table:: + :widths: 30 70 + + * - ``await`` :func:`open_connection` + - Establish a TCP connection. + + * - ``await`` :func:`open_unix_connection` + - Establish a Unix socket connection. + + * - ``await`` :func:`start_server` + - Start a TCP server. + + * - ``await`` :func:`start_unix_server` + - Start a Unix socket server. + + * - :class:`StreamReader` + - High-level async/await object to receive network data. + + * - :class:`StreamWriter` + - High-level async/await object to send network data. + + +.. rubric:: Examples + +* :ref:`Example TCP client `. + +* See also the :ref:`streams APIs ` + documentation. + + +Synchronization +=============== + +Threading-like synchronization primitives that can be used in Tasks. + +.. list-table:: + :widths: 30 70 + + * - :class:`Lock` + - A mutex lock. + + * - :class:`Event` + - An event object. + + * - :class:`Condition` + - A condition object. + + * - :class:`Semaphore` + - A semaphore. + + * - :class:`BoundedSemaphore` + - A bounded semaphore. + + +.. rubric:: Examples + +* :ref:`Using asyncio.Event `. + +* See also the documentation of asyncio + :ref:`synchronization primitives `. + + +Exceptions +========== + +.. list-table:: + :widths: 30 70 + + + * - :exc:`asyncio.TimeoutError` + - Raised on timeout by functions like :func:`wait_for`. + Keep in mind that ``asyncio.TimeoutError`` is **unrelated** + to the built-in :exc:`TimeoutError` exception. + + * - :exc:`asyncio.CancelledError` + - Raised when a Task is cancelled. See also :meth:`Task.cancel`. + + +.. rubric:: Examples + +* :ref:`Handling CancelledError to run code on cancellation request + `. + +* See also the full list of + :ref:`asyncio-specific exceptions `. diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index e1b47d2588cd..19a0acdfd55e 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1466,7 +1466,7 @@ during 5 seconds, and then stops the event loop:: .. seealso:: - A similar :ref:`current date ` example + A similar :ref:`current date ` example created with a coroutine and the :func:`run` function. diff --git a/Doc/library/asyncio-exceptions.rst b/Doc/library/asyncio-exceptions.rst index 7f3ed857943d..31bc1edf0142 100644 --- a/Doc/library/asyncio-exceptions.rst +++ b/Doc/library/asyncio-exceptions.rst @@ -1,6 +1,8 @@ .. currentmodule:: asyncio +.. _asyncio-exceptions: + ========== Exceptions ========== @@ -10,7 +12,7 @@ Exceptions The operation has exceeded the given deadline. - .. note:: + .. important:: This exception is different from the builtin :exc:`TimeoutError` exception. @@ -23,7 +25,7 @@ Exceptions when asyncio Tasks are cancelled. In almost all situations the exception must always be re-raised. - .. note:: + .. important:: This exception is a subclass of :exc:`Exception`, so it can be accidentally suppressed by ``try..except`` block:: diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index fcad751fe19d..23d2f3cb1d7c 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -140,6 +140,8 @@ Exceptions Examples ======== +.. _asyncio_example_queue_dist: + Queues can be used to distribute workload between several concurrent tasks:: diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 80b76253d065..29163a217dce 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -10,6 +10,8 @@ Streams are high-level async/await-ready primitives to work with network connections. Streams allow sending and receiving data without using callbacks or low-level protocols and transports. +.. _asyncio_example_stream: + Here is an example of a TCP echo client written using asyncio streams:: diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 92c081b70bfc..ef8a1cbb4b1c 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -9,6 +9,8 @@ Subprocesses This section describes high-level async/await asyncio APIs to create and manage subprocesses. +.. _asyncio_example_subprocess_shell: + Here's an example of how asyncio can run a shell command and communicate its result back:: diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index 8da5aa81ab9e..f29988554cee 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -94,6 +94,8 @@ Event :meth:`clear` method. The :meth:`wait` method blocks until the flag is set to *true*. The flag is set to *false* initially. + .. _asyncio_example_sync_event: + Example:: async def waiter(event): diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 7e09b166ebc9..d3cfd5ff31ce 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -165,7 +165,7 @@ Sleeping If *result* is provided, it is returned to the caller when the coroutine completes. - .. _asyncio-date-coroutine: + .. _asyncio_example_sleep: Example of coroutine displaying the current date every second during 5 seconds:: @@ -219,6 +219,8 @@ Running Tasks Concurrently If the *gather* itself is cancelled, the cancellation is propagated regardless of *return_exceptions*. + .. _asyncio_example_gather: + Example:: import asyncio @@ -316,6 +318,8 @@ Timeouts If the wait is cancelled, the future *fut* is also cancelled. + .. _asyncio_example_waitfor: + Example:: async def eternity(): @@ -539,6 +543,8 @@ Task Object suppressing cancellation completely is not common and is actively discouraged. + .. _asyncio_example_task_cancel: + The following example illustrates how coroutines can intercept the cancellation request:: diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index c1b04086c35f..25daeb6b8531 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -42,8 +42,8 @@ as well as **low-level** APIs for *library and framework developers* to: with async/await syntax. -Contents --------- +Reference +--------- .. rubric:: High-level APIs @@ -73,6 +73,7 @@ Contents .. toctree:: :maxdepth: 1 + asyncio-api-index.rst asyncio-dev.rst From webhook-mailer at python.org Fri Sep 14 18:12:25 2018 From: webhook-mailer at python.org (Pablo Galindo) Date: Fri, 14 Sep 2018 22:12:25 -0000 Subject: [Python-checkins] bpo-34685: Skip posix_spawn scheduler tests on BSD (GH-9316) Message-ID: https://github.com/python/cpython/commit/3d18b50a12e639e018d49d7b85663164d60bfd2b commit: 3d18b50a12e639e018d49d7b85663164d60bfd2b branch: master author: Pablo Galindo committer: GitHub date: 2018-09-14T15:12:22-07:00 summary: bpo-34685: Skip posix_spawn scheduler tests on BSD (GH-9316) * Skip posix_spawn scheduler tests on BSD. We were already skyping similar tests as the behaviour can depend on the implementation in some cases. files: M Lib/test/test_posix.py diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py index d402d4fb088c..86c04b9f324a 100644 --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -1659,15 +1659,17 @@ def test_setsigdef_wrong_type(self): os.environ, setsigdef=[signal.NSIG, signal.NSIG+1]) @requires_sched + @unittest.skipIf(sys.platform.startswith(('freebsd', 'netbsd')), + "bpo-34685: test can fail on BSD") def test_setscheduler_only_param(self): policy = os.sched_getscheduler(0) priority = os.sched_get_priority_min(policy) code = textwrap.dedent(f"""\ - import os + import os, sys if os.sched_getscheduler(0) != {policy}: - os.exit(101) + sys.exit(101) if os.sched_getparam(0).sched_priority != {priority}: - os.exit(102)""") + sys.exit(102)""") pid = posix.posix_spawn( sys.executable, [sys.executable, '-c', code], @@ -1677,15 +1679,17 @@ def test_setscheduler_only_param(self): self.assertEqual(os.waitpid(pid, 0), (pid, 0)) @requires_sched + @unittest.skipIf(sys.platform.startswith(('freebsd', 'netbsd')), + "bpo-34685: test can fail on BSD") def test_setscheduler_with_policy(self): policy = os.sched_getscheduler(0) priority = os.sched_get_priority_min(policy) code = textwrap.dedent(f"""\ - import os + import os, sys if os.sched_getscheduler(0) != {policy}: - os.exit(101) + sys.exit(101) if os.sched_getparam(0).sched_priority != {priority}: - os.exit(102)""") + sys.exit(102)""") pid = posix.posix_spawn( sys.executable, [sys.executable, '-c', code], From webhook-mailer at python.org Fri Sep 14 18:55:23 2018 From: webhook-mailer at python.org (Petr Viktorin) Date: Fri, 14 Sep 2018 22:55:23 -0000 Subject: [Python-checkins] bpo-33486: regen autotools files using autoupdate+autoreconf (GH-6853) Message-ID: https://github.com/python/cpython/commit/24f684692070f53b6f6e4dc67b9fe23dbd58655f commit: 24f684692070f53b6f6e4dc67b9fe23dbd58655f branch: master author: Eitan Adler committer: Petr Viktorin date: 2018-09-14T15:55:20-07:00 summary: bpo-33486: regen autotools files using autoupdate+autoreconf (GH-6853) files: M configure.ac diff --git a/configure.ac b/configure.ac index 3e5388ab0804..03638f8ae9bc 100644 --- a/configure.ac +++ b/configure.ac @@ -5,9 +5,9 @@ dnl *********************************************** # Set VERSION so we only need to edit in one place (i.e., here) m4_define(PYTHON_VERSION, 3.8) -AC_PREREQ(2.65) +AC_PREREQ([2.69]) -AC_INIT(python, PYTHON_VERSION, https://bugs.python.org/) +AC_INIT([python],[PYTHON_VERSION],[https://bugs.python.org/]) AC_CONFIG_MACRO_DIR(m4) @@ -1219,7 +1219,7 @@ fi], assertions='false' AC_MSG_CHECKING(for --with-assertions) AC_ARG_WITH(assertions, - AC_HELP_STRING([--with-assertions], [build with C assertions enabled]), + AS_HELP_STRING([--with-assertions],[build with C assertions enabled]), [ if test "$withval" != no then @@ -3336,7 +3336,7 @@ fi # Check for DTrace support AC_MSG_CHECKING(for --with-dtrace) AC_ARG_WITH(dtrace, - AC_HELP_STRING(--with(out)-dtrace, [disable/enable DTrace support]),, + AS_HELP_STRING([--with(out)-dtrace],[disable/enable DTrace support]),, with_dtrace=no) AC_MSG_RESULT($with_dtrace) From webhook-mailer at python.org Fri Sep 14 19:21:35 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 14 Sep 2018 23:21:35 -0000 Subject: [Python-checkins] Simplify PyInit_timezone. (GH-9323) Message-ID: https://github.com/python/cpython/commit/afde1c1a05cc8a1e8adf6403c451f6708509a605 commit: afde1c1a05cc8a1e8adf6403c451f6708509a605 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-14T16:21:32-07:00 summary: Simplify PyInit_timezone. (GH-9323) Assume tzname exists. Only use a hack to compute altzone if it's not defined. files: M Modules/timemodule.c diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 1a4cff23d65e..b9a71b48414a 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1522,29 +1522,6 @@ PyDoc_STRVAR(get_clock_info_doc, \n\ Get information of the specified clock."); -#if !defined(HAVE_TZNAME) || defined(__GLIBC__) || defined(__CYGWIN__) -static void -get_zone(char *zone, int n, struct tm *p) -{ -#ifdef HAVE_STRUCT_TM_TM_ZONE - strncpy(zone, p->tm_zone ? p->tm_zone : " ", n); -#else - tzset(); - strftime(zone, n, "%Z", p); -#endif -} - -static int -get_gmtoff(time_t t, struct tm *p) -{ -#ifdef HAVE_STRUCT_TM_TM_ZONE - return p->tm_gmtoff; -#else - return timegm(p) - t; -#endif -} -#endif /* !defined(HAVE_TZNAME) || defined(__GLIBC__) || defined(__CYGWIN__) */ - static void PyInit_timezone(PyObject *m) { /* This code moved from PyInit_time wholesale to allow calling it from @@ -1563,65 +1540,35 @@ PyInit_timezone(PyObject *m) { And I'm lazy and hate C so nyer. */ -#if defined(HAVE_TZNAME) && !defined(__GLIBC__) && !defined(__CYGWIN__) PyObject *otz0, *otz1; tzset(); PyModule_AddIntConstant(m, "timezone", timezone); #ifdef HAVE_ALTZONE PyModule_AddIntConstant(m, "altzone", altzone); -#else - PyModule_AddIntConstant(m, "altzone", timezone-3600); -#endif - PyModule_AddIntConstant(m, "daylight", daylight); - otz0 = PyUnicode_DecodeLocale(tzname[0], "surrogateescape"); - otz1 = PyUnicode_DecodeLocale(tzname[1], "surrogateescape"); - PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1)); -#else /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/ +#elif defined(HAVE_STRUCT_TM_TM_ZONE) { -#define YEAR ((time_t)((365 * 24 + 6) * 3600)) + static const time_t YEAR = (365 * 24 + 6) * 3600; time_t t; struct tm p; long janzone, julyzone; - char janname[10], julyname[10]; t = (time((time_t *)0) / YEAR) * YEAR; _PyTime_localtime(t, &p); - get_zone(janname, 9, &p); - janzone = -get_gmtoff(t, &p); - janname[9] = '\0'; + janzone = -p.tm_gmtoff; t += YEAR/2; _PyTime_localtime(t, &p); - get_zone(julyname, 9, &p); - julyzone = -get_gmtoff(t, &p); - julyname[9] = '\0'; - - if( janzone < julyzone ) { - /* DST is reversed in the southern hemisphere */ - PyModule_AddIntConstant(m, "timezone", julyzone); - PyModule_AddIntConstant(m, "altzone", janzone); - PyModule_AddIntConstant(m, "daylight", - janzone != julyzone); - PyModule_AddObject(m, "tzname", - Py_BuildValue("(zz)", - julyname, janname)); - } else { - PyModule_AddIntConstant(m, "timezone", janzone); - PyModule_AddIntConstant(m, "altzone", julyzone); - PyModule_AddIntConstant(m, "daylight", - janzone != julyzone); - PyModule_AddObject(m, "tzname", - Py_BuildValue("(zz)", - janname, julyname)); - } + julyzone = -p.tm_gmtoff; + + // DST is reversed in the southern hemisphere. + PyModule_AddIntConstant(m, "altzone", + (janzone < julyzone) ? janzone : julyzone); } -#ifdef __CYGWIN__ - tzset(); - PyModule_AddIntConstant(m, "timezone", _timezone); - PyModule_AddIntConstant(m, "altzone", _timezone-3600); - PyModule_AddIntConstant(m, "daylight", _daylight); - PyModule_AddObject(m, "tzname", - Py_BuildValue("(zz)", _tzname[0], _tzname[1])); -#endif /* __CYGWIN__ */ -#endif /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/ +#else + PyModule_AddIntConstant(m, "altzone", timezone-3600); +#endif + PyModule_AddIntConstant(m, "daylight", daylight); + otz0 = PyUnicode_DecodeLocale(tzname[0], "surrogateescape"); + otz1 = PyUnicode_DecodeLocale(tzname[1], "surrogateescape"); + PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1)); } From webhook-mailer at python.org Fri Sep 14 19:28:10 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 14 Sep 2018 23:28:10 -0000 Subject: [Python-checkins] [2.7] closes bpo-8450: a better error message when http status line isn't received (GH-2825) Message-ID: https://github.com/python/cpython/commit/ee78ba2c819b0cd7671a575e584d8fe1b7adb3e4 commit: ee78ba2c819b0cd7671a575e584d8fe1b7adb3e4 branch: 2.7 author: Shoham Peller committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-14T16:28:08-07:00 summary: [2.7] closes bpo-8450: a better error message when http status line isn't received (GH-2825) When the server has closed the connection before sending a status-line, the client's error message should have a more descriptive error message https://bugs.python.org/issue8450 https://bugs.python.org/issue8450 files: M Lib/httplib.py diff --git a/Lib/httplib.py b/Lib/httplib.py index f3bb22c2b6ff..60a8fb4e355f 100644 --- a/Lib/httplib.py +++ b/Lib/httplib.py @@ -399,7 +399,7 @@ def _read_status(self): if not line: # Presumably, the server closed the connection before # sending a valid response. - raise BadStatusLine(line) + raise BadStatusLine("No status line received - the server has closed the connection") try: [version, status, reason] = line.split(None, 2) except ValueError: From webhook-mailer at python.org Fri Sep 14 19:57:16 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Fri, 14 Sep 2018 23:57:16 -0000 Subject: [Python-checkins] bpo-33649: Fix asyncio-dev (GH-9324) Message-ID: https://github.com/python/cpython/commit/805e27eff65d51f7aea2c00ccbb4f5d44f4499f2 commit: 805e27eff65d51f7aea2c00ccbb4f5d44f4499f2 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-14T16:57:11-07:00 summary: bpo-33649: Fix asyncio-dev (GH-9324) files: M Doc/library/asyncio-api-index.rst M Doc/library/asyncio-dev.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-future.rst M Doc/library/asyncio.rst M Doc/library/ipc.rst diff --git a/Doc/library/asyncio-api-index.rst b/Doc/library/asyncio-api-index.rst index a4d0e1ba32e1..dc691000471c 100644 --- a/Doc/library/asyncio-api-index.rst +++ b/Doc/library/asyncio-api-index.rst @@ -15,7 +15,7 @@ Utilities to run asyncio programs, create Tasks, and await on multiple things with timeouts. .. list-table:: - :widths: 30 70 + :widths: 50 50 * - :func:`run` - Create event loop, run a coroutine, close the loop. @@ -71,7 +71,7 @@ implement connection pools, and pub/sub patterns. .. list-table:: - :widths: 30 70 + :widths: 50 50 * - :class:`Queue` - A FIFO queue. @@ -97,7 +97,7 @@ Subprocesses Utilities to spawn subprocesses and run shell commands. .. list-table:: - :widths: 30 70 + :widths: 50 50 * - ``await`` :func:`create_subprocess_exec` - Create a subprocess. @@ -120,7 +120,7 @@ Streams High-level APIs to work with network IO. .. list-table:: - :widths: 30 70 + :widths: 50 50 * - ``await`` :func:`open_connection` - Establish a TCP connection. @@ -155,7 +155,7 @@ Synchronization Threading-like synchronization primitives that can be used in Tasks. .. list-table:: - :widths: 30 70 + :widths: 50 50 * - :class:`Lock` - A mutex lock. @@ -185,7 +185,7 @@ Exceptions ========== .. list-table:: - :widths: 30 70 + :widths: 50 50 * - :exc:`asyncio.TimeoutError` diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index 87abe92c2944..e8153186760b 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -2,415 +2,237 @@ .. _asyncio-dev: -Develop with asyncio -==================== +======================= +Developing with asyncio +======================= -Asynchronous programming is different than classical "sequential" programming. -This page lists common traps and explains how to avoid them. +Asynchronous programming is different from classical "sequential" +programming. + +This page lists common mistakes and traps and explains how +to avoid them. .. _asyncio-debug-mode: -Debug mode of asyncio ---------------------- - -The implementation of :mod:`asyncio` has been written for performance. -In order to ease the development of asynchronous code, you may wish to -enable *debug mode*. - -To enable all debug checks for an application: - -* Enable the asyncio debug mode globally by setting the environment variable - :envvar:`PYTHONASYNCIODEBUG` to ``1``, using ``-X dev`` command line option - (see the :option:`-X` option), or by calling - :meth:`loop.set_debug`. -* Set the log level of the :ref:`asyncio logger ` to - :py:data:`logging.DEBUG`. For example, call - ``logging.basicConfig(level=logging.DEBUG)`` at startup. -* Configure the :mod:`warnings` module to display :exc:`ResourceWarning` - warnings. For example, use the ``-Wdefault`` command line option of Python to - display them. - -Examples debug checks: - -* Log :ref:`coroutines defined but never "yielded from" - ` -* :meth:`loop.call_soon` and :meth:`loop.call_at` methods - raise an exception if they are called from the wrong thread. -* Log the execution time of the selector -* Log callbacks taking more than 100 ms to be executed. The - :attr:`loop.slow_callback_duration` attribute is the minimum - duration in seconds of "slow" callbacks. -* :exc:`ResourceWarning` warnings are emitted when transports and event loops - are :ref:`not closed explicitly `. +Debug Mode +========== -.. versionchanged:: 3.7 +By default asyncio runs in production mode. In order to ease +the development asyncio has a *debug mode*. - The new ``-X dev`` command line option can now also be used to enable - the debug mode. +To enable debugging for an application: -.. seealso:: +* Enable the debug mode globally by setting the environment variable + :envvar:`PYTHONASYNCIODEBUG` to ``1``. - The :meth:`loop.set_debug` method and the :ref:`asyncio logger - `. +* Alternatively, the debug mode can be enabled by using the ``-X dev`` + command line option for Python (see the :option:`-X` option). +* Yet another way to enable the debug mode is by calling + :meth:`loop.set_debug` or by passing ``debug=True`` to + :func:`asyncio.run`. -Cancellation ------------- +In addition to enabling debug mode, consider also: -Cancellation of tasks is not common in classic programming. In asynchronous -programming, not only is it something common, but you have to prepare your -code to handle it. +* setting the log level of the :ref:`asyncio logger ` to + :py:data:`logging.DEBUG`, for example the following snippet of code + can be run at startup of the application:: -Futures and tasks can be cancelled explicitly with their :meth:`Future.cancel` -method. The :func:`wait_for` function cancels the waited task when the timeout -occurs. There are many other cases where a task can be cancelled indirectly. + logging.basicConfig(level=logging.DEBUG) -Don't call :meth:`~Future.set_result` or :meth:`~Future.set_exception` method -of :class:`Future` if the future is cancelled: it would fail with an exception. -For example, write:: +* configuring the :mod:`warnings` module to display + :exc:`ResourceWarning` warnings. One way of doing that is by + using the ``-Wdefault`` command line option. - if not fut.cancelled(): - fut.set_result('done') -Don't schedule directly a call to the :meth:`~Future.set_result` or the -:meth:`~Future.set_exception` method of a future with -:meth:`loop.call_soon`: the future can be cancelled before its method -is called. +In asyncio debug mode the following checks are performed: -If you wait for a future, you should check early if the future was cancelled to -avoid useless operations. Example:: +* Log :ref:`coroutines that were not awaited + `; this mitigates the "forgotten + await" pitfall. - async def slow_operation(fut): - if fut.cancelled(): - return - # ... slow computation ... - await fut - # ... +* Many non-treadsafe asyncio APIs (such as :meth:`loop.call_soon` and + :meth:`loop.call_at` methods) raise an exception if they are called + from a wrong thread. -The :func:`shield` function can also be used to ignore cancellation. +* Log the execution time of the IO selector if it takes too long to + perform an IO operation. + +* Log callbacks taking longer than 100 ms to be executed. The + :attr:`loop.slow_callback_duration` attribute is the minimum + duration in seconds of "slow" callbacks. .. _asyncio-multithreading: -Concurrency and multithreading ------------------------------- +Concurrency and Multithreading +============================== An event loop runs in a thread (typically the main thread) and executes -all callbacks and tasks in its thread. While a task is running in the -event loop, no other tasks may run in the same thread. When a task -executes an ``await`` expression, the running task gets suspended, and the -event loop executes the next task. Prior to suspending the task, the awaiting -chain is checked, and if the chain ends with a future, the running task is -not suspended. - -To schedule a callback from a different thread, the +all callbacks and Tasks in its thread. While a Task is running in the +event loop, no other Tasks can run in the same thread. When a Task +executes an ``await`` expression, the running Task gets suspended, and +the event loop executes the next Task. + +To schedule a callback from a different OS thread, the :meth:`loop.call_soon_threadsafe` method should be used. Example:: loop.call_soon_threadsafe(callback, *args) -Most asyncio objects are not thread safe. You should only worry if you access -objects outside the event loop. For example, to cancel a future, don't call -directly its :meth:`Future.cancel` method, but:: +Almost all asyncio objects are not thread safe, which is typically +not a problem unless there is code that works with them from outside +of a Task or a callback. If there's a need for such code to call a +low-level asyncio API, the :meth:`loop.call_soon_threadsafe` method +should be used, e.g.:: loop.call_soon_threadsafe(fut.cancel) -To handle signals and to execute subprocesses, the event loop must be run in -the main thread. - -To schedule a coroutine object from a different thread, the +To schedule a coroutine object from a different OS thread, the :func:`run_coroutine_threadsafe` function should be used. It returns a :class:`concurrent.futures.Future` to access the result:: - future = asyncio.run_coroutine_threadsafe(coro_func(), loop) - result = future.result(timeout) # Wait for the result with a timeout - -The :meth:`loop.run_in_executor` method can be used with a -:class:`concurrent.futures.ThreadPoolExecutor` to execute a callback in -different thread so as not to block the event loop's main thread. + async def coro_func(): + return await asyncio.sleep(1, 42) -.. seealso:: + # Later in another OS thread: - The :ref:`Synchronization primitives ` section describes ways - to synchronize tasks. - - The :ref:`Subprocess and threads ` section lists - asyncio limitations to run subprocesses from different threads. + future = asyncio.run_coroutine_threadsafe(coro_func(), loop) + # Wait for the result: + result = future.result() +To handle signals and to execute subprocesses, the event loop must be +run in the main thread. +The :meth:`loop.run_in_executor` method can be used with a +:class:`concurrent.futures.ThreadPoolExecutor` to execute +blocking code in a different OS thread without blocking the OS thread +that the event loop runs in. .. _asyncio-handle-blocking: -Handle blocking functions correctly ------------------------------------ - -Blocking functions should not be called directly. For example, if a function -blocks for 1 second, other tasks are delayed by 1 second which can have an -important impact on reactivity. - -For networking and subprocesses, the :mod:`asyncio` module provides high-level -APIs like :ref:`protocols `. +Running Blocking Code +===================== -An executor can be used to run a task in a different thread or even in a -different process, to not block the thread of the event loop. See the -:meth:`loop.run_in_executor` method. +Blocking (CPU-bound) code should not be called directly. For example, +if a function performs a CPU-intensive calculation for 1 second, +all concurrent asyncio Tasks and IO operations would be delayed +by 1 second. -.. seealso:: - - The :ref:`Delayed calls ` section details how the - event loop handles time. +An executor can be used to run a task in a different thread or even in +a different process to avoid blocking block the OS thread with the +event loop. See the :meth:`loop.run_in_executor` method for more +details. .. _asyncio-logger: Logging -------- - -The :mod:`asyncio` module logs information with the :mod:`logging` module in -the logger ``'asyncio'``. +======= -The default log level for the :mod:`asyncio` module is :py:data:`logging.INFO`. -For those not wanting such verbosity from :mod:`asyncio` the log level can -be changed. For example, to change the level to :py:data:`logging.WARNING`: +asyncio uses the :mod:`logging` module and all logging is performed +via the ``"asyncio"`` logger. -.. code-block:: none +The default log level is :py:data:`logging.INFO`, which can easily be +adjusted:: - logging.getLogger('asyncio').setLevel(logging.WARNING) + logging.getLogger("asyncio").setLevel(logging.WARNING) .. _asyncio-coroutine-not-scheduled: -Detect coroutine objects never scheduled ----------------------------------------- - -When a coroutine function is called and its result is not passed to -:meth:`asyncio.create_task` the execution of the coroutine object will -never be scheduled which is probably a bug. Using ``asyncio.create_task`` is -preferred to the low level :func:`ensure_future` and :meth:`loop.create_task` -methods. :ref:`Enable the debug mode of asyncio ` -to :ref:`log a warning ` to detect it. +Detect never awaited coroutines +=============================== -Example with the bug:: +When a coroutine is called (e.g. ``coro()`` instead of ``await coro()``) +the call is not wrapped with :meth:`asyncio.create_task`, the execution +of the coroutine object will never be scheduled. For example:: import asyncio async def test(): print("never scheduled") + async def main(): + test() + + asyncio.run(main()) + +Output:: + + test.py:7: RuntimeWarning: coroutine 'test' was never awaited test() Output in debug mode:: - Coroutine test() at test.py:3 was never yielded from - Coroutine object created at (most recent call last): - File "test.py", line 7, in - test() + test.py:7: RuntimeWarning: coroutine 'test' was never awaited + Coroutine created at (most recent call last) + File "../t.py", line 9, in + asyncio.run(main(), debug=True) + + < .. > -The fix is to call the :meth:`asyncio.create_task` function. Using -``asyncio.create_task`` is preferred to the low level :func:`ensure_future` and -:meth:`loop.create_task` methods. + File "../t.py", line 7, in main + test() + test() -.. seealso:: +The usual fix is to either await the coroutine or call the +:meth:`asyncio.create_task` function:: - :ref:`Pending task destroyed `. + async def main(): + await test() -Detect exceptions never consumed --------------------------------- +Detect never consumed exceptions +================================ -Python usually calls :func:`sys.excepthook` on unhandled exceptions. If -:meth:`Future.set_exception` is called, but the exception is never consumed, -:func:`sys.excepthook` is not called. Instead, :ref:`a log is emitted -` when the future is deleted by the garbage collector, with the -traceback where the exception was raised. +If a :meth:`Future.set_exception` is called but the Future object is +never awaited on, the exception would never be propagated to the +user code. In this case, asyncio would emit a log message when the +Future object is garbage collected. -Example of unhandled exception:: +Example of an unhandled exception:: import asyncio - @asyncio.coroutine - def bug(): + async def bug(): raise Exception("not consumed") - loop = asyncio.get_event_loop() - asyncio.ensure_future(bug()) - loop.run_forever() - loop.close() + async def main(): + asyncio.create_task(bug()) + + asyncio.run(main()) Output:: Task exception was never retrieved - future: exception=Exception('not consumed',)> - Traceback (most recent call last): - File "asyncio/tasks.py", line 237, in _step - result = next(coro) - File "asyncio/coroutines.py", line 141, in coro - res = func(*args, **kw) - File "test.py", line 5, in bug - raise Exception("not consumed") - Exception: not consumed + future: + exception=Exception('not consumed')> -:ref:`Enable the debug mode of asyncio ` to get the -traceback where the task was created. Output in debug mode:: - - Task exception was never retrieved - future: exception=Exception('not consumed',) created at test.py:8> - source_traceback: Object created at (most recent call last): - File "test.py", line 8, in - asyncio.ensure_future(bug()) Traceback (most recent call last): - File "asyncio/tasks.py", line 237, in _step - result = next(coro) - File "asyncio/coroutines.py", line 79, in __next__ - return next(self.gen) - File "asyncio/coroutines.py", line 141, in coro - res = func(*args, **kw) - File "test.py", line 5, in bug + File "test.py", line 4, in bug raise Exception("not consumed") Exception: not consumed -There are different options to fix this issue. The first option is to chain the -coroutine in another coroutine and use classic try/except:: - - async def handle_exception(): - try: - await bug() - except Exception: - print("exception consumed") - - loop = asyncio.get_event_loop() - asyncio.ensure_future(handle_exception()) - loop.run_forever() - loop.close() - -Another option is to use the :meth:`asyncio.run` function:: - - asyncio.run(bug()) - -.. seealso:: - - The :meth:`Future.exception` method. - +:ref:`Enable the debug mode ` to get the +traceback where the task was created:: -Chain coroutines correctly --------------------------- + asyncio.run(main(), debug=True) -When a coroutine function calls other coroutine functions and tasks, they -should be chained explicitly with ``await``. Otherwise, the execution is -not guaranteed to be sequential. - -Example with different bugs using :func:`asyncio.sleep` to simulate slow -operations:: - - import asyncio - - async def create(): - await asyncio.sleep(3.0) - print("(1) create file") - - async def write(): - await asyncio.sleep(1.0) - print("(2) write into file") - - async def close(): - print("(3) close file") - - async def test(): - asyncio.ensure_future(create()) - asyncio.ensure_future(write()) - asyncio.ensure_future(close()) - await asyncio.sleep(2.0) - loop.stop() - - loop = asyncio.get_event_loop() - asyncio.ensure_future(test()) - loop.run_forever() - print("Pending tasks at exit: %s" % asyncio.Task.all_tasks(loop)) - loop.close() - -Expected output: - -.. code-block:: none - - (1) create file - (2) write into file - (3) close file - Pending tasks at exit: set() - -Actual output: - -.. code-block:: none - - (3) close file - (2) write into file - Pending tasks at exit: {>} - Task was destroyed but it is pending! - task: > - -The loop stopped before the ``create()`` finished, ``close()`` has been called -before ``write()``, whereas coroutine functions were called in this order: -``create()``, ``write()``, ``close()``. - -To fix the example, tasks must be marked with ``await``:: - - async def test(): - await asyncio.ensure_future(create()) - await asyncio.ensure_future(write()) - await asyncio.ensure_future(close()) - await asyncio.sleep(2.0) - loop.stop() - -Or without ``asyncio.ensure_future()``:: - - async def test(): - await create() - await write() - await close() - await asyncio.sleep(2.0) - loop.stop() - - -.. _asyncio-pending-task-destroyed: - -Pending task destroyed ----------------------- - -If a pending task is destroyed, the execution of its wrapped :ref:`coroutine -` did not complete. It is probably a bug and so a warning is logged. - -Example of log: - -.. code-block:: none - - Task was destroyed but it is pending! - task: wait_for=> - -:ref:`Enable the debug mode of asyncio ` to get the -traceback where the task was created. Example of log in debug mode: +Output in debug mode:: -.. code-block:: none + Task exception was never retrieved + future: + exception=Exception('not consumed') created at asyncio/tasks.py:321> - Task was destroyed but it is pending! source_traceback: Object created at (most recent call last): - File "test.py", line 15, in - task = asyncio.ensure_future(coro, loop=loop) - task: wait_for= created at test.py:15> - - -.. seealso:: - - :ref:`Detect coroutine objects never scheduled `. + File "../t.py", line 9, in + asyncio.run(main(), debug=True) -.. _asyncio-close-transports: + < .. > -Close transports and event loops --------------------------------- - -When a transport is no longer needed, call its ``close()`` method to release -resources. Event loops must also be closed explicitly. - -If a transport or an event loop is not closed explicitly, a -:exc:`ResourceWarning` warning will be emitted in its destructor. By default, -:exc:`ResourceWarning` warnings are ignored. The :ref:`Debug mode of asyncio -` section explains how to display them. + Traceback (most recent call last): + File "../t.py", line 4, in bug + raise Exception("not consumed") + Exception: not consumed diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 19a0acdfd55e..718c32277c27 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1075,6 +1075,11 @@ Enabling debug mode Set the debug mode of the event loop. + .. versionchanged:: 3.7 + + The new ``-X dev`` command line option can now also be used + to enable the debug mode. + .. seealso:: The :ref:`debug mode of asyncio `. diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index ff04339d8580..19bf8a6e058b 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -121,6 +121,16 @@ Future Object or an exception set with :meth:`set_result` or :meth:`set_exception` calls. + .. method:: cancelled() + + Return ``True`` if the Future was *cancelled*. + + The method is usually used to check if a Future is not + *cancelled* before setting a result or an exception for it:: + + if not fut.cancelled(): + fut.set_result(42) + .. method:: add_done_callback(callback, *, context=None) Add a callback to be run when the Future is *done*. @@ -180,10 +190,6 @@ Future Object .. versionadded:: 3.7 - .. method:: cancelled() - - Return ``True`` if the Future was *cancelled*. - This example creates a Future object, creates and schedules an asynchronous Task to set result for the Future, and waits until diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 25daeb6b8531..9f45f7458c37 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -32,8 +32,9 @@ asyncio provides a set of **high-level** APIs to: as well as **low-level** APIs for *library and framework developers* to: * create and manage :ref:`event loops `, which - provide asynchronous APIs for networking, subprocesses, OS signals, - etc; + provide asynchronous APIs for :meth:`networking `, + running :meth:`subprocesses `, + handling :meth:`OS signals `, etc; * implement efficient protocols using :ref:`transports `; @@ -75,11 +76,3 @@ Reference asyncio-api-index.rst asyncio-dev.rst - - -.. seealso:: - - The :mod:`asyncio` module was proposed in :PEP:`3156`. - Since the acceptance of the PEP many new APIs were added and many - original APIs were altered. The PEP should be treated as a - historical document. diff --git a/Doc/library/ipc.rst b/Doc/library/ipc.rst index 8f5b3b216c78..b88a174eb97f 100644 --- a/Doc/library/ipc.rst +++ b/Doc/library/ipc.rst @@ -1,11 +1,11 @@ .. _ipc: ***************************************** -Interprocess Communication and Networking +Networking and Interprocess Communication ***************************************** -The modules described in this chapter provide mechanisms for different processes -to communicate. +The modules described in this chapter provide mechanisms for +networking and inter-processes communication. Some modules only work for two processes that are on the same machine, e.g. :mod:`signal` and :mod:`mmap`. Other modules support networking protocols @@ -15,6 +15,7 @@ The list of modules described in this chapter is: .. toctree:: + :maxdepth: 1 asyncio.rst socket.rst From webhook-mailer at python.org Fri Sep 14 23:14:19 2018 From: webhook-mailer at python.org (Ezio Melotti) Date: Sat, 15 Sep 2018 03:14:19 -0000 Subject: [Python-checkins] Add myself for HTML-related modules (#9325) Message-ID: https://github.com/python/cpython/commit/3d07349d19dda1ff82167ad363b25e0181d5388e commit: 3d07349d19dda1ff82167ad363b25e0181d5388e branch: master author: Ezio Melotti committer: GitHub date: 2018-09-14T20:14:16-07:00 summary: Add myself for HTML-related modules (#9325) files: M .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 17d7ef4d7a24..2e98e14a3d6f 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -16,6 +16,11 @@ **/*hashlib* @python/crypto-team **/*pyhash* @python/crypto-team +# HTML +/Lib/html/ @ezio-melotti +/Lib/_markupbase.py @ezio-melotti +/Lib/test/test_html*.py @ezio-melotti + # Import (including importlib). # Ignoring importlib.h so as to not get flagged on # all pull requests that change the the emitted From solipsis at pitrou.net Sat Sep 15 05:08:15 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 15 Sep 2018 09:08:15 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=11 Message-ID: <20180915090815.1.DD1BE4C786F46751@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [0, 7, 0] memory blocks, sum=7 test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogNKImLI', '--timeout', '7200'] From webhook-mailer at python.org Sat Sep 15 05:28:35 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Sat, 15 Sep 2018 09:28:35 -0000 Subject: [Python-checkins] bpo-34579: Fix test_embed DEFAULT_CON AIX (GH-9063) Message-ID: https://github.com/python/cpython/commit/d2067318c79f66cfdabc53333715a02d5fa5ff81 commit: d2067318c79f66cfdabc53333715a02d5fa5ff81 branch: master author: Michael Felt committer: Victor Stinner date: 2018-09-15T02:28:31-07:00 summary: bpo-34579: Fix test_embed DEFAULT_CON AIX (GH-9063) * Modify DEFAULT_CONFIG for AIX * bedevere/news did not like old name * Modify NEWS entry * Modified per peer review * Define and use NULL_STR constant to account for AIX libc behavior * Modify per peer review * Modify NEWS files: A Misc/NEWS.d/next/Tests/2018-09-04-15-16-42.bpo-34579.bp4HdM.rst M Lib/test/test_embed.py diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 9155c40f405e..80233a54b0b0 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -4,11 +4,15 @@ from collections import namedtuple import os +import platform import re import subprocess import sys +# AIX libc prints an empty string as '' rather than the string '(null)' +NULL_STR = '' if platform.system() == 'AIX' else '(null)' + class EmbeddingTestsMixin: def setUp(self): here = os.path.abspath(__file__) @@ -258,7 +262,7 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): 'use_environment': 1, 'use_hash_seed': 0, 'hash_seed': 0, - 'allocator': '(null)', + 'allocator': NULL_STR, 'dev_mode': 0, 'faulthandler': 0, 'tracemalloc': 0, @@ -276,11 +280,11 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): 'coerce_c_locale': 0, 'coerce_c_locale_warn': 0, - 'pycache_prefix': '(null)', + 'pycache_prefix': NULL_STR, 'program_name': './_testembed', 'argc': 0, 'argv': '[]', - 'program': '(null)', + 'program': NULL_STR, 'isolated': 0, 'site_import': 1, diff --git a/Misc/NEWS.d/next/Tests/2018-09-04-15-16-42.bpo-34579.bp4HdM.rst b/Misc/NEWS.d/next/Tests/2018-09-04-15-16-42.bpo-34579.bp4HdM.rst new file mode 100644 index 000000000000..9e01cc9cb232 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-04-15-16-42.bpo-34579.bp4HdM.rst @@ -0,0 +1,2 @@ +Fix test_embed for AIX +Patch by Michael Felt From webhook-mailer at python.org Sat Sep 15 13:32:33 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Sat, 15 Sep 2018 17:32:33 -0000 Subject: [Python-checkins] closes bpo-34515: Support non-ASCII identifiers in lib2to3. (GH-8950) Message-ID: https://github.com/python/cpython/commit/10a428b64b3f224e2ccd40ff2afb141b9b3425b1 commit: 10a428b64b3f224e2ccd40ff2afb141b9b3425b1 branch: master author: Monson Shao committer: Benjamin Peterson date: 2018-09-15T10:32:29-07:00 summary: closes bpo-34515: Support non-ASCII identifiers in lib2to3. (GH-8950) files: A Misc/NEWS.d/next/Library/2018-08-27-16-01-22.bpo-34515.S0Irst.rst M Lib/lib2to3/pgen2/tokenize.py M Lib/lib2to3/tests/test_parser.py diff --git a/Lib/lib2to3/pgen2/tokenize.py b/Lib/lib2to3/pgen2/tokenize.py index bf83ab8c7d9d..c07b34f7c645 100644 --- a/Lib/lib2to3/pgen2/tokenize.py +++ b/Lib/lib2to3/pgen2/tokenize.py @@ -56,7 +56,7 @@ def _combinations(*l): Whitespace = r'[ \f\t]*' Comment = r'#[^\r\n]*' Ignore = Whitespace + any(r'\\\r?\n' + Whitespace) + maybe(Comment) -Name = r'[a-zA-Z_]\w*' +Name = r'\w+' Binnumber = r'0[bB]_?[01]+(?:_[01]+)*' Hexnumber = r'0[xX]_?[\da-fA-F]+(?:_[\da-fA-F]+)*[lL]?' @@ -107,8 +107,8 @@ def _combinations(*l): PseudoExtras = group(r'\\\r?\n', Comment, Triple) PseudoToken = Whitespace + group(PseudoExtras, Number, Funny, ContStr, Name) -tokenprog, pseudoprog, single3prog, double3prog = list(map( - re.compile, (Token, PseudoToken, Single3, Double3))) +tokenprog, pseudoprog, single3prog, double3prog = map( + re.compile, (Token, PseudoToken, Single3, Double3)) _strprefixes = ( _combinations('r', 'R', 'f', 'F') | @@ -349,7 +349,6 @@ def generate_tokens(readline): logical line; continuation lines are included. """ lnum = parenlev = continued = 0 - namechars, numchars = string.ascii_letters + '_', '0123456789' contstr, needcont = '', 0 contline = None indents = [0] @@ -451,7 +450,7 @@ def generate_tokens(readline): spos, epos, pos = (lnum, start), (lnum, end), end token, initial = line[start:end], line[start] - if initial in numchars or \ + if initial in string.digits or \ (initial == '.' and token != '.'): # ordinary number yield (NUMBER, token, spos, epos, line) elif initial in '\r\n': @@ -501,7 +500,7 @@ def generate_tokens(readline): yield stashed stashed = None yield (STRING, token, spos, epos, line) - elif initial in namechars: # ordinary name + elif initial.isidentifier(): # ordinary name if token in ('async', 'await'): if async_def: yield (ASYNC if token == 'async' else AWAIT, diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py index 7eb9afac46e0..829e5a72924a 100644 --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -529,6 +529,16 @@ def test_4(self): self.validate("""x = {2, 3, 4,}""") +# Adapted from Python 3's Lib/test/test_unicode_identifiers.py and +# Lib/test/test_tokenize.py:TokenizeTest.test_non_ascii_identifiers +class TestIdentfier(GrammarTest): + def test_non_ascii_identifiers(self): + self.validate("?rter = 'places'\ngr?n = 'green'") + self.validate("? = a? = ?? = 1") + self.validate("? = a? = ?? = 1") + self.validate("??????? = a_??????? = 1") + + class TestNumericLiterals(GrammarTest): def test_new_octal_notation(self): self.validate("""0o7777777777777""") diff --git a/Misc/NEWS.d/next/Library/2018-08-27-16-01-22.bpo-34515.S0Irst.rst b/Misc/NEWS.d/next/Library/2018-08-27-16-01-22.bpo-34515.S0Irst.rst new file mode 100644 index 000000000000..7ace7ba8218c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-08-27-16-01-22.bpo-34515.S0Irst.rst @@ -0,0 +1 @@ +Fix parsing non-ASCII identifiers in :mod:`lib2to3.pgen2.tokenize` (PEP 3131). From webhook-mailer at python.org Sat Sep 15 13:53:02 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 15 Sep 2018 17:53:02 -0000 Subject: [Python-checkins] closes bpo-34515: Support non-ASCII identifiers in lib2to3. (GH-8950) Message-ID: https://github.com/python/cpython/commit/51dbae867e82014f9af89662977e4981463c51e8 commit: 51dbae867e82014f9af89662977e4981463c51e8 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-15T10:52:57-07:00 summary: closes bpo-34515: Support non-ASCII identifiers in lib2to3. (GH-8950) (cherry picked from commit 10a428b64b3f224e2ccd40ff2afb141b9b3425b1) Co-authored-by: Monson Shao files: A Misc/NEWS.d/next/Library/2018-08-27-16-01-22.bpo-34515.S0Irst.rst M Lib/lib2to3/pgen2/tokenize.py M Lib/lib2to3/tests/test_parser.py diff --git a/Lib/lib2to3/pgen2/tokenize.py b/Lib/lib2to3/pgen2/tokenize.py index bf83ab8c7d9d..c07b34f7c645 100644 --- a/Lib/lib2to3/pgen2/tokenize.py +++ b/Lib/lib2to3/pgen2/tokenize.py @@ -56,7 +56,7 @@ def _combinations(*l): Whitespace = r'[ \f\t]*' Comment = r'#[^\r\n]*' Ignore = Whitespace + any(r'\\\r?\n' + Whitespace) + maybe(Comment) -Name = r'[a-zA-Z_]\w*' +Name = r'\w+' Binnumber = r'0[bB]_?[01]+(?:_[01]+)*' Hexnumber = r'0[xX]_?[\da-fA-F]+(?:_[\da-fA-F]+)*[lL]?' @@ -107,8 +107,8 @@ def _combinations(*l): PseudoExtras = group(r'\\\r?\n', Comment, Triple) PseudoToken = Whitespace + group(PseudoExtras, Number, Funny, ContStr, Name) -tokenprog, pseudoprog, single3prog, double3prog = list(map( - re.compile, (Token, PseudoToken, Single3, Double3))) +tokenprog, pseudoprog, single3prog, double3prog = map( + re.compile, (Token, PseudoToken, Single3, Double3)) _strprefixes = ( _combinations('r', 'R', 'f', 'F') | @@ -349,7 +349,6 @@ def generate_tokens(readline): logical line; continuation lines are included. """ lnum = parenlev = continued = 0 - namechars, numchars = string.ascii_letters + '_', '0123456789' contstr, needcont = '', 0 contline = None indents = [0] @@ -451,7 +450,7 @@ def generate_tokens(readline): spos, epos, pos = (lnum, start), (lnum, end), end token, initial = line[start:end], line[start] - if initial in numchars or \ + if initial in string.digits or \ (initial == '.' and token != '.'): # ordinary number yield (NUMBER, token, spos, epos, line) elif initial in '\r\n': @@ -501,7 +500,7 @@ def generate_tokens(readline): yield stashed stashed = None yield (STRING, token, spos, epos, line) - elif initial in namechars: # ordinary name + elif initial.isidentifier(): # ordinary name if token in ('async', 'await'): if async_def: yield (ASYNC if token == 'async' else AWAIT, diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py index 7eb9afac46e0..829e5a72924a 100644 --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -529,6 +529,16 @@ def test_4(self): self.validate("""x = {2, 3, 4,}""") +# Adapted from Python 3's Lib/test/test_unicode_identifiers.py and +# Lib/test/test_tokenize.py:TokenizeTest.test_non_ascii_identifiers +class TestIdentfier(GrammarTest): + def test_non_ascii_identifiers(self): + self.validate("?rter = 'places'\ngr?n = 'green'") + self.validate("? = a? = ?? = 1") + self.validate("? = a? = ?? = 1") + self.validate("??????? = a_??????? = 1") + + class TestNumericLiterals(GrammarTest): def test_new_octal_notation(self): self.validate("""0o7777777777777""") diff --git a/Misc/NEWS.d/next/Library/2018-08-27-16-01-22.bpo-34515.S0Irst.rst b/Misc/NEWS.d/next/Library/2018-08-27-16-01-22.bpo-34515.S0Irst.rst new file mode 100644 index 000000000000..7ace7ba8218c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-08-27-16-01-22.bpo-34515.S0Irst.rst @@ -0,0 +1 @@ +Fix parsing non-ASCII identifiers in :mod:`lib2to3.pgen2.tokenize` (PEP 3131). From webhook-mailer at python.org Sun Sep 16 01:36:33 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Sun, 16 Sep 2018 05:36:33 -0000 Subject: [Python-checkins] bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) Message-ID: https://github.com/python/cpython/commit/7843caeb909bd907e903606414e238db4082315a commit: 7843caeb909bd907e903606414e238db4082315a branch: master author: Vladimir Matveev committer: Victor Stinner date: 2018-09-15T22:36:29-07:00 summary: bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) files: A Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst M Lib/ctypes/test/test_win32.py M Modules/_ctypes/_ctypes_test.c M Modules/_ctypes/callproc.c M Modules/_ctypes/libffi_msvc/ffi.c M Modules/_ctypes/libffi_msvc/ffi.h M Modules/_ctypes/libffi_msvc/prep_cif.c diff --git a/Lib/ctypes/test/test_win32.py b/Lib/ctypes/test/test_win32.py index 5d85ad6200b3..ee722704a35d 100644 --- a/Lib/ctypes/test/test_win32.py +++ b/Lib/ctypes/test/test_win32.py @@ -54,6 +54,24 @@ def test_noargs(self): windll.user32.GetDesktopWindow() + at unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') +class ReturnStructSizesTestCase(unittest.TestCase): + def test_sizes(self): + dll = CDLL(_ctypes_test.__file__) + for i in range(1, 11): + fields = [ (f"f{f}", c_char) for f in range(1, i + 1)] + class S(Structure): + _fields_ = fields + f = getattr(dll, f"TestSize{i}") + f.restype = S + res = f() + for i, f in enumerate(fields): + value = getattr(res, f[0]) + expected = bytes([ord('a') + i]) + self.assertEquals(value, expected) + + + @unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') class TestWintypes(unittest.TestCase): def test_HWND(self): diff --git a/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst b/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst new file mode 100644 index 000000000000..86ae1cd06171 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst @@ -0,0 +1 @@ +Fix returning structs from functions produced by MSVC diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c index 620a3c6aea6e..0152945ca1ad 100644 --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -660,6 +660,200 @@ EXPORT(void) TwoOutArgs(int a, int *pi, int b, int *pj) *pj += b; } +#ifdef MS_WIN32 + +typedef struct { + char f1; +} Size1; + +typedef struct { + char f1; + char f2; +} Size2; + +typedef struct { + char f1; + char f2; + char f3; +} Size3; + +typedef struct { + char f1; + char f2; + char f3; + char f4; +} Size4; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; +} Size5; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; +} Size6; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; +} Size7; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; +} Size8; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; + char f9; +} Size9; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; + char f9; + char f10; +} Size10; + +EXPORT(Size1) TestSize1() { + Size1 f; + f.f1 = 'a'; + return f; +} + +EXPORT(Size2) TestSize2() { + Size2 f; + f.f1 = 'a'; + f.f2 = 'b'; + return f; +} + +EXPORT(Size3) TestSize3() { + Size3 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + return f; +} + +EXPORT(Size4) TestSize4() { + Size4 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + return f; +} + +EXPORT(Size5) TestSize5() { + Size5 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + return f; +} + +EXPORT(Size6) TestSize6() { + Size6 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + return f; +} + +EXPORT(Size7) TestSize7() { + Size7 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + return f; +} + +EXPORT(Size8) TestSize8() { + Size8 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + return f; +} + +EXPORT(Size9) TestSize9() { + Size9 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + f.f9 = 'i'; + return f; +} + +EXPORT(Size10) TestSize10() { + Size10 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + f.f9 = 'i'; + f.f10 = 'j'; + return f; +} + +#endif + #ifdef MS_WIN32 EXPORT(S2H) __stdcall s_ret_2h_func(S2H inp) { return ret_2h_func(inp); } EXPORT(S8I) __stdcall s_ret_8i_func(S8I inp) { return ret_8i_func(inp); } diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index bdc372811598..ba154fe61b6a 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -715,9 +715,9 @@ ffi_type *_ctypes_get_ffi_type(PyObject *obj) It returns small structures in registers */ if (dict->ffi_type_pointer.type == FFI_TYPE_STRUCT) { - if (dict->ffi_type_pointer.size <= 4) + if (can_return_struct_as_int(dict->ffi_type_pointer.size)) return &ffi_type_sint32; - else if (dict->ffi_type_pointer.size <= 8) + else if (can_return_struct_as_sint64 (dict->ffi_type_pointer.size)) return &ffi_type_sint64; } #endif diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c index 91a27dce3f25..d202b158b079 100644 --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -145,6 +145,21 @@ void ffi_prep_args(char *stack, extended_cif *ecif) return; } +/* +Per: https://msdn.microsoft.com/en-us/library/7572ztz4.aspx +To be returned by value in RAX, user-defined types must have a length +of 1, 2, 4, 8, 16, 32, or 64 bits +*/ +int can_return_struct_as_int(size_t s) +{ + return s == 1 || s == 2 || s == 4; +} + +int can_return_struct_as_sint64(size_t s) +{ + return s == 8; +} + /* Perform machine dependent cif processing */ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) { @@ -163,9 +178,9 @@ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) /* MSVC returns small structures in registers. Put in cif->flags the value FFI_TYPE_STRUCT only if the structure is big enough; otherwise, put the 4- or 8-bytes integer type. */ - if (cif->rtype->size <= 4) + if (can_return_struct_as_int(cif->rtype->size)) cif->flags = FFI_TYPE_INT; - else if (cif->rtype->size <= 8) + else if (can_return_struct_as_sint64(cif->rtype->size)) cif->flags = FFI_TYPE_SINT64; else cif->flags = FFI_TYPE_STRUCT; diff --git a/Modules/_ctypes/libffi_msvc/ffi.h b/Modules/_ctypes/libffi_msvc/ffi.h index efb14c5f6f3a..ba74202720a6 100644 --- a/Modules/_ctypes/libffi_msvc/ffi.h +++ b/Modules/_ctypes/libffi_msvc/ffi.h @@ -136,6 +136,9 @@ typedef struct _ffi_type /*@null@*/ struct _ffi_type **elements; } ffi_type; +int can_return_struct_as_int(size_t); +int can_return_struct_as_sint64(size_t); + /* These are defined in types.c */ extern ffi_type ffi_type_void; extern ffi_type ffi_type_uint8; diff --git a/Modules/_ctypes/libffi_msvc/prep_cif.c b/Modules/_ctypes/libffi_msvc/prep_cif.c index b07a2e6db271..022435e53fcd 100644 --- a/Modules/_ctypes/libffi_msvc/prep_cif.c +++ b/Modules/_ctypes/libffi_msvc/prep_cif.c @@ -117,7 +117,8 @@ ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, /* Make space for the return structure pointer */ if (cif->rtype->type == FFI_TYPE_STRUCT #ifdef _WIN32 - && (cif->rtype->size > 8) /* MSVC returns small structs in registers */ + && !can_return_struct_as_int(cif->rtype->size) /* MSVC returns small structs in registers */ + && !can_return_struct_as_sint64(cif->rtype->size) #endif #ifdef SPARC && (cif->abi != FFI_V9 || cif->rtype->size > 32) @@ -146,7 +147,9 @@ ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, bytes += sizeof(void*); else #elif defined (_WIN64) - if ((*ptr)->type == FFI_TYPE_STRUCT && ((*ptr)->size > 8)) + if ((*ptr)->type == FFI_TYPE_STRUCT && + !can_return_struct_as_int((*ptr)->size) && + !can_return_struct_as_sint64((*ptr)->size)) bytes += sizeof(void*); else #endif From webhook-mailer at python.org Sun Sep 16 01:53:16 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 16 Sep 2018 05:53:16 -0000 Subject: [Python-checkins] bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) Message-ID: https://github.com/python/cpython/commit/e3f6aa7fe48b91f4ff619b2a51d473249d620bcb commit: e3f6aa7fe48b91f4ff619b2a51d473249d620bcb branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-15T22:53:13-07:00 summary: bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) (cherry picked from commit 7843caeb909bd907e903606414e238db4082315a) Co-authored-by: Vladimir Matveev files: A Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst M Lib/ctypes/test/test_win32.py M Modules/_ctypes/_ctypes_test.c M Modules/_ctypes/callproc.c M Modules/_ctypes/libffi_msvc/ffi.c M Modules/_ctypes/libffi_msvc/ffi.h M Modules/_ctypes/libffi_msvc/prep_cif.c diff --git a/Lib/ctypes/test/test_win32.py b/Lib/ctypes/test/test_win32.py index 5d85ad6200b3..ee722704a35d 100644 --- a/Lib/ctypes/test/test_win32.py +++ b/Lib/ctypes/test/test_win32.py @@ -54,6 +54,24 @@ def test_noargs(self): windll.user32.GetDesktopWindow() + at unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') +class ReturnStructSizesTestCase(unittest.TestCase): + def test_sizes(self): + dll = CDLL(_ctypes_test.__file__) + for i in range(1, 11): + fields = [ (f"f{f}", c_char) for f in range(1, i + 1)] + class S(Structure): + _fields_ = fields + f = getattr(dll, f"TestSize{i}") + f.restype = S + res = f() + for i, f in enumerate(fields): + value = getattr(res, f[0]) + expected = bytes([ord('a') + i]) + self.assertEquals(value, expected) + + + @unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') class TestWintypes(unittest.TestCase): def test_HWND(self): diff --git a/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst b/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst new file mode 100644 index 000000000000..86ae1cd06171 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst @@ -0,0 +1 @@ +Fix returning structs from functions produced by MSVC diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c index 620a3c6aea6e..0152945ca1ad 100644 --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -660,6 +660,200 @@ EXPORT(void) TwoOutArgs(int a, int *pi, int b, int *pj) *pj += b; } +#ifdef MS_WIN32 + +typedef struct { + char f1; +} Size1; + +typedef struct { + char f1; + char f2; +} Size2; + +typedef struct { + char f1; + char f2; + char f3; +} Size3; + +typedef struct { + char f1; + char f2; + char f3; + char f4; +} Size4; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; +} Size5; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; +} Size6; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; +} Size7; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; +} Size8; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; + char f9; +} Size9; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; + char f9; + char f10; +} Size10; + +EXPORT(Size1) TestSize1() { + Size1 f; + f.f1 = 'a'; + return f; +} + +EXPORT(Size2) TestSize2() { + Size2 f; + f.f1 = 'a'; + f.f2 = 'b'; + return f; +} + +EXPORT(Size3) TestSize3() { + Size3 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + return f; +} + +EXPORT(Size4) TestSize4() { + Size4 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + return f; +} + +EXPORT(Size5) TestSize5() { + Size5 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + return f; +} + +EXPORT(Size6) TestSize6() { + Size6 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + return f; +} + +EXPORT(Size7) TestSize7() { + Size7 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + return f; +} + +EXPORT(Size8) TestSize8() { + Size8 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + return f; +} + +EXPORT(Size9) TestSize9() { + Size9 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + f.f9 = 'i'; + return f; +} + +EXPORT(Size10) TestSize10() { + Size10 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + f.f9 = 'i'; + f.f10 = 'j'; + return f; +} + +#endif + #ifdef MS_WIN32 EXPORT(S2H) __stdcall s_ret_2h_func(S2H inp) { return ret_2h_func(inp); } EXPORT(S8I) __stdcall s_ret_8i_func(S8I inp) { return ret_8i_func(inp); } diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index bdc372811598..ba154fe61b6a 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -715,9 +715,9 @@ ffi_type *_ctypes_get_ffi_type(PyObject *obj) It returns small structures in registers */ if (dict->ffi_type_pointer.type == FFI_TYPE_STRUCT) { - if (dict->ffi_type_pointer.size <= 4) + if (can_return_struct_as_int(dict->ffi_type_pointer.size)) return &ffi_type_sint32; - else if (dict->ffi_type_pointer.size <= 8) + else if (can_return_struct_as_sint64 (dict->ffi_type_pointer.size)) return &ffi_type_sint64; } #endif diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c index 91a27dce3f25..d202b158b079 100644 --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -145,6 +145,21 @@ void ffi_prep_args(char *stack, extended_cif *ecif) return; } +/* +Per: https://msdn.microsoft.com/en-us/library/7572ztz4.aspx +To be returned by value in RAX, user-defined types must have a length +of 1, 2, 4, 8, 16, 32, or 64 bits +*/ +int can_return_struct_as_int(size_t s) +{ + return s == 1 || s == 2 || s == 4; +} + +int can_return_struct_as_sint64(size_t s) +{ + return s == 8; +} + /* Perform machine dependent cif processing */ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) { @@ -163,9 +178,9 @@ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) /* MSVC returns small structures in registers. Put in cif->flags the value FFI_TYPE_STRUCT only if the structure is big enough; otherwise, put the 4- or 8-bytes integer type. */ - if (cif->rtype->size <= 4) + if (can_return_struct_as_int(cif->rtype->size)) cif->flags = FFI_TYPE_INT; - else if (cif->rtype->size <= 8) + else if (can_return_struct_as_sint64(cif->rtype->size)) cif->flags = FFI_TYPE_SINT64; else cif->flags = FFI_TYPE_STRUCT; diff --git a/Modules/_ctypes/libffi_msvc/ffi.h b/Modules/_ctypes/libffi_msvc/ffi.h index efb14c5f6f3a..ba74202720a6 100644 --- a/Modules/_ctypes/libffi_msvc/ffi.h +++ b/Modules/_ctypes/libffi_msvc/ffi.h @@ -136,6 +136,9 @@ typedef struct _ffi_type /*@null@*/ struct _ffi_type **elements; } ffi_type; +int can_return_struct_as_int(size_t); +int can_return_struct_as_sint64(size_t); + /* These are defined in types.c */ extern ffi_type ffi_type_void; extern ffi_type ffi_type_uint8; diff --git a/Modules/_ctypes/libffi_msvc/prep_cif.c b/Modules/_ctypes/libffi_msvc/prep_cif.c index b07a2e6db271..022435e53fcd 100644 --- a/Modules/_ctypes/libffi_msvc/prep_cif.c +++ b/Modules/_ctypes/libffi_msvc/prep_cif.c @@ -117,7 +117,8 @@ ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, /* Make space for the return structure pointer */ if (cif->rtype->type == FFI_TYPE_STRUCT #ifdef _WIN32 - && (cif->rtype->size > 8) /* MSVC returns small structs in registers */ + && !can_return_struct_as_int(cif->rtype->size) /* MSVC returns small structs in registers */ + && !can_return_struct_as_sint64(cif->rtype->size) #endif #ifdef SPARC && (cif->abi != FFI_V9 || cif->rtype->size > 32) @@ -146,7 +147,9 @@ ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, bytes += sizeof(void*); else #elif defined (_WIN64) - if ((*ptr)->type == FFI_TYPE_STRUCT && ((*ptr)->size > 8)) + if ((*ptr)->type == FFI_TYPE_STRUCT && + !can_return_struct_as_int((*ptr)->size) && + !can_return_struct_as_sint64((*ptr)->size)) bytes += sizeof(void*); else #endif From webhook-mailer at python.org Sun Sep 16 01:57:49 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 16 Sep 2018 05:57:49 -0000 Subject: [Python-checkins] bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) Message-ID: https://github.com/python/cpython/commit/e53632019816749ffd5be0afab2a99d744dbbe35 commit: e53632019816749ffd5be0afab2a99d744dbbe35 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-15T22:57:46-07:00 summary: bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) (cherry picked from commit 7843caeb909bd907e903606414e238db4082315a) Co-authored-by: Vladimir Matveev files: A Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst M Lib/ctypes/test/test_win32.py M Modules/_ctypes/_ctypes_test.c M Modules/_ctypes/callproc.c M Modules/_ctypes/libffi_msvc/ffi.c M Modules/_ctypes/libffi_msvc/ffi.h M Modules/_ctypes/libffi_msvc/prep_cif.c diff --git a/Lib/ctypes/test/test_win32.py b/Lib/ctypes/test/test_win32.py index 5d85ad6200b3..ee722704a35d 100644 --- a/Lib/ctypes/test/test_win32.py +++ b/Lib/ctypes/test/test_win32.py @@ -54,6 +54,24 @@ def test_noargs(self): windll.user32.GetDesktopWindow() + at unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') +class ReturnStructSizesTestCase(unittest.TestCase): + def test_sizes(self): + dll = CDLL(_ctypes_test.__file__) + for i in range(1, 11): + fields = [ (f"f{f}", c_char) for f in range(1, i + 1)] + class S(Structure): + _fields_ = fields + f = getattr(dll, f"TestSize{i}") + f.restype = S + res = f() + for i, f in enumerate(fields): + value = getattr(res, f[0]) + expected = bytes([ord('a') + i]) + self.assertEquals(value, expected) + + + @unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') class TestWintypes(unittest.TestCase): def test_HWND(self): diff --git a/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst b/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst new file mode 100644 index 000000000000..86ae1cd06171 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst @@ -0,0 +1 @@ +Fix returning structs from functions produced by MSVC diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c index f6af49aea385..6ffbeab8bf63 100644 --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -662,6 +662,200 @@ EXPORT(void) TwoOutArgs(int a, int *pi, int b, int *pj) *pj += b; } +#ifdef MS_WIN32 + +typedef struct { + char f1; +} Size1; + +typedef struct { + char f1; + char f2; +} Size2; + +typedef struct { + char f1; + char f2; + char f3; +} Size3; + +typedef struct { + char f1; + char f2; + char f3; + char f4; +} Size4; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; +} Size5; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; +} Size6; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; +} Size7; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; +} Size8; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; + char f9; +} Size9; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; + char f9; + char f10; +} Size10; + +EXPORT(Size1) TestSize1() { + Size1 f; + f.f1 = 'a'; + return f; +} + +EXPORT(Size2) TestSize2() { + Size2 f; + f.f1 = 'a'; + f.f2 = 'b'; + return f; +} + +EXPORT(Size3) TestSize3() { + Size3 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + return f; +} + +EXPORT(Size4) TestSize4() { + Size4 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + return f; +} + +EXPORT(Size5) TestSize5() { + Size5 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + return f; +} + +EXPORT(Size6) TestSize6() { + Size6 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + return f; +} + +EXPORT(Size7) TestSize7() { + Size7 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + return f; +} + +EXPORT(Size8) TestSize8() { + Size8 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + return f; +} + +EXPORT(Size9) TestSize9() { + Size9 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + f.f9 = 'i'; + return f; +} + +EXPORT(Size10) TestSize10() { + Size10 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + f.f9 = 'i'; + f.f10 = 'j'; + return f; +} + +#endif + #ifdef MS_WIN32 EXPORT(S2H) __stdcall s_ret_2h_func(S2H inp) { return ret_2h_func(inp); } EXPORT(S8I) __stdcall s_ret_8i_func(S8I inp) { return ret_8i_func(inp); } diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index 8edffabcdf09..b8c2c746a0fe 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -715,9 +715,9 @@ ffi_type *_ctypes_get_ffi_type(PyObject *obj) It returns small structures in registers */ if (dict->ffi_type_pointer.type == FFI_TYPE_STRUCT) { - if (dict->ffi_type_pointer.size <= 4) + if (can_return_struct_as_int(dict->ffi_type_pointer.size)) return &ffi_type_sint32; - else if (dict->ffi_type_pointer.size <= 8) + else if (can_return_struct_as_sint64 (dict->ffi_type_pointer.size)) return &ffi_type_sint64; } #endif diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c index 91a27dce3f25..d202b158b079 100644 --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -145,6 +145,21 @@ void ffi_prep_args(char *stack, extended_cif *ecif) return; } +/* +Per: https://msdn.microsoft.com/en-us/library/7572ztz4.aspx +To be returned by value in RAX, user-defined types must have a length +of 1, 2, 4, 8, 16, 32, or 64 bits +*/ +int can_return_struct_as_int(size_t s) +{ + return s == 1 || s == 2 || s == 4; +} + +int can_return_struct_as_sint64(size_t s) +{ + return s == 8; +} + /* Perform machine dependent cif processing */ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) { @@ -163,9 +178,9 @@ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) /* MSVC returns small structures in registers. Put in cif->flags the value FFI_TYPE_STRUCT only if the structure is big enough; otherwise, put the 4- or 8-bytes integer type. */ - if (cif->rtype->size <= 4) + if (can_return_struct_as_int(cif->rtype->size)) cif->flags = FFI_TYPE_INT; - else if (cif->rtype->size <= 8) + else if (can_return_struct_as_sint64(cif->rtype->size)) cif->flags = FFI_TYPE_SINT64; else cif->flags = FFI_TYPE_STRUCT; diff --git a/Modules/_ctypes/libffi_msvc/ffi.h b/Modules/_ctypes/libffi_msvc/ffi.h index efb14c5f6f3a..ba74202720a6 100644 --- a/Modules/_ctypes/libffi_msvc/ffi.h +++ b/Modules/_ctypes/libffi_msvc/ffi.h @@ -136,6 +136,9 @@ typedef struct _ffi_type /*@null@*/ struct _ffi_type **elements; } ffi_type; +int can_return_struct_as_int(size_t); +int can_return_struct_as_sint64(size_t); + /* These are defined in types.c */ extern ffi_type ffi_type_void; extern ffi_type ffi_type_uint8; diff --git a/Modules/_ctypes/libffi_msvc/prep_cif.c b/Modules/_ctypes/libffi_msvc/prep_cif.c index b07a2e6db271..022435e53fcd 100644 --- a/Modules/_ctypes/libffi_msvc/prep_cif.c +++ b/Modules/_ctypes/libffi_msvc/prep_cif.c @@ -117,7 +117,8 @@ ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, /* Make space for the return structure pointer */ if (cif->rtype->type == FFI_TYPE_STRUCT #ifdef _WIN32 - && (cif->rtype->size > 8) /* MSVC returns small structs in registers */ + && !can_return_struct_as_int(cif->rtype->size) /* MSVC returns small structs in registers */ + && !can_return_struct_as_sint64(cif->rtype->size) #endif #ifdef SPARC && (cif->abi != FFI_V9 || cif->rtype->size > 32) @@ -146,7 +147,9 @@ ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, bytes += sizeof(void*); else #elif defined (_WIN64) - if ((*ptr)->type == FFI_TYPE_STRUCT && ((*ptr)->size > 8)) + if ((*ptr)->type == FFI_TYPE_STRUCT && + !can_return_struct_as_int((*ptr)->size) && + !can_return_struct_as_sint64((*ptr)->size)) bytes += sizeof(void*); else #endif From webhook-mailer at python.org Sun Sep 16 02:09:17 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 16 Sep 2018 06:09:17 -0000 Subject: [Python-checkins] [2.7] bpo-19417: Add test_bdb.py (GH-5217) (GH-6156) Message-ID: https://github.com/python/cpython/commit/57e70d3802a2a78e638999c6923053c63fe373f8 commit: 57e70d3802a2a78e638999c6923053c63fe373f8 branch: 2.7 author: xdegaye committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-15T23:09:14-07:00 summary: [2.7] bpo-19417: Add test_bdb.py (GH-5217) (GH-6156) (cherry picked from commit 3fe33043ee83d19e15551094fc1e0984617ded3c) https://bugs.python.org/issue19417 files: A Lib/test/test_bdb.py A Misc/NEWS.d/next/Tests/2018-01-08-13-33-47.bpo-19417.2asoXy.rst M Lib/test/test_sundry.py diff --git a/Lib/test/test_bdb.py b/Lib/test/test_bdb.py new file mode 100644 index 000000000000..4fb7c79cdcc5 --- /dev/null +++ b/Lib/test/test_bdb.py @@ -0,0 +1,1034 @@ +""" Test the bdb module. + + A test defines a list of tuples that may be seen as paired tuples, each + pair being defined by 'expect_tuple, set_tuple' as follows: + + ([event, [lineno[, co_name[, eargs]]]]), (set_type, [sargs]) + + * 'expect_tuple' describes the expected current state of the Bdb instance. + It may be the empty tuple and no check is done in that case. + * 'set_tuple' defines the set_*() method to be invoked when the Bdb + instance reaches this state. + + Example of an 'expect_tuple, set_tuple' pair: + + ('line', 2, 'tfunc_main'), ('step', ) + + Definitions of the members of the 'expect_tuple': + event: + Name of the trace event. The set methods that do not give back + control to the tracer [1] do not trigger a tracer event and in + that case the next 'event' may be 'None' by convention, its value + is not checked. + [1] Methods that trigger a trace event are set_step(), set_next(), + set_return(), set_until() and set_continue(). + lineno: + Line number. Line numbers are relative to the start of the + function when tracing a function in the test_bdb module (i.e. this + module). + co_name: + Name of the function being currently traced. + eargs: + A tuple: + * On an 'exception' event the tuple holds a class object, the + current exception must be an instance of this class. + * On a 'line' event, the tuple holds a dictionary and a list. The + dictionary maps each breakpoint number that has been hit on this + line to its hits count. The list holds the list of breakpoint + number temporaries that are being deleted. + + Definitions of the members of the 'set_tuple': + set_type: + The type of the set method to be invoked. This may + be the type of one of the Bdb set methods: 'step', 'next', + 'until', 'return', 'continue', 'break', 'quit', or the type of one + of the set methods added by test_bdb.Bdb: 'ignore', 'enable', + 'disable', 'clear', 'up', 'down'. + sargs: + The arguments of the set method if any, packed in a tuple. +""" + +from __future__ import absolute_import +from __future__ import print_function + +import bdb as _bdb +import sys +import os +import unittest +import textwrap +import importlib +import linecache +from contextlib import contextmanager +from itertools import islice, repeat +import test.support + +class BdbException(Exception): pass +class BdbError(BdbException): """Error raised by the Bdb instance.""" +class BdbSyntaxError(BdbException): """Syntax error in the test case.""" +class BdbNotExpectedError(BdbException): """Unexpected result.""" + +# When 'dry_run' is set to true, expect tuples are ignored and the actual +# state of the tracer is printed after running each set_*() method of the test +# case. The full list of breakpoints and their attributes is also printed +# after each 'line' event where a breakpoint has been hit. +dry_run = 0 + +__file__ = os.path.splitext(__file__)[0] + '.py' + +def reset_Breakpoint(): + _bdb.Breakpoint.next = 1 + _bdb.Breakpoint.bplist = {} + _bdb.Breakpoint.bpbynumber = [None] + +def info_breakpoints(): + bp_list = [bp for bp in _bdb.Breakpoint.bpbynumber if bp] + if not bp_list: + return '' + + header_added = False + for bp in bp_list: + if not header_added: + info = 'BpNum Temp Enb Hits Ignore Where\n' + header_added = True + + disp = 'yes ' if bp.temporary else 'no ' + enab = 'yes' if bp.enabled else 'no ' + info += ('%-5d %s %s %-4d %-6d at %s:%d' % + (bp.number, disp, enab, bp.hits, bp.ignore, + os.path.basename(bp.file), bp.line)) + if bp.cond: + info += '\n\tstop only if %s' % (bp.cond,) + info += '\n' + return info + +class Bdb(_bdb.Bdb, object): + """Extend Bdb to enhance test coverage.""" + + def trace_dispatch(self, frame, event, arg): + self.currentbp = None + return super(Bdb, self).trace_dispatch(frame, event, arg) + + def set_break(self, filename, lineno, temporary=False, cond=None, + funcname=None): + if isinstance(funcname, str): + if filename == __file__: + globals_ = globals() + else: + module = importlib.import_module(filename[:-3]) + globals_ = module.__dict__ + func = eval(funcname, globals_) + code = func.__code__ + filename = code.co_filename + lineno = code.co_firstlineno + funcname = code.co_name + + res = super(Bdb, self).set_break(filename, lineno, + temporary=temporary, cond=cond, funcname=funcname) + if isinstance(res, str): + raise BdbError(res) + return res + + # Back port of get_bpbynumber() from bdb.Bdb in Python 3. + def get_bpbynumber(self, arg): + """Return a breakpoint by its index in Breakpoint.bybpnumber. + + For invalid arg values or if the breakpoint doesn't exist, + raise a ValueError. + """ + if not arg: + raise ValueError('Breakpoint number expected') + try: + number = int(arg) + except ValueError: + raise ValueError('Non-numeric breakpoint number %s' % arg) + try: + bp = _bdb.Breakpoint.bpbynumber[number] + except IndexError: + raise ValueError('Breakpoint number %d out of range' % number) + if bp is None: + raise ValueError('Breakpoint %d already deleted' % number) + return bp + + def get_stack(self, f, t): + self.stack, self.index = super(Bdb, self).get_stack(f, t) + self.frame = self.stack[self.index][0] + return self.stack, self.index + + def set_ignore(self, bpnum): + """Increment the ignore count of Breakpoint number 'bpnum'.""" + bp = self.get_bpbynumber(bpnum) + bp.ignore += 1 + + def set_enable(self, bpnum): + bp = self.get_bpbynumber(bpnum) + bp.enabled = True + + def set_disable(self, bpnum): + bp = self.get_bpbynumber(bpnum) + bp.enabled = False + + def set_clear(self, fname, lineno): + err = self.clear_break(fname, lineno) + if err: + raise BdbError(err) + + def set_up(self): + """Move up in the frame stack.""" + if not self.index: + raise BdbError('Oldest frame') + self.index -= 1 + self.frame = self.stack[self.index][0] + + def set_down(self): + """Move down in the frame stack.""" + if self.index + 1 == len(self.stack): + raise BdbError('Newest frame') + self.index += 1 + self.frame = self.stack[self.index][0] + +class Tracer(Bdb): + """A tracer for testing the bdb module.""" + + def __init__(self, expect_set, skip=None, dry_run=False, test_case=None): + super(Tracer, self).__init__(skip=skip) + self.expect_set = expect_set + self.dry_run = dry_run + self.header = ('Dry-run results for %s:' % test_case if + test_case is not None else None) + self.init_test() + + def init_test(self): + self.cur_except = None + self.expect_set_no = 0 + self.breakpoint_hits = None + self.expected_list = list(islice(self.expect_set, 0, None, 2)) + self.set_list = list(islice(self.expect_set, 1, None, 2)) + + def trace_dispatch(self, frame, event, arg): + # On an 'exception' event, call_exc_trace() in Python/ceval.c discards + # a BdbException raised by the Tracer instance, so we raise it on the + # next trace_dispatch() call that occurs unless the set_quit() or + # set_continue() method has been invoked on the 'exception' event. + if self.cur_except is not None: + raise self.cur_except + + if event == 'exception': + try: + res = super(Tracer, self).trace_dispatch(frame, event, arg) + return res + except BdbException as e: + self.cur_except = e + return self.trace_dispatch + else: + return super(Tracer, self).trace_dispatch(frame, event, arg) + + def user_call(self, frame, argument_list): + # Adopt the same behavior as pdb and, as a side effect, skip also the + # first 'call' event when the Tracer is started with Tracer.runcall() + # which may be possibly considered as a bug. + if not self.stop_here(frame): + return + self.process_event('call', frame, argument_list) + self.next_set_method() + + def user_line(self, frame): + self.process_event('line', frame) + + if self.dry_run and self.breakpoint_hits: + info = info_breakpoints().strip('\n') + # Indent each line. + for line in info.split('\n'): + print(' ' + line) + self.delete_temporaries() + self.breakpoint_hits = None + + self.next_set_method() + + def user_return(self, frame, return_value): + self.process_event('return', frame, return_value) + self.next_set_method() + + def user_exception(self, frame, exc_info): + self.exc_info = exc_info + self.process_event('exception', frame) + self.next_set_method() + + def do_clear(self, arg): + # The temporary breakpoints are deleted in user_line(). + bp_list = [self.currentbp] + self.breakpoint_hits = (bp_list, bp_list) + + def delete_temporaries(self): + if self.breakpoint_hits: + for n in self.breakpoint_hits[1]: + self.clear_bpbynumber(n) + + def pop_next(self): + self.expect_set_no += 1 + try: + self.expect = self.expected_list.pop(0) + except IndexError: + raise BdbNotExpectedError( + 'expect_set list exhausted, cannot pop item %d' % + self.expect_set_no) + self.set_tuple = self.set_list.pop(0) + + def process_event(self, event, frame, *args): + # Call get_stack() to enable walking the stack with set_up() and + # set_down(). + tb = None + if event == 'exception': + tb = self.exc_info[2] + self.get_stack(frame, tb) + + # A breakpoint has been hit and it is not a temporary. + if self.currentbp is not None and not self.breakpoint_hits: + bp_list = [self.currentbp] + self.breakpoint_hits = (bp_list, []) + + # Pop next event. + self.event= event + self.pop_next() + if self.dry_run: + self.print_state(self.header) + return + + # Validate the expected results. + if self.expect: + self.check_equal(self.expect[0], event, 'Wrong event type') + self.check_lno_name() + + if event in ('call', 'return'): + self.check_expect_max_size(3) + elif len(self.expect) > 3: + if event == 'line': + bps, temporaries = self.expect[3] + bpnums = sorted(bps.keys()) + if not self.breakpoint_hits: + self.raise_not_expected( + 'No breakpoints hit at expect_set item %d' % + self.expect_set_no) + self.check_equal(bpnums, self.breakpoint_hits[0], + 'Breakpoint numbers do not match') + self.check_equal([bps[n] for n in bpnums], + [self.get_bpbynumber(n).hits for + n in self.breakpoint_hits[0]], + 'Wrong breakpoint hit count') + self.check_equal(sorted(temporaries), self.breakpoint_hits[1], + 'Wrong temporary breakpoints') + + elif event == 'exception': + if not isinstance(self.exc_info[1], self.expect[3]): + self.raise_not_expected( + "Wrong exception at expect_set item %d, got '%s'" % + (self.expect_set_no, self.exc_info)) + + def check_equal(self, expected, result, msg): + if expected == result: + return + self.raise_not_expected("%s at expect_set item %d, got '%s'" % + (msg, self.expect_set_no, result)) + + def check_lno_name(self): + """Check the line number and function co_name.""" + s = len(self.expect) + if s > 1: + lineno = self.lno_abs2rel() + self.check_equal(self.expect[1], lineno, 'Wrong line number') + if s > 2: + self.check_equal(self.expect[2], self.frame.f_code.co_name, + 'Wrong function name') + + def check_expect_max_size(self, size): + if len(self.expect) > size: + raise BdbSyntaxError('Invalid size of the %s expect tuple: %s' % + (self.event, self.expect)) + + def lno_abs2rel(self): + fname = self.canonic(self.frame.f_code.co_filename) + lineno = self.frame.f_lineno + return ((lineno - self.frame.f_code.co_firstlineno + 1) + if fname == self.canonic(__file__) else lineno) + + def lno_rel2abs(self, fname, lineno): + return (self.frame.f_code.co_firstlineno + lineno - 1 + if (lineno and self.canonic(fname) == self.canonic(__file__)) + else lineno) + + def get_state(self): + lineno = self.lno_abs2rel() + co_name = self.frame.f_code.co_name + state = "('%s', %d, '%s'" % (self.event, lineno, co_name) + if self.breakpoint_hits: + bps = '{' + for n in self.breakpoint_hits[0]: + if bps != '{': + bps += ', ' + bps += '%s: %s' % (n, self.get_bpbynumber(n).hits) + bps += '}' + bps = '(' + bps + ', ' + str(self.breakpoint_hits[1]) + ')' + state += ', ' + bps + elif self.event == 'exception': + state += ', ' + self.exc_info[0].__name__ + state += '), ' + return state.ljust(32) + str(self.set_tuple) + ',' + + def print_state(self, header=None): + if header is not None and self.expect_set_no == 1: + print() + print(header) + print('%d: %s' % (self.expect_set_no, self.get_state())) + + def raise_not_expected(self, msg): + msg += '\n' + msg += ' Expected: %s\n' % str(self.expect) + msg += ' Got: ' + self.get_state() + raise BdbNotExpectedError(msg) + + def next_set_method(self): + set_type = self.set_tuple[0] + args = self.set_tuple[1] if len(self.set_tuple) == 2 else None + set_method = getattr(self, 'set_' + set_type) + + # The following set methods give back control to the tracer. + if set_type in ('step', 'continue', 'quit'): + set_method() + return + elif set_type in ('next', 'return'): + set_method(self.frame) + return + elif set_type == 'until': + set_method(self.frame) + return + + # The following set methods do not give back control to the tracer and + # next_set_method() is called recursively. + if (args and set_type in ('break', 'clear', 'ignore', 'enable', + 'disable')) or set_type in ('up', 'down'): + if set_type in ('break', 'clear'): + fname = args[0] + lineno = args[1] + lineno = self.lno_rel2abs(fname, lineno) + args = [fname, lineno] + list(args[2:]) + set_method(*args) + elif set_type in ('ignore', 'enable', 'disable'): + set_method(*args) + elif set_type in ('up', 'down'): + set_method() + + # Process the next expect_set item. + # It is not expected that a test may reach the recursion limit. + self.event= None + self.pop_next() + if self.dry_run: + self.print_state() + else: + if self.expect: + self.check_lno_name() + self.check_expect_max_size(3) + self.next_set_method() + else: + raise BdbSyntaxError('"%s" is an invalid set_tuple' % + self.set_tuple) + +class TracerRun(): + """Provide a context for running a Tracer instance with a test case.""" + + def __init__(self, test_case, skip=None): + self.test_case = test_case + self.dry_run = test_case.dry_run + self.tracer = Tracer(test_case.expect_set, skip=skip, + dry_run=self.dry_run, test_case=test_case.id()) + + def __enter__(self): + # test_pdb does not reset Breakpoint class attributes on exit :-( + reset_Breakpoint() + return self.tracer + + def __exit__(self, type_=None, value=None, traceback=None): + reset_Breakpoint() + sys.settrace(None) + + not_empty = '' + if self.tracer.set_list: + not_empty += 'All paired tuples have not been processed, ' + not_empty += ('the last one was number %d' % + self.tracer.expect_set_no) + + # Make a BdbNotExpectedError a unittest failure. + if type_ is not None and issubclass(BdbNotExpectedError, type_): + if isinstance(value, BaseException) and value.args: + err_msg = value.args[0] + if not_empty: + err_msg += '\n' + not_empty + if self.dry_run: + print(err_msg) + return True + else: + self.test_case.fail(err_msg) + else: + assert False, 'BdbNotExpectedError with empty args' + + if not_empty: + if self.dry_run: + print(not_empty) + else: + self.test_case.fail(not_empty) + +def run_test(modules, set_list, skip=None): + """Run a test and print the dry-run results. + + 'modules': A dictionary mapping module names to their source code as a + string. The dictionary MUST include one module named + 'test_module' with a main() function. + 'set_list': A list of set_type tuples to be run on the module. + + For example, running the following script outputs the following results: + + ***************************** SCRIPT ******************************** + + from test.test_bdb import run_test, break_in_func + + code = ''' + def func(): + lno = 3 + + def main(): + func() + lno = 7 + ''' + + set_list = [ + break_in_func('func', 'test_module.py'), + ('continue', ), + ('step', ), + ('step', ), + ('step', ), + ('quit', ), + ] + + modules = { 'test_module': code } + run_test(modules, set_list) + + **************************** results ******************************** + + 1: ('line', 2, 'tfunc_import'), ('next',), + 2: ('line', 3, 'tfunc_import'), ('step',), + 3: ('call', 5, 'main'), ('break', ('test_module.py', None, False, None, 'func')), + 4: ('None', 5, 'main'), ('continue',), + 5: ('line', 3, 'func', ({1: 1}, [])), ('step',), + BpNum Temp Enb Hits Ignore Where + 1 no yes 1 0 at test_module.py:2 + 6: ('return', 3, 'func'), ('step',), + 7: ('line', 7, 'main'), ('step',), + 8: ('return', 7, 'main'), ('quit',), + + ************************************************************************* + + """ + def gen(a, b): + try: + while 1: + x = next(a) + y = next(b) + yield x + yield y + except StopIteration: + return + + # Step over the import statement in tfunc_import using 'next' and step + # into main() in test_module. + sl = [('next', ), ('step', )] + sl.extend(set_list) + + test = BaseTestCase() + test.dry_run = True + test.id = lambda : None + test.expect_set = list(gen(repeat(()), iter(sl))) + with create_modules(modules): + sys.path.append(os.getcwd()) + with TracerRun(test, skip=skip) as tracer: + tracer.runcall(tfunc_import) + + at contextmanager +def create_modules(modules): + with test.support.temp_cwd(): + try: + sys.path.insert(0, os.getcwd()) + for m in modules: + fname = m + '.py' + with open(fname, 'w') as f: + f.write(textwrap.dedent(modules[m])) + linecache.checkcache(fname) + yield + finally: + sys.path.pop(0) + for m in modules: + test.support.forget(m) + +def break_in_func(funcname, fname=__file__, temporary=False, cond=None): + return 'break', (fname, None, temporary, cond, funcname) + +TEST_MODULE = 'test_module' +TEST_MODULE_FNAME = TEST_MODULE + '.py' +def tfunc_import(): + import test_module + test_module.main() + +def tfunc_main(): + lno = 2 + tfunc_first() + tfunc_second() + lno = 5 + lno = 6 + lno = 7 + +def tfunc_first(): + lno = 2 + lno = 3 + lno = 4 + +def tfunc_second(): + lno = 2 + +class BaseTestCase(unittest.TestCase): + """Base class for all tests.""" + dry_run = dry_run + +class StateTestCase(BaseTestCase): + """Test the step, next, return, until and quit 'set_' methods.""" + + def test_step(self): + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('step', ), + ('line', 2, 'tfunc_first'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_step_on_last_statement(self): + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('break', (__file__, 3)), + ('None', 1, 'tfunc_first'), ('continue', ), + ('line', 3, 'tfunc_first', ({1:1}, [])), ('step', ), + ('line', 4, 'tfunc_first'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_next_on_last_statement(self): + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('break', (__file__, 3)), + ('None', 1, 'tfunc_first'), ('continue', ), + ('line', 3, 'tfunc_first', ({1:1}, [])), ('next', ), + ('line', 4, 'tfunc_first'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_next(self): + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('next', ), + ('line', 4, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_second'), ('step', ), + ('line', 2, 'tfunc_second'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_next_over_import(self): + code = """ + def main(): + lno = 3 + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), ('next', ), + ('line', 3, 'tfunc_import'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_import) + + def test_next_on_plain_statement(self): + # Check that set_next() is equivalent to set_step() on a plain + # statement. + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('next', ), + ('line', 2, 'tfunc_first'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_next_in_caller_frame(self): + # Check that set_next() in the caller frame causes the tracer + # to stop next in the caller frame. + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('up', ), + ('None', 3, 'tfunc_main'), ('next', ), + ('line', 4, 'tfunc_main'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_return(self): + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('step', ), + ('line', 2, 'tfunc_first'), ('return', ), + ('return', 4, 'tfunc_first'), ('step', ), + ('line', 4, 'tfunc_main'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_return_in_caller_frame(self): + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('up', ), + ('None', 3, 'tfunc_main'), ('return', ), + ('return', 7, 'tfunc_main'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_until(self): + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('step', ), + ('line', 2, 'tfunc_first'), ('until', ), + ('line', 3, 'tfunc_first'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_until_in_caller_frame(self): + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('up', ), + ('None', 3, 'tfunc_main'), ('until', ), + ('line', 4, 'tfunc_main'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + + def test_skip(self): + # Check that tracing is skipped over the import statement in + # 'tfunc_import()'. + code = """ + def main(): + lno = 3 + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), ('step', ), + ('line', 3, 'tfunc_import'), ('quit', ), + ] + skip = ('importlib*', TEST_MODULE) + with TracerRun(self, skip=skip) as tracer: + tracer.runcall(tfunc_import) + + def test_down(self): + # Check that set_down() raises BdbError at the newest frame. + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('down', ), + ] + with TracerRun(self) as tracer: + self.assertRaises(BdbError, tracer.runcall, tfunc_main) + + def test_up(self): + self.expect_set = [ + ('line', 2, 'tfunc_main'), ('step', ), + ('line', 3, 'tfunc_main'), ('step', ), + ('call', 1, 'tfunc_first'), ('up', ), + ('None', 3, 'tfunc_main'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_main) + +class BreakpointTestCase(BaseTestCase): + """Test the breakpoint set method.""" + + def test_bp_on_non_existent_module(self): + self.expect_set = [ + ('line', 2, 'tfunc_import'), ('break', ('/non/existent/module.py', 1)) + ] + with TracerRun(self) as tracer: + self.assertRaises(BdbError, tracer.runcall, tfunc_import) + + def test_bp_after_last_statement(self): + code = """ + def main(): + lno = 3 + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), ('break', (TEST_MODULE_FNAME, 4)) + ] + with TracerRun(self) as tracer: + self.assertRaises(BdbError, tracer.runcall, tfunc_import) + + def test_temporary_bp(self): + code = """ + def func(): + lno = 3 + + def main(): + for i in range(2): + func() + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), + break_in_func('func', TEST_MODULE_FNAME, True), + ('None', 2, 'tfunc_import'), + break_in_func('func', TEST_MODULE_FNAME, True), + ('None', 2, 'tfunc_import'), ('continue', ), + ('line', 3, 'func', ({1:1}, [1])), ('continue', ), + ('line', 3, 'func', ({2:1}, [2])), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_import) + + def test_disabled_temporary_bp(self): + code = """ + def func(): + lno = 3 + + def main(): + for i in range(3): + func() + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), + break_in_func('func', TEST_MODULE_FNAME), + ('None', 2, 'tfunc_import'), + break_in_func('func', TEST_MODULE_FNAME, True), + ('None', 2, 'tfunc_import'), ('disable', (2, )), + ('None', 2, 'tfunc_import'), ('continue', ), + ('line', 3, 'func', ({1:1}, [])), ('enable', (2, )), + ('None', 3, 'func'), ('disable', (1, )), + ('None', 3, 'func'), ('continue', ), + ('line', 3, 'func', ({2:1}, [2])), ('enable', (1, )), + ('None', 3, 'func'), ('continue', ), + ('line', 3, 'func', ({1:2}, [])), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_import) + + def test_bp_condition(self): + code = """ + def func(a): + lno = 3 + + def main(): + for i in range(3): + func(i) + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), + break_in_func('func', TEST_MODULE_FNAME, False, 'a == 2'), + ('None', 2, 'tfunc_import'), ('continue', ), + ('line', 3, 'func', ({1:3}, [])), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_import) + + def test_bp_exception_on_condition_evaluation(self): + code = """ + def func(a): + lno = 3 + + def main(): + func(0) + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), + break_in_func('func', TEST_MODULE_FNAME, False, '1 / 0'), + ('None', 2, 'tfunc_import'), ('continue', ), + ('line', 3, 'func', ({1:1}, [])), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_import) + + def test_bp_ignore_count(self): + code = """ + def func(): + lno = 3 + + def main(): + for i in range(2): + func() + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), + break_in_func('func', TEST_MODULE_FNAME), + ('None', 2, 'tfunc_import'), ('ignore', (1, )), + ('None', 2, 'tfunc_import'), ('continue', ), + ('line', 3, 'func', ({1:2}, [])), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_import) + + def test_ignore_count_on_disabled_bp(self): + code = """ + def func(): + lno = 3 + + def main(): + for i in range(3): + func() + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), + break_in_func('func', TEST_MODULE_FNAME), + ('None', 2, 'tfunc_import'), + break_in_func('func', TEST_MODULE_FNAME), + ('None', 2, 'tfunc_import'), ('ignore', (1, )), + ('None', 2, 'tfunc_import'), ('disable', (1, )), + ('None', 2, 'tfunc_import'), ('continue', ), + ('line', 3, 'func', ({2:1}, [])), ('enable', (1, )), + ('None', 3, 'func'), ('continue', ), + ('line', 3, 'func', ({2:2}, [])), ('continue', ), + ('line', 3, 'func', ({1:2}, [])), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_import) + + def test_clear_two_bp_on_same_line(self): + code = """ + def func(): + lno = 3 + lno = 4 + + def main(): + for i in range(3): + func() + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), ('break', (TEST_MODULE_FNAME, 3)), + ('None', 2, 'tfunc_import'), ('break', (TEST_MODULE_FNAME, 3)), + ('None', 2, 'tfunc_import'), ('break', (TEST_MODULE_FNAME, 4)), + ('None', 2, 'tfunc_import'), ('continue', ), + ('line', 3, 'func', ({1:1}, [])), ('continue', ), + ('line', 4, 'func', ({3:1}, [])), ('clear', (TEST_MODULE_FNAME, 3)), + ('None', 4, 'func'), ('continue', ), + ('line', 4, 'func', ({3:2}, [])), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_import) + + def test_clear_at_no_bp(self): + self.expect_set = [ + ('line', 2, 'tfunc_import'), ('clear', (__file__, 1)) + ] + with TracerRun(self) as tracer: + self.assertRaises(BdbError, tracer.runcall, tfunc_import) + +class RunTestCase(BaseTestCase): + """Test run, runeval and set_trace.""" + + def test_run_step(self): + # Check that the bdb 'run' method stops at the first line event. + code = """ + lno = 2 + """ + self.expect_set = [ + ('line', 2, ''), ('step', ), + ('return', 2, ''), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.run(compile(textwrap.dedent(code), '', 'exec')) + + def test_runeval_step(self): + # Test bdb 'runeval'. + code = """ + def main(): + lno = 3 + """ + modules = { TEST_MODULE: code } + with create_modules(modules): + self.expect_set = [ + ('line', 1, ''), ('step', ), + ('call', 2, 'main'), ('step', ), + ('line', 3, 'main'), ('step', ), + ('return', 3, 'main'), ('step', ), + ('return', 1, ''), ('quit', ), + ] + import test_module + with TracerRun(self) as tracer: + tracer.runeval('test_module.main()', globals(), locals()) + +class IssuesTestCase(BaseTestCase): + """Test fixed bdb issues.""" + + def test_step_at_return_with_no_trace_in_caller(self): + # Issue #13183. + # Check that the tracer does step into the caller frame when the + # trace function is not set in that frame. + code_1 = """ + from test_module_2 import func + def main(): + func() + lno = 5 + """ + code_2 = """ + def func(): + lno = 3 + """ + modules = { + TEST_MODULE: code_1, + 'test_module_2': code_2, + } + with create_modules(modules): + self.expect_set = [ + ('line', 2, 'tfunc_import'), + break_in_func('func', 'test_module_2.py'), + ('None', 2, 'tfunc_import'), ('continue', ), + ('line', 3, 'func', ({1:1}, [])), ('step', ), + ('return', 3, 'func'), ('step', ), + ('line', 5, 'main'), ('quit', ), + ] + with TracerRun(self) as tracer: + tracer.runcall(tfunc_import) + +def test_main(): + test.support.run_unittest( + StateTestCase, + RunTestCase, + BreakpointTestCase, + IssuesTestCase, + ) + +if __name__ == "__main__": + test_main() diff --git a/Lib/test/test_sundry.py b/Lib/test/test_sundry.py index 8fe89953b737..d4a97c82d9d8 100644 --- a/Lib/test/test_sundry.py +++ b/Lib/test/test_sundry.py @@ -10,7 +10,6 @@ def test_at_least_import_untested_modules(self): with test_support.check_warnings(quiet=True): import CGIHTTPServer import audiodev - import bdb import cgitb import code import compileall diff --git a/Misc/NEWS.d/next/Tests/2018-01-08-13-33-47.bpo-19417.2asoXy.rst b/Misc/NEWS.d/next/Tests/2018-01-08-13-33-47.bpo-19417.2asoXy.rst new file mode 100644 index 000000000000..739352fcdd67 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-01-08-13-33-47.bpo-19417.2asoXy.rst @@ -0,0 +1 @@ +Add test_bdb.py. From solipsis at pitrou.net Sun Sep 16 05:11:22 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 16 Sep 2018 09:11:22 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=5 Message-ID: <20180916091122.1.A39042358312797C@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [2, -1, 0] memory blocks, sum=1 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogzKAFeK', '--timeout', '7200'] From webhook-mailer at python.org Sun Sep 16 19:40:47 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Sun, 16 Sep 2018 23:40:47 -0000 Subject: [Python-checkins] bpo-33649: Clarify protocol_factory as a method parameter (GH-9330) Message-ID: https://github.com/python/cpython/commit/5cc583d94095ed3bb543fa2f032f9593a6315a52 commit: 5cc583d94095ed3bb543fa2f032f9593a6315a52 branch: master author: Bumsik Kim committer: Yury Selivanov date: 2018-09-16T16:40:44-07:00 summary: bpo-33649: Clarify protocol_factory as a method parameter (GH-9330) files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 718c32277c27..3e1571f72453 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -535,6 +535,9 @@ Creating network servers Arguments: + * *protocol_factory* must be a callable returning a + :ref:`protocol ` implementation. + * The *host* parameter can be set to several types which determine behavior: - If *host* is a string, the TCP server is bound to *host* and *port*. - if *host* is a sequence of strings, the TCP server is bound to all @@ -617,6 +620,9 @@ Creating network servers :class:`str`, :class:`bytes`, and :class:`~pathlib.Path` paths are supported. + See the documentation of the :meth:`loop.create_server` method + for information about arguments to this method. + Availability: UNIX. .. versionadded:: 3.7 @@ -637,6 +643,9 @@ Creating network servers Parameters: + * *protocol_factory* must be a callable returning a + :ref:`protocol ` implementation. + * *sock* is a preexisting socket object returned from :meth:`socket.accept `. @@ -909,7 +918,8 @@ Working with pipes *pipe* is a :term:`file-like object `. Return pair ``(transport, protocol)``, where *transport* supports - the :class:`ReadTransport` interface. + the :class:`ReadTransport` interface and *protocol* is an object + instantiated by the *protocol_factory*. With :class:`SelectorEventLoop` event loop, the *pipe* is set to non-blocking mode. @@ -924,7 +934,8 @@ Working with pipes *pipe* is :term:`file-like object `. Return pair ``(transport, protocol)``, where *transport* supports - :class:`WriteTransport` interface. + :class:`WriteTransport` interface and *protocol* is an object + instantiated by the *protocol_factory*. With :class:`SelectorEventLoop` event loop, the *pipe* is set to non-blocking mode. @@ -1159,7 +1170,8 @@ async/await code consider using high-level convenient for documentation on other arguments. Returns a pair of ``(transport, protocol)``, where *transport* - conforms to the :class:`asyncio.SubprocessTransport` base class. + conforms to the :class:`asyncio.SubprocessTransport` base class and + *protocol* is an object instantiated by the *protocol_factory*. .. coroutinemethod:: loop.subprocess_shell(protocol_factory, cmd, \*, \ stdin=subprocess.PIPE, stdout=subprocess.PIPE, \ @@ -1180,7 +1192,8 @@ async/await code consider using high-level convenient the remaining arguments. Returns a pair of ``(transport, protocol)``, where *transport* - conforms to the :class:`SubprocessTransport` base class. + conforms to the :class:`SubprocessTransport` base class and + *protocol* is an object instantiated by the *protocol_factory*. .. note:: It is the application's responsibility to ensure that all whitespace From webhook-mailer at python.org Mon Sep 17 01:38:13 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Mon, 17 Sep 2018 05:38:13 -0000 Subject: [Python-checkins] closes bpo-34673: Tweaks to make ceval more editable. (GH-9289) Message-ID: https://github.com/python/cpython/commit/ddd1949fea59f256e51191540a4446f75ed608fa commit: ddd1949fea59f256e51191540a4446f75ed608fa branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-16T22:38:02-07:00 summary: closes bpo-34673: Tweaks to make ceval more editable. (GH-9289) Two major changes: - Move case statements out of the TARGET macro. - Move PREDICT macro invocations after the case label. files: M Python/ceval.c diff --git a/Python/ceval.c b/Python/ceval.c index f3a74b00a2b6..4a82bd3198af 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -629,8 +629,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) #include "opcode_targets.h" #define TARGET(op) \ - TARGET_##op: \ - case op: + op: \ + TARGET_##op #define DISPATCH() \ { \ @@ -663,8 +663,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) #endif #else -#define TARGET(op) \ - case op: +#define TARGET(op) op #define DISPATCH() continue #define FAST_DISPATCH() goto fast_next_opcode @@ -1064,10 +1063,11 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) It is essential that any operation that fails must goto error and that all operation that succeed call [FAST_]DISPATCH() ! */ - TARGET(NOP) + case TARGET(NOP): { FAST_DISPATCH(); + } - TARGET(LOAD_FAST) { + case TARGET(LOAD_FAST): { PyObject *value = GETLOCAL(oparg); if (value == NULL) { format_exc_check_arg(PyExc_UnboundLocalError, @@ -1080,28 +1080,28 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) FAST_DISPATCH(); } - PREDICTED(LOAD_CONST); - TARGET(LOAD_CONST) { + case TARGET(LOAD_CONST): { + PREDICTED(LOAD_CONST); PyObject *value = GETITEM(consts, oparg); Py_INCREF(value); PUSH(value); FAST_DISPATCH(); } - PREDICTED(STORE_FAST); - TARGET(STORE_FAST) { + case TARGET(STORE_FAST): { + PREDICTED(STORE_FAST); PyObject *value = POP(); SETLOCAL(oparg, value); FAST_DISPATCH(); } - TARGET(POP_TOP) { + case TARGET(POP_TOP): { PyObject *value = POP(); Py_DECREF(value); FAST_DISPATCH(); } - TARGET(ROT_TWO) { + case TARGET(ROT_TWO): { PyObject *top = TOP(); PyObject *second = SECOND(); SET_TOP(second); @@ -1109,7 +1109,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) FAST_DISPATCH(); } - TARGET(ROT_THREE) { + case TARGET(ROT_THREE): { PyObject *top = TOP(); PyObject *second = SECOND(); PyObject *third = THIRD(); @@ -1119,7 +1119,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) FAST_DISPATCH(); } - TARGET(ROT_FOUR) { + case TARGET(ROT_FOUR): { PyObject *top = TOP(); PyObject *second = SECOND(); PyObject *third = THIRD(); @@ -1131,14 +1131,14 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) FAST_DISPATCH(); } - TARGET(DUP_TOP) { + case TARGET(DUP_TOP): { PyObject *top = TOP(); Py_INCREF(top); PUSH(top); FAST_DISPATCH(); } - TARGET(DUP_TOP_TWO) { + case TARGET(DUP_TOP_TWO): { PyObject *top = TOP(); PyObject *second = SECOND(); Py_INCREF(top); @@ -1149,7 +1149,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) FAST_DISPATCH(); } - TARGET(UNARY_POSITIVE) { + case TARGET(UNARY_POSITIVE): { PyObject *value = TOP(); PyObject *res = PyNumber_Positive(value); Py_DECREF(value); @@ -1159,7 +1159,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(UNARY_NEGATIVE) { + case TARGET(UNARY_NEGATIVE): { PyObject *value = TOP(); PyObject *res = PyNumber_Negative(value); Py_DECREF(value); @@ -1169,7 +1169,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(UNARY_NOT) { + case TARGET(UNARY_NOT): { PyObject *value = TOP(); int err = PyObject_IsTrue(value); Py_DECREF(value); @@ -1187,7 +1187,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) goto error; } - TARGET(UNARY_INVERT) { + case TARGET(UNARY_INVERT): { PyObject *value = TOP(); PyObject *res = PyNumber_Invert(value); Py_DECREF(value); @@ -1197,7 +1197,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_POWER) { + case TARGET(BINARY_POWER): { PyObject *exp = POP(); PyObject *base = TOP(); PyObject *res = PyNumber_Power(base, exp, Py_None); @@ -1209,7 +1209,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_MULTIPLY) { + case TARGET(BINARY_MULTIPLY): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_Multiply(left, right); @@ -1221,7 +1221,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_MATRIX_MULTIPLY) { + case TARGET(BINARY_MATRIX_MULTIPLY): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_MatrixMultiply(left, right); @@ -1233,7 +1233,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_TRUE_DIVIDE) { + case TARGET(BINARY_TRUE_DIVIDE): { PyObject *divisor = POP(); PyObject *dividend = TOP(); PyObject *quotient = PyNumber_TrueDivide(dividend, divisor); @@ -1245,7 +1245,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_FLOOR_DIVIDE) { + case TARGET(BINARY_FLOOR_DIVIDE): { PyObject *divisor = POP(); PyObject *dividend = TOP(); PyObject *quotient = PyNumber_FloorDivide(dividend, divisor); @@ -1257,7 +1257,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_MODULO) { + case TARGET(BINARY_MODULO): { PyObject *divisor = POP(); PyObject *dividend = TOP(); PyObject *res; @@ -1277,7 +1277,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_ADD) { + case TARGET(BINARY_ADD): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *sum; @@ -1303,7 +1303,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_SUBTRACT) { + case TARGET(BINARY_SUBTRACT): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *diff = PyNumber_Subtract(left, right); @@ -1315,7 +1315,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_SUBSCR) { + case TARGET(BINARY_SUBSCR): { PyObject *sub = POP(); PyObject *container = TOP(); PyObject *res = PyObject_GetItem(container, sub); @@ -1327,7 +1327,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_LSHIFT) { + case TARGET(BINARY_LSHIFT): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_Lshift(left, right); @@ -1339,7 +1339,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_RSHIFT) { + case TARGET(BINARY_RSHIFT): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_Rshift(left, right); @@ -1351,7 +1351,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_AND) { + case TARGET(BINARY_AND): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_And(left, right); @@ -1363,7 +1363,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_XOR) { + case TARGET(BINARY_XOR): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_Xor(left, right); @@ -1375,7 +1375,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BINARY_OR) { + case TARGET(BINARY_OR): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_Or(left, right); @@ -1387,7 +1387,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(LIST_APPEND) { + case TARGET(LIST_APPEND): { PyObject *v = POP(); PyObject *list = PEEK(oparg); int err; @@ -1399,7 +1399,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(SET_ADD) { + case TARGET(SET_ADD): { PyObject *v = POP(); PyObject *set = PEEK(oparg); int err; @@ -1411,7 +1411,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_POWER) { + case TARGET(INPLACE_POWER): { PyObject *exp = POP(); PyObject *base = TOP(); PyObject *res = PyNumber_InPlacePower(base, exp, Py_None); @@ -1423,7 +1423,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_MULTIPLY) { + case TARGET(INPLACE_MULTIPLY): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_InPlaceMultiply(left, right); @@ -1435,7 +1435,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_MATRIX_MULTIPLY) { + case TARGET(INPLACE_MATRIX_MULTIPLY): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_InPlaceMatrixMultiply(left, right); @@ -1447,7 +1447,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_TRUE_DIVIDE) { + case TARGET(INPLACE_TRUE_DIVIDE): { PyObject *divisor = POP(); PyObject *dividend = TOP(); PyObject *quotient = PyNumber_InPlaceTrueDivide(dividend, divisor); @@ -1459,7 +1459,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_FLOOR_DIVIDE) { + case TARGET(INPLACE_FLOOR_DIVIDE): { PyObject *divisor = POP(); PyObject *dividend = TOP(); PyObject *quotient = PyNumber_InPlaceFloorDivide(dividend, divisor); @@ -1471,7 +1471,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_MODULO) { + case TARGET(INPLACE_MODULO): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *mod = PyNumber_InPlaceRemainder(left, right); @@ -1483,7 +1483,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_ADD) { + case TARGET(INPLACE_ADD): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *sum; @@ -1502,7 +1502,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_SUBTRACT) { + case TARGET(INPLACE_SUBTRACT): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *diff = PyNumber_InPlaceSubtract(left, right); @@ -1514,7 +1514,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_LSHIFT) { + case TARGET(INPLACE_LSHIFT): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_InPlaceLshift(left, right); @@ -1526,7 +1526,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_RSHIFT) { + case TARGET(INPLACE_RSHIFT): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_InPlaceRshift(left, right); @@ -1538,7 +1538,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_AND) { + case TARGET(INPLACE_AND): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_InPlaceAnd(left, right); @@ -1550,7 +1550,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_XOR) { + case TARGET(INPLACE_XOR): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_InPlaceXor(left, right); @@ -1562,7 +1562,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(INPLACE_OR) { + case TARGET(INPLACE_OR): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = PyNumber_InPlaceOr(left, right); @@ -1574,7 +1574,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(STORE_SUBSCR) { + case TARGET(STORE_SUBSCR): { PyObject *sub = TOP(); PyObject *container = SECOND(); PyObject *v = THIRD(); @@ -1590,7 +1590,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(DELETE_SUBSCR) { + case TARGET(DELETE_SUBSCR): { PyObject *sub = TOP(); PyObject *container = SECOND(); int err; @@ -1604,7 +1604,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(PRINT_EXPR) { + case TARGET(PRINT_EXPR): { _Py_IDENTIFIER(displayhook); PyObject *value = POP(); PyObject *hook = _PySys_GetObjectId(&PyId_displayhook); @@ -1623,7 +1623,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(RAISE_VARARGS) { + case TARGET(RAISE_VARARGS): { PyObject *cause = NULL, *exc = NULL; switch (oparg) { case 2: @@ -1645,13 +1645,13 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) goto error; } - TARGET(RETURN_VALUE) { + case TARGET(RETURN_VALUE): { retval = POP(); assert(f->f_iblock == 0); goto return_or_yield; } - TARGET(GET_AITER) { + case TARGET(GET_AITER): { unaryfunc getter = NULL; PyObject *iter = NULL; PyObject *obj = TOP(); @@ -1697,7 +1697,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(GET_ANEXT) { + case TARGET(GET_ANEXT): { unaryfunc getter = NULL; PyObject *next_iter = NULL; PyObject *awaitable = NULL; @@ -1749,8 +1749,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - PREDICTED(GET_AWAITABLE); - TARGET(GET_AWAITABLE) { + case TARGET(GET_AWAITABLE): { + PREDICTED(GET_AWAITABLE); PyObject *iterable = TOP(); PyObject *iter = _PyCoro_GetAwaitableIter(iterable); @@ -1786,7 +1786,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(YIELD_FROM) { + case TARGET(YIELD_FROM): { PyObject *v = POP(); PyObject *receiver = TOP(); int err; @@ -1820,7 +1820,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) goto return_or_yield; } - TARGET(YIELD_VALUE) { + case TARGET(YIELD_VALUE): { retval = POP(); if (co->co_flags & CO_ASYNC_GENERATOR) { @@ -1837,7 +1837,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) goto return_or_yield; } - TARGET(POP_EXCEPT) { + case TARGET(POP_EXCEPT): { PyObject *type, *value, *traceback; _PyErr_StackItem *exc_info; PyTryBlock *b = PyFrame_BlockPop(f); @@ -1861,13 +1861,13 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - PREDICTED(POP_BLOCK); - TARGET(POP_BLOCK) { + case TARGET(POP_BLOCK): { + PREDICTED(POP_BLOCK); PyFrame_BlockPop(f); DISPATCH(); } - TARGET(POP_FINALLY) { + case TARGET(POP_FINALLY): { /* If oparg is 0 at the top of the stack are 1 or 6 values: Either: - TOP = NULL or an integer @@ -1918,7 +1918,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(CALL_FINALLY) { + case TARGET(CALL_FINALLY): { PyObject *ret = PyLong_FromLong(INSTR_OFFSET()); if (ret == NULL) { goto error; @@ -1928,7 +1928,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) FAST_DISPATCH(); } - TARGET(BEGIN_FINALLY) { + case TARGET(BEGIN_FINALLY): { /* Push NULL onto the stack for using it in END_FINALLY, POP_FINALLY, WITH_CLEANUP_START and WITH_CLEANUP_FINISH. */ @@ -1936,8 +1936,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) FAST_DISPATCH(); } - PREDICTED(END_FINALLY); - TARGET(END_FINALLY) { + case TARGET(END_FINALLY): { + PREDICTED(END_FINALLY); /* At the top of the stack are 1 or 6 values: Either: - TOP = NULL or an integer @@ -1967,7 +1967,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) } } - TARGET(END_ASYNC_FOR) { + case TARGET(END_ASYNC_FOR): { PyObject *exc = POP(); assert(PyExceptionClass_Check(exc)); if (PyErr_GivenExceptionMatches(exc, PyExc_StopAsyncIteration)) { @@ -1987,7 +1987,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) } } - TARGET(LOAD_BUILD_CLASS) { + case TARGET(LOAD_BUILD_CLASS): { _Py_IDENTIFIER(__build_class__); PyObject *bc; @@ -2016,7 +2016,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(STORE_NAME) { + case TARGET(STORE_NAME): { PyObject *name = GETITEM(names, oparg); PyObject *v = POP(); PyObject *ns = f->f_locals; @@ -2037,7 +2037,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(DELETE_NAME) { + case TARGET(DELETE_NAME): { PyObject *name = GETITEM(names, oparg); PyObject *ns = f->f_locals; int err; @@ -2056,8 +2056,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - PREDICTED(UNPACK_SEQUENCE); - TARGET(UNPACK_SEQUENCE) { + case TARGET(UNPACK_SEQUENCE): { + PREDICTED(UNPACK_SEQUENCE); PyObject *seq = POP(), *item, **items; if (PyTuple_CheckExact(seq) && PyTuple_GET_SIZE(seq) == oparg) { @@ -2087,7 +2087,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(UNPACK_EX) { + case TARGET(UNPACK_EX): { int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject *seq = POP(); @@ -2102,7 +2102,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(STORE_ATTR) { + case TARGET(STORE_ATTR): { PyObject *name = GETITEM(names, oparg); PyObject *owner = TOP(); PyObject *v = SECOND(); @@ -2116,7 +2116,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(DELETE_ATTR) { + case TARGET(DELETE_ATTR): { PyObject *name = GETITEM(names, oparg); PyObject *owner = POP(); int err; @@ -2127,7 +2127,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(STORE_GLOBAL) { + case TARGET(STORE_GLOBAL): { PyObject *name = GETITEM(names, oparg); PyObject *v = POP(); int err; @@ -2138,7 +2138,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(DELETE_GLOBAL) { + case TARGET(DELETE_GLOBAL): { PyObject *name = GETITEM(names, oparg); int err; err = PyDict_DelItem(f->f_globals, name); @@ -2150,7 +2150,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(LOAD_NAME) { + case TARGET(LOAD_NAME): { PyObject *name = GETITEM(names, oparg); PyObject *locals = f->f_locals; PyObject *v; @@ -2201,7 +2201,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(LOAD_GLOBAL) { + case TARGET(LOAD_GLOBAL): { PyObject *name = GETITEM(names, oparg); PyObject *v; if (PyDict_CheckExact(f->f_globals) @@ -2246,7 +2246,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(DELETE_FAST) { + case TARGET(DELETE_FAST): { PyObject *v = GETLOCAL(oparg); if (v != NULL) { SETLOCAL(oparg, NULL); @@ -2260,7 +2260,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) goto error; } - TARGET(DELETE_DEREF) { + case TARGET(DELETE_DEREF): { PyObject *cell = freevars[oparg]; PyObject *oldobj = PyCell_GET(cell); if (oldobj != NULL) { @@ -2272,14 +2272,14 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) goto error; } - TARGET(LOAD_CLOSURE) { + case TARGET(LOAD_CLOSURE): { PyObject *cell = freevars[oparg]; Py_INCREF(cell); PUSH(cell); DISPATCH(); } - TARGET(LOAD_CLASSDEREF) { + case TARGET(LOAD_CLASSDEREF): { PyObject *name, *value, *locals = f->f_locals; Py_ssize_t idx; assert(locals); @@ -2312,7 +2312,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(LOAD_DEREF) { + case TARGET(LOAD_DEREF): { PyObject *cell = freevars[oparg]; PyObject *value = PyCell_GET(cell); if (value == NULL) { @@ -2324,7 +2324,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(STORE_DEREF) { + case TARGET(STORE_DEREF): { PyObject *v = POP(); PyObject *cell = freevars[oparg]; PyObject *oldobj = PyCell_GET(cell); @@ -2333,7 +2333,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_STRING) { + case TARGET(BUILD_STRING): { PyObject *str; PyObject *empty = PyUnicode_New(0, 0); if (empty == NULL) { @@ -2351,7 +2351,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_TUPLE) { + case TARGET(BUILD_TUPLE): { PyObject *tup = PyTuple_New(oparg); if (tup == NULL) goto error; @@ -2363,7 +2363,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_LIST) { + case TARGET(BUILD_LIST): { PyObject *list = PyList_New(oparg); if (list == NULL) goto error; @@ -2375,9 +2375,9 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_TUPLE_UNPACK_WITH_CALL) - TARGET(BUILD_TUPLE_UNPACK) - TARGET(BUILD_LIST_UNPACK) { + case TARGET(BUILD_TUPLE_UNPACK_WITH_CALL): + case TARGET(BUILD_TUPLE_UNPACK): + case TARGET(BUILD_LIST_UNPACK): { int convert_to_tuple = opcode != BUILD_LIST_UNPACK; Py_ssize_t i; PyObject *sum = PyList_New(0); @@ -2418,7 +2418,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_SET) { + case TARGET(BUILD_SET): { PyObject *set = PySet_New(NULL); int err = 0; int i; @@ -2439,7 +2439,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_SET_UNPACK) { + case TARGET(BUILD_SET_UNPACK): { Py_ssize_t i; PyObject *sum = PySet_New(NULL); if (sum == NULL) @@ -2458,7 +2458,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_MAP) { + case TARGET(BUILD_MAP): { Py_ssize_t i; PyObject *map = _PyDict_NewPresized((Py_ssize_t)oparg); if (map == NULL) @@ -2482,7 +2482,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(SETUP_ANNOTATIONS) { + case TARGET(SETUP_ANNOTATIONS): { _Py_IDENTIFIER(__annotations__); int err; PyObject *ann_dict; @@ -2538,7 +2538,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_CONST_KEY_MAP) { + case TARGET(BUILD_CONST_KEY_MAP): { Py_ssize_t i; PyObject *map; PyObject *keys = TOP(); @@ -2571,7 +2571,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_MAP_UNPACK) { + case TARGET(BUILD_MAP_UNPACK): { Py_ssize_t i; PyObject *sum = PyDict_New(); if (sum == NULL) @@ -2596,7 +2596,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_MAP_UNPACK_WITH_CALL) { + case TARGET(BUILD_MAP_UNPACK_WITH_CALL): { Py_ssize_t i; PyObject *sum = PyDict_New(); if (sum == NULL) @@ -2646,7 +2646,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(MAP_ADD) { + case TARGET(MAP_ADD): { PyObject *key = TOP(); PyObject *value = SECOND(); PyObject *map; @@ -2663,7 +2663,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(LOAD_ATTR) { + case TARGET(LOAD_ATTR): { PyObject *name = GETITEM(names, oparg); PyObject *owner = TOP(); PyObject *res = PyObject_GetAttr(owner, name); @@ -2674,7 +2674,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(COMPARE_OP) { + case TARGET(COMPARE_OP): { PyObject *right = POP(); PyObject *left = TOP(); PyObject *res = cmp_outcome(oparg, left, right); @@ -2688,7 +2688,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(IMPORT_NAME) { + case TARGET(IMPORT_NAME): { PyObject *name = GETITEM(names, oparg); PyObject *fromlist = POP(); PyObject *level = TOP(); @@ -2702,7 +2702,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(IMPORT_STAR) { + case TARGET(IMPORT_STAR): { PyObject *from = POP(), *locals; int err; if (PyFrame_FastToLocalsWithError(f) < 0) { @@ -2725,7 +2725,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(IMPORT_FROM) { + case TARGET(IMPORT_FROM): { PyObject *name = GETITEM(names, oparg); PyObject *from = TOP(); PyObject *res; @@ -2736,13 +2736,13 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(JUMP_FORWARD) { + case TARGET(JUMP_FORWARD): { JUMPBY(oparg); FAST_DISPATCH(); } - PREDICTED(POP_JUMP_IF_FALSE); - TARGET(POP_JUMP_IF_FALSE) { + case TARGET(POP_JUMP_IF_FALSE): { + PREDICTED(POP_JUMP_IF_FALSE); PyObject *cond = POP(); int err; if (cond == Py_True) { @@ -2765,8 +2765,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - PREDICTED(POP_JUMP_IF_TRUE); - TARGET(POP_JUMP_IF_TRUE) { + case TARGET(POP_JUMP_IF_TRUE): { + PREDICTED(POP_JUMP_IF_TRUE); PyObject *cond = POP(); int err; if (cond == Py_False) { @@ -2790,7 +2790,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(JUMP_IF_FALSE_OR_POP) { + case TARGET(JUMP_IF_FALSE_OR_POP): { PyObject *cond = TOP(); int err; if (cond == Py_True) { @@ -2814,7 +2814,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(JUMP_IF_TRUE_OR_POP) { + case TARGET(JUMP_IF_TRUE_OR_POP): { PyObject *cond = TOP(); int err; if (cond == Py_False) { @@ -2839,8 +2839,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - PREDICTED(JUMP_ABSOLUTE); - TARGET(JUMP_ABSOLUTE) { + case TARGET(JUMP_ABSOLUTE): { + PREDICTED(JUMP_ABSOLUTE); JUMPTO(oparg); #if FAST_LOOPS /* Enabling this path speeds-up all while and for-loops by bypassing @@ -2856,7 +2856,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) #endif } - TARGET(GET_ITER) { + case TARGET(GET_ITER): { /* before: [obj]; after [getiter(obj)] */ PyObject *iterable = TOP(); PyObject *iter = PyObject_GetIter(iterable); @@ -2869,7 +2869,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(GET_YIELD_FROM_ITER) { + case TARGET(GET_YIELD_FROM_ITER): { /* before: [obj]; after [getiter(obj)] */ PyObject *iterable = TOP(); PyObject *iter; @@ -2898,8 +2898,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - PREDICTED(FOR_ITER); - TARGET(FOR_ITER) { + case TARGET(FOR_ITER): { + PREDICTED(FOR_ITER); /* before: [iter]; after: [iter, iter()] *or* [] */ PyObject *iter = TOP(); PyObject *next = (*iter->ob_type->tp_iternext)(iter); @@ -2924,7 +2924,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(SETUP_FINALLY) { + case TARGET(SETUP_FINALLY): { /* NOTE: If you add any new block-setup opcodes that are not try/except/finally handlers, you may need to update the PyGen_NeedsFinalizing() function. @@ -2935,7 +2935,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BEFORE_ASYNC_WITH) { + case TARGET(BEFORE_ASYNC_WITH): { _Py_IDENTIFIER(__aexit__); _Py_IDENTIFIER(__aenter__); @@ -2959,7 +2959,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(SETUP_ASYNC_WITH) { + case TARGET(SETUP_ASYNC_WITH): { PyObject *res = POP(); /* Setup the finally block before pushing the result of __aenter__ on the stack. */ @@ -2969,7 +2969,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(SETUP_WITH) { + case TARGET(SETUP_WITH): { _Py_IDENTIFIER(__exit__); _Py_IDENTIFIER(__enter__); PyObject *mgr = TOP(); @@ -2997,7 +2997,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(WITH_CLEANUP_START) { + case TARGET(WITH_CLEANUP_START): { /* At the top of the stack are 1 or 6 values indicating how/why we entered the finally clause: - TOP = NULL @@ -3070,8 +3070,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - PREDICTED(WITH_CLEANUP_FINISH); - TARGET(WITH_CLEANUP_FINISH) { + case TARGET(WITH_CLEANUP_FINISH): { + PREDICTED(WITH_CLEANUP_FINISH); /* TOP = the result of calling the context.__exit__ bound method SECOND = either None or exception type @@ -3107,7 +3107,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(LOAD_METHOD) { + case TARGET(LOAD_METHOD): { /* Designed to work in tamdem with CALL_METHOD. */ PyObject *name = GETITEM(names, oparg); PyObject *obj = TOP(); @@ -3144,7 +3144,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(CALL_METHOD) { + case TARGET(CALL_METHOD): { /* Designed to work in tamdem with LOAD_METHOD. */ PyObject **sp, *res, *meth; @@ -3193,8 +3193,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - PREDICTED(CALL_FUNCTION); - TARGET(CALL_FUNCTION) { + case TARGET(CALL_FUNCTION): { + PREDICTED(CALL_FUNCTION); PyObject **sp, *res; sp = stack_pointer; res = call_function(&sp, oparg, NULL); @@ -3206,7 +3206,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(CALL_FUNCTION_KW) { + case TARGET(CALL_FUNCTION_KW): { PyObject **sp, *res, *names; names = POP(); @@ -3223,7 +3223,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(CALL_FUNCTION_EX) { + case TARGET(CALL_FUNCTION_EX): { PyObject *func, *callargs, *kwargs = NULL, *result; if (oparg & 0x01) { kwargs = POP(); @@ -3276,7 +3276,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(MAKE_FUNCTION) { + case TARGET(MAKE_FUNCTION): { PyObject *qualname = POP(); PyObject *codeobj = POP(); PyFunctionObject *func = (PyFunctionObject *) @@ -3309,7 +3309,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(BUILD_SLICE) { + case TARGET(BUILD_SLICE): { PyObject *start, *stop, *step, *slice; if (oparg == 3) step = POP(); @@ -3327,7 +3327,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(FORMAT_VALUE) { + case TARGET(FORMAT_VALUE): { /* Handles f-string value formatting. */ PyObject *result; PyObject *fmt_spec; @@ -3385,7 +3385,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) DISPATCH(); } - TARGET(EXTENDED_ARG) { + case TARGET(EXTENDED_ARG): { int oldoparg = oparg; NEXTOPARG(); oparg |= oldoparg << 8; From solipsis at pitrou.net Mon Sep 17 05:09:30 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 17 Sep 2018 09:09:30 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=1 Message-ID: <20180917090930.1.BB457B61D625910B@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 3] memory blocks, sum=3 test_collections leaked [-7, 1, 0] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogznxs8M', '--timeout', '7200'] From webhook-mailer at python.org Mon Sep 17 07:24:06 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Mon, 17 Sep 2018 11:24:06 -0000 Subject: [Python-checkins] bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113) Message-ID: https://github.com/python/cpython/commit/e0e5065daef36dafe10a46eaa8b7800274d73062 commit: e0e5065daef36dafe10a46eaa8b7800274d73062 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-17T14:24:01+03:00 summary: bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113) files: A Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst M Lib/multiprocessing/managers.py M Lib/test/_test_multiprocessing.py diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py index 3f263802bc7c..dbed993a38d6 100644 --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -1134,10 +1134,13 @@ def __imul__(self, value): DictProxy = MakeProxyType('DictProxy', ( - '__contains__', '__delitem__', '__getitem__', '__len__', + '__contains__', '__delitem__', '__getitem__', '__iter__', '__len__', '__setitem__', 'clear', 'copy', 'get', 'has_key', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values' )) +DictProxy._method_to_typeid_ = { + '__iter__': 'Iterator', + } ArrayProxy = MakeProxyType('ArrayProxy', ( diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index a5509ce9135a..bb9eb240a8a2 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -2080,6 +2080,16 @@ def test_list(self): a.append('hello') self.assertEqual(f[0][:], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 'hello']) + def test_list_iter(self): + a = self.list(list(range(10))) + it = iter(a) + self.assertEqual(list(it), list(range(10))) + self.assertEqual(list(it), []) # exhausted + # list modified during iteration + it = iter(a) + a[0] = 100 + self.assertEqual(next(it), 100) + def test_list_proxy_in_list(self): a = self.list([self.list(range(3)) for _i in range(3)]) self.assertEqual([inner[:] for inner in a], [[0, 1, 2]] * 3) @@ -2110,6 +2120,19 @@ def test_dict(self): self.assertEqual(sorted(d.values()), [chr(i) for i in indices]) self.assertEqual(sorted(d.items()), [(i, chr(i)) for i in indices]) + def test_dict_iter(self): + d = self.dict() + indices = list(range(65, 70)) + for i in indices: + d[i] = chr(i) + it = iter(d) + self.assertEqual(list(it), indices) + self.assertEqual(list(it), []) # exhausted + # dictionary changed size during iteration + it = iter(d) + d.clear() + self.assertRaises(RuntimeError, next, it) + def test_dict_proxy_nested(self): pets = self.dict(ferrets=2, hamsters=4) supplies = self.dict(water=10, feed=3) diff --git a/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst b/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst new file mode 100644 index 000000000000..bffb355ea2ab --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst @@ -0,0 +1 @@ +Fixed iterator of :class:`multiprocessing.managers.DictProxy`. From webhook-mailer at python.org Mon Sep 17 07:53:36 2018 From: webhook-mailer at python.org (Christian Heimes) Date: Mon, 17 Sep 2018 11:53:36 -0000 Subject: [Python-checkins] bpo-34710: fix SSL module build (GH-9347) Message-ID: https://github.com/python/cpython/commit/b3a271fc0ce3e13e427be8914decfc205a220ca8 commit: b3a271fc0ce3e13e427be8914decfc205a220ca8 branch: master author: Alexandru Ardelean committer: Christian Heimes date: 2018-09-17T04:53:31-07:00 summary: bpo-34710: fix SSL module build (GH-9347) Include ``openssl/dh.h`` header file to fix implicit function declaration of ``DH_free()``. Signed-off-by: Alexandru Ardelean files: A Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst b/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst new file mode 100644 index 000000000000..b06289d091e5 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst @@ -0,0 +1 @@ +Fixed SSL module build with OpenSSL & pedantic CFLAGS. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 2bce4816d26f..3c9388480941 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -63,6 +63,7 @@ static PySocketModule_APIObject PySocketModule; #include "openssl/err.h" #include "openssl/rand.h" #include "openssl/bio.h" +#include "openssl/dh.h" #ifndef HAVE_X509_VERIFY_PARAM_SET1_HOST # ifdef LIBRESSL_VERSION_NUMBER From webhook-mailer at python.org Mon Sep 17 08:11:02 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 12:11:02 -0000 Subject: [Python-checkins] bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113) Message-ID: https://github.com/python/cpython/commit/1d3078849381335008379e3de96d5a81d2131113 commit: 1d3078849381335008379e3de96d5a81d2131113 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T05:10:56-07:00 summary: bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113) (cherry picked from commit e0e5065daef36dafe10a46eaa8b7800274d73062) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst M Lib/multiprocessing/managers.py M Lib/test/_test_multiprocessing.py diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py index 3f263802bc7c..dbed993a38d6 100644 --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -1134,10 +1134,13 @@ def __imul__(self, value): DictProxy = MakeProxyType('DictProxy', ( - '__contains__', '__delitem__', '__getitem__', '__len__', + '__contains__', '__delitem__', '__getitem__', '__iter__', '__len__', '__setitem__', 'clear', 'copy', 'get', 'has_key', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values' )) +DictProxy._method_to_typeid_ = { + '__iter__': 'Iterator', + } ArrayProxy = MakeProxyType('ArrayProxy', ( diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index 2ece58b305d3..754a26f47067 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -2079,6 +2079,16 @@ def test_list(self): a.append('hello') self.assertEqual(f[0][:], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 'hello']) + def test_list_iter(self): + a = self.list(list(range(10))) + it = iter(a) + self.assertEqual(list(it), list(range(10))) + self.assertEqual(list(it), []) # exhausted + # list modified during iteration + it = iter(a) + a[0] = 100 + self.assertEqual(next(it), 100) + def test_list_proxy_in_list(self): a = self.list([self.list(range(3)) for _i in range(3)]) self.assertEqual([inner[:] for inner in a], [[0, 1, 2]] * 3) @@ -2109,6 +2119,19 @@ def test_dict(self): self.assertEqual(sorted(d.values()), [chr(i) for i in indices]) self.assertEqual(sorted(d.items()), [(i, chr(i)) for i in indices]) + def test_dict_iter(self): + d = self.dict() + indices = list(range(65, 70)) + for i in indices: + d[i] = chr(i) + it = iter(d) + self.assertEqual(list(it), indices) + self.assertEqual(list(it), []) # exhausted + # dictionary changed size during iteration + it = iter(d) + d.clear() + self.assertRaises(RuntimeError, next, it) + def test_dict_proxy_nested(self): pets = self.dict(ferrets=2, hamsters=4) supplies = self.dict(water=10, feed=3) diff --git a/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst b/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst new file mode 100644 index 000000000000..bffb355ea2ab --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst @@ -0,0 +1 @@ +Fixed iterator of :class:`multiprocessing.managers.DictProxy`. From webhook-mailer at python.org Mon Sep 17 08:11:07 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 12:11:07 -0000 Subject: [Python-checkins] bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113) Message-ID: https://github.com/python/cpython/commit/f35e4d5851555d66f1e7a36b2a37f0d82b3d1706 commit: f35e4d5851555d66f1e7a36b2a37f0d82b3d1706 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T05:11:04-07:00 summary: bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113) (cherry picked from commit e0e5065daef36dafe10a46eaa8b7800274d73062) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst M Lib/multiprocessing/managers.py M Lib/test/_test_multiprocessing.py diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py index 27f26fdf6fec..4d74401398cc 100644 --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -1095,10 +1095,13 @@ def __imul__(self, value): DictProxy = MakeProxyType('DictProxy', ( - '__contains__', '__delitem__', '__getitem__', '__len__', + '__contains__', '__delitem__', '__getitem__', '__iter__', '__len__', '__setitem__', 'clear', 'copy', 'get', 'has_key', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values' )) +DictProxy._method_to_typeid_ = { + '__iter__': 'Iterator', + } ArrayProxy = MakeProxyType('ArrayProxy', ( diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index 5c625dd49541..9d4917076c94 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -1813,6 +1813,16 @@ def test_list(self): a.append('hello') self.assertEqual(f[0][:], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 'hello']) + def test_list_iter(self): + a = self.list(list(range(10))) + it = iter(a) + self.assertEqual(list(it), list(range(10))) + self.assertEqual(list(it), []) # exhausted + # list modified during iteration + it = iter(a) + a[0] = 100 + self.assertEqual(next(it), 100) + def test_list_proxy_in_list(self): a = self.list([self.list(range(3)) for _i in range(3)]) self.assertEqual([inner[:] for inner in a], [[0, 1, 2]] * 3) @@ -1843,6 +1853,19 @@ def test_dict(self): self.assertEqual(sorted(d.values()), [chr(i) for i in indices]) self.assertEqual(sorted(d.items()), [(i, chr(i)) for i in indices]) + def test_dict_iter(self): + d = self.dict() + indices = list(range(65, 70)) + for i in indices: + d[i] = chr(i) + it = iter(d) + self.assertEqual(list(it), indices) + self.assertEqual(list(it), []) # exhausted + # dictionary changed size during iteration + it = iter(d) + d.clear() + self.assertRaises(RuntimeError, next, it) + def test_dict_proxy_nested(self): pets = self.dict(ferrets=2, hamsters=4) supplies = self.dict(water=10, feed=3) diff --git a/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst b/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst new file mode 100644 index 000000000000..bffb355ea2ab --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst @@ -0,0 +1 @@ +Fixed iterator of :class:`multiprocessing.managers.DictProxy`. From webhook-mailer at python.org Mon Sep 17 08:15:06 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Mon, 17 Sep 2018 12:15:06 -0000 Subject: [Python-checkins] bpo-33216: Improve the documentation for CALL_FUNCTION_* (GH-8338) (GH-8784) Message-ID: https://github.com/python/cpython/commit/5e99b56d6b249995a4fa2bc09c0bb03841f49572 commit: 5e99b56d6b249995a4fa2bc09c0bb03841f49572 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-17T15:15:03+03:00 summary: bpo-33216: Improve the documentation for CALL_FUNCTION_* (GH-8338) (GH-8784) files: M Doc/library/dis.rst diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 8f505e65bd51..95649379ac6d 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -1104,18 +1104,20 @@ All of the following opcodes use their arguments. .. opcode:: RAISE_VARARGS (argc) - Raises an exception. *argc* indicates the number of parameters to the raise + Raises an exception. *argc* indicates the number of arguments to the raise statement, ranging from 0 to 3. The handler will find the traceback as TOS2, the parameter as TOS1, and the exception as TOS. .. opcode:: CALL_FUNCTION (argc) - Calls a function. *argc* indicates the number of positional arguments. - The positional arguments are on the stack, with the right-most argument - on top. Below the arguments, the function object to call is on the stack. - Pops all function arguments, and the function itself off the stack, and - pushes the return value. + Calls a callable object with positional arguments. + *argc* indicates the number of positional arguments. + The top of the stack contains positional arguments, with the right-most + argument on top. Below the arguments is a callable object to call. + ``CALL_FUNCTION`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. versionchanged:: 3.6 This opcode is used only for calls with positional arguments. @@ -1123,31 +1125,36 @@ All of the following opcodes use their arguments. .. opcode:: CALL_FUNCTION_KW (argc) - Calls a function. *argc* indicates the number of arguments (positional - and keyword). The top element on the stack contains a tuple of keyword - argument names. Below the tuple, keyword arguments are on the stack, in - the order corresponding to the tuple. Below the keyword arguments, the - positional arguments are on the stack, with the right-most parameter on - top. Below the arguments, the function object to call is on the stack. - Pops all function arguments, and the function itself off the stack, and - pushes the return value. + Calls a callable object with positional (if any) and keyword arguments. + *argc* indicates the total number of positional and keyword arguments. + The top element on the stack contains a tuple of keyword argument names. + Below that are keyword arguments in the order corresponding to the tuple. + Below that are positional arguments, with the right-most parameter on + top. Below the arguments is a callable object to call. + ``CALL_FUNCTION_KW`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. versionchanged:: 3.6 Keyword arguments are packed in a tuple instead of a dictionary, - *argc* indicates the total number of arguments + *argc* indicates the total number of arguments. .. opcode:: CALL_FUNCTION_EX (flags) - Calls a function. The lowest bit of *flags* indicates whether the - var-keyword argument is placed at the top of the stack. Below the - var-keyword argument, the var-positional argument is on the stack. - Below the arguments, the function object to call is placed. - Pops all function arguments, and the function itself off the stack, and - pushes the return value. Note that this opcode pops at most three items - from the stack. Var-positional and var-keyword arguments are packed - by :opcode:`BUILD_TUPLE_UNPACK_WITH_CALL` and - :opcode:`BUILD_MAP_UNPACK_WITH_CALL`. + Calls a callable object with variable set of positional and keyword + arguments. If the lowest bit of *flags* is set, the top of the stack + contains a mapping object containing additional keyword arguments. + Below that is an iterable object containing positional arguments and + a callable object to call. :opcode:`BUILD_MAP_UNPACK_WITH_CALL` and + :opcode:`BUILD_TUPLE_UNPACK_WITH_CALL` can be used for merging multiple + mapping objects and iterables containing arguments. + Before the callable is called, the mapping object and iterable object + are each "unpacked" and their contents passed in as keyword and + positional arguments respectively. + ``CALL_FUNCTION_EX`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. versionadded:: 3.6 @@ -1179,7 +1186,8 @@ All of the following opcodes use their arguments. Pushes a new function object on the stack. From bottom to top, the consumed stack must consist of values if the argument carries a specified flag value - * ``0x01`` a tuple of default argument objects in positional order + * ``0x01`` a tuple of default values for positional-only and + positional-or-keyword parameters in positional order * ``0x02`` a dictionary of keyword-only parameters' default values * ``0x04`` an annotation dictionary * ``0x08`` a tuple containing cells for free variables, making a closure @@ -1262,7 +1270,7 @@ instructions: .. data:: hasconst - Sequence of bytecodes that have a constant parameter. + Sequence of bytecodes that access a constant. .. data:: hasfree From webhook-mailer at python.org Mon Sep 17 08:15:51 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Mon, 17 Sep 2018 12:15:51 -0000 Subject: [Python-checkins] [2.7] bpo-33216: Clarify the documentation for CALL_FUNCTION_* (GH-8338) (GH-8783) Message-ID: https://github.com/python/cpython/commit/afa591d7748c66831e6e650108024f3293be33f1 commit: afa591d7748c66831e6e650108024f3293be33f1 branch: 2.7 author: Serhiy Storchaka committer: GitHub date: 2018-09-17T15:15:48+03:00 summary: [2.7] bpo-33216: Clarify the documentation for CALL_FUNCTION_* (GH-8338) (GH-8783) (cherry picked from commit 76aa2c0a9a8dd3ac90b91e7342c8ce8125bf21f9) Co-authored-by: larryhastings files: M Doc/library/dis.rst diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 04b9b1538633..e773ab8090fc 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -107,7 +107,7 @@ The :mod:`dis` module defines the following functions and constants: .. data:: hasconst - Sequence of bytecodes that have a constant parameter. + Sequence of bytecodes that access a constant. .. data:: hasfree @@ -796,21 +796,25 @@ the more significant byte last. .. opcode:: RAISE_VARARGS (argc) - Raises an exception. *argc* indicates the number of parameters to the raise + Raises an exception. *argc* indicates the number of arguments to the raise statement, ranging from 0 to 3. The handler will find the traceback as TOS2, the parameter as TOS1, and the exception as TOS. .. opcode:: CALL_FUNCTION (argc) - Calls a function. The low byte of *argc* indicates the number of positional - parameters, the high byte the number of keyword parameters. On the stack, the - opcode finds the keyword parameters first. For each keyword argument, the - value is on top of the key. Below the keyword parameters, the positional - parameters are on the stack, with the right-most parameter on top. Below the - parameters, the function object to call is on the stack. Pops all function - arguments, and the function itself off the stack, and pushes the return - value. + Calls a callable object. The low byte of *argc* indicates the number of + positional arguments, the high byte the number of keyword arguments. + The stack contains keyword arguments on top (if any), then the positional + arguments below that (if any), then the callable object to call below that. + Each keyword argument is represented with two values on the stack: + the argument's name, and its value, with the argument's value above the + name on the stack. + The positional arguments are pushed in the order that they are passed in + to the callable object, with the right-most positional argument on top. + ``CALL_FUNCTION`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. opcode:: MAKE_FUNCTION (argc) @@ -847,24 +851,53 @@ the more significant byte last. .. opcode:: CALL_FUNCTION_VAR (argc) - Calls a function. *argc* is interpreted as in :opcode:`CALL_FUNCTION`. The - top element on the stack contains the variable argument list, followed by - keyword and positional arguments. + Calls a callable object, similarly to :opcode:`CALL_FUNCTION`. + *argc* represents the number of keyword and positional + arguments, identically to :opcode:`CALL_FUNCTION`. + The top of the stack contains an iterable object containing + additional positional arguments. + Below that are keyword arguments (if any), positional arguments (if any) + and a callable object, identically to :opcode:`CALL_FUNCTION`. + Before the callable object is called, the iterable object + is "unpacked" and its contents are appended to the positional + arguments passed in. + The iterable object is ignored when computing + the value of ``argc``. .. opcode:: CALL_FUNCTION_KW (argc) - Calls a function. *argc* is interpreted as in :opcode:`CALL_FUNCTION`. The - top element on the stack contains the keyword arguments dictionary, followed - by explicit keyword and positional arguments. + Calls a callable object, similarly to :opcode:`CALL_FUNCTION`. + *argc* represents the number of keyword and positional + arguments, identically to :opcode:`CALL_FUNCTION`. + The top of the stack contains a mapping object containing additional keyword + arguments. + Below that are keyword arguments (if any), positional arguments (if any) + and a callable object, identically to :opcode:`CALL_FUNCTION`. + Before the callable is called, the mapping object at the top of the stack is + "unpacked" and its contents are appended to the keyword arguments passed in. + The mapping object at the top of the stack is ignored when computing + the value of ``argc``. .. opcode:: CALL_FUNCTION_VAR_KW (argc) - Calls a function. *argc* is interpreted as in :opcode:`CALL_FUNCTION`. The - top element on the stack contains the keyword arguments dictionary, followed - by the variable-arguments tuple, followed by explicit keyword and positional - arguments. + Calls a callable object, similarly to :opcode:`CALL_FUNCTION_VAR` and + :opcode:`CALL_FUNCTION_KW`. + *argc* represents the number of keyword and positional + arguments, identically to :opcode:`CALL_FUNCTION`. + The top of the stack contains a mapping object, as per + :opcode:`CALL_FUNCTION_KW`. + Below that is an iterable object, as per + :opcode:`CALL_FUNCTION_VAR`. + Below that are keyword arguments (if any), positional arguments (if any) + and a callable object, identically to :opcode:`CALL_FUNCTION`. + Before the callable is called, the mapping object and iterable object + are each "unpacked" and their contents passed in as keyword and + positional arguments respectively, + identically to :opcode:`CALL_FUNCTION_VAR` and :opcode:`CALL_FUNCTION_KW`. + The mapping object and iterable object are both ignored when computing + the value of ``argc``. .. opcode:: HAVE_ARGUMENT () From webhook-mailer at python.org Mon Sep 17 08:17:33 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Mon, 17 Sep 2018 12:17:33 -0000 Subject: [Python-checkins] bpo-12458: Fix line numbers for multiline expressions. (GH-8774) Message-ID: https://github.com/python/cpython/commit/da8d72c953369b872a12c13f136ada77a786714a commit: da8d72c953369b872a12c13f136ada77a786714a branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-17T15:17:29+03:00 summary: bpo-12458: Fix line numbers for multiline expressions. (GH-8774) files: A Misc/NEWS.d/next/Core and Builtins/2018-08-15-20-46-49.bpo-12458.ApHbx5.rst M Lib/test/test_dis.py M Lib/test/test_doctest.py M Lib/test/test_faulthandler.py M Lib/test/test_traceback.py M Lib/test/test_tracemalloc.py M Python/compile.c M Python/importlib.h M Python/importlib_external.h diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index c86f61f236bd..8d1912b6ee58 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -124,7 +124,8 @@ def bug708901(): 2 LOAD_CONST 1 (1) %3d 4 LOAD_CONST 2 (10) - 6 CALL_FUNCTION 2 + +%3d 6 CALL_FUNCTION 2 8 GET_ITER >> 10 FOR_ITER 4 (to 16) 12 STORE_FAST 0 (res) @@ -134,6 +135,7 @@ def bug708901(): 18 RETURN_VALUE """ % (bug708901.__code__.co_firstlineno + 1, bug708901.__code__.co_firstlineno + 2, + bug708901.__code__.co_firstlineno + 1, bug708901.__code__.co_firstlineno + 3) @@ -154,7 +156,8 @@ def bug1333982(x=[]): 16 CALL_FUNCTION 1 %3d 18 LOAD_CONST 4 (1) - 20 BINARY_ADD + +%3d 20 BINARY_ADD 22 CALL_FUNCTION 1 24 RAISE_VARARGS 1 @@ -164,6 +167,7 @@ def bug1333982(x=[]): __file__, bug1333982.__code__.co_firstlineno + 1, bug1333982.__code__.co_firstlineno + 2, + bug1333982.__code__.co_firstlineno + 1, bug1333982.__code__.co_firstlineno + 3) _BIG_LINENO_FORMAT = """\ diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py index 797bdb847190..efdce3c90c26 100644 --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest.py @@ -2719,7 +2719,7 @@ def test_unicode(): """ Exception raised: Traceback (most recent call last): File ... - compileflags, 1), test.globs) + exec(compile(example.source, filename, "single", File "", line 1, in raise Exception('cl?') Exception: cl? diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py index d6dc4ba55d9b..59289d026e39 100644 --- a/Lib/test/test_faulthandler.py +++ b/Lib/test/test_faulthandler.py @@ -400,7 +400,7 @@ def funcA(): if filename: lineno = 9 elif fd is not None: - lineno = 12 + lineno = 11 else: lineno = 14 expected = [ diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index 3af85b81721e..a8240b41c084 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -995,11 +995,11 @@ def some_inner(k, v): s = some_inner(3, 4) self.assertEqual( [' File "%s", line %d, in some_inner\n' - ' traceback.walk_stack(None), capture_locals=True, limit=1)\n' + ' return traceback.StackSummary.extract(\n' ' a = 1\n' ' b = 2\n' ' k = 3\n' - ' v = 4\n' % (__file__, some_inner.__code__.co_firstlineno + 4) + ' v = 4\n' % (__file__, some_inner.__code__.co_firstlineno + 3) ], s.format()) class TestTracebackException(unittest.TestCase): diff --git a/Lib/test/test_tracemalloc.py b/Lib/test/test_tracemalloc.py index 491cd057ef87..76a6159c7aa4 100644 --- a/Lib/test/test_tracemalloc.py +++ b/Lib/test/test_tracemalloc.py @@ -936,7 +936,7 @@ def get_traceback(self): return None def track(self, release_gil=False, nframe=1): - frames = get_frames(nframe, 2) + frames = get_frames(nframe, 1) _testcapi.tracemalloc_track(self.domain, self.ptr, self.size, release_gil) return frames diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-08-15-20-46-49.bpo-12458.ApHbx5.rst b/Misc/NEWS.d/next/Core and Builtins/2018-08-15-20-46-49.bpo-12458.ApHbx5.rst new file mode 100644 index 000000000000..7479654fad7a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-08-15-20-46-49.bpo-12458.ApHbx5.rst @@ -0,0 +1,3 @@ +Tracebacks show now correct line number for subexpressions in multiline +expressions. Tracebacks show now the line number of the first line for +multiline expressions instead of the line number of the last subexpression. diff --git a/Python/compile.c b/Python/compile.c index c6a667c29efe..ebd73fb72319 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -4089,10 +4089,6 @@ compiler_comprehension(struct compiler *c, expr_ty e, int type, is_async_generator = c->u->u_ste->ste_coroutine; if (is_async_generator && !is_async_function && type != COMP_GENEXP) { - if (e->lineno > c->u->u_lineno) { - c->u->u_lineno = e->lineno; - c->u->u_lineno_set = 0; - } compiler_error(c, "asynchronous comprehension outside of " "an asynchronous function"); goto error_in_scope; @@ -4430,17 +4426,8 @@ compiler_with(struct compiler *c, stmt_ty s, int pos) } static int -compiler_visit_expr(struct compiler *c, expr_ty e) +compiler_visit_expr1(struct compiler *c, expr_ty e) { - /* If expr e has a different line number than the last expr/stmt, - set a new line number for the next instruction. - */ - if (e->lineno > c->u->u_lineno) { - c->u->u_lineno = e->lineno; - c->u->u_lineno_set = 0; - } - /* Updating the column offset is always harmless. */ - c->u->u_col_offset = e->col_offset; switch (e->kind) { case BoolOp_kind: return compiler_boolop(c, e); @@ -4609,6 +4596,31 @@ compiler_visit_expr(struct compiler *c, expr_ty e) return 1; } +static int +compiler_visit_expr(struct compiler *c, expr_ty e) +{ + /* If expr e has a different line number than the last expr/stmt, + set a new line number for the next instruction. + */ + int old_lineno = c->u->u_lineno; + int old_col_offset = c->u->u_col_offset; + if (e->lineno != c->u->u_lineno) { + c->u->u_lineno = e->lineno; + c->u->u_lineno_set = 0; + } + /* Updating the column offset is always harmless. */ + c->u->u_col_offset = e->col_offset; + + int res = compiler_visit_expr1(c, e); + + if (old_lineno != c->u->u_lineno) { + c->u->u_lineno = old_lineno; + c->u->u_lineno_set = 0; + } + c->u->u_col_offset = old_col_offset; + return res; +} + static int compiler_augassign(struct compiler *c, stmt_ty s) { diff --git a/Python/importlib.h b/Python/importlib.h index f006b87606f2..1d2576005e9e 100644 --- a/Python/importlib.h +++ b/Python/importlib.h @@ -430,397 +430,399 @@ const unsigned char _Py_M__importlib[] = { 101,41,1,218,3,102,120,110,114,10,0,0,0,114,11,0, 0,0,218,25,95,114,101,113,117,105,114,101,115,95,98,117, 105,108,116,105,110,95,119,114,97,112,112,101,114,232,0,0, - 0,115,8,0,0,0,0,1,10,1,10,1,8,1,122,52, - 95,114,101,113,117,105,114,101,115,95,98,117,105,108,116,105, - 110,46,60,108,111,99,97,108,115,62,46,95,114,101,113,117, - 105,114,101,115,95,98,117,105,108,116,105,110,95,119,114,97, - 112,112,101,114,41,1,114,12,0,0,0,41,2,114,72,0, - 0,0,114,73,0,0,0,114,10,0,0,0,41,1,114,72, - 0,0,0,114,11,0,0,0,218,17,95,114,101,113,117,105, - 114,101,115,95,98,117,105,108,116,105,110,230,0,0,0,115, - 6,0,0,0,0,2,12,5,10,1,114,74,0,0,0,99, - 1,0,0,0,0,0,0,0,2,0,0,0,3,0,0,0, - 3,0,0,0,115,26,0,0,0,135,0,102,1,100,1,100, - 2,132,8,125,1,116,0,124,1,136,0,131,2,1,0,124, - 1,83,0,41,3,122,47,68,101,99,111,114,97,116,111,114, - 32,116,111,32,118,101,114,105,102,121,32,116,104,101,32,110, - 97,109,101,100,32,109,111,100,117,108,101,32,105,115,32,102, - 114,111,122,101,110,46,99,2,0,0,0,0,0,0,0,2, - 0,0,0,4,0,0,0,19,0,0,0,115,38,0,0,0, - 116,0,160,1,124,1,161,1,115,28,116,2,100,1,160,3, - 124,1,161,1,124,1,100,2,141,2,130,1,136,0,124,0, - 124,1,131,2,83,0,41,3,78,122,27,123,33,114,125,32, - 105,115,32,110,111,116,32,97,32,102,114,111,122,101,110,32, - 109,111,100,117,108,101,41,1,114,15,0,0,0,41,4,114, - 49,0,0,0,218,9,105,115,95,102,114,111,122,101,110,114, - 70,0,0,0,114,38,0,0,0,41,2,114,26,0,0,0, - 114,71,0,0,0,41,1,114,72,0,0,0,114,10,0,0, - 0,114,11,0,0,0,218,24,95,114,101,113,117,105,114,101, - 115,95,102,114,111,122,101,110,95,119,114,97,112,112,101,114, - 243,0,0,0,115,8,0,0,0,0,1,10,1,10,1,8, - 1,122,50,95,114,101,113,117,105,114,101,115,95,102,114,111, - 122,101,110,46,60,108,111,99,97,108,115,62,46,95,114,101, - 113,117,105,114,101,115,95,102,114,111,122,101,110,95,119,114, - 97,112,112,101,114,41,1,114,12,0,0,0,41,2,114,72, - 0,0,0,114,76,0,0,0,114,10,0,0,0,41,1,114, - 72,0,0,0,114,11,0,0,0,218,16,95,114,101,113,117, - 105,114,101,115,95,102,114,111,122,101,110,241,0,0,0,115, - 6,0,0,0,0,2,12,5,10,1,114,77,0,0,0,99, - 2,0,0,0,0,0,0,0,4,0,0,0,3,0,0,0, - 67,0,0,0,115,62,0,0,0,116,0,124,1,124,0,131, - 2,125,2,124,1,116,1,106,2,107,6,114,50,116,1,106, - 2,124,1,25,0,125,3,116,3,124,2,124,3,131,2,1, - 0,116,1,106,2,124,1,25,0,83,0,116,4,124,2,131, - 1,83,0,100,1,83,0,41,2,122,128,76,111,97,100,32, - 116,104,101,32,115,112,101,99,105,102,105,101,100,32,109,111, - 100,117,108,101,32,105,110,116,111,32,115,121,115,46,109,111, - 100,117,108,101,115,32,97,110,100,32,114,101,116,117,114,110, - 32,105,116,46,10,10,32,32,32,32,84,104,105,115,32,109, - 101,116,104,111,100,32,105,115,32,100,101,112,114,101,99,97, - 116,101,100,46,32,32,85,115,101,32,108,111,97,100,101,114, - 46,101,120,101,99,95,109,111,100,117,108,101,32,105,110,115, - 116,101,97,100,46,10,10,32,32,32,32,78,41,5,218,16, - 115,112,101,99,95,102,114,111,109,95,108,111,97,100,101,114, - 114,14,0,0,0,218,7,109,111,100,117,108,101,115,218,5, - 95,101,120,101,99,218,5,95,108,111,97,100,41,4,114,26, - 0,0,0,114,71,0,0,0,218,4,115,112,101,99,218,6, - 109,111,100,117,108,101,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,17,95,108,111,97,100,95,109,111,100, - 117,108,101,95,115,104,105,109,253,0,0,0,115,12,0,0, - 0,0,6,10,1,10,1,10,1,10,1,10,2,114,84,0, - 0,0,99,1,0,0,0,0,0,0,0,5,0,0,0,8, - 0,0,0,67,0,0,0,115,226,0,0,0,116,0,124,0, - 100,1,100,0,131,3,125,1,116,1,124,1,100,2,131,2, - 114,56,122,12,124,1,160,2,124,0,161,1,87,0,83,0, - 4,0,116,3,107,10,114,54,1,0,1,0,1,0,89,0, - 110,2,88,0,122,10,124,0,106,4,125,2,87,0,110,20, - 4,0,116,5,107,10,114,86,1,0,1,0,1,0,89,0, - 110,18,88,0,124,2,100,0,107,9,114,104,116,6,124,2, - 131,1,83,0,122,10,124,0,106,7,125,3,87,0,110,24, - 4,0,116,5,107,10,114,138,1,0,1,0,1,0,100,3, - 125,3,89,0,110,2,88,0,122,10,124,0,106,8,125,4, - 87,0,110,58,4,0,116,5,107,10,114,208,1,0,1,0, - 1,0,124,1,100,0,107,8,114,188,100,4,160,9,124,3, - 161,1,6,0,89,0,83,0,100,5,160,9,124,3,124,1, - 161,2,6,0,89,0,83,0,89,0,110,14,88,0,100,6, - 160,9,124,3,124,4,161,2,83,0,100,0,83,0,41,7, - 78,218,10,95,95,108,111,97,100,101,114,95,95,218,11,109, - 111,100,117,108,101,95,114,101,112,114,250,1,63,122,13,60, - 109,111,100,117,108,101,32,123,33,114,125,62,122,20,60,109, - 111,100,117,108,101,32,123,33,114,125,32,40,123,33,114,125, - 41,62,122,23,60,109,111,100,117,108,101,32,123,33,114,125, - 32,102,114,111,109,32,123,33,114,125,62,41,10,114,6,0, - 0,0,114,4,0,0,0,114,86,0,0,0,218,9,69,120, - 99,101,112,116,105,111,110,218,8,95,95,115,112,101,99,95, - 95,218,14,65,116,116,114,105,98,117,116,101,69,114,114,111, - 114,218,22,95,109,111,100,117,108,101,95,114,101,112,114,95, - 102,114,111,109,95,115,112,101,99,114,1,0,0,0,218,8, - 95,95,102,105,108,101,95,95,114,38,0,0,0,41,5,114, - 83,0,0,0,218,6,108,111,97,100,101,114,114,82,0,0, - 0,114,15,0,0,0,218,8,102,105,108,101,110,97,109,101, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 12,95,109,111,100,117,108,101,95,114,101,112,114,13,1,0, - 0,115,46,0,0,0,0,2,12,1,10,4,2,1,12,1, - 14,1,6,1,2,1,10,1,14,1,6,2,8,1,8,4, - 2,1,10,1,14,1,10,1,2,1,10,1,14,1,8,1, - 14,2,22,2,114,95,0,0,0,99,0,0,0,0,0,0, - 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,36, - 0,0,0,101,0,90,1,100,0,90,2,100,1,100,2,132, - 0,90,3,100,3,100,4,132,0,90,4,100,5,100,6,132, - 0,90,5,100,7,83,0,41,8,218,17,95,105,110,115,116, - 97,108,108,101,100,95,115,97,102,101,108,121,99,2,0,0, - 0,0,0,0,0,2,0,0,0,2,0,0,0,67,0,0, - 0,115,18,0,0,0,124,1,124,0,95,0,124,1,106,1, - 124,0,95,2,100,0,83,0,41,1,78,41,3,218,7,95, - 109,111,100,117,108,101,114,89,0,0,0,218,5,95,115,112, - 101,99,41,2,114,26,0,0,0,114,83,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,27,0, - 0,0,51,1,0,0,115,4,0,0,0,0,1,6,1,122, - 26,95,105,110,115,116,97,108,108,101,100,95,115,97,102,101, - 108,121,46,95,95,105,110,105,116,95,95,99,1,0,0,0, - 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, - 115,28,0,0,0,100,1,124,0,106,0,95,1,124,0,106, - 2,116,3,106,4,124,0,106,0,106,5,60,0,100,0,83, - 0,41,2,78,84,41,6,114,98,0,0,0,218,13,95,105, - 110,105,116,105,97,108,105,122,105,110,103,114,97,0,0,0, - 114,14,0,0,0,114,79,0,0,0,114,15,0,0,0,41, - 1,114,26,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,114,46,0,0,0,55,1,0,0,115,4, - 0,0,0,0,4,8,1,122,27,95,105,110,115,116,97,108, - 108,101,100,95,115,97,102,101,108,121,46,95,95,101,110,116, - 101,114,95,95,99,1,0,0,0,0,0,0,0,3,0,0, - 0,8,0,0,0,71,0,0,0,115,98,0,0,0,122,82, - 124,0,106,0,125,2,116,1,100,1,100,2,132,0,124,1, - 68,0,131,1,131,1,114,64,122,14,116,2,106,3,124,2, - 106,4,61,0,87,0,113,80,4,0,116,5,107,10,114,60, - 1,0,1,0,1,0,89,0,113,80,88,0,110,16,116,6, - 100,3,124,2,106,4,124,2,106,7,131,3,1,0,87,0, - 53,0,100,4,124,0,106,0,95,8,88,0,100,0,83,0, - 41,5,78,99,1,0,0,0,0,0,0,0,2,0,0,0, - 3,0,0,0,115,0,0,0,115,22,0,0,0,124,0,93, - 14,125,1,124,1,100,0,107,9,86,0,1,0,113,2,100, - 0,83,0,41,1,78,114,10,0,0,0,41,2,90,2,46, - 48,90,3,97,114,103,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,9,60,103,101,110,101,120,112,114,62, - 65,1,0,0,115,2,0,0,0,4,0,122,45,95,105,110, + 0,115,10,0,0,0,0,1,10,1,10,1,2,255,6,2, + 122,52,95,114,101,113,117,105,114,101,115,95,98,117,105,108, + 116,105,110,46,60,108,111,99,97,108,115,62,46,95,114,101, + 113,117,105,114,101,115,95,98,117,105,108,116,105,110,95,119, + 114,97,112,112,101,114,41,1,114,12,0,0,0,41,2,114, + 72,0,0,0,114,73,0,0,0,114,10,0,0,0,41,1, + 114,72,0,0,0,114,11,0,0,0,218,17,95,114,101,113, + 117,105,114,101,115,95,98,117,105,108,116,105,110,230,0,0, + 0,115,6,0,0,0,0,2,12,5,10,1,114,74,0,0, + 0,99,1,0,0,0,0,0,0,0,2,0,0,0,3,0, + 0,0,3,0,0,0,115,26,0,0,0,135,0,102,1,100, + 1,100,2,132,8,125,1,116,0,124,1,136,0,131,2,1, + 0,124,1,83,0,41,3,122,47,68,101,99,111,114,97,116, + 111,114,32,116,111,32,118,101,114,105,102,121,32,116,104,101, + 32,110,97,109,101,100,32,109,111,100,117,108,101,32,105,115, + 32,102,114,111,122,101,110,46,99,2,0,0,0,0,0,0, + 0,2,0,0,0,4,0,0,0,19,0,0,0,115,38,0, + 0,0,116,0,160,1,124,1,161,1,115,28,116,2,100,1, + 160,3,124,1,161,1,124,1,100,2,141,2,130,1,136,0, + 124,0,124,1,131,2,83,0,41,3,78,122,27,123,33,114, + 125,32,105,115,32,110,111,116,32,97,32,102,114,111,122,101, + 110,32,109,111,100,117,108,101,41,1,114,15,0,0,0,41, + 4,114,49,0,0,0,218,9,105,115,95,102,114,111,122,101, + 110,114,70,0,0,0,114,38,0,0,0,41,2,114,26,0, + 0,0,114,71,0,0,0,41,1,114,72,0,0,0,114,10, + 0,0,0,114,11,0,0,0,218,24,95,114,101,113,117,105, + 114,101,115,95,102,114,111,122,101,110,95,119,114,97,112,112, + 101,114,243,0,0,0,115,10,0,0,0,0,1,10,1,10, + 1,2,255,6,2,122,50,95,114,101,113,117,105,114,101,115, + 95,102,114,111,122,101,110,46,60,108,111,99,97,108,115,62, + 46,95,114,101,113,117,105,114,101,115,95,102,114,111,122,101, + 110,95,119,114,97,112,112,101,114,41,1,114,12,0,0,0, + 41,2,114,72,0,0,0,114,76,0,0,0,114,10,0,0, + 0,41,1,114,72,0,0,0,114,11,0,0,0,218,16,95, + 114,101,113,117,105,114,101,115,95,102,114,111,122,101,110,241, + 0,0,0,115,6,0,0,0,0,2,12,5,10,1,114,77, + 0,0,0,99,2,0,0,0,0,0,0,0,4,0,0,0, + 3,0,0,0,67,0,0,0,115,62,0,0,0,116,0,124, + 1,124,0,131,2,125,2,124,1,116,1,106,2,107,6,114, + 50,116,1,106,2,124,1,25,0,125,3,116,3,124,2,124, + 3,131,2,1,0,116,1,106,2,124,1,25,0,83,0,116, + 4,124,2,131,1,83,0,100,1,83,0,41,2,122,128,76, + 111,97,100,32,116,104,101,32,115,112,101,99,105,102,105,101, + 100,32,109,111,100,117,108,101,32,105,110,116,111,32,115,121, + 115,46,109,111,100,117,108,101,115,32,97,110,100,32,114,101, + 116,117,114,110,32,105,116,46,10,10,32,32,32,32,84,104, + 105,115,32,109,101,116,104,111,100,32,105,115,32,100,101,112, + 114,101,99,97,116,101,100,46,32,32,85,115,101,32,108,111, + 97,100,101,114,46,101,120,101,99,95,109,111,100,117,108,101, + 32,105,110,115,116,101,97,100,46,10,10,32,32,32,32,78, + 41,5,218,16,115,112,101,99,95,102,114,111,109,95,108,111, + 97,100,101,114,114,14,0,0,0,218,7,109,111,100,117,108, + 101,115,218,5,95,101,120,101,99,218,5,95,108,111,97,100, + 41,4,114,26,0,0,0,114,71,0,0,0,218,4,115,112, + 101,99,218,6,109,111,100,117,108,101,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,218,17,95,108,111,97,100, + 95,109,111,100,117,108,101,95,115,104,105,109,253,0,0,0, + 115,12,0,0,0,0,6,10,1,10,1,10,1,10,1,10, + 2,114,84,0,0,0,99,1,0,0,0,0,0,0,0,5, + 0,0,0,8,0,0,0,67,0,0,0,115,226,0,0,0, + 116,0,124,0,100,1,100,0,131,3,125,1,116,1,124,1, + 100,2,131,2,114,56,122,12,124,1,160,2,124,0,161,1, + 87,0,83,0,4,0,116,3,107,10,114,54,1,0,1,0, + 1,0,89,0,110,2,88,0,122,10,124,0,106,4,125,2, + 87,0,110,20,4,0,116,5,107,10,114,86,1,0,1,0, + 1,0,89,0,110,18,88,0,124,2,100,0,107,9,114,104, + 116,6,124,2,131,1,83,0,122,10,124,0,106,7,125,3, + 87,0,110,24,4,0,116,5,107,10,114,138,1,0,1,0, + 1,0,100,3,125,3,89,0,110,2,88,0,122,10,124,0, + 106,8,125,4,87,0,110,58,4,0,116,5,107,10,114,208, + 1,0,1,0,1,0,124,1,100,0,107,8,114,188,100,4, + 160,9,124,3,161,1,6,0,89,0,83,0,100,5,160,9, + 124,3,124,1,161,2,6,0,89,0,83,0,89,0,110,14, + 88,0,100,6,160,9,124,3,124,4,161,2,83,0,100,0, + 83,0,41,7,78,218,10,95,95,108,111,97,100,101,114,95, + 95,218,11,109,111,100,117,108,101,95,114,101,112,114,250,1, + 63,122,13,60,109,111,100,117,108,101,32,123,33,114,125,62, + 122,20,60,109,111,100,117,108,101,32,123,33,114,125,32,40, + 123,33,114,125,41,62,122,23,60,109,111,100,117,108,101,32, + 123,33,114,125,32,102,114,111,109,32,123,33,114,125,62,41, + 10,114,6,0,0,0,114,4,0,0,0,114,86,0,0,0, + 218,9,69,120,99,101,112,116,105,111,110,218,8,95,95,115, + 112,101,99,95,95,218,14,65,116,116,114,105,98,117,116,101, + 69,114,114,111,114,218,22,95,109,111,100,117,108,101,95,114, + 101,112,114,95,102,114,111,109,95,115,112,101,99,114,1,0, + 0,0,218,8,95,95,102,105,108,101,95,95,114,38,0,0, + 0,41,5,114,83,0,0,0,218,6,108,111,97,100,101,114, + 114,82,0,0,0,114,15,0,0,0,218,8,102,105,108,101, + 110,97,109,101,114,10,0,0,0,114,10,0,0,0,114,11, + 0,0,0,218,12,95,109,111,100,117,108,101,95,114,101,112, + 114,13,1,0,0,115,46,0,0,0,0,2,12,1,10,4, + 2,1,12,1,14,1,6,1,2,1,10,1,14,1,6,2, + 8,1,8,4,2,1,10,1,14,1,10,1,2,1,10,1, + 14,1,8,1,14,2,22,2,114,95,0,0,0,99,0,0, + 0,0,0,0,0,0,0,0,0,0,2,0,0,0,64,0, + 0,0,115,36,0,0,0,101,0,90,1,100,0,90,2,100, + 1,100,2,132,0,90,3,100,3,100,4,132,0,90,4,100, + 5,100,6,132,0,90,5,100,7,83,0,41,8,218,17,95, + 105,110,115,116,97,108,108,101,100,95,115,97,102,101,108,121, + 99,2,0,0,0,0,0,0,0,2,0,0,0,2,0,0, + 0,67,0,0,0,115,18,0,0,0,124,1,124,0,95,0, + 124,1,106,1,124,0,95,2,100,0,83,0,41,1,78,41, + 3,218,7,95,109,111,100,117,108,101,114,89,0,0,0,218, + 5,95,115,112,101,99,41,2,114,26,0,0,0,114,83,0, + 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, + 0,114,27,0,0,0,51,1,0,0,115,4,0,0,0,0, + 1,6,1,122,26,95,105,110,115,116,97,108,108,101,100,95, + 115,97,102,101,108,121,46,95,95,105,110,105,116,95,95,99, + 1,0,0,0,0,0,0,0,1,0,0,0,3,0,0,0, + 67,0,0,0,115,28,0,0,0,100,1,124,0,106,0,95, + 1,124,0,106,2,116,3,106,4,124,0,106,0,106,5,60, + 0,100,0,83,0,41,2,78,84,41,6,114,98,0,0,0, + 218,13,95,105,110,105,116,105,97,108,105,122,105,110,103,114, + 97,0,0,0,114,14,0,0,0,114,79,0,0,0,114,15, + 0,0,0,41,1,114,26,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,114,46,0,0,0,55,1, + 0,0,115,4,0,0,0,0,4,8,1,122,27,95,105,110, 115,116,97,108,108,101,100,95,115,97,102,101,108,121,46,95, - 95,101,120,105,116,95,95,46,60,108,111,99,97,108,115,62, - 46,60,103,101,110,101,120,112,114,62,122,18,105,109,112,111, - 114,116,32,123,33,114,125,32,35,32,123,33,114,125,70,41, - 9,114,98,0,0,0,218,3,97,110,121,114,14,0,0,0, - 114,79,0,0,0,114,15,0,0,0,114,55,0,0,0,114, - 68,0,0,0,114,93,0,0,0,114,99,0,0,0,41,3, - 114,26,0,0,0,114,47,0,0,0,114,82,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,48, - 0,0,0,62,1,0,0,115,18,0,0,0,0,1,2,1, - 6,1,18,1,2,1,14,1,14,1,8,2,20,2,122,26, - 95,105,110,115,116,97,108,108,101,100,95,115,97,102,101,108, - 121,46,95,95,101,120,105,116,95,95,78,41,6,114,1,0, - 0,0,114,0,0,0,0,114,2,0,0,0,114,27,0,0, - 0,114,46,0,0,0,114,48,0,0,0,114,10,0,0,0, + 95,101,110,116,101,114,95,95,99,1,0,0,0,0,0,0, + 0,3,0,0,0,8,0,0,0,71,0,0,0,115,98,0, + 0,0,122,82,124,0,106,0,125,2,116,1,100,1,100,2, + 132,0,124,1,68,0,131,1,131,1,114,64,122,14,116,2, + 106,3,124,2,106,4,61,0,87,0,113,80,4,0,116,5, + 107,10,114,60,1,0,1,0,1,0,89,0,113,80,88,0, + 110,16,116,6,100,3,124,2,106,4,124,2,106,7,131,3, + 1,0,87,0,53,0,100,4,124,0,106,0,95,8,88,0, + 100,0,83,0,41,5,78,99,1,0,0,0,0,0,0,0, + 2,0,0,0,3,0,0,0,115,0,0,0,115,22,0,0, + 0,124,0,93,14,125,1,124,1,100,0,107,9,86,0,1, + 0,113,2,100,0,83,0,41,1,78,114,10,0,0,0,41, + 2,90,2,46,48,90,3,97,114,103,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,218,9,60,103,101,110,101, + 120,112,114,62,65,1,0,0,115,4,0,0,0,4,0,2, + 0,122,45,95,105,110,115,116,97,108,108,101,100,95,115,97, + 102,101,108,121,46,95,95,101,120,105,116,95,95,46,60,108, + 111,99,97,108,115,62,46,60,103,101,110,101,120,112,114,62, + 122,18,105,109,112,111,114,116,32,123,33,114,125,32,35,32, + 123,33,114,125,70,41,9,114,98,0,0,0,218,3,97,110, + 121,114,14,0,0,0,114,79,0,0,0,114,15,0,0,0, + 114,55,0,0,0,114,68,0,0,0,114,93,0,0,0,114, + 99,0,0,0,41,3,114,26,0,0,0,114,47,0,0,0, + 114,82,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,114,48,0,0,0,62,1,0,0,115,18,0, + 0,0,0,1,2,1,6,1,18,1,2,1,14,1,14,1, + 8,2,20,2,122,26,95,105,110,115,116,97,108,108,101,100, + 95,115,97,102,101,108,121,46,95,95,101,120,105,116,95,95, + 78,41,6,114,1,0,0,0,114,0,0,0,0,114,2,0, + 0,0,114,27,0,0,0,114,46,0,0,0,114,48,0,0, + 0,114,10,0,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,114,96,0,0,0,49,1,0,0,115,6, + 0,0,0,8,2,8,4,8,7,114,96,0,0,0,99,0, + 0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,64, + 0,0,0,115,114,0,0,0,101,0,90,1,100,0,90,2, + 100,1,90,3,100,2,100,2,100,2,100,3,156,3,100,4, + 100,5,132,2,90,4,100,6,100,7,132,0,90,5,100,8, + 100,9,132,0,90,6,101,7,100,10,100,11,132,0,131,1, + 90,8,101,8,106,9,100,12,100,11,132,0,131,1,90,8, + 101,7,100,13,100,14,132,0,131,1,90,10,101,7,100,15, + 100,16,132,0,131,1,90,11,101,11,106,9,100,17,100,16, + 132,0,131,1,90,11,100,2,83,0,41,18,218,10,77,111, + 100,117,108,101,83,112,101,99,97,208,5,0,0,84,104,101, + 32,115,112,101,99,105,102,105,99,97,116,105,111,110,32,102, + 111,114,32,97,32,109,111,100,117,108,101,44,32,117,115,101, + 100,32,102,111,114,32,108,111,97,100,105,110,103,46,10,10, + 32,32,32,32,65,32,109,111,100,117,108,101,39,115,32,115, + 112,101,99,32,105,115,32,116,104,101,32,115,111,117,114,99, + 101,32,102,111,114,32,105,110,102,111,114,109,97,116,105,111, + 110,32,97,98,111,117,116,32,116,104,101,32,109,111,100,117, + 108,101,46,32,32,70,111,114,10,32,32,32,32,100,97,116, + 97,32,97,115,115,111,99,105,97,116,101,100,32,119,105,116, + 104,32,116,104,101,32,109,111,100,117,108,101,44,32,105,110, + 99,108,117,100,105,110,103,32,115,111,117,114,99,101,44,32, + 117,115,101,32,116,104,101,32,115,112,101,99,39,115,10,32, + 32,32,32,108,111,97,100,101,114,46,10,10,32,32,32,32, + 96,110,97,109,101,96,32,105,115,32,116,104,101,32,97,98, + 115,111,108,117,116,101,32,110,97,109,101,32,111,102,32,116, + 104,101,32,109,111,100,117,108,101,46,32,32,96,108,111,97, + 100,101,114,96,32,105,115,32,116,104,101,32,108,111,97,100, + 101,114,10,32,32,32,32,116,111,32,117,115,101,32,119,104, + 101,110,32,108,111,97,100,105,110,103,32,116,104,101,32,109, + 111,100,117,108,101,46,32,32,96,112,97,114,101,110,116,96, + 32,105,115,32,116,104,101,32,110,97,109,101,32,111,102,32, + 116,104,101,10,32,32,32,32,112,97,99,107,97,103,101,32, + 116,104,101,32,109,111,100,117,108,101,32,105,115,32,105,110, + 46,32,32,84,104,101,32,112,97,114,101,110,116,32,105,115, + 32,100,101,114,105,118,101,100,32,102,114,111,109,32,116,104, + 101,32,110,97,109,101,46,10,10,32,32,32,32,96,105,115, + 95,112,97,99,107,97,103,101,96,32,100,101,116,101,114,109, + 105,110,101,115,32,105,102,32,116,104,101,32,109,111,100,117, + 108,101,32,105,115,32,99,111,110,115,105,100,101,114,101,100, + 32,97,32,112,97,99,107,97,103,101,32,111,114,10,32,32, + 32,32,110,111,116,46,32,32,79,110,32,109,111,100,117,108, + 101,115,32,116,104,105,115,32,105,115,32,114,101,102,108,101, + 99,116,101,100,32,98,121,32,116,104,101,32,96,95,95,112, + 97,116,104,95,95,96,32,97,116,116,114,105,98,117,116,101, + 46,10,10,32,32,32,32,96,111,114,105,103,105,110,96,32, + 105,115,32,116,104,101,32,115,112,101,99,105,102,105,99,32, + 108,111,99,97,116,105,111,110,32,117,115,101,100,32,98,121, + 32,116,104,101,32,108,111,97,100,101,114,32,102,114,111,109, + 32,119,104,105,99,104,32,116,111,10,32,32,32,32,108,111, + 97,100,32,116,104,101,32,109,111,100,117,108,101,44,32,105, + 102,32,116,104,97,116,32,105,110,102,111,114,109,97,116,105, + 111,110,32,105,115,32,97,118,97,105,108,97,98,108,101,46, + 32,32,87,104,101,110,32,102,105,108,101,110,97,109,101,32, + 105,115,10,32,32,32,32,115,101,116,44,32,111,114,105,103, + 105,110,32,119,105,108,108,32,109,97,116,99,104,46,10,10, + 32,32,32,32,96,104,97,115,95,108,111,99,97,116,105,111, + 110,96,32,105,110,100,105,99,97,116,101,115,32,116,104,97, + 116,32,97,32,115,112,101,99,39,115,32,34,111,114,105,103, + 105,110,34,32,114,101,102,108,101,99,116,115,32,97,32,108, + 111,99,97,116,105,111,110,46,10,32,32,32,32,87,104,101, + 110,32,116,104,105,115,32,105,115,32,84,114,117,101,44,32, + 96,95,95,102,105,108,101,95,95,96,32,97,116,116,114,105, + 98,117,116,101,32,111,102,32,116,104,101,32,109,111,100,117, + 108,101,32,105,115,32,115,101,116,46,10,10,32,32,32,32, + 96,99,97,99,104,101,100,96,32,105,115,32,116,104,101,32, + 108,111,99,97,116,105,111,110,32,111,102,32,116,104,101,32, + 99,97,99,104,101,100,32,98,121,116,101,99,111,100,101,32, + 102,105,108,101,44,32,105,102,32,97,110,121,46,32,32,73, + 116,10,32,32,32,32,99,111,114,114,101,115,112,111,110,100, + 115,32,116,111,32,116,104,101,32,96,95,95,99,97,99,104, + 101,100,95,95,96,32,97,116,116,114,105,98,117,116,101,46, + 10,10,32,32,32,32,96,115,117,98,109,111,100,117,108,101, + 95,115,101,97,114,99,104,95,108,111,99,97,116,105,111,110, + 115,96,32,105,115,32,116,104,101,32,115,101,113,117,101,110, + 99,101,32,111,102,32,112,97,116,104,32,101,110,116,114,105, + 101,115,32,116,111,10,32,32,32,32,115,101,97,114,99,104, + 32,119,104,101,110,32,105,109,112,111,114,116,105,110,103,32, + 115,117,98,109,111,100,117,108,101,115,46,32,32,73,102,32, + 115,101,116,44,32,105,115,95,112,97,99,107,97,103,101,32, + 115,104,111,117,108,100,32,98,101,10,32,32,32,32,84,114, + 117,101,45,45,97,110,100,32,70,97,108,115,101,32,111,116, + 104,101,114,119,105,115,101,46,10,10,32,32,32,32,80,97, + 99,107,97,103,101,115,32,97,114,101,32,115,105,109,112,108, + 121,32,109,111,100,117,108,101,115,32,116,104,97,116,32,40, + 109,97,121,41,32,104,97,118,101,32,115,117,98,109,111,100, + 117,108,101,115,46,32,32,73,102,32,97,32,115,112,101,99, + 10,32,32,32,32,104,97,115,32,97,32,110,111,110,45,78, + 111,110,101,32,118,97,108,117,101,32,105,110,32,96,115,117, + 98,109,111,100,117,108,101,95,115,101,97,114,99,104,95,108, + 111,99,97,116,105,111,110,115,96,44,32,116,104,101,32,105, + 109,112,111,114,116,10,32,32,32,32,115,121,115,116,101,109, + 32,119,105,108,108,32,99,111,110,115,105,100,101,114,32,109, + 111,100,117,108,101,115,32,108,111,97,100,101,100,32,102,114, + 111,109,32,116,104,101,32,115,112,101,99,32,97,115,32,112, + 97,99,107,97,103,101,115,46,10,10,32,32,32,32,79,110, + 108,121,32,102,105,110,100,101,114,115,32,40,115,101,101,32, + 105,109,112,111,114,116,108,105,98,46,97,98,99,46,77,101, + 116,97,80,97,116,104,70,105,110,100,101,114,32,97,110,100, + 10,32,32,32,32,105,109,112,111,114,116,108,105,98,46,97, + 98,99,46,80,97,116,104,69,110,116,114,121,70,105,110,100, + 101,114,41,32,115,104,111,117,108,100,32,109,111,100,105,102, + 121,32,77,111,100,117,108,101,83,112,101,99,32,105,110,115, + 116,97,110,99,101,115,46,10,10,32,32,32,32,78,41,3, + 218,6,111,114,105,103,105,110,218,12,108,111,97,100,101,114, + 95,115,116,97,116,101,218,10,105,115,95,112,97,99,107,97, + 103,101,99,3,0,0,0,3,0,0,0,6,0,0,0,2, + 0,0,0,67,0,0,0,115,54,0,0,0,124,1,124,0, + 95,0,124,2,124,0,95,1,124,3,124,0,95,2,124,4, + 124,0,95,3,124,5,114,32,103,0,110,2,100,0,124,0, + 95,4,100,1,124,0,95,5,100,0,124,0,95,6,100,0, + 83,0,41,2,78,70,41,7,114,15,0,0,0,114,93,0, + 0,0,114,103,0,0,0,114,104,0,0,0,218,26,115,117, + 98,109,111,100,117,108,101,95,115,101,97,114,99,104,95,108, + 111,99,97,116,105,111,110,115,218,13,95,115,101,116,95,102, + 105,108,101,97,116,116,114,218,7,95,99,97,99,104,101,100, + 41,6,114,26,0,0,0,114,15,0,0,0,114,93,0,0, + 0,114,103,0,0,0,114,104,0,0,0,114,105,0,0,0, 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 96,0,0,0,49,1,0,0,115,6,0,0,0,8,2,8, - 4,8,7,114,96,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,4,0,0,0,64,0,0,0,115,114,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 100,2,100,2,100,3,156,3,100,4,100,5,132,2,90,4, - 100,6,100,7,132,0,90,5,100,8,100,9,132,0,90,6, - 101,7,100,10,100,11,132,0,131,1,90,8,101,8,106,9, - 100,12,100,11,132,0,131,1,90,8,101,7,100,13,100,14, - 132,0,131,1,90,10,101,7,100,15,100,16,132,0,131,1, - 90,11,101,11,106,9,100,17,100,16,132,0,131,1,90,11, - 100,2,83,0,41,18,218,10,77,111,100,117,108,101,83,112, - 101,99,97,208,5,0,0,84,104,101,32,115,112,101,99,105, - 102,105,99,97,116,105,111,110,32,102,111,114,32,97,32,109, - 111,100,117,108,101,44,32,117,115,101,100,32,102,111,114,32, - 108,111,97,100,105,110,103,46,10,10,32,32,32,32,65,32, - 109,111,100,117,108,101,39,115,32,115,112,101,99,32,105,115, - 32,116,104,101,32,115,111,117,114,99,101,32,102,111,114,32, - 105,110,102,111,114,109,97,116,105,111,110,32,97,98,111,117, - 116,32,116,104,101,32,109,111,100,117,108,101,46,32,32,70, - 111,114,10,32,32,32,32,100,97,116,97,32,97,115,115,111, - 99,105,97,116,101,100,32,119,105,116,104,32,116,104,101,32, - 109,111,100,117,108,101,44,32,105,110,99,108,117,100,105,110, - 103,32,115,111,117,114,99,101,44,32,117,115,101,32,116,104, - 101,32,115,112,101,99,39,115,10,32,32,32,32,108,111,97, - 100,101,114,46,10,10,32,32,32,32,96,110,97,109,101,96, - 32,105,115,32,116,104,101,32,97,98,115,111,108,117,116,101, - 32,110,97,109,101,32,111,102,32,116,104,101,32,109,111,100, - 117,108,101,46,32,32,96,108,111,97,100,101,114,96,32,105, - 115,32,116,104,101,32,108,111,97,100,101,114,10,32,32,32, - 32,116,111,32,117,115,101,32,119,104,101,110,32,108,111,97, - 100,105,110,103,32,116,104,101,32,109,111,100,117,108,101,46, - 32,32,96,112,97,114,101,110,116,96,32,105,115,32,116,104, - 101,32,110,97,109,101,32,111,102,32,116,104,101,10,32,32, - 32,32,112,97,99,107,97,103,101,32,116,104,101,32,109,111, - 100,117,108,101,32,105,115,32,105,110,46,32,32,84,104,101, - 32,112,97,114,101,110,116,32,105,115,32,100,101,114,105,118, - 101,100,32,102,114,111,109,32,116,104,101,32,110,97,109,101, - 46,10,10,32,32,32,32,96,105,115,95,112,97,99,107,97, - 103,101,96,32,100,101,116,101,114,109,105,110,101,115,32,105, - 102,32,116,104,101,32,109,111,100,117,108,101,32,105,115,32, - 99,111,110,115,105,100,101,114,101,100,32,97,32,112,97,99, - 107,97,103,101,32,111,114,10,32,32,32,32,110,111,116,46, - 32,32,79,110,32,109,111,100,117,108,101,115,32,116,104,105, - 115,32,105,115,32,114,101,102,108,101,99,116,101,100,32,98, - 121,32,116,104,101,32,96,95,95,112,97,116,104,95,95,96, - 32,97,116,116,114,105,98,117,116,101,46,10,10,32,32,32, - 32,96,111,114,105,103,105,110,96,32,105,115,32,116,104,101, - 32,115,112,101,99,105,102,105,99,32,108,111,99,97,116,105, - 111,110,32,117,115,101,100,32,98,121,32,116,104,101,32,108, - 111,97,100,101,114,32,102,114,111,109,32,119,104,105,99,104, - 32,116,111,10,32,32,32,32,108,111,97,100,32,116,104,101, - 32,109,111,100,117,108,101,44,32,105,102,32,116,104,97,116, - 32,105,110,102,111,114,109,97,116,105,111,110,32,105,115,32, - 97,118,97,105,108,97,98,108,101,46,32,32,87,104,101,110, - 32,102,105,108,101,110,97,109,101,32,105,115,10,32,32,32, - 32,115,101,116,44,32,111,114,105,103,105,110,32,119,105,108, - 108,32,109,97,116,99,104,46,10,10,32,32,32,32,96,104, - 97,115,95,108,111,99,97,116,105,111,110,96,32,105,110,100, - 105,99,97,116,101,115,32,116,104,97,116,32,97,32,115,112, - 101,99,39,115,32,34,111,114,105,103,105,110,34,32,114,101, - 102,108,101,99,116,115,32,97,32,108,111,99,97,116,105,111, - 110,46,10,32,32,32,32,87,104,101,110,32,116,104,105,115, - 32,105,115,32,84,114,117,101,44,32,96,95,95,102,105,108, - 101,95,95,96,32,97,116,116,114,105,98,117,116,101,32,111, - 102,32,116,104,101,32,109,111,100,117,108,101,32,105,115,32, - 115,101,116,46,10,10,32,32,32,32,96,99,97,99,104,101, - 100,96,32,105,115,32,116,104,101,32,108,111,99,97,116,105, - 111,110,32,111,102,32,116,104,101,32,99,97,99,104,101,100, - 32,98,121,116,101,99,111,100,101,32,102,105,108,101,44,32, - 105,102,32,97,110,121,46,32,32,73,116,10,32,32,32,32, - 99,111,114,114,101,115,112,111,110,100,115,32,116,111,32,116, - 104,101,32,96,95,95,99,97,99,104,101,100,95,95,96,32, - 97,116,116,114,105,98,117,116,101,46,10,10,32,32,32,32, - 96,115,117,98,109,111,100,117,108,101,95,115,101,97,114,99, - 104,95,108,111,99,97,116,105,111,110,115,96,32,105,115,32, - 116,104,101,32,115,101,113,117,101,110,99,101,32,111,102,32, - 112,97,116,104,32,101,110,116,114,105,101,115,32,116,111,10, - 32,32,32,32,115,101,97,114,99,104,32,119,104,101,110,32, - 105,109,112,111,114,116,105,110,103,32,115,117,98,109,111,100, - 117,108,101,115,46,32,32,73,102,32,115,101,116,44,32,105, - 115,95,112,97,99,107,97,103,101,32,115,104,111,117,108,100, - 32,98,101,10,32,32,32,32,84,114,117,101,45,45,97,110, - 100,32,70,97,108,115,101,32,111,116,104,101,114,119,105,115, - 101,46,10,10,32,32,32,32,80,97,99,107,97,103,101,115, - 32,97,114,101,32,115,105,109,112,108,121,32,109,111,100,117, - 108,101,115,32,116,104,97,116,32,40,109,97,121,41,32,104, - 97,118,101,32,115,117,98,109,111,100,117,108,101,115,46,32, - 32,73,102,32,97,32,115,112,101,99,10,32,32,32,32,104, - 97,115,32,97,32,110,111,110,45,78,111,110,101,32,118,97, - 108,117,101,32,105,110,32,96,115,117,98,109,111,100,117,108, - 101,95,115,101,97,114,99,104,95,108,111,99,97,116,105,111, - 110,115,96,44,32,116,104,101,32,105,109,112,111,114,116,10, - 32,32,32,32,115,121,115,116,101,109,32,119,105,108,108,32, - 99,111,110,115,105,100,101,114,32,109,111,100,117,108,101,115, - 32,108,111,97,100,101,100,32,102,114,111,109,32,116,104,101, - 32,115,112,101,99,32,97,115,32,112,97,99,107,97,103,101, - 115,46,10,10,32,32,32,32,79,110,108,121,32,102,105,110, - 100,101,114,115,32,40,115,101,101,32,105,109,112,111,114,116, - 108,105,98,46,97,98,99,46,77,101,116,97,80,97,116,104, - 70,105,110,100,101,114,32,97,110,100,10,32,32,32,32,105, - 109,112,111,114,116,108,105,98,46,97,98,99,46,80,97,116, - 104,69,110,116,114,121,70,105,110,100,101,114,41,32,115,104, - 111,117,108,100,32,109,111,100,105,102,121,32,77,111,100,117, - 108,101,83,112,101,99,32,105,110,115,116,97,110,99,101,115, - 46,10,10,32,32,32,32,78,41,3,218,6,111,114,105,103, - 105,110,218,12,108,111,97,100,101,114,95,115,116,97,116,101, - 218,10,105,115,95,112,97,99,107,97,103,101,99,3,0,0, - 0,3,0,0,0,6,0,0,0,2,0,0,0,67,0,0, - 0,115,54,0,0,0,124,1,124,0,95,0,124,2,124,0, - 95,1,124,3,124,0,95,2,124,4,124,0,95,3,124,5, - 114,32,103,0,110,2,100,0,124,0,95,4,100,1,124,0, - 95,5,100,0,124,0,95,6,100,0,83,0,41,2,78,70, - 41,7,114,15,0,0,0,114,93,0,0,0,114,103,0,0, - 0,114,104,0,0,0,218,26,115,117,98,109,111,100,117,108, - 101,95,115,101,97,114,99,104,95,108,111,99,97,116,105,111, - 110,115,218,13,95,115,101,116,95,102,105,108,101,97,116,116, - 114,218,7,95,99,97,99,104,101,100,41,6,114,26,0,0, - 0,114,15,0,0,0,114,93,0,0,0,114,103,0,0,0, - 114,104,0,0,0,114,105,0,0,0,114,10,0,0,0,114, - 10,0,0,0,114,11,0,0,0,114,27,0,0,0,113,1, - 0,0,115,14,0,0,0,0,2,6,1,6,1,6,1,6, - 1,14,3,6,1,122,19,77,111,100,117,108,101,83,112,101, - 99,46,95,95,105,110,105,116,95,95,99,1,0,0,0,0, - 0,0,0,2,0,0,0,6,0,0,0,67,0,0,0,115, - 102,0,0,0,100,1,160,0,124,0,106,1,161,1,100,2, - 160,0,124,0,106,2,161,1,103,2,125,1,124,0,106,3, - 100,0,107,9,114,52,124,1,160,4,100,3,160,0,124,0, - 106,3,161,1,161,1,1,0,124,0,106,5,100,0,107,9, - 114,80,124,1,160,4,100,4,160,0,124,0,106,5,161,1, - 161,1,1,0,100,5,160,0,124,0,106,6,106,7,100,6, - 160,8,124,1,161,1,161,2,83,0,41,7,78,122,9,110, - 97,109,101,61,123,33,114,125,122,11,108,111,97,100,101,114, - 61,123,33,114,125,122,11,111,114,105,103,105,110,61,123,33, - 114,125,122,29,115,117,98,109,111,100,117,108,101,95,115,101, - 97,114,99,104,95,108,111,99,97,116,105,111,110,115,61,123, - 125,122,6,123,125,40,123,125,41,122,2,44,32,41,9,114, - 38,0,0,0,114,15,0,0,0,114,93,0,0,0,114,103, - 0,0,0,218,6,97,112,112,101,110,100,114,106,0,0,0, - 218,9,95,95,99,108,97,115,115,95,95,114,1,0,0,0, - 218,4,106,111,105,110,41,2,114,26,0,0,0,114,47,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,40,0,0,0,125,1,0,0,115,16,0,0,0,0, - 1,10,1,14,1,10,1,18,1,10,1,8,1,10,1,122, - 19,77,111,100,117,108,101,83,112,101,99,46,95,95,114,101, - 112,114,95,95,99,2,0,0,0,0,0,0,0,3,0,0, - 0,8,0,0,0,67,0,0,0,115,106,0,0,0,124,0, - 106,0,125,2,122,72,124,0,106,1,124,1,106,1,107,2, - 111,76,124,0,106,2,124,1,106,2,107,2,111,76,124,0, - 106,3,124,1,106,3,107,2,111,76,124,2,124,1,106,0, - 107,2,111,76,124,0,106,4,124,1,106,4,107,2,111,76, - 124,0,106,5,124,1,106,5,107,2,87,0,83,0,4,0, - 116,6,107,10,114,100,1,0,1,0,1,0,89,0,100,1, - 83,0,88,0,100,0,83,0,41,2,78,70,41,7,114,106, - 0,0,0,114,15,0,0,0,114,93,0,0,0,114,103,0, - 0,0,218,6,99,97,99,104,101,100,218,12,104,97,115,95, - 108,111,99,97,116,105,111,110,114,90,0,0,0,41,3,114, - 26,0,0,0,90,5,111,116,104,101,114,90,4,115,109,115, - 108,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 218,6,95,95,101,113,95,95,135,1,0,0,115,20,0,0, - 0,0,1,6,1,2,1,12,1,12,1,12,1,10,1,12, - 1,14,1,14,1,122,17,77,111,100,117,108,101,83,112,101, - 99,46,95,95,101,113,95,95,99,1,0,0,0,0,0,0, - 0,1,0,0,0,3,0,0,0,67,0,0,0,115,58,0, - 0,0,124,0,106,0,100,0,107,8,114,52,124,0,106,1, - 100,0,107,9,114,52,124,0,106,2,114,52,116,3,100,0, - 107,8,114,38,116,4,130,1,116,3,160,5,124,0,106,1, - 161,1,124,0,95,0,124,0,106,0,83,0,41,1,78,41, - 6,114,108,0,0,0,114,103,0,0,0,114,107,0,0,0, - 218,19,95,98,111,111,116,115,116,114,97,112,95,101,120,116, - 101,114,110,97,108,218,19,78,111,116,73,109,112,108,101,109, - 101,110,116,101,100,69,114,114,111,114,90,11,95,103,101,116, - 95,99,97,99,104,101,100,41,1,114,26,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,112,0, - 0,0,147,1,0,0,115,12,0,0,0,0,2,10,1,16, - 1,8,1,4,1,14,1,122,17,77,111,100,117,108,101,83, - 112,101,99,46,99,97,99,104,101,100,99,2,0,0,0,0, - 0,0,0,2,0,0,0,2,0,0,0,67,0,0,0,115, - 10,0,0,0,124,1,124,0,95,0,100,0,83,0,41,1, - 78,41,1,114,108,0,0,0,41,2,114,26,0,0,0,114, - 112,0,0,0,114,10,0,0,0,114,10,0,0,0,114,11, - 0,0,0,114,112,0,0,0,156,1,0,0,115,2,0,0, - 0,0,2,99,1,0,0,0,0,0,0,0,1,0,0,0, - 3,0,0,0,67,0,0,0,115,36,0,0,0,124,0,106, - 0,100,1,107,8,114,26,124,0,106,1,160,2,100,2,161, - 1,100,3,25,0,83,0,124,0,106,1,83,0,100,1,83, - 0,41,4,122,32,84,104,101,32,110,97,109,101,32,111,102, - 32,116,104,101,32,109,111,100,117,108,101,39,115,32,112,97, - 114,101,110,116,46,78,218,1,46,114,19,0,0,0,41,3, - 114,106,0,0,0,114,15,0,0,0,218,10,114,112,97,114, - 116,105,116,105,111,110,41,1,114,26,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,218,6,112,97, - 114,101,110,116,160,1,0,0,115,6,0,0,0,0,3,10, - 1,16,2,122,17,77,111,100,117,108,101,83,112,101,99,46, - 112,97,114,101,110,116,99,1,0,0,0,0,0,0,0,1, - 0,0,0,1,0,0,0,67,0,0,0,115,6,0,0,0, - 124,0,106,0,83,0,41,1,78,41,1,114,107,0,0,0, - 41,1,114,26,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,114,113,0,0,0,168,1,0,0,115, - 2,0,0,0,0,2,122,23,77,111,100,117,108,101,83,112, - 101,99,46,104,97,115,95,108,111,99,97,116,105,111,110,99, - 2,0,0,0,0,0,0,0,2,0,0,0,2,0,0,0, - 67,0,0,0,115,14,0,0,0,116,0,124,1,131,1,124, - 0,95,1,100,0,83,0,41,1,78,41,2,218,4,98,111, - 111,108,114,107,0,0,0,41,2,114,26,0,0,0,218,5, - 118,97,108,117,101,114,10,0,0,0,114,10,0,0,0,114, - 11,0,0,0,114,113,0,0,0,172,1,0,0,115,2,0, - 0,0,0,2,41,12,114,1,0,0,0,114,0,0,0,0, - 114,2,0,0,0,114,3,0,0,0,114,27,0,0,0,114, - 40,0,0,0,114,114,0,0,0,218,8,112,114,111,112,101, - 114,116,121,114,112,0,0,0,218,6,115,101,116,116,101,114, - 114,119,0,0,0,114,113,0,0,0,114,10,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,102, - 0,0,0,76,1,0,0,115,20,0,0,0,8,35,4,2, - 4,1,14,11,8,10,8,12,12,9,14,4,12,8,12,4, - 114,102,0,0,0,41,2,114,103,0,0,0,114,105,0,0, - 0,99,2,0,0,0,2,0,0,0,6,0,0,0,8,0, - 0,0,67,0,0,0,115,154,0,0,0,116,0,124,1,100, - 1,131,2,114,74,116,1,100,2,107,8,114,22,116,2,130, - 1,116,1,106,3,125,4,124,3,100,2,107,8,114,48,124, - 4,124,0,124,1,100,3,141,2,83,0,124,3,114,56,103, - 0,110,2,100,2,125,5,124,4,124,0,124,1,124,5,100, - 4,141,3,83,0,124,3,100,2,107,8,114,138,116,0,124, - 1,100,5,131,2,114,134,122,14,124,1,160,4,124,0,161, - 1,125,3,87,0,113,138,4,0,116,5,107,10,114,130,1, - 0,1,0,1,0,100,2,125,3,89,0,113,138,88,0,110, - 4,100,6,125,3,116,6,124,0,124,1,124,2,124,3,100, - 7,141,4,83,0,41,8,122,53,82,101,116,117,114,110,32, - 97,32,109,111,100,117,108,101,32,115,112,101,99,32,98,97, - 115,101,100,32,111,110,32,118,97,114,105,111,117,115,32,108, - 111,97,100,101,114,32,109,101,116,104,111,100,115,46,90,12, - 103,101,116,95,102,105,108,101,110,97,109,101,78,41,1,114, - 93,0,0,0,41,2,114,93,0,0,0,114,106,0,0,0, - 114,105,0,0,0,70,41,2,114,103,0,0,0,114,105,0, - 0,0,41,7,114,4,0,0,0,114,115,0,0,0,114,116, - 0,0,0,218,23,115,112,101,99,95,102,114,111,109,95,102, - 105,108,101,95,108,111,99,97,116,105,111,110,114,105,0,0, - 0,114,70,0,0,0,114,102,0,0,0,41,6,114,15,0, - 0,0,114,93,0,0,0,114,103,0,0,0,114,105,0,0, - 0,114,124,0,0,0,90,6,115,101,97,114,99,104,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,78,0, - 0,0,177,1,0,0,115,34,0,0,0,0,2,10,1,8, - 1,4,1,6,2,8,1,12,1,12,1,6,1,8,2,8, + 27,0,0,0,113,1,0,0,115,14,0,0,0,0,2,6, + 1,6,1,6,1,6,1,14,3,6,1,122,19,77,111,100, + 117,108,101,83,112,101,99,46,95,95,105,110,105,116,95,95, + 99,1,0,0,0,0,0,0,0,2,0,0,0,6,0,0, + 0,67,0,0,0,115,102,0,0,0,100,1,160,0,124,0, + 106,1,161,1,100,2,160,0,124,0,106,2,161,1,103,2, + 125,1,124,0,106,3,100,0,107,9,114,52,124,1,160,4, + 100,3,160,0,124,0,106,3,161,1,161,1,1,0,124,0, + 106,5,100,0,107,9,114,80,124,1,160,4,100,4,160,0, + 124,0,106,5,161,1,161,1,1,0,100,5,160,0,124,0, + 106,6,106,7,100,6,160,8,124,1,161,1,161,2,83,0, + 41,7,78,122,9,110,97,109,101,61,123,33,114,125,122,11, + 108,111,97,100,101,114,61,123,33,114,125,122,11,111,114,105, + 103,105,110,61,123,33,114,125,122,29,115,117,98,109,111,100, + 117,108,101,95,115,101,97,114,99,104,95,108,111,99,97,116, + 105,111,110,115,61,123,125,122,6,123,125,40,123,125,41,122, + 2,44,32,41,9,114,38,0,0,0,114,15,0,0,0,114, + 93,0,0,0,114,103,0,0,0,218,6,97,112,112,101,110, + 100,114,106,0,0,0,218,9,95,95,99,108,97,115,115,95, + 95,114,1,0,0,0,218,4,106,111,105,110,41,2,114,26, + 0,0,0,114,47,0,0,0,114,10,0,0,0,114,10,0, + 0,0,114,11,0,0,0,114,40,0,0,0,125,1,0,0, + 115,20,0,0,0,0,1,10,1,10,255,4,2,10,1,18, + 1,10,1,8,1,4,255,6,2,122,19,77,111,100,117,108, + 101,83,112,101,99,46,95,95,114,101,112,114,95,95,99,2, + 0,0,0,0,0,0,0,3,0,0,0,8,0,0,0,67, + 0,0,0,115,114,0,0,0,124,0,106,0,125,2,122,76, + 124,0,106,1,124,1,106,1,107,2,111,76,124,0,106,2, + 124,1,106,2,107,2,111,76,124,0,106,3,124,1,106,3, + 107,2,111,76,124,2,124,1,106,0,107,2,111,76,124,0, + 106,4,124,1,106,4,107,2,111,76,124,0,106,5,124,1, + 106,5,107,2,87,0,83,0,87,0,110,26,4,0,116,6, + 107,10,114,108,1,0,1,0,1,0,89,0,100,1,83,0, + 89,0,110,2,88,0,100,0,83,0,41,2,78,70,41,7, + 114,106,0,0,0,114,15,0,0,0,114,93,0,0,0,114, + 103,0,0,0,218,6,99,97,99,104,101,100,218,12,104,97, + 115,95,108,111,99,97,116,105,111,110,114,90,0,0,0,41, + 3,114,26,0,0,0,90,5,111,116,104,101,114,90,4,115, + 109,115,108,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,218,6,95,95,101,113,95,95,135,1,0,0,115,30, + 0,0,0,0,1,6,1,2,1,12,1,10,255,2,2,10, + 254,2,3,8,253,2,4,10,252,2,5,10,251,8,6,14, + 1,122,17,77,111,100,117,108,101,83,112,101,99,46,95,95, + 101,113,95,95,99,1,0,0,0,0,0,0,0,1,0,0, + 0,3,0,0,0,67,0,0,0,115,58,0,0,0,124,0, + 106,0,100,0,107,8,114,52,124,0,106,1,100,0,107,9, + 114,52,124,0,106,2,114,52,116,3,100,0,107,8,114,38, + 116,4,130,1,116,3,160,5,124,0,106,1,161,1,124,0, + 95,0,124,0,106,0,83,0,41,1,78,41,6,114,108,0, + 0,0,114,103,0,0,0,114,107,0,0,0,218,19,95,98, + 111,111,116,115,116,114,97,112,95,101,120,116,101,114,110,97, + 108,218,19,78,111,116,73,109,112,108,101,109,101,110,116,101, + 100,69,114,114,111,114,90,11,95,103,101,116,95,99,97,99, + 104,101,100,41,1,114,26,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,114,112,0,0,0,147,1, + 0,0,115,12,0,0,0,0,2,10,1,16,1,8,1,4, + 1,14,1,122,17,77,111,100,117,108,101,83,112,101,99,46, + 99,97,99,104,101,100,99,2,0,0,0,0,0,0,0,2, + 0,0,0,2,0,0,0,67,0,0,0,115,10,0,0,0, + 124,1,124,0,95,0,100,0,83,0,41,1,78,41,1,114, + 108,0,0,0,41,2,114,26,0,0,0,114,112,0,0,0, + 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, + 112,0,0,0,156,1,0,0,115,2,0,0,0,0,2,99, + 1,0,0,0,0,0,0,0,1,0,0,0,3,0,0,0, + 67,0,0,0,115,36,0,0,0,124,0,106,0,100,1,107, + 8,114,26,124,0,106,1,160,2,100,2,161,1,100,3,25, + 0,83,0,124,0,106,1,83,0,100,1,83,0,41,4,122, + 32,84,104,101,32,110,97,109,101,32,111,102,32,116,104,101, + 32,109,111,100,117,108,101,39,115,32,112,97,114,101,110,116, + 46,78,218,1,46,114,19,0,0,0,41,3,114,106,0,0, + 0,114,15,0,0,0,218,10,114,112,97,114,116,105,116,105, + 111,110,41,1,114,26,0,0,0,114,10,0,0,0,114,10, + 0,0,0,114,11,0,0,0,218,6,112,97,114,101,110,116, + 160,1,0,0,115,6,0,0,0,0,3,10,1,16,2,122, + 17,77,111,100,117,108,101,83,112,101,99,46,112,97,114,101, + 110,116,99,1,0,0,0,0,0,0,0,1,0,0,0,1, + 0,0,0,67,0,0,0,115,6,0,0,0,124,0,106,0, + 83,0,41,1,78,41,1,114,107,0,0,0,41,1,114,26, + 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,114,113,0,0,0,168,1,0,0,115,2,0,0,0, + 0,2,122,23,77,111,100,117,108,101,83,112,101,99,46,104, + 97,115,95,108,111,99,97,116,105,111,110,99,2,0,0,0, + 0,0,0,0,2,0,0,0,2,0,0,0,67,0,0,0, + 115,14,0,0,0,116,0,124,1,131,1,124,0,95,1,100, + 0,83,0,41,1,78,41,2,218,4,98,111,111,108,114,107, + 0,0,0,41,2,114,26,0,0,0,218,5,118,97,108,117, + 101,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, + 114,113,0,0,0,172,1,0,0,115,2,0,0,0,0,2, + 41,12,114,1,0,0,0,114,0,0,0,0,114,2,0,0, + 0,114,3,0,0,0,114,27,0,0,0,114,40,0,0,0, + 114,114,0,0,0,218,8,112,114,111,112,101,114,116,121,114, + 112,0,0,0,218,6,115,101,116,116,101,114,114,119,0,0, + 0,114,113,0,0,0,114,10,0,0,0,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,114,102,0,0,0,76, + 1,0,0,115,22,0,0,0,8,35,4,2,4,1,2,255, + 12,12,8,10,8,12,12,9,14,4,12,8,12,4,114,102, + 0,0,0,41,2,114,103,0,0,0,114,105,0,0,0,99, + 2,0,0,0,2,0,0,0,6,0,0,0,8,0,0,0, + 67,0,0,0,115,154,0,0,0,116,0,124,1,100,1,131, + 2,114,74,116,1,100,2,107,8,114,22,116,2,130,1,116, + 1,106,3,125,4,124,3,100,2,107,8,114,48,124,4,124, + 0,124,1,100,3,141,2,83,0,124,3,114,56,103,0,110, + 2,100,2,125,5,124,4,124,0,124,1,124,5,100,4,141, + 3,83,0,124,3,100,2,107,8,114,138,116,0,124,1,100, + 5,131,2,114,134,122,14,124,1,160,4,124,0,161,1,125, + 3,87,0,110,24,4,0,116,5,107,10,114,130,1,0,1, + 0,1,0,100,2,125,3,89,0,110,2,88,0,110,4,100, + 6,125,3,116,6,124,0,124,1,124,2,124,3,100,7,141, + 4,83,0,41,8,122,53,82,101,116,117,114,110,32,97,32, + 109,111,100,117,108,101,32,115,112,101,99,32,98,97,115,101, + 100,32,111,110,32,118,97,114,105,111,117,115,32,108,111,97, + 100,101,114,32,109,101,116,104,111,100,115,46,90,12,103,101, + 116,95,102,105,108,101,110,97,109,101,78,41,1,114,93,0, + 0,0,41,2,114,93,0,0,0,114,106,0,0,0,114,105, + 0,0,0,70,41,2,114,103,0,0,0,114,105,0,0,0, + 41,7,114,4,0,0,0,114,115,0,0,0,114,116,0,0, + 0,218,23,115,112,101,99,95,102,114,111,109,95,102,105,108, + 101,95,108,111,99,97,116,105,111,110,114,105,0,0,0,114, + 70,0,0,0,114,102,0,0,0,41,6,114,15,0,0,0, + 114,93,0,0,0,114,103,0,0,0,114,105,0,0,0,114, + 124,0,0,0,90,6,115,101,97,114,99,104,114,10,0,0, + 0,114,10,0,0,0,114,11,0,0,0,114,78,0,0,0, + 177,1,0,0,115,36,0,0,0,0,2,10,1,8,1,4, + 1,6,2,8,1,12,1,12,1,6,1,2,255,6,3,8, 1,10,1,2,1,14,1,14,1,12,3,4,2,114,78,0, 0,0,99,3,0,0,0,0,0,0,0,8,0,0,0,8, 0,0,0,67,0,0,0,115,56,1,0,0,122,10,124,0, @@ -1059,19 +1061,162 @@ const unsigned char _Py_M__importlib[] = { 0,0,0,114,11,0,0,0,114,81,0,0,0,174,2,0, 0,115,4,0,0,0,0,9,12,1,114,81,0,0,0,99, 0,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0, - 64,0,0,0,115,136,0,0,0,101,0,90,1,100,0,90, + 64,0,0,0,115,142,0,0,0,101,0,90,1,100,0,90, 2,100,1,90,3,101,4,100,2,100,3,132,0,131,1,90, - 5,101,6,100,19,100,5,100,6,132,1,131,1,90,7,101, - 6,100,20,100,7,100,8,132,1,131,1,90,8,101,6,100, - 9,100,10,132,0,131,1,90,9,101,6,100,11,100,12,132, - 0,131,1,90,10,101,6,101,11,100,13,100,14,132,0,131, - 1,131,1,90,12,101,6,101,11,100,15,100,16,132,0,131, - 1,131,1,90,13,101,6,101,11,100,17,100,18,132,0,131, - 1,131,1,90,14,101,6,101,15,131,1,90,16,100,4,83, - 0,41,21,218,15,66,117,105,108,116,105,110,73,109,112,111, - 114,116,101,114,122,144,77,101,116,97,32,112,97,116,104,32, - 105,109,112,111,114,116,32,102,111,114,32,98,117,105,108,116, - 45,105,110,32,109,111,100,117,108,101,115,46,10,10,32,32, + 5,101,6,100,4,100,4,102,2,100,5,100,6,132,1,131, + 1,90,7,101,6,100,4,102,1,100,7,100,8,132,1,131, + 1,90,8,101,6,100,9,100,10,132,0,131,1,90,9,101, + 6,100,11,100,12,132,0,131,1,90,10,101,6,101,11,100, + 13,100,14,132,0,131,1,131,1,90,12,101,6,101,11,100, + 15,100,16,132,0,131,1,131,1,90,13,101,6,101,11,100, + 17,100,18,132,0,131,1,131,1,90,14,101,6,101,15,131, + 1,90,16,100,4,83,0,41,19,218,15,66,117,105,108,116, + 105,110,73,109,112,111,114,116,101,114,122,144,77,101,116,97, + 32,112,97,116,104,32,105,109,112,111,114,116,32,102,111,114, + 32,98,117,105,108,116,45,105,110,32,109,111,100,117,108,101, + 115,46,10,10,32,32,32,32,65,108,108,32,109,101,116,104, + 111,100,115,32,97,114,101,32,101,105,116,104,101,114,32,99, + 108,97,115,115,32,111,114,32,115,116,97,116,105,99,32,109, + 101,116,104,111,100,115,32,116,111,32,97,118,111,105,100,32, + 116,104,101,32,110,101,101,100,32,116,111,10,32,32,32,32, + 105,110,115,116,97,110,116,105,97,116,101,32,116,104,101,32, + 99,108,97,115,115,46,10,10,32,32,32,32,99,1,0,0, + 0,0,0,0,0,1,0,0,0,3,0,0,0,67,0,0, + 0,115,12,0,0,0,100,1,160,0,124,0,106,1,161,1, + 83,0,41,2,122,115,82,101,116,117,114,110,32,114,101,112, + 114,32,102,111,114,32,116,104,101,32,109,111,100,117,108,101, + 46,10,10,32,32,32,32,32,32,32,32,84,104,101,32,109, + 101,116,104,111,100,32,105,115,32,100,101,112,114,101,99,97, + 116,101,100,46,32,32,84,104,101,32,105,109,112,111,114,116, + 32,109,97,99,104,105,110,101,114,121,32,100,111,101,115,32, + 116,104,101,32,106,111,98,32,105,116,115,101,108,102,46,10, + 10,32,32,32,32,32,32,32,32,122,24,60,109,111,100,117, + 108,101,32,123,33,114,125,32,40,98,117,105,108,116,45,105, + 110,41,62,41,2,114,38,0,0,0,114,1,0,0,0,41, + 1,114,83,0,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,114,86,0,0,0,198,2,0,0,115,2, + 0,0,0,0,7,122,27,66,117,105,108,116,105,110,73,109, + 112,111,114,116,101,114,46,109,111,100,117,108,101,95,114,101, + 112,114,78,99,4,0,0,0,0,0,0,0,4,0,0,0, + 5,0,0,0,67,0,0,0,115,44,0,0,0,124,2,100, + 0,107,9,114,12,100,0,83,0,116,0,160,1,124,1,161, + 1,114,36,116,2,124,1,124,0,100,1,100,2,141,3,83, + 0,100,0,83,0,100,0,83,0,41,3,78,122,8,98,117, + 105,108,116,45,105,110,41,1,114,103,0,0,0,41,3,114, + 49,0,0,0,90,10,105,115,95,98,117,105,108,116,105,110, + 114,78,0,0,0,41,4,218,3,99,108,115,114,71,0,0, + 0,218,4,112,97,116,104,218,6,116,97,114,103,101,116,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,9, + 102,105,110,100,95,115,112,101,99,207,2,0,0,115,10,0, + 0,0,0,2,8,1,4,1,10,1,14,2,122,25,66,117, + 105,108,116,105,110,73,109,112,111,114,116,101,114,46,102,105, + 110,100,95,115,112,101,99,99,3,0,0,0,0,0,0,0, + 4,0,0,0,4,0,0,0,67,0,0,0,115,30,0,0, + 0,124,0,160,0,124,1,124,2,161,2,125,3,124,3,100, + 1,107,9,114,26,124,3,106,1,83,0,100,1,83,0,41, + 2,122,175,70,105,110,100,32,116,104,101,32,98,117,105,108, + 116,45,105,110,32,109,111,100,117,108,101,46,10,10,32,32, + 32,32,32,32,32,32,73,102,32,39,112,97,116,104,39,32, + 105,115,32,101,118,101,114,32,115,112,101,99,105,102,105,101, + 100,32,116,104,101,110,32,116,104,101,32,115,101,97,114,99, + 104,32,105,115,32,99,111,110,115,105,100,101,114,101,100,32, + 97,32,102,97,105,108,117,114,101,46,10,10,32,32,32,32, + 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, + 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, + 85,115,101,32,102,105,110,100,95,115,112,101,99,40,41,32, + 105,110,115,116,101,97,100,46,10,10,32,32,32,32,32,32, + 32,32,78,41,2,114,145,0,0,0,114,93,0,0,0,41, + 4,114,142,0,0,0,114,71,0,0,0,114,143,0,0,0, + 114,82,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,218,11,102,105,110,100,95,109,111,100,117,108, + 101,216,2,0,0,115,4,0,0,0,0,9,12,1,122,27, + 66,117,105,108,116,105,110,73,109,112,111,114,116,101,114,46, + 102,105,110,100,95,109,111,100,117,108,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,4,0,0,0,67,0,0,0, + 115,46,0,0,0,124,1,106,0,116,1,106,2,107,7,114, + 34,116,3,100,1,160,4,124,1,106,0,161,1,124,1,106, + 0,100,2,141,2,130,1,116,5,116,6,106,7,124,1,131, + 2,83,0,41,3,122,24,67,114,101,97,116,101,32,97,32, + 98,117,105,108,116,45,105,110,32,109,111,100,117,108,101,122, + 29,123,33,114,125,32,105,115,32,110,111,116,32,97,32,98, + 117,105,108,116,45,105,110,32,109,111,100,117,108,101,41,1, + 114,15,0,0,0,41,8,114,15,0,0,0,114,14,0,0, + 0,114,69,0,0,0,114,70,0,0,0,114,38,0,0,0, + 114,59,0,0,0,114,49,0,0,0,90,14,99,114,101,97, + 116,101,95,98,117,105,108,116,105,110,41,2,114,26,0,0, + 0,114,82,0,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,114,134,0,0,0,228,2,0,0,115,10, + 0,0,0,0,3,12,1,12,1,4,255,6,2,122,29,66, + 117,105,108,116,105,110,73,109,112,111,114,116,101,114,46,99, + 114,101,97,116,101,95,109,111,100,117,108,101,99,2,0,0, + 0,0,0,0,0,2,0,0,0,3,0,0,0,67,0,0, + 0,115,16,0,0,0,116,0,116,1,106,2,124,1,131,2, + 1,0,100,1,83,0,41,2,122,22,69,120,101,99,32,97, + 32,98,117,105,108,116,45,105,110,32,109,111,100,117,108,101, + 78,41,3,114,59,0,0,0,114,49,0,0,0,90,12,101, + 120,101,99,95,98,117,105,108,116,105,110,41,2,114,26,0, + 0,0,114,83,0,0,0,114,10,0,0,0,114,10,0,0, + 0,114,11,0,0,0,114,135,0,0,0,236,2,0,0,115, + 2,0,0,0,0,3,122,27,66,117,105,108,116,105,110,73, + 109,112,111,114,116,101,114,46,101,120,101,99,95,109,111,100, + 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, + 0,41,2,122,57,82,101,116,117,114,110,32,78,111,110,101, + 32,97,115,32,98,117,105,108,116,45,105,110,32,109,111,100, + 117,108,101,115,32,100,111,32,110,111,116,32,104,97,118,101, + 32,99,111,100,101,32,111,98,106,101,99,116,115,46,78,114, + 10,0,0,0,41,2,114,142,0,0,0,114,71,0,0,0, + 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, + 8,103,101,116,95,99,111,100,101,241,2,0,0,115,2,0, + 0,0,0,4,122,24,66,117,105,108,116,105,110,73,109,112, + 111,114,116,101,114,46,103,101,116,95,99,111,100,101,99,2, + 0,0,0,0,0,0,0,2,0,0,0,1,0,0,0,67, + 0,0,0,115,4,0,0,0,100,1,83,0,41,2,122,56, + 82,101,116,117,114,110,32,78,111,110,101,32,97,115,32,98, + 117,105,108,116,45,105,110,32,109,111,100,117,108,101,115,32, + 100,111,32,110,111,116,32,104,97,118,101,32,115,111,117,114, + 99,101,32,99,111,100,101,46,78,114,10,0,0,0,41,2, + 114,142,0,0,0,114,71,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,218,10,103,101,116,95,115, + 111,117,114,99,101,247,2,0,0,115,2,0,0,0,0,4, + 122,26,66,117,105,108,116,105,110,73,109,112,111,114,116,101, + 114,46,103,101,116,95,115,111,117,114,99,101,99,2,0,0, + 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, + 0,115,4,0,0,0,100,1,83,0,41,2,122,52,82,101, + 116,117,114,110,32,70,97,108,115,101,32,97,115,32,98,117, + 105,108,116,45,105,110,32,109,111,100,117,108,101,115,32,97, + 114,101,32,110,101,118,101,114,32,112,97,99,107,97,103,101, + 115,46,70,114,10,0,0,0,41,2,114,142,0,0,0,114, + 71,0,0,0,114,10,0,0,0,114,10,0,0,0,114,11, + 0,0,0,114,105,0,0,0,253,2,0,0,115,2,0,0, + 0,0,4,122,26,66,117,105,108,116,105,110,73,109,112,111, + 114,116,101,114,46,105,115,95,112,97,99,107,97,103,101,41, + 17,114,1,0,0,0,114,0,0,0,0,114,2,0,0,0, + 114,3,0,0,0,218,12,115,116,97,116,105,99,109,101,116, + 104,111,100,114,86,0,0,0,218,11,99,108,97,115,115,109, + 101,116,104,111,100,114,145,0,0,0,114,146,0,0,0,114, + 134,0,0,0,114,135,0,0,0,114,74,0,0,0,114,147, + 0,0,0,114,148,0,0,0,114,105,0,0,0,114,84,0, + 0,0,114,137,0,0,0,114,10,0,0,0,114,10,0,0, + 0,114,10,0,0,0,114,11,0,0,0,114,141,0,0,0, + 189,2,0,0,115,42,0,0,0,8,7,4,2,12,9,2, + 1,2,0,2,255,12,9,2,1,2,255,12,12,12,8,12, + 5,2,1,2,255,12,6,2,1,2,255,12,6,2,1,2, + 255,12,6,114,141,0,0,0,99,0,0,0,0,0,0,0, + 0,0,0,0,0,4,0,0,0,64,0,0,0,115,146,0, + 0,0,101,0,90,1,100,0,90,2,100,1,90,3,101,4, + 100,2,100,3,132,0,131,1,90,5,101,6,100,4,100,4, + 102,2,100,5,100,6,132,1,131,1,90,7,101,6,100,4, + 102,1,100,7,100,8,132,1,131,1,90,8,101,6,100,9, + 100,10,132,0,131,1,90,9,101,4,100,11,100,12,132,0, + 131,1,90,10,101,6,100,13,100,14,132,0,131,1,90,11, + 101,6,101,12,100,15,100,16,132,0,131,1,131,1,90,13, + 101,6,101,12,100,17,100,18,132,0,131,1,131,1,90,14, + 101,6,101,12,100,19,100,20,132,0,131,1,131,1,90,15, + 100,4,83,0,41,21,218,14,70,114,111,122,101,110,73,109, + 112,111,114,116,101,114,122,142,77,101,116,97,32,112,97,116, + 104,32,105,109,112,111,114,116,32,102,111,114,32,102,114,111, + 122,101,110,32,109,111,100,117,108,101,115,46,10,10,32,32, 32,32,65,108,108,32,109,101,116,104,111,100,115,32,97,114, 101,32,101,105,116,104,101,114,32,99,108,97,115,115,32,111, 114,32,115,116,97,116,105,99,32,109,101,116,104,111,100,115, @@ -1088,535 +1233,394 @@ const unsigned char _Py_M__importlib[] = { 84,104,101,32,105,109,112,111,114,116,32,109,97,99,104,105, 110,101,114,121,32,100,111,101,115,32,116,104,101,32,106,111, 98,32,105,116,115,101,108,102,46,10,10,32,32,32,32,32, - 32,32,32,122,24,60,109,111,100,117,108,101,32,123,33,114, - 125,32,40,98,117,105,108,116,45,105,110,41,62,41,2,114, - 38,0,0,0,114,1,0,0,0,41,1,114,83,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 86,0,0,0,198,2,0,0,115,2,0,0,0,0,7,122, - 27,66,117,105,108,116,105,110,73,109,112,111,114,116,101,114, - 46,109,111,100,117,108,101,95,114,101,112,114,78,99,4,0, - 0,0,0,0,0,0,4,0,0,0,5,0,0,0,67,0, - 0,0,115,44,0,0,0,124,2,100,0,107,9,114,12,100, - 0,83,0,116,0,160,1,124,1,161,1,114,36,116,2,124, - 1,124,0,100,1,100,2,141,3,83,0,100,0,83,0,100, - 0,83,0,41,3,78,122,8,98,117,105,108,116,45,105,110, - 41,1,114,103,0,0,0,41,3,114,49,0,0,0,90,10, - 105,115,95,98,117,105,108,116,105,110,114,78,0,0,0,41, - 4,218,3,99,108,115,114,71,0,0,0,218,4,112,97,116, - 104,218,6,116,97,114,103,101,116,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,218,9,102,105,110,100,95,115, - 112,101,99,207,2,0,0,115,10,0,0,0,0,2,8,1, - 4,1,10,1,14,2,122,25,66,117,105,108,116,105,110,73, - 109,112,111,114,116,101,114,46,102,105,110,100,95,115,112,101, - 99,99,3,0,0,0,0,0,0,0,4,0,0,0,4,0, - 0,0,67,0,0,0,115,30,0,0,0,124,0,160,0,124, - 1,124,2,161,2,125,3,124,3,100,1,107,9,114,26,124, - 3,106,1,83,0,100,1,83,0,41,2,122,175,70,105,110, - 100,32,116,104,101,32,98,117,105,108,116,45,105,110,32,109, - 111,100,117,108,101,46,10,10,32,32,32,32,32,32,32,32, - 73,102,32,39,112,97,116,104,39,32,105,115,32,101,118,101, - 114,32,115,112,101,99,105,102,105,101,100,32,116,104,101,110, - 32,116,104,101,32,115,101,97,114,99,104,32,105,115,32,99, - 111,110,115,105,100,101,114,101,100,32,97,32,102,97,105,108, - 117,114,101,46,10,10,32,32,32,32,32,32,32,32,84,104, - 105,115,32,109,101,116,104,111,100,32,105,115,32,100,101,112, - 114,101,99,97,116,101,100,46,32,32,85,115,101,32,102,105, - 110,100,95,115,112,101,99,40,41,32,105,110,115,116,101,97, - 100,46,10,10,32,32,32,32,32,32,32,32,78,41,2,114, - 145,0,0,0,114,93,0,0,0,41,4,114,142,0,0,0, - 114,71,0,0,0,114,143,0,0,0,114,82,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,11, - 102,105,110,100,95,109,111,100,117,108,101,216,2,0,0,115, - 4,0,0,0,0,9,12,1,122,27,66,117,105,108,116,105, - 110,73,109,112,111,114,116,101,114,46,102,105,110,100,95,109, - 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, - 0,0,4,0,0,0,67,0,0,0,115,46,0,0,0,124, - 1,106,0,116,1,106,2,107,7,114,34,116,3,100,1,160, - 4,124,1,106,0,161,1,124,1,106,0,100,2,141,2,130, - 1,116,5,116,6,106,7,124,1,131,2,83,0,41,3,122, - 24,67,114,101,97,116,101,32,97,32,98,117,105,108,116,45, - 105,110,32,109,111,100,117,108,101,122,29,123,33,114,125,32, - 105,115,32,110,111,116,32,97,32,98,117,105,108,116,45,105, - 110,32,109,111,100,117,108,101,41,1,114,15,0,0,0,41, - 8,114,15,0,0,0,114,14,0,0,0,114,69,0,0,0, - 114,70,0,0,0,114,38,0,0,0,114,59,0,0,0,114, - 49,0,0,0,90,14,99,114,101,97,116,101,95,98,117,105, - 108,116,105,110,41,2,114,26,0,0,0,114,82,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 134,0,0,0,228,2,0,0,115,8,0,0,0,0,3,12, - 1,12,1,10,1,122,29,66,117,105,108,116,105,110,73,109, - 112,111,114,116,101,114,46,99,114,101,97,116,101,95,109,111, - 100,117,108,101,99,2,0,0,0,0,0,0,0,2,0,0, - 0,3,0,0,0,67,0,0,0,115,16,0,0,0,116,0, - 116,1,106,2,124,1,131,2,1,0,100,1,83,0,41,2, - 122,22,69,120,101,99,32,97,32,98,117,105,108,116,45,105, - 110,32,109,111,100,117,108,101,78,41,3,114,59,0,0,0, - 114,49,0,0,0,90,12,101,120,101,99,95,98,117,105,108, - 116,105,110,41,2,114,26,0,0,0,114,83,0,0,0,114, + 32,32,32,122,22,60,109,111,100,117,108,101,32,123,33,114, + 125,32,40,102,114,111,122,101,110,41,62,41,2,114,38,0, + 0,0,114,1,0,0,0,41,1,218,1,109,114,10,0,0, + 0,114,10,0,0,0,114,11,0,0,0,114,86,0,0,0, + 15,3,0,0,115,2,0,0,0,0,7,122,26,70,114,111, + 122,101,110,73,109,112,111,114,116,101,114,46,109,111,100,117, + 108,101,95,114,101,112,114,78,99,4,0,0,0,0,0,0, + 0,4,0,0,0,5,0,0,0,67,0,0,0,115,32,0, + 0,0,116,0,160,1,124,1,161,1,114,24,116,2,124,1, + 124,0,100,1,100,2,141,3,83,0,100,0,83,0,100,0, + 83,0,41,3,78,90,6,102,114,111,122,101,110,41,1,114, + 103,0,0,0,41,3,114,49,0,0,0,114,75,0,0,0, + 114,78,0,0,0,41,4,114,142,0,0,0,114,71,0,0, + 0,114,143,0,0,0,114,144,0,0,0,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,114,145,0,0,0,24, + 3,0,0,115,6,0,0,0,0,2,10,1,14,2,122,24, + 70,114,111,122,101,110,73,109,112,111,114,116,101,114,46,102, + 105,110,100,95,115,112,101,99,99,3,0,0,0,0,0,0, + 0,3,0,0,0,3,0,0,0,67,0,0,0,115,18,0, + 0,0,116,0,160,1,124,1,161,1,114,14,124,0,83,0, + 100,1,83,0,41,2,122,93,70,105,110,100,32,97,32,102, + 114,111,122,101,110,32,109,111,100,117,108,101,46,10,10,32, + 32,32,32,32,32,32,32,84,104,105,115,32,109,101,116,104, + 111,100,32,105,115,32,100,101,112,114,101,99,97,116,101,100, + 46,32,32,85,115,101,32,102,105,110,100,95,115,112,101,99, + 40,41,32,105,110,115,116,101,97,100,46,10,10,32,32,32, + 32,32,32,32,32,78,41,2,114,49,0,0,0,114,75,0, + 0,0,41,3,114,142,0,0,0,114,71,0,0,0,114,143, + 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,114,146,0,0,0,31,3,0,0,115,2,0,0,0, + 0,7,122,26,70,114,111,122,101,110,73,109,112,111,114,116, + 101,114,46,102,105,110,100,95,109,111,100,117,108,101,99,2, + 0,0,0,0,0,0,0,2,0,0,0,1,0,0,0,67, + 0,0,0,115,4,0,0,0,100,1,83,0,41,2,122,42, + 85,115,101,32,100,101,102,97,117,108,116,32,115,101,109,97, + 110,116,105,99,115,32,102,111,114,32,109,111,100,117,108,101, + 32,99,114,101,97,116,105,111,110,46,78,114,10,0,0,0, + 41,2,114,142,0,0,0,114,82,0,0,0,114,10,0,0, + 0,114,10,0,0,0,114,11,0,0,0,114,134,0,0,0, + 40,3,0,0,115,2,0,0,0,0,2,122,28,70,114,111, + 122,101,110,73,109,112,111,114,116,101,114,46,99,114,101,97, + 116,101,95,109,111,100,117,108,101,99,1,0,0,0,0,0, + 0,0,3,0,0,0,4,0,0,0,67,0,0,0,115,64, + 0,0,0,124,0,106,0,106,1,125,1,116,2,160,3,124, + 1,161,1,115,36,116,4,100,1,160,5,124,1,161,1,124, + 1,100,2,141,2,130,1,116,6,116,2,106,7,124,1,131, + 2,125,2,116,8,124,2,124,0,106,9,131,2,1,0,100, + 0,83,0,41,3,78,122,27,123,33,114,125,32,105,115,32, + 110,111,116,32,97,32,102,114,111,122,101,110,32,109,111,100, + 117,108,101,41,1,114,15,0,0,0,41,10,114,89,0,0, + 0,114,15,0,0,0,114,49,0,0,0,114,75,0,0,0, + 114,70,0,0,0,114,38,0,0,0,114,59,0,0,0,218, + 17,103,101,116,95,102,114,111,122,101,110,95,111,98,106,101, + 99,116,218,4,101,120,101,99,114,7,0,0,0,41,3,114, + 83,0,0,0,114,15,0,0,0,218,4,99,111,100,101,114, 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,135, - 0,0,0,236,2,0,0,115,2,0,0,0,0,3,122,27, - 66,117,105,108,116,105,110,73,109,112,111,114,116,101,114,46, - 101,120,101,99,95,109,111,100,117,108,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,83,0,41,2,122,57,82,101,116, - 117,114,110,32,78,111,110,101,32,97,115,32,98,117,105,108, - 116,45,105,110,32,109,111,100,117,108,101,115,32,100,111,32, - 110,111,116,32,104,97,118,101,32,99,111,100,101,32,111,98, - 106,101,99,116,115,46,78,114,10,0,0,0,41,2,114,142, + 0,0,0,44,3,0,0,115,14,0,0,0,0,2,8,1, + 10,1,10,1,2,255,6,2,12,1,122,26,70,114,111,122, + 101,110,73,109,112,111,114,116,101,114,46,101,120,101,99,95, + 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,2, + 0,0,0,3,0,0,0,67,0,0,0,115,10,0,0,0, + 116,0,124,0,124,1,131,2,83,0,41,1,122,95,76,111, + 97,100,32,97,32,102,114,111,122,101,110,32,109,111,100,117, + 108,101,46,10,10,32,32,32,32,32,32,32,32,84,104,105, + 115,32,109,101,116,104,111,100,32,105,115,32,100,101,112,114, + 101,99,97,116,101,100,46,32,32,85,115,101,32,101,120,101, + 99,95,109,111,100,117,108,101,40,41,32,105,110,115,116,101, + 97,100,46,10,10,32,32,32,32,32,32,32,32,41,1,114, + 84,0,0,0,41,2,114,142,0,0,0,114,71,0,0,0, + 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, + 137,0,0,0,53,3,0,0,115,2,0,0,0,0,7,122, + 26,70,114,111,122,101,110,73,109,112,111,114,116,101,114,46, + 108,111,97,100,95,109,111,100,117,108,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,3,0,0,0,67,0,0,0, + 115,10,0,0,0,116,0,160,1,124,1,161,1,83,0,41, + 1,122,45,82,101,116,117,114,110,32,116,104,101,32,99,111, + 100,101,32,111,98,106,101,99,116,32,102,111,114,32,116,104, + 101,32,102,114,111,122,101,110,32,109,111,100,117,108,101,46, + 41,2,114,49,0,0,0,114,153,0,0,0,41,2,114,142, 0,0,0,114,71,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,218,8,103,101,116,95,99,111,100, - 101,241,2,0,0,115,2,0,0,0,0,4,122,24,66,117, - 105,108,116,105,110,73,109,112,111,114,116,101,114,46,103,101, - 116,95,99,111,100,101,99,2,0,0,0,0,0,0,0,2, - 0,0,0,1,0,0,0,67,0,0,0,115,4,0,0,0, - 100,1,83,0,41,2,122,56,82,101,116,117,114,110,32,78, - 111,110,101,32,97,115,32,98,117,105,108,116,45,105,110,32, - 109,111,100,117,108,101,115,32,100,111,32,110,111,116,32,104, - 97,118,101,32,115,111,117,114,99,101,32,99,111,100,101,46, - 78,114,10,0,0,0,41,2,114,142,0,0,0,114,71,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,218,10,103,101,116,95,115,111,117,114,99,101,247,2,0, - 0,115,2,0,0,0,0,4,122,26,66,117,105,108,116,105, + 0,0,114,11,0,0,0,114,147,0,0,0,62,3,0,0, + 115,2,0,0,0,0,4,122,23,70,114,111,122,101,110,73, + 109,112,111,114,116,101,114,46,103,101,116,95,99,111,100,101, + 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, + 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, + 122,54,82,101,116,117,114,110,32,78,111,110,101,32,97,115, + 32,102,114,111,122,101,110,32,109,111,100,117,108,101,115,32, + 100,111,32,110,111,116,32,104,97,118,101,32,115,111,117,114, + 99,101,32,99,111,100,101,46,78,114,10,0,0,0,41,2, + 114,142,0,0,0,114,71,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,114,148,0,0,0,68,3, + 0,0,115,2,0,0,0,0,4,122,25,70,114,111,122,101, 110,73,109,112,111,114,116,101,114,46,103,101,116,95,115,111, 117,114,99,101,99,2,0,0,0,0,0,0,0,2,0,0, - 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, - 83,0,41,2,122,52,82,101,116,117,114,110,32,70,97,108, - 115,101,32,97,115,32,98,117,105,108,116,45,105,110,32,109, - 111,100,117,108,101,115,32,97,114,101,32,110,101,118,101,114, - 32,112,97,99,107,97,103,101,115,46,70,114,10,0,0,0, - 41,2,114,142,0,0,0,114,71,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,105,0,0,0, - 253,2,0,0,115,2,0,0,0,0,4,122,26,66,117,105, - 108,116,105,110,73,109,112,111,114,116,101,114,46,105,115,95, - 112,97,99,107,97,103,101,41,2,78,78,41,1,78,41,17, - 114,1,0,0,0,114,0,0,0,0,114,2,0,0,0,114, - 3,0,0,0,218,12,115,116,97,116,105,99,109,101,116,104, - 111,100,114,86,0,0,0,218,11,99,108,97,115,115,109,101, - 116,104,111,100,114,145,0,0,0,114,146,0,0,0,114,134, - 0,0,0,114,135,0,0,0,114,74,0,0,0,114,147,0, - 0,0,114,148,0,0,0,114,105,0,0,0,114,84,0,0, - 0,114,137,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,114,141,0,0,0,189, - 2,0,0,115,30,0,0,0,8,7,4,2,12,9,2,1, - 12,8,2,1,12,11,12,8,12,5,2,1,14,5,2,1, - 14,5,2,1,14,5,114,141,0,0,0,99,0,0,0,0, - 0,0,0,0,0,0,0,0,4,0,0,0,64,0,0,0, - 115,140,0,0,0,101,0,90,1,100,0,90,2,100,1,90, - 3,101,4,100,2,100,3,132,0,131,1,90,5,101,6,100, - 21,100,5,100,6,132,1,131,1,90,7,101,6,100,22,100, - 7,100,8,132,1,131,1,90,8,101,6,100,9,100,10,132, - 0,131,1,90,9,101,4,100,11,100,12,132,0,131,1,90, - 10,101,6,100,13,100,14,132,0,131,1,90,11,101,6,101, - 12,100,15,100,16,132,0,131,1,131,1,90,13,101,6,101, - 12,100,17,100,18,132,0,131,1,131,1,90,14,101,6,101, - 12,100,19,100,20,132,0,131,1,131,1,90,15,100,4,83, - 0,41,23,218,14,70,114,111,122,101,110,73,109,112,111,114, - 116,101,114,122,142,77,101,116,97,32,112,97,116,104,32,105, - 109,112,111,114,116,32,102,111,114,32,102,114,111,122,101,110, - 32,109,111,100,117,108,101,115,46,10,10,32,32,32,32,65, - 108,108,32,109,101,116,104,111,100,115,32,97,114,101,32,101, - 105,116,104,101,114,32,99,108,97,115,115,32,111,114,32,115, - 116,97,116,105,99,32,109,101,116,104,111,100,115,32,116,111, - 32,97,118,111,105,100,32,116,104,101,32,110,101,101,100,32, - 116,111,10,32,32,32,32,105,110,115,116,97,110,116,105,97, - 116,101,32,116,104,101,32,99,108,97,115,115,46,10,10,32, - 32,32,32,99,1,0,0,0,0,0,0,0,1,0,0,0, - 3,0,0,0,67,0,0,0,115,12,0,0,0,100,1,160, - 0,124,0,106,1,161,1,83,0,41,2,122,115,82,101,116, - 117,114,110,32,114,101,112,114,32,102,111,114,32,116,104,101, - 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, - 32,32,84,104,101,32,109,101,116,104,111,100,32,105,115,32, - 100,101,112,114,101,99,97,116,101,100,46,32,32,84,104,101, - 32,105,109,112,111,114,116,32,109,97,99,104,105,110,101,114, - 121,32,100,111,101,115,32,116,104,101,32,106,111,98,32,105, - 116,115,101,108,102,46,10,10,32,32,32,32,32,32,32,32, - 122,22,60,109,111,100,117,108,101,32,123,33,114,125,32,40, - 102,114,111,122,101,110,41,62,41,2,114,38,0,0,0,114, - 1,0,0,0,41,1,218,1,109,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,86,0,0,0,15,3,0, - 0,115,2,0,0,0,0,7,122,26,70,114,111,122,101,110, - 73,109,112,111,114,116,101,114,46,109,111,100,117,108,101,95, - 114,101,112,114,78,99,4,0,0,0,0,0,0,0,4,0, - 0,0,5,0,0,0,67,0,0,0,115,32,0,0,0,116, - 0,160,1,124,1,161,1,114,24,116,2,124,1,124,0,100, - 1,100,2,141,3,83,0,100,0,83,0,100,0,83,0,41, - 3,78,90,6,102,114,111,122,101,110,41,1,114,103,0,0, - 0,41,3,114,49,0,0,0,114,75,0,0,0,114,78,0, - 0,0,41,4,114,142,0,0,0,114,71,0,0,0,114,143, - 0,0,0,114,144,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,114,145,0,0,0,24,3,0,0, - 115,6,0,0,0,0,2,10,1,14,2,122,24,70,114,111, - 122,101,110,73,109,112,111,114,116,101,114,46,102,105,110,100, - 95,115,112,101,99,99,3,0,0,0,0,0,0,0,3,0, - 0,0,3,0,0,0,67,0,0,0,115,18,0,0,0,116, - 0,160,1,124,1,161,1,114,14,124,0,83,0,100,1,83, - 0,41,2,122,93,70,105,110,100,32,97,32,102,114,111,122, - 101,110,32,109,111,100,117,108,101,46,10,10,32,32,32,32, - 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, - 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, - 85,115,101,32,102,105,110,100,95,115,112,101,99,40,41,32, - 105,110,115,116,101,97,100,46,10,10,32,32,32,32,32,32, - 32,32,78,41,2,114,49,0,0,0,114,75,0,0,0,41, - 3,114,142,0,0,0,114,71,0,0,0,114,143,0,0,0, + 0,3,0,0,0,67,0,0,0,115,10,0,0,0,116,0, + 160,1,124,1,161,1,83,0,41,1,122,46,82,101,116,117, + 114,110,32,84,114,117,101,32,105,102,32,116,104,101,32,102, + 114,111,122,101,110,32,109,111,100,117,108,101,32,105,115,32, + 97,32,112,97,99,107,97,103,101,46,41,2,114,49,0,0, + 0,90,17,105,115,95,102,114,111,122,101,110,95,112,97,99, + 107,97,103,101,41,2,114,142,0,0,0,114,71,0,0,0, 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 146,0,0,0,31,3,0,0,115,2,0,0,0,0,7,122, - 26,70,114,111,122,101,110,73,109,112,111,114,116,101,114,46, - 102,105,110,100,95,109,111,100,117,108,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,83,0,41,2,122,42,85,115,101, - 32,100,101,102,97,117,108,116,32,115,101,109,97,110,116,105, - 99,115,32,102,111,114,32,109,111,100,117,108,101,32,99,114, - 101,97,116,105,111,110,46,78,114,10,0,0,0,41,2,114, - 142,0,0,0,114,82,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,134,0,0,0,40,3,0, - 0,115,2,0,0,0,0,2,122,28,70,114,111,122,101,110, - 73,109,112,111,114,116,101,114,46,99,114,101,97,116,101,95, - 109,111,100,117,108,101,99,1,0,0,0,0,0,0,0,3, - 0,0,0,4,0,0,0,67,0,0,0,115,64,0,0,0, - 124,0,106,0,106,1,125,1,116,2,160,3,124,1,161,1, - 115,36,116,4,100,1,160,5,124,1,161,1,124,1,100,2, - 141,2,130,1,116,6,116,2,106,7,124,1,131,2,125,2, - 116,8,124,2,124,0,106,9,131,2,1,0,100,0,83,0, - 41,3,78,122,27,123,33,114,125,32,105,115,32,110,111,116, - 32,97,32,102,114,111,122,101,110,32,109,111,100,117,108,101, - 41,1,114,15,0,0,0,41,10,114,89,0,0,0,114,15, - 0,0,0,114,49,0,0,0,114,75,0,0,0,114,70,0, - 0,0,114,38,0,0,0,114,59,0,0,0,218,17,103,101, - 116,95,102,114,111,122,101,110,95,111,98,106,101,99,116,218, - 4,101,120,101,99,114,7,0,0,0,41,3,114,83,0,0, - 0,114,15,0,0,0,218,4,99,111,100,101,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,135,0,0,0, - 44,3,0,0,115,12,0,0,0,0,2,8,1,10,1,10, - 1,8,1,12,1,122,26,70,114,111,122,101,110,73,109,112, - 111,114,116,101,114,46,101,120,101,99,95,109,111,100,117,108, - 101,99,2,0,0,0,0,0,0,0,2,0,0,0,3,0, - 0,0,67,0,0,0,115,10,0,0,0,116,0,124,0,124, - 1,131,2,83,0,41,1,122,95,76,111,97,100,32,97,32, - 102,114,111,122,101,110,32,109,111,100,117,108,101,46,10,10, - 32,32,32,32,32,32,32,32,84,104,105,115,32,109,101,116, - 104,111,100,32,105,115,32,100,101,112,114,101,99,97,116,101, - 100,46,32,32,85,115,101,32,101,120,101,99,95,109,111,100, - 117,108,101,40,41,32,105,110,115,116,101,97,100,46,10,10, - 32,32,32,32,32,32,32,32,41,1,114,84,0,0,0,41, - 2,114,142,0,0,0,114,71,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,114,137,0,0,0,53, - 3,0,0,115,2,0,0,0,0,7,122,26,70,114,111,122, - 101,110,73,109,112,111,114,116,101,114,46,108,111,97,100,95, - 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,2, - 0,0,0,3,0,0,0,67,0,0,0,115,10,0,0,0, - 116,0,160,1,124,1,161,1,83,0,41,1,122,45,82,101, - 116,117,114,110,32,116,104,101,32,99,111,100,101,32,111,98, - 106,101,99,116,32,102,111,114,32,116,104,101,32,102,114,111, - 122,101,110,32,109,111,100,117,108,101,46,41,2,114,49,0, - 0,0,114,153,0,0,0,41,2,114,142,0,0,0,114,71, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,147,0,0,0,62,3,0,0,115,2,0,0,0, - 0,4,122,23,70,114,111,122,101,110,73,109,112,111,114,116, - 101,114,46,103,101,116,95,99,111,100,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,83,0,41,2,122,54,82,101,116, - 117,114,110,32,78,111,110,101,32,97,115,32,102,114,111,122, - 101,110,32,109,111,100,117,108,101,115,32,100,111,32,110,111, - 116,32,104,97,118,101,32,115,111,117,114,99,101,32,99,111, - 100,101,46,78,114,10,0,0,0,41,2,114,142,0,0,0, - 114,71,0,0,0,114,10,0,0,0,114,10,0,0,0,114, - 11,0,0,0,114,148,0,0,0,68,3,0,0,115,2,0, - 0,0,0,4,122,25,70,114,111,122,101,110,73,109,112,111, - 114,116,101,114,46,103,101,116,95,115,111,117,114,99,101,99, - 2,0,0,0,0,0,0,0,2,0,0,0,3,0,0,0, - 67,0,0,0,115,10,0,0,0,116,0,160,1,124,1,161, - 1,83,0,41,1,122,46,82,101,116,117,114,110,32,84,114, - 117,101,32,105,102,32,116,104,101,32,102,114,111,122,101,110, - 32,109,111,100,117,108,101,32,105,115,32,97,32,112,97,99, - 107,97,103,101,46,41,2,114,49,0,0,0,90,17,105,115, - 95,102,114,111,122,101,110,95,112,97,99,107,97,103,101,41, - 2,114,142,0,0,0,114,71,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,114,105,0,0,0,74, - 3,0,0,115,2,0,0,0,0,4,122,25,70,114,111,122, - 101,110,73,109,112,111,114,116,101,114,46,105,115,95,112,97, - 99,107,97,103,101,41,2,78,78,41,1,78,41,16,114,1, - 0,0,0,114,0,0,0,0,114,2,0,0,0,114,3,0, - 0,0,114,149,0,0,0,114,86,0,0,0,114,150,0,0, - 0,114,145,0,0,0,114,146,0,0,0,114,134,0,0,0, - 114,135,0,0,0,114,137,0,0,0,114,77,0,0,0,114, - 147,0,0,0,114,148,0,0,0,114,105,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,151,0,0,0,6,3,0,0,115,30,0,0,0, - 8,7,4,2,12,9,2,1,12,6,2,1,12,8,12,4, - 12,9,12,9,2,1,14,5,2,1,14,5,2,1,114,151, - 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, - 2,0,0,0,64,0,0,0,115,32,0,0,0,101,0,90, - 1,100,0,90,2,100,1,90,3,100,2,100,3,132,0,90, - 4,100,4,100,5,132,0,90,5,100,6,83,0,41,7,218, - 18,95,73,109,112,111,114,116,76,111,99,107,67,111,110,116, - 101,120,116,122,36,67,111,110,116,101,120,116,32,109,97,110, - 97,103,101,114,32,102,111,114,32,116,104,101,32,105,109,112, - 111,114,116,32,108,111,99,107,46,99,1,0,0,0,0,0, - 0,0,1,0,0,0,2,0,0,0,67,0,0,0,115,12, - 0,0,0,116,0,160,1,161,0,1,0,100,1,83,0,41, - 2,122,24,65,99,113,117,105,114,101,32,116,104,101,32,105, - 109,112,111,114,116,32,108,111,99,107,46,78,41,2,114,49, - 0,0,0,114,50,0,0,0,41,1,114,26,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,46, - 0,0,0,87,3,0,0,115,2,0,0,0,0,2,122,28, - 95,73,109,112,111,114,116,76,111,99,107,67,111,110,116,101, - 120,116,46,95,95,101,110,116,101,114,95,95,99,4,0,0, - 0,0,0,0,0,4,0,0,0,2,0,0,0,67,0,0, - 0,115,12,0,0,0,116,0,160,1,161,0,1,0,100,1, - 83,0,41,2,122,60,82,101,108,101,97,115,101,32,116,104, - 101,32,105,109,112,111,114,116,32,108,111,99,107,32,114,101, - 103,97,114,100,108,101,115,115,32,111,102,32,97,110,121,32, - 114,97,105,115,101,100,32,101,120,99,101,112,116,105,111,110, - 115,46,78,41,2,114,49,0,0,0,114,52,0,0,0,41, - 4,114,26,0,0,0,90,8,101,120,99,95,116,121,112,101, - 90,9,101,120,99,95,118,97,108,117,101,90,13,101,120,99, - 95,116,114,97,99,101,98,97,99,107,114,10,0,0,0,114, - 10,0,0,0,114,11,0,0,0,114,48,0,0,0,91,3, - 0,0,115,2,0,0,0,0,2,122,27,95,73,109,112,111, - 114,116,76,111,99,107,67,111,110,116,101,120,116,46,95,95, - 101,120,105,116,95,95,78,41,6,114,1,0,0,0,114,0, - 0,0,0,114,2,0,0,0,114,3,0,0,0,114,46,0, - 0,0,114,48,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,156,0,0,0, - 83,3,0,0,115,6,0,0,0,8,2,4,2,8,4,114, - 156,0,0,0,99,3,0,0,0,0,0,0,0,5,0,0, - 0,5,0,0,0,67,0,0,0,115,64,0,0,0,124,1, - 160,0,100,1,124,2,100,2,24,0,161,2,125,3,116,1, - 124,3,131,1,124,2,107,0,114,36,116,2,100,3,131,1, - 130,1,124,3,100,4,25,0,125,4,124,0,114,60,100,5, - 160,3,124,4,124,0,161,2,83,0,124,4,83,0,41,6, - 122,50,82,101,115,111,108,118,101,32,97,32,114,101,108,97, - 116,105,118,101,32,109,111,100,117,108,101,32,110,97,109,101, - 32,116,111,32,97,110,32,97,98,115,111,108,117,116,101,32, - 111,110,101,46,114,117,0,0,0,114,33,0,0,0,122,50, - 97,116,116,101,109,112,116,101,100,32,114,101,108,97,116,105, - 118,101,32,105,109,112,111,114,116,32,98,101,121,111,110,100, - 32,116,111,112,45,108,101,118,101,108,32,112,97,99,107,97, - 103,101,114,19,0,0,0,122,5,123,125,46,123,125,41,4, - 218,6,114,115,112,108,105,116,218,3,108,101,110,218,10,86, - 97,108,117,101,69,114,114,111,114,114,38,0,0,0,41,5, - 114,15,0,0,0,218,7,112,97,99,107,97,103,101,218,5, - 108,101,118,101,108,90,4,98,105,116,115,90,4,98,97,115, - 101,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 218,13,95,114,101,115,111,108,118,101,95,110,97,109,101,96, - 3,0,0,115,10,0,0,0,0,2,16,1,12,1,8,1, - 8,1,114,162,0,0,0,99,3,0,0,0,0,0,0,0, - 4,0,0,0,4,0,0,0,67,0,0,0,115,34,0,0, - 0,124,0,160,0,124,1,124,2,161,2,125,3,124,3,100, - 0,107,8,114,24,100,0,83,0,116,1,124,1,124,3,131, - 2,83,0,41,1,78,41,2,114,146,0,0,0,114,78,0, - 0,0,41,4,218,6,102,105,110,100,101,114,114,15,0,0, - 0,114,143,0,0,0,114,93,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,218,17,95,102,105,110, - 100,95,115,112,101,99,95,108,101,103,97,99,121,105,3,0, - 0,115,8,0,0,0,0,3,12,1,8,1,4,1,114,164, - 0,0,0,99,3,0,0,0,0,0,0,0,10,0,0,0, - 10,0,0,0,67,0,0,0,115,12,1,0,0,116,0,106, - 1,125,3,124,3,100,1,107,8,114,22,116,2,100,2,131, - 1,130,1,124,3,115,38,116,3,160,4,100,3,116,5,161, - 2,1,0,124,0,116,0,106,6,107,6,125,4,124,3,68, - 0,93,210,125,5,116,7,131,0,143,84,1,0,122,10,124, - 5,106,8,125,6,87,0,110,54,4,0,116,9,107,10,114, - 128,1,0,1,0,1,0,116,10,124,5,124,0,124,1,131, - 3,125,7,124,7,100,1,107,8,114,124,89,0,87,0,53, - 0,81,0,82,0,163,0,113,52,89,0,110,14,88,0,124, - 6,124,0,124,1,124,2,131,3,125,7,87,0,53,0,81, - 0,82,0,88,0,124,7,100,1,107,9,114,52,124,4,144, - 0,115,254,124,0,116,0,106,6,107,6,144,0,114,254,116, - 0,106,6,124,0,25,0,125,8,122,10,124,8,106,11,125, - 9,87,0,110,28,4,0,116,9,107,10,114,226,1,0,1, - 0,1,0,124,7,6,0,89,0,2,0,1,0,83,0,88, - 0,124,9,100,1,107,8,114,244,124,7,2,0,1,0,83, - 0,124,9,2,0,1,0,83,0,113,52,124,7,2,0,1, - 0,83,0,113,52,100,1,83,0,41,4,122,21,70,105,110, - 100,32,97,32,109,111,100,117,108,101,39,115,32,115,112,101, - 99,46,78,122,53,115,121,115,46,109,101,116,97,95,112,97, - 116,104,32,105,115,32,78,111,110,101,44,32,80,121,116,104, - 111,110,32,105,115,32,108,105,107,101,108,121,32,115,104,117, - 116,116,105,110,103,32,100,111,119,110,122,22,115,121,115,46, - 109,101,116,97,95,112,97,116,104,32,105,115,32,101,109,112, - 116,121,41,12,114,14,0,0,0,218,9,109,101,116,97,95, - 112,97,116,104,114,70,0,0,0,218,9,95,119,97,114,110, - 105,110,103,115,218,4,119,97,114,110,218,13,73,109,112,111, - 114,116,87,97,114,110,105,110,103,114,79,0,0,0,114,156, - 0,0,0,114,145,0,0,0,114,90,0,0,0,114,164,0, - 0,0,114,89,0,0,0,41,10,114,15,0,0,0,114,143, - 0,0,0,114,144,0,0,0,114,165,0,0,0,90,9,105, - 115,95,114,101,108,111,97,100,114,163,0,0,0,114,145,0, - 0,0,114,82,0,0,0,114,83,0,0,0,114,89,0,0, + 105,0,0,0,74,3,0,0,115,2,0,0,0,0,4,122, + 25,70,114,111,122,101,110,73,109,112,111,114,116,101,114,46, + 105,115,95,112,97,99,107,97,103,101,41,16,114,1,0,0, + 0,114,0,0,0,0,114,2,0,0,0,114,3,0,0,0, + 114,149,0,0,0,114,86,0,0,0,114,150,0,0,0,114, + 145,0,0,0,114,146,0,0,0,114,134,0,0,0,114,135, + 0,0,0,114,137,0,0,0,114,77,0,0,0,114,147,0, + 0,0,114,148,0,0,0,114,105,0,0,0,114,10,0,0, 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 218,10,95,102,105,110,100,95,115,112,101,99,114,3,0,0, - 115,54,0,0,0,0,2,6,1,8,2,8,3,4,1,12, - 5,10,1,8,1,8,1,2,1,10,1,14,1,12,1,8, - 1,20,2,22,1,8,2,18,1,10,1,2,1,10,1,14, - 4,14,2,8,1,8,2,10,2,10,2,114,169,0,0,0, - 99,3,0,0,0,0,0,0,0,3,0,0,0,5,0,0, - 0,67,0,0,0,115,108,0,0,0,116,0,124,0,116,1, - 131,2,115,28,116,2,100,1,160,3,116,4,124,0,131,1, - 161,1,131,1,130,1,124,2,100,2,107,0,114,44,116,5, - 100,3,131,1,130,1,124,2,100,2,107,4,114,84,116,0, - 124,1,116,1,131,2,115,72,116,2,100,4,131,1,130,1, - 110,12,124,1,115,84,116,6,100,5,131,1,130,1,124,0, - 115,104,124,2,100,2,107,2,114,104,116,5,100,6,131,1, - 130,1,100,7,83,0,41,8,122,28,86,101,114,105,102,121, - 32,97,114,103,117,109,101,110,116,115,32,97,114,101,32,34, - 115,97,110,101,34,46,122,31,109,111,100,117,108,101,32,110, - 97,109,101,32,109,117,115,116,32,98,101,32,115,116,114,44, - 32,110,111,116,32,123,125,114,19,0,0,0,122,18,108,101, - 118,101,108,32,109,117,115,116,32,98,101,32,62,61,32,48, - 122,31,95,95,112,97,99,107,97,103,101,95,95,32,110,111, - 116,32,115,101,116,32,116,111,32,97,32,115,116,114,105,110, - 103,122,54,97,116,116,101,109,112,116,101,100,32,114,101,108, - 97,116,105,118,101,32,105,109,112,111,114,116,32,119,105,116, - 104,32,110,111,32,107,110,111,119,110,32,112,97,114,101,110, - 116,32,112,97,99,107,97,103,101,122,17,69,109,112,116,121, - 32,109,111,100,117,108,101,32,110,97,109,101,78,41,7,218, - 10,105,115,105,110,115,116,97,110,99,101,218,3,115,116,114, - 218,9,84,121,112,101,69,114,114,111,114,114,38,0,0,0, - 114,13,0,0,0,114,159,0,0,0,114,70,0,0,0,41, - 3,114,15,0,0,0,114,160,0,0,0,114,161,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 13,95,115,97,110,105,116,121,95,99,104,101,99,107,161,3, - 0,0,115,22,0,0,0,0,2,10,1,18,1,8,1,8, - 1,8,1,10,1,10,1,4,1,8,2,12,1,114,173,0, - 0,0,122,16,78,111,32,109,111,100,117,108,101,32,110,97, - 109,101,100,32,122,4,123,33,114,125,99,2,0,0,0,0, - 0,0,0,8,0,0,0,8,0,0,0,67,0,0,0,115, - 220,0,0,0,100,0,125,2,124,0,160,0,100,1,161,1, - 100,2,25,0,125,3,124,3,114,134,124,3,116,1,106,2, - 107,7,114,42,116,3,124,1,124,3,131,2,1,0,124,0, - 116,1,106,2,107,6,114,62,116,1,106,2,124,0,25,0, - 83,0,116,1,106,2,124,3,25,0,125,4,122,10,124,4, - 106,4,125,2,87,0,110,50,4,0,116,5,107,10,114,132, - 1,0,1,0,1,0,116,6,100,3,23,0,160,7,124,0, - 124,3,161,2,125,5,116,8,124,5,124,0,100,4,141,2, - 100,0,130,2,89,0,110,2,88,0,116,9,124,0,124,2, - 131,2,125,6,124,6,100,0,107,8,114,172,116,8,116,6, - 160,7,124,0,161,1,124,0,100,4,141,2,130,1,110,8, - 116,10,124,6,131,1,125,7,124,3,114,216,116,1,106,2, - 124,3,25,0,125,4,116,11,124,4,124,0,160,0,100,1, - 161,1,100,5,25,0,124,7,131,3,1,0,124,7,83,0, - 41,6,78,114,117,0,0,0,114,19,0,0,0,122,23,59, - 32,123,33,114,125,32,105,115,32,110,111,116,32,97,32,112, - 97,99,107,97,103,101,41,1,114,15,0,0,0,233,2,0, - 0,0,41,12,114,118,0,0,0,114,14,0,0,0,114,79, - 0,0,0,114,59,0,0,0,114,127,0,0,0,114,90,0, - 0,0,218,8,95,69,82,82,95,77,83,71,114,38,0,0, - 0,218,19,77,111,100,117,108,101,78,111,116,70,111,117,110, - 100,69,114,114,111,114,114,169,0,0,0,114,140,0,0,0, - 114,5,0,0,0,41,8,114,15,0,0,0,218,7,105,109, - 112,111,114,116,95,114,143,0,0,0,114,119,0,0,0,90, - 13,112,97,114,101,110,116,95,109,111,100,117,108,101,114,138, - 0,0,0,114,82,0,0,0,114,83,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,218,23,95,102, - 105,110,100,95,97,110,100,95,108,111,97,100,95,117,110,108, - 111,99,107,101,100,180,3,0,0,115,42,0,0,0,0,1, - 4,1,14,1,4,1,10,1,10,2,10,1,10,1,10,1, - 2,1,10,1,14,1,16,1,20,1,10,1,8,1,20,2, - 8,1,4,2,10,1,22,1,114,178,0,0,0,99,2,0, - 0,0,0,0,0,0,4,0,0,0,10,0,0,0,67,0, - 0,0,115,106,0,0,0,116,0,124,0,131,1,143,50,1, - 0,116,1,106,2,160,3,124,0,116,4,161,2,125,2,124, - 2,116,4,107,8,114,54,116,5,124,0,124,1,131,2,87, - 0,2,0,53,0,81,0,82,0,163,0,83,0,87,0,53, - 0,81,0,82,0,88,0,124,2,100,1,107,8,114,94,100, - 2,160,6,124,0,161,1,125,3,116,7,124,3,124,0,100, - 3,141,2,130,1,116,8,124,0,131,1,1,0,124,2,83, - 0,41,4,122,25,70,105,110,100,32,97,110,100,32,108,111, - 97,100,32,116,104,101,32,109,111,100,117,108,101,46,78,122, - 40,105,109,112,111,114,116,32,111,102,32,123,125,32,104,97, - 108,116,101,100,59,32,78,111,110,101,32,105,110,32,115,121, - 115,46,109,111,100,117,108,101,115,41,1,114,15,0,0,0, - 41,9,114,42,0,0,0,114,14,0,0,0,114,79,0,0, - 0,114,30,0,0,0,218,14,95,78,69,69,68,83,95,76, - 79,65,68,73,78,71,114,178,0,0,0,114,38,0,0,0, - 114,176,0,0,0,114,57,0,0,0,41,4,114,15,0,0, - 0,114,177,0,0,0,114,83,0,0,0,114,67,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 14,95,102,105,110,100,95,97,110,100,95,108,111,97,100,210, - 3,0,0,115,20,0,0,0,0,2,10,1,14,1,8,1, - 32,2,8,1,4,1,6,1,12,2,8,1,114,180,0,0, - 0,114,19,0,0,0,99,3,0,0,0,0,0,0,0,3, - 0,0,0,4,0,0,0,67,0,0,0,115,42,0,0,0, - 116,0,124,0,124,1,124,2,131,3,1,0,124,2,100,1, - 107,4,114,32,116,1,124,0,124,1,124,2,131,3,125,0, - 116,2,124,0,116,3,131,2,83,0,41,2,97,50,1,0, - 0,73,109,112,111,114,116,32,97,110,100,32,114,101,116,117, - 114,110,32,116,104,101,32,109,111,100,117,108,101,32,98,97, - 115,101,100,32,111,110,32,105,116,115,32,110,97,109,101,44, - 32,116,104,101,32,112,97,99,107,97,103,101,32,116,104,101, - 32,99,97,108,108,32,105,115,10,32,32,32,32,98,101,105, - 110,103,32,109,97,100,101,32,102,114,111,109,44,32,97,110, - 100,32,116,104,101,32,108,101,118,101,108,32,97,100,106,117, - 115,116,109,101,110,116,46,10,10,32,32,32,32,84,104,105, - 115,32,102,117,110,99,116,105,111,110,32,114,101,112,114,101, - 115,101,110,116,115,32,116,104,101,32,103,114,101,97,116,101, - 115,116,32,99,111,109,109,111,110,32,100,101,110,111,109,105, - 110,97,116,111,114,32,111,102,32,102,117,110,99,116,105,111, - 110,97,108,105,116,121,10,32,32,32,32,98,101,116,119,101, - 101,110,32,105,109,112,111,114,116,95,109,111,100,117,108,101, - 32,97,110,100,32,95,95,105,109,112,111,114,116,95,95,46, - 32,84,104,105,115,32,105,110,99,108,117,100,101,115,32,115, - 101,116,116,105,110,103,32,95,95,112,97,99,107,97,103,101, - 95,95,32,105,102,10,32,32,32,32,116,104,101,32,108,111, - 97,100,101,114,32,100,105,100,32,110,111,116,46,10,10,32, - 32,32,32,114,19,0,0,0,41,4,114,173,0,0,0,114, - 162,0,0,0,114,180,0,0,0,218,11,95,103,99,100,95, - 105,109,112,111,114,116,41,3,114,15,0,0,0,114,160,0, - 0,0,114,161,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,114,181,0,0,0,226,3,0,0,115, - 8,0,0,0,0,9,12,1,8,1,12,1,114,181,0,0, - 0,41,1,218,9,114,101,99,117,114,115,105,118,101,99,3, - 0,0,0,1,0,0,0,8,0,0,0,11,0,0,0,67, - 0,0,0,115,226,0,0,0,124,1,68,0,93,216,125,4, - 116,0,124,4,116,1,131,2,115,66,124,3,114,34,124,0, - 106,2,100,1,23,0,125,5,110,4,100,2,125,5,116,3, - 100,3,124,5,155,0,100,4,116,4,124,4,131,1,106,2, - 155,0,157,4,131,1,130,1,113,4,124,4,100,5,107,2, - 114,108,124,3,115,220,116,5,124,0,100,6,131,2,114,220, - 116,6,124,0,124,0,106,7,124,2,100,7,100,8,141,4, - 1,0,113,4,116,5,124,0,124,4,131,2,115,4,100,9, - 160,8,124,0,106,2,124,4,161,2,125,6,122,14,116,9, - 124,2,124,6,131,2,1,0,87,0,113,4,4,0,116,10, - 107,10,114,218,1,0,125,7,1,0,122,42,124,7,106,11, - 124,6,107,2,114,200,116,12,106,13,160,14,124,6,116,15, - 161,2,100,10,107,9,114,200,87,0,89,0,162,8,113,4, - 130,0,87,0,53,0,100,10,125,7,126,7,88,0,89,0, - 113,4,88,0,113,4,124,0,83,0,41,11,122,238,70,105, - 103,117,114,101,32,111,117,116,32,119,104,97,116,32,95,95, - 105,109,112,111,114,116,95,95,32,115,104,111,117,108,100,32, - 114,101,116,117,114,110,46,10,10,32,32,32,32,84,104,101, - 32,105,109,112,111,114,116,95,32,112,97,114,97,109,101,116, - 101,114,32,105,115,32,97,32,99,97,108,108,97,98,108,101, - 32,119,104,105,99,104,32,116,97,107,101,115,32,116,104,101, - 32,110,97,109,101,32,111,102,32,109,111,100,117,108,101,32, - 116,111,10,32,32,32,32,105,109,112,111,114,116,46,32,73, - 116,32,105,115,32,114,101,113,117,105,114,101,100,32,116,111, - 32,100,101,99,111,117,112,108,101,32,116,104,101,32,102,117, - 110,99,116,105,111,110,32,102,114,111,109,32,97,115,115,117, - 109,105,110,103,32,105,109,112,111,114,116,108,105,98,39,115, - 10,32,32,32,32,105,109,112,111,114,116,32,105,109,112,108, - 101,109,101,110,116,97,116,105,111,110,32,105,115,32,100,101, - 115,105,114,101,100,46,10,10,32,32,32,32,122,8,46,95, - 95,97,108,108,95,95,122,13,96,96,102,114,111,109,32,108, - 105,115,116,39,39,122,8,73,116,101,109,32,105,110,32,122, - 18,32,109,117,115,116,32,98,101,32,115,116,114,44,32,110, - 111,116,32,250,1,42,218,7,95,95,97,108,108,95,95,84, - 41,1,114,182,0,0,0,122,5,123,125,46,123,125,78,41, - 16,114,170,0,0,0,114,171,0,0,0,114,1,0,0,0, - 114,172,0,0,0,114,13,0,0,0,114,4,0,0,0,218, - 16,95,104,97,110,100,108,101,95,102,114,111,109,108,105,115, - 116,114,184,0,0,0,114,38,0,0,0,114,59,0,0,0, - 114,176,0,0,0,114,15,0,0,0,114,14,0,0,0,114, - 79,0,0,0,114,30,0,0,0,114,179,0,0,0,41,8, - 114,83,0,0,0,218,8,102,114,111,109,108,105,115,116,114, - 177,0,0,0,114,182,0,0,0,218,1,120,90,5,119,104, - 101,114,101,90,9,102,114,111,109,95,110,97,109,101,90,3, - 101,120,99,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,185,0,0,0,241,3,0,0,115,40,0,0,0, - 0,10,8,1,10,1,4,1,12,2,4,1,28,2,8,1, - 14,1,10,1,10,1,10,1,14,1,2,1,14,1,16,4, - 10,1,18,1,8,1,22,1,114,185,0,0,0,99,1,0, + 114,151,0,0,0,6,3,0,0,115,42,0,0,0,8,7, + 4,2,12,9,2,1,2,0,2,255,12,7,2,1,2,255, + 12,9,12,4,12,9,12,9,2,1,2,255,12,6,2,1, + 2,255,12,6,2,1,2,255,114,151,0,0,0,99,0,0, + 0,0,0,0,0,0,0,0,0,0,2,0,0,0,64,0, + 0,0,115,32,0,0,0,101,0,90,1,100,0,90,2,100, + 1,90,3,100,2,100,3,132,0,90,4,100,4,100,5,132, + 0,90,5,100,6,83,0,41,7,218,18,95,73,109,112,111, + 114,116,76,111,99,107,67,111,110,116,101,120,116,122,36,67, + 111,110,116,101,120,116,32,109,97,110,97,103,101,114,32,102, + 111,114,32,116,104,101,32,105,109,112,111,114,116,32,108,111, + 99,107,46,99,1,0,0,0,0,0,0,0,1,0,0,0, + 2,0,0,0,67,0,0,0,115,12,0,0,0,116,0,160, + 1,161,0,1,0,100,1,83,0,41,2,122,24,65,99,113, + 117,105,114,101,32,116,104,101,32,105,109,112,111,114,116,32, + 108,111,99,107,46,78,41,2,114,49,0,0,0,114,50,0, + 0,0,41,1,114,26,0,0,0,114,10,0,0,0,114,10, + 0,0,0,114,11,0,0,0,114,46,0,0,0,87,3,0, + 0,115,2,0,0,0,0,2,122,28,95,73,109,112,111,114, + 116,76,111,99,107,67,111,110,116,101,120,116,46,95,95,101, + 110,116,101,114,95,95,99,4,0,0,0,0,0,0,0,4, + 0,0,0,2,0,0,0,67,0,0,0,115,12,0,0,0, + 116,0,160,1,161,0,1,0,100,1,83,0,41,2,122,60, + 82,101,108,101,97,115,101,32,116,104,101,32,105,109,112,111, + 114,116,32,108,111,99,107,32,114,101,103,97,114,100,108,101, + 115,115,32,111,102,32,97,110,121,32,114,97,105,115,101,100, + 32,101,120,99,101,112,116,105,111,110,115,46,78,41,2,114, + 49,0,0,0,114,52,0,0,0,41,4,114,26,0,0,0, + 90,8,101,120,99,95,116,121,112,101,90,9,101,120,99,95, + 118,97,108,117,101,90,13,101,120,99,95,116,114,97,99,101, + 98,97,99,107,114,10,0,0,0,114,10,0,0,0,114,11, + 0,0,0,114,48,0,0,0,91,3,0,0,115,2,0,0, + 0,0,2,122,27,95,73,109,112,111,114,116,76,111,99,107, + 67,111,110,116,101,120,116,46,95,95,101,120,105,116,95,95, + 78,41,6,114,1,0,0,0,114,0,0,0,0,114,2,0, + 0,0,114,3,0,0,0,114,46,0,0,0,114,48,0,0, + 0,114,10,0,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,114,156,0,0,0,83,3,0,0,115,6, + 0,0,0,8,2,4,2,8,4,114,156,0,0,0,99,3, + 0,0,0,0,0,0,0,5,0,0,0,5,0,0,0,67, + 0,0,0,115,64,0,0,0,124,1,160,0,100,1,124,2, + 100,2,24,0,161,2,125,3,116,1,124,3,131,1,124,2, + 107,0,114,36,116,2,100,3,131,1,130,1,124,3,100,4, + 25,0,125,4,124,0,114,60,100,5,160,3,124,4,124,0, + 161,2,83,0,124,4,83,0,41,6,122,50,82,101,115,111, + 108,118,101,32,97,32,114,101,108,97,116,105,118,101,32,109, + 111,100,117,108,101,32,110,97,109,101,32,116,111,32,97,110, + 32,97,98,115,111,108,117,116,101,32,111,110,101,46,114,117, + 0,0,0,114,33,0,0,0,122,50,97,116,116,101,109,112, + 116,101,100,32,114,101,108,97,116,105,118,101,32,105,109,112, + 111,114,116,32,98,101,121,111,110,100,32,116,111,112,45,108, + 101,118,101,108,32,112,97,99,107,97,103,101,114,19,0,0, + 0,122,5,123,125,46,123,125,41,4,218,6,114,115,112,108, + 105,116,218,3,108,101,110,218,10,86,97,108,117,101,69,114, + 114,111,114,114,38,0,0,0,41,5,114,15,0,0,0,218, + 7,112,97,99,107,97,103,101,218,5,108,101,118,101,108,90, + 4,98,105,116,115,90,4,98,97,115,101,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,218,13,95,114,101,115, + 111,108,118,101,95,110,97,109,101,96,3,0,0,115,10,0, + 0,0,0,2,16,1,12,1,8,1,8,1,114,162,0,0, + 0,99,3,0,0,0,0,0,0,0,4,0,0,0,4,0, + 0,0,67,0,0,0,115,34,0,0,0,124,0,160,0,124, + 1,124,2,161,2,125,3,124,3,100,0,107,8,114,24,100, + 0,83,0,116,1,124,1,124,3,131,2,83,0,41,1,78, + 41,2,114,146,0,0,0,114,78,0,0,0,41,4,218,6, + 102,105,110,100,101,114,114,15,0,0,0,114,143,0,0,0, + 114,93,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,218,17,95,102,105,110,100,95,115,112,101,99, + 95,108,101,103,97,99,121,105,3,0,0,115,8,0,0,0, + 0,3,12,1,8,1,4,1,114,164,0,0,0,99,3,0, + 0,0,0,0,0,0,10,0,0,0,10,0,0,0,67,0, + 0,0,115,12,1,0,0,116,0,106,1,125,3,124,3,100, + 1,107,8,114,22,116,2,100,2,131,1,130,1,124,3,115, + 38,116,3,160,4,100,3,116,5,161,2,1,0,124,0,116, + 0,106,6,107,6,125,4,124,3,68,0,93,210,125,5,116, + 7,131,0,143,84,1,0,122,10,124,5,106,8,125,6,87, + 0,110,54,4,0,116,9,107,10,114,128,1,0,1,0,1, + 0,116,10,124,5,124,0,124,1,131,3,125,7,124,7,100, + 1,107,8,114,124,89,0,87,0,53,0,81,0,82,0,163, + 0,113,52,89,0,110,14,88,0,124,6,124,0,124,1,124, + 2,131,3,125,7,87,0,53,0,81,0,82,0,88,0,124, + 7,100,1,107,9,114,52,124,4,144,0,115,254,124,0,116, + 0,106,6,107,6,144,0,114,254,116,0,106,6,124,0,25, + 0,125,8,122,10,124,8,106,11,125,9,87,0,110,28,4, + 0,116,9,107,10,114,226,1,0,1,0,1,0,124,7,6, + 0,89,0,2,0,1,0,83,0,88,0,124,9,100,1,107, + 8,114,244,124,7,2,0,1,0,83,0,124,9,2,0,1, + 0,83,0,113,52,124,7,2,0,1,0,83,0,113,52,100, + 1,83,0,41,4,122,21,70,105,110,100,32,97,32,109,111, + 100,117,108,101,39,115,32,115,112,101,99,46,78,122,53,115, + 121,115,46,109,101,116,97,95,112,97,116,104,32,105,115,32, + 78,111,110,101,44,32,80,121,116,104,111,110,32,105,115,32, + 108,105,107,101,108,121,32,115,104,117,116,116,105,110,103,32, + 100,111,119,110,122,22,115,121,115,46,109,101,116,97,95,112, + 97,116,104,32,105,115,32,101,109,112,116,121,41,12,114,14, + 0,0,0,218,9,109,101,116,97,95,112,97,116,104,114,70, + 0,0,0,218,9,95,119,97,114,110,105,110,103,115,218,4, + 119,97,114,110,218,13,73,109,112,111,114,116,87,97,114,110, + 105,110,103,114,79,0,0,0,114,156,0,0,0,114,145,0, + 0,0,114,90,0,0,0,114,164,0,0,0,114,89,0,0, + 0,41,10,114,15,0,0,0,114,143,0,0,0,114,144,0, + 0,0,114,165,0,0,0,90,9,105,115,95,114,101,108,111, + 97,100,114,163,0,0,0,114,145,0,0,0,114,82,0,0, + 0,114,83,0,0,0,114,89,0,0,0,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,218,10,95,102,105,110, + 100,95,115,112,101,99,114,3,0,0,115,54,0,0,0,0, + 2,6,1,8,2,8,3,4,1,12,5,10,1,8,1,8, + 1,2,1,10,1,14,1,12,1,8,1,20,2,22,1,8, + 2,18,1,10,1,2,1,10,1,14,4,14,2,8,1,8, + 2,10,2,10,2,114,169,0,0,0,99,3,0,0,0,0, + 0,0,0,3,0,0,0,5,0,0,0,67,0,0,0,115, + 108,0,0,0,116,0,124,0,116,1,131,2,115,28,116,2, + 100,1,160,3,116,4,124,0,131,1,161,1,131,1,130,1, + 124,2,100,2,107,0,114,44,116,5,100,3,131,1,130,1, + 124,2,100,2,107,4,114,84,116,0,124,1,116,1,131,2, + 115,72,116,2,100,4,131,1,130,1,110,12,124,1,115,84, + 116,6,100,5,131,1,130,1,124,0,115,104,124,2,100,2, + 107,2,114,104,116,5,100,6,131,1,130,1,100,7,83,0, + 41,8,122,28,86,101,114,105,102,121,32,97,114,103,117,109, + 101,110,116,115,32,97,114,101,32,34,115,97,110,101,34,46, + 122,31,109,111,100,117,108,101,32,110,97,109,101,32,109,117, + 115,116,32,98,101,32,115,116,114,44,32,110,111,116,32,123, + 125,114,19,0,0,0,122,18,108,101,118,101,108,32,109,117, + 115,116,32,98,101,32,62,61,32,48,122,31,95,95,112,97, + 99,107,97,103,101,95,95,32,110,111,116,32,115,101,116,32, + 116,111,32,97,32,115,116,114,105,110,103,122,54,97,116,116, + 101,109,112,116,101,100,32,114,101,108,97,116,105,118,101,32, + 105,109,112,111,114,116,32,119,105,116,104,32,110,111,32,107, + 110,111,119,110,32,112,97,114,101,110,116,32,112,97,99,107, + 97,103,101,122,17,69,109,112,116,121,32,109,111,100,117,108, + 101,32,110,97,109,101,78,41,7,218,10,105,115,105,110,115, + 116,97,110,99,101,218,3,115,116,114,218,9,84,121,112,101, + 69,114,114,111,114,114,38,0,0,0,114,13,0,0,0,114, + 159,0,0,0,114,70,0,0,0,41,3,114,15,0,0,0, + 114,160,0,0,0,114,161,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,218,13,95,115,97,110,105, + 116,121,95,99,104,101,99,107,161,3,0,0,115,22,0,0, + 0,0,2,10,1,18,1,8,1,8,1,8,1,10,1,10, + 1,4,1,8,2,12,1,114,173,0,0,0,122,16,78,111, + 32,109,111,100,117,108,101,32,110,97,109,101,100,32,122,4, + 123,33,114,125,99,2,0,0,0,0,0,0,0,8,0,0, + 0,8,0,0,0,67,0,0,0,115,220,0,0,0,100,0, + 125,2,124,0,160,0,100,1,161,1,100,2,25,0,125,3, + 124,3,114,134,124,3,116,1,106,2,107,7,114,42,116,3, + 124,1,124,3,131,2,1,0,124,0,116,1,106,2,107,6, + 114,62,116,1,106,2,124,0,25,0,83,0,116,1,106,2, + 124,3,25,0,125,4,122,10,124,4,106,4,125,2,87,0, + 110,50,4,0,116,5,107,10,114,132,1,0,1,0,1,0, + 116,6,100,3,23,0,160,7,124,0,124,3,161,2,125,5, + 116,8,124,5,124,0,100,4,141,2,100,0,130,2,89,0, + 110,2,88,0,116,9,124,0,124,2,131,2,125,6,124,6, + 100,0,107,8,114,172,116,8,116,6,160,7,124,0,161,1, + 124,0,100,4,141,2,130,1,110,8,116,10,124,6,131,1, + 125,7,124,3,114,216,116,1,106,2,124,3,25,0,125,4, + 116,11,124,4,124,0,160,0,100,1,161,1,100,5,25,0, + 124,7,131,3,1,0,124,7,83,0,41,6,78,114,117,0, + 0,0,114,19,0,0,0,122,23,59,32,123,33,114,125,32, + 105,115,32,110,111,116,32,97,32,112,97,99,107,97,103,101, + 41,1,114,15,0,0,0,233,2,0,0,0,41,12,114,118, + 0,0,0,114,14,0,0,0,114,79,0,0,0,114,59,0, + 0,0,114,127,0,0,0,114,90,0,0,0,218,8,95,69, + 82,82,95,77,83,71,114,38,0,0,0,218,19,77,111,100, + 117,108,101,78,111,116,70,111,117,110,100,69,114,114,111,114, + 114,169,0,0,0,114,140,0,0,0,114,5,0,0,0,41, + 8,114,15,0,0,0,218,7,105,109,112,111,114,116,95,114, + 143,0,0,0,114,119,0,0,0,90,13,112,97,114,101,110, + 116,95,109,111,100,117,108,101,114,138,0,0,0,114,82,0, + 0,0,114,83,0,0,0,114,10,0,0,0,114,10,0,0, + 0,114,11,0,0,0,218,23,95,102,105,110,100,95,97,110, + 100,95,108,111,97,100,95,117,110,108,111,99,107,101,100,180, + 3,0,0,115,42,0,0,0,0,1,4,1,14,1,4,1, + 10,1,10,2,10,1,10,1,10,1,2,1,10,1,14,1, + 16,1,20,1,10,1,8,1,20,2,8,1,4,2,10,1, + 22,1,114,178,0,0,0,99,2,0,0,0,0,0,0,0, + 4,0,0,0,10,0,0,0,67,0,0,0,115,106,0,0, + 0,116,0,124,0,131,1,143,50,1,0,116,1,106,2,160, + 3,124,0,116,4,161,2,125,2,124,2,116,4,107,8,114, + 54,116,5,124,0,124,1,131,2,87,0,2,0,53,0,81, + 0,82,0,163,0,83,0,87,0,53,0,81,0,82,0,88, + 0,124,2,100,1,107,8,114,94,100,2,160,6,124,0,161, + 1,125,3,116,7,124,3,124,0,100,3,141,2,130,1,116, + 8,124,0,131,1,1,0,124,2,83,0,41,4,122,25,70, + 105,110,100,32,97,110,100,32,108,111,97,100,32,116,104,101, + 32,109,111,100,117,108,101,46,78,122,40,105,109,112,111,114, + 116,32,111,102,32,123,125,32,104,97,108,116,101,100,59,32, + 78,111,110,101,32,105,110,32,115,121,115,46,109,111,100,117, + 108,101,115,41,1,114,15,0,0,0,41,9,114,42,0,0, + 0,114,14,0,0,0,114,79,0,0,0,114,30,0,0,0, + 218,14,95,78,69,69,68,83,95,76,79,65,68,73,78,71, + 114,178,0,0,0,114,38,0,0,0,114,176,0,0,0,114, + 57,0,0,0,41,4,114,15,0,0,0,114,177,0,0,0, + 114,83,0,0,0,114,67,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,218,14,95,102,105,110,100, + 95,97,110,100,95,108,111,97,100,210,3,0,0,115,22,0, + 0,0,0,2,10,1,14,1,8,1,32,2,8,1,4,1, + 2,255,4,2,12,2,8,1,114,180,0,0,0,114,19,0, + 0,0,99,3,0,0,0,0,0,0,0,3,0,0,0,4, + 0,0,0,67,0,0,0,115,42,0,0,0,116,0,124,0, + 124,1,124,2,131,3,1,0,124,2,100,1,107,4,114,32, + 116,1,124,0,124,1,124,2,131,3,125,0,116,2,124,0, + 116,3,131,2,83,0,41,2,97,50,1,0,0,73,109,112, + 111,114,116,32,97,110,100,32,114,101,116,117,114,110,32,116, + 104,101,32,109,111,100,117,108,101,32,98,97,115,101,100,32, + 111,110,32,105,116,115,32,110,97,109,101,44,32,116,104,101, + 32,112,97,99,107,97,103,101,32,116,104,101,32,99,97,108, + 108,32,105,115,10,32,32,32,32,98,101,105,110,103,32,109, + 97,100,101,32,102,114,111,109,44,32,97,110,100,32,116,104, + 101,32,108,101,118,101,108,32,97,100,106,117,115,116,109,101, + 110,116,46,10,10,32,32,32,32,84,104,105,115,32,102,117, + 110,99,116,105,111,110,32,114,101,112,114,101,115,101,110,116, + 115,32,116,104,101,32,103,114,101,97,116,101,115,116,32,99, + 111,109,109,111,110,32,100,101,110,111,109,105,110,97,116,111, + 114,32,111,102,32,102,117,110,99,116,105,111,110,97,108,105, + 116,121,10,32,32,32,32,98,101,116,119,101,101,110,32,105, + 109,112,111,114,116,95,109,111,100,117,108,101,32,97,110,100, + 32,95,95,105,109,112,111,114,116,95,95,46,32,84,104,105, + 115,32,105,110,99,108,117,100,101,115,32,115,101,116,116,105, + 110,103,32,95,95,112,97,99,107,97,103,101,95,95,32,105, + 102,10,32,32,32,32,116,104,101,32,108,111,97,100,101,114, + 32,100,105,100,32,110,111,116,46,10,10,32,32,32,32,114, + 19,0,0,0,41,4,114,173,0,0,0,114,162,0,0,0, + 114,180,0,0,0,218,11,95,103,99,100,95,105,109,112,111, + 114,116,41,3,114,15,0,0,0,114,160,0,0,0,114,161, + 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,114,181,0,0,0,226,3,0,0,115,8,0,0,0, + 0,9,12,1,8,1,12,1,114,181,0,0,0,41,1,218, + 9,114,101,99,117,114,115,105,118,101,99,3,0,0,0,1, + 0,0,0,8,0,0,0,11,0,0,0,67,0,0,0,115, + 226,0,0,0,124,1,68,0,93,216,125,4,116,0,124,4, + 116,1,131,2,115,66,124,3,114,34,124,0,106,2,100,1, + 23,0,125,5,110,4,100,2,125,5,116,3,100,3,124,5, + 155,0,100,4,116,4,124,4,131,1,106,2,155,0,157,4, + 131,1,130,1,110,154,124,4,100,5,107,2,114,108,124,3, + 115,106,116,5,124,0,100,6,131,2,114,106,116,6,124,0, + 124,0,106,7,124,2,100,7,100,8,141,4,1,0,110,112, + 116,5,124,0,124,4,131,2,115,220,100,9,160,8,124,0, + 106,2,124,4,161,2,125,6,122,14,116,9,124,2,124,6, + 131,2,1,0,87,0,110,72,4,0,116,10,107,10,114,218, + 1,0,125,7,1,0,122,42,124,7,106,11,124,6,107,2, + 114,200,116,12,106,13,160,14,124,6,116,15,161,2,100,10, + 107,9,114,200,87,0,89,0,162,8,113,4,130,0,87,0, + 53,0,100,10,125,7,126,7,88,0,89,0,110,2,88,0, + 113,4,124,0,83,0,41,11,122,238,70,105,103,117,114,101, + 32,111,117,116,32,119,104,97,116,32,95,95,105,109,112,111, + 114,116,95,95,32,115,104,111,117,108,100,32,114,101,116,117, + 114,110,46,10,10,32,32,32,32,84,104,101,32,105,109,112, + 111,114,116,95,32,112,97,114,97,109,101,116,101,114,32,105, + 115,32,97,32,99,97,108,108,97,98,108,101,32,119,104,105, + 99,104,32,116,97,107,101,115,32,116,104,101,32,110,97,109, + 101,32,111,102,32,109,111,100,117,108,101,32,116,111,10,32, + 32,32,32,105,109,112,111,114,116,46,32,73,116,32,105,115, + 32,114,101,113,117,105,114,101,100,32,116,111,32,100,101,99, + 111,117,112,108,101,32,116,104,101,32,102,117,110,99,116,105, + 111,110,32,102,114,111,109,32,97,115,115,117,109,105,110,103, + 32,105,109,112,111,114,116,108,105,98,39,115,10,32,32,32, + 32,105,109,112,111,114,116,32,105,109,112,108,101,109,101,110, + 116,97,116,105,111,110,32,105,115,32,100,101,115,105,114,101, + 100,46,10,10,32,32,32,32,122,8,46,95,95,97,108,108, + 95,95,122,13,96,96,102,114,111,109,32,108,105,115,116,39, + 39,122,8,73,116,101,109,32,105,110,32,122,18,32,109,117, + 115,116,32,98,101,32,115,116,114,44,32,110,111,116,32,250, + 1,42,218,7,95,95,97,108,108,95,95,84,41,1,114,182, + 0,0,0,122,5,123,125,46,123,125,78,41,16,114,170,0, + 0,0,114,171,0,0,0,114,1,0,0,0,114,172,0,0, + 0,114,13,0,0,0,114,4,0,0,0,218,16,95,104,97, + 110,100,108,101,95,102,114,111,109,108,105,115,116,114,184,0, + 0,0,114,38,0,0,0,114,59,0,0,0,114,176,0,0, + 0,114,15,0,0,0,114,14,0,0,0,114,79,0,0,0, + 114,30,0,0,0,114,179,0,0,0,41,8,114,83,0,0, + 0,218,8,102,114,111,109,108,105,115,116,114,177,0,0,0, + 114,182,0,0,0,218,1,120,90,5,119,104,101,114,101,90, + 9,102,114,111,109,95,110,97,109,101,90,3,101,120,99,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,185, + 0,0,0,241,3,0,0,115,44,0,0,0,0,10,8,1, + 10,1,4,1,12,2,4,1,28,2,8,1,14,1,10,1, + 2,255,8,2,10,1,14,1,2,1,14,1,16,4,10,1, + 16,255,2,2,8,1,22,1,114,185,0,0,0,99,1,0, 0,0,0,0,0,0,3,0,0,0,6,0,0,0,67,0, 0,0,115,146,0,0,0,124,0,160,0,100,1,161,1,125, 1,124,0,160,0,100,2,161,1,125,2,124,1,100,3,107, @@ -1655,175 +1659,176 @@ const unsigned char _Py_M__importlib[] = { 103,108,111,98,97,108,115,114,160,0,0,0,114,82,0,0, 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, 218,17,95,99,97,108,99,95,95,95,112,97,99,107,97,103, - 101,95,95,22,4,0,0,115,30,0,0,0,0,7,10,1, - 10,1,8,1,18,1,22,2,10,1,4,1,8,1,6,2, - 6,2,10,1,8,1,8,1,14,1,114,191,0,0,0,114, - 10,0,0,0,99,5,0,0,0,0,0,0,0,9,0,0, - 0,5,0,0,0,67,0,0,0,115,180,0,0,0,124,4, - 100,1,107,2,114,18,116,0,124,0,131,1,125,5,110,36, - 124,1,100,2,107,9,114,30,124,1,110,2,105,0,125,6, - 116,1,124,6,131,1,125,7,116,0,124,0,124,7,124,4, - 131,3,125,5,124,3,115,150,124,4,100,1,107,2,114,84, - 116,0,124,0,160,2,100,3,161,1,100,1,25,0,131,1, - 83,0,124,0,115,92,124,5,83,0,116,3,124,0,131,1, - 116,3,124,0,160,2,100,3,161,1,100,1,25,0,131,1, - 24,0,125,8,116,4,106,5,124,5,106,6,100,2,116,3, - 124,5,106,6,131,1,124,8,24,0,133,2,25,0,25,0, - 83,0,110,26,116,7,124,5,100,4,131,2,114,172,116,8, - 124,5,124,3,116,0,131,3,83,0,124,5,83,0,100,2, - 83,0,41,5,97,215,1,0,0,73,109,112,111,114,116,32, - 97,32,109,111,100,117,108,101,46,10,10,32,32,32,32,84, - 104,101,32,39,103,108,111,98,97,108,115,39,32,97,114,103, - 117,109,101,110,116,32,105,115,32,117,115,101,100,32,116,111, - 32,105,110,102,101,114,32,119,104,101,114,101,32,116,104,101, - 32,105,109,112,111,114,116,32,105,115,32,111,99,99,117,114, - 114,105,110,103,32,102,114,111,109,10,32,32,32,32,116,111, - 32,104,97,110,100,108,101,32,114,101,108,97,116,105,118,101, - 32,105,109,112,111,114,116,115,46,32,84,104,101,32,39,108, - 111,99,97,108,115,39,32,97,114,103,117,109,101,110,116,32, - 105,115,32,105,103,110,111,114,101,100,46,32,84,104,101,10, - 32,32,32,32,39,102,114,111,109,108,105,115,116,39,32,97, - 114,103,117,109,101,110,116,32,115,112,101,99,105,102,105,101, - 115,32,119,104,97,116,32,115,104,111,117,108,100,32,101,120, - 105,115,116,32,97,115,32,97,116,116,114,105,98,117,116,101, - 115,32,111,110,32,116,104,101,32,109,111,100,117,108,101,10, - 32,32,32,32,98,101,105,110,103,32,105,109,112,111,114,116, - 101,100,32,40,101,46,103,46,32,96,96,102,114,111,109,32, - 109,111,100,117,108,101,32,105,109,112,111,114,116,32,60,102, - 114,111,109,108,105,115,116,62,96,96,41,46,32,32,84,104, - 101,32,39,108,101,118,101,108,39,10,32,32,32,32,97,114, - 103,117,109,101,110,116,32,114,101,112,114,101,115,101,110,116, - 115,32,116,104,101,32,112,97,99,107,97,103,101,32,108,111, - 99,97,116,105,111,110,32,116,111,32,105,109,112,111,114,116, - 32,102,114,111,109,32,105,110,32,97,32,114,101,108,97,116, - 105,118,101,10,32,32,32,32,105,109,112,111,114,116,32,40, - 101,46,103,46,32,96,96,102,114,111,109,32,46,46,112,107, - 103,32,105,109,112,111,114,116,32,109,111,100,96,96,32,119, - 111,117,108,100,32,104,97,118,101,32,97,32,39,108,101,118, - 101,108,39,32,111,102,32,50,41,46,10,10,32,32,32,32, - 114,19,0,0,0,78,114,117,0,0,0,114,127,0,0,0, - 41,9,114,181,0,0,0,114,191,0,0,0,218,9,112,97, - 114,116,105,116,105,111,110,114,158,0,0,0,114,14,0,0, - 0,114,79,0,0,0,114,1,0,0,0,114,4,0,0,0, - 114,185,0,0,0,41,9,114,15,0,0,0,114,190,0,0, - 0,218,6,108,111,99,97,108,115,114,186,0,0,0,114,161, - 0,0,0,114,83,0,0,0,90,8,103,108,111,98,97,108, - 115,95,114,160,0,0,0,90,7,99,117,116,95,111,102,102, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 10,95,95,105,109,112,111,114,116,95,95,49,4,0,0,115, - 30,0,0,0,0,11,8,1,10,2,16,1,8,1,12,1, - 4,3,8,1,18,1,4,1,4,4,26,3,32,1,10,1, - 12,2,114,194,0,0,0,99,1,0,0,0,0,0,0,0, + 101,95,95,22,4,0,0,115,38,0,0,0,0,7,10,1, + 10,1,8,1,18,1,22,2,2,0,2,254,6,3,4,1, + 8,1,6,2,6,2,2,0,2,254,6,3,8,1,8,1, + 14,1,114,191,0,0,0,114,10,0,0,0,99,5,0,0, + 0,0,0,0,0,9,0,0,0,5,0,0,0,67,0,0, + 0,115,180,0,0,0,124,4,100,1,107,2,114,18,116,0, + 124,0,131,1,125,5,110,36,124,1,100,2,107,9,114,30, + 124,1,110,2,105,0,125,6,116,1,124,6,131,1,125,7, + 116,0,124,0,124,7,124,4,131,3,125,5,124,3,115,150, + 124,4,100,1,107,2,114,84,116,0,124,0,160,2,100,3, + 161,1,100,1,25,0,131,1,83,0,124,0,115,92,124,5, + 83,0,116,3,124,0,131,1,116,3,124,0,160,2,100,3, + 161,1,100,1,25,0,131,1,24,0,125,8,116,4,106,5, + 124,5,106,6,100,2,116,3,124,5,106,6,131,1,124,8, + 24,0,133,2,25,0,25,0,83,0,110,26,116,7,124,5, + 100,4,131,2,114,172,116,8,124,5,124,3,116,0,131,3, + 83,0,124,5,83,0,100,2,83,0,41,5,97,215,1,0, + 0,73,109,112,111,114,116,32,97,32,109,111,100,117,108,101, + 46,10,10,32,32,32,32,84,104,101,32,39,103,108,111,98, + 97,108,115,39,32,97,114,103,117,109,101,110,116,32,105,115, + 32,117,115,101,100,32,116,111,32,105,110,102,101,114,32,119, + 104,101,114,101,32,116,104,101,32,105,109,112,111,114,116,32, + 105,115,32,111,99,99,117,114,114,105,110,103,32,102,114,111, + 109,10,32,32,32,32,116,111,32,104,97,110,100,108,101,32, + 114,101,108,97,116,105,118,101,32,105,109,112,111,114,116,115, + 46,32,84,104,101,32,39,108,111,99,97,108,115,39,32,97, + 114,103,117,109,101,110,116,32,105,115,32,105,103,110,111,114, + 101,100,46,32,84,104,101,10,32,32,32,32,39,102,114,111, + 109,108,105,115,116,39,32,97,114,103,117,109,101,110,116,32, + 115,112,101,99,105,102,105,101,115,32,119,104,97,116,32,115, + 104,111,117,108,100,32,101,120,105,115,116,32,97,115,32,97, + 116,116,114,105,98,117,116,101,115,32,111,110,32,116,104,101, + 32,109,111,100,117,108,101,10,32,32,32,32,98,101,105,110, + 103,32,105,109,112,111,114,116,101,100,32,40,101,46,103,46, + 32,96,96,102,114,111,109,32,109,111,100,117,108,101,32,105, + 109,112,111,114,116,32,60,102,114,111,109,108,105,115,116,62, + 96,96,41,46,32,32,84,104,101,32,39,108,101,118,101,108, + 39,10,32,32,32,32,97,114,103,117,109,101,110,116,32,114, + 101,112,114,101,115,101,110,116,115,32,116,104,101,32,112,97, + 99,107,97,103,101,32,108,111,99,97,116,105,111,110,32,116, + 111,32,105,109,112,111,114,116,32,102,114,111,109,32,105,110, + 32,97,32,114,101,108,97,116,105,118,101,10,32,32,32,32, + 105,109,112,111,114,116,32,40,101,46,103,46,32,96,96,102, + 114,111,109,32,46,46,112,107,103,32,105,109,112,111,114,116, + 32,109,111,100,96,96,32,119,111,117,108,100,32,104,97,118, + 101,32,97,32,39,108,101,118,101,108,39,32,111,102,32,50, + 41,46,10,10,32,32,32,32,114,19,0,0,0,78,114,117, + 0,0,0,114,127,0,0,0,41,9,114,181,0,0,0,114, + 191,0,0,0,218,9,112,97,114,116,105,116,105,111,110,114, + 158,0,0,0,114,14,0,0,0,114,79,0,0,0,114,1, + 0,0,0,114,4,0,0,0,114,185,0,0,0,41,9,114, + 15,0,0,0,114,190,0,0,0,218,6,108,111,99,97,108, + 115,114,186,0,0,0,114,161,0,0,0,114,83,0,0,0, + 90,8,103,108,111,98,97,108,115,95,114,160,0,0,0,90, + 7,99,117,116,95,111,102,102,114,10,0,0,0,114,10,0, + 0,0,114,11,0,0,0,218,10,95,95,105,109,112,111,114, + 116,95,95,49,4,0,0,115,30,0,0,0,0,11,8,1, + 10,2,16,1,8,1,12,1,4,3,8,1,18,1,4,1, + 4,4,26,3,32,1,10,1,12,2,114,194,0,0,0,99, + 1,0,0,0,0,0,0,0,2,0,0,0,3,0,0,0, + 67,0,0,0,115,38,0,0,0,116,0,160,1,124,0,161, + 1,125,1,124,1,100,0,107,8,114,30,116,2,100,1,124, + 0,23,0,131,1,130,1,116,3,124,1,131,1,83,0,41, + 2,78,122,25,110,111,32,98,117,105,108,116,45,105,110,32, + 109,111,100,117,108,101,32,110,97,109,101,100,32,41,4,114, + 141,0,0,0,114,145,0,0,0,114,70,0,0,0,114,140, + 0,0,0,41,2,114,15,0,0,0,114,82,0,0,0,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,18, + 95,98,117,105,108,116,105,110,95,102,114,111,109,95,110,97, + 109,101,86,4,0,0,115,8,0,0,0,0,1,10,1,8, + 1,12,1,114,195,0,0,0,99,2,0,0,0,0,0,0, + 0,10,0,0,0,5,0,0,0,67,0,0,0,115,166,0, + 0,0,124,1,97,0,124,0,97,1,116,2,116,1,131,1, + 125,2,116,1,106,3,160,4,161,0,68,0,93,72,92,2, + 125,3,125,4,116,5,124,4,124,2,131,2,114,26,124,3, + 116,1,106,6,107,6,114,60,116,7,125,5,110,18,116,0, + 160,8,124,3,161,1,114,26,116,9,125,5,110,2,113,26, + 116,10,124,4,124,5,131,2,125,6,116,11,124,6,124,4, + 131,2,1,0,113,26,116,1,106,3,116,12,25,0,125,7, + 100,1,68,0,93,46,125,8,124,8,116,1,106,3,107,7, + 114,138,116,13,124,8,131,1,125,9,110,10,116,1,106,3, + 124,8,25,0,125,9,116,14,124,7,124,8,124,9,131,3, + 1,0,113,114,100,2,83,0,41,3,122,250,83,101,116,117, + 112,32,105,109,112,111,114,116,108,105,98,32,98,121,32,105, + 109,112,111,114,116,105,110,103,32,110,101,101,100,101,100,32, + 98,117,105,108,116,45,105,110,32,109,111,100,117,108,101,115, + 32,97,110,100,32,105,110,106,101,99,116,105,110,103,32,116, + 104,101,109,10,32,32,32,32,105,110,116,111,32,116,104,101, + 32,103,108,111,98,97,108,32,110,97,109,101,115,112,97,99, + 101,46,10,10,32,32,32,32,65,115,32,115,121,115,32,105, + 115,32,110,101,101,100,101,100,32,102,111,114,32,115,121,115, + 46,109,111,100,117,108,101,115,32,97,99,99,101,115,115,32, + 97,110,100,32,95,105,109,112,32,105,115,32,110,101,101,100, + 101,100,32,116,111,32,108,111,97,100,32,98,117,105,108,116, + 45,105,110,10,32,32,32,32,109,111,100,117,108,101,115,44, + 32,116,104,111,115,101,32,116,119,111,32,109,111,100,117,108, + 101,115,32,109,117,115,116,32,98,101,32,101,120,112,108,105, + 99,105,116,108,121,32,112,97,115,115,101,100,32,105,110,46, + 10,10,32,32,32,32,41,3,114,20,0,0,0,114,166,0, + 0,0,114,56,0,0,0,78,41,15,114,49,0,0,0,114, + 14,0,0,0,114,13,0,0,0,114,79,0,0,0,218,5, + 105,116,101,109,115,114,170,0,0,0,114,69,0,0,0,114, + 141,0,0,0,114,75,0,0,0,114,151,0,0,0,114,128, + 0,0,0,114,133,0,0,0,114,1,0,0,0,114,195,0, + 0,0,114,5,0,0,0,41,10,218,10,115,121,115,95,109, + 111,100,117,108,101,218,11,95,105,109,112,95,109,111,100,117, + 108,101,90,11,109,111,100,117,108,101,95,116,121,112,101,114, + 15,0,0,0,114,83,0,0,0,114,93,0,0,0,114,82, + 0,0,0,90,11,115,101,108,102,95,109,111,100,117,108,101, + 90,12,98,117,105,108,116,105,110,95,110,97,109,101,90,14, + 98,117,105,108,116,105,110,95,109,111,100,117,108,101,114,10, + 0,0,0,114,10,0,0,0,114,11,0,0,0,218,6,95, + 115,101,116,117,112,93,4,0,0,115,36,0,0,0,0,9, + 4,1,4,3,8,1,18,1,10,1,10,1,6,1,10,1, + 6,2,2,1,10,1,12,3,10,1,8,1,10,1,10,2, + 10,1,114,199,0,0,0,99,2,0,0,0,0,0,0,0, 2,0,0,0,3,0,0,0,67,0,0,0,115,38,0,0, - 0,116,0,160,1,124,0,161,1,125,1,124,1,100,0,107, - 8,114,30,116,2,100,1,124,0,23,0,131,1,130,1,116, - 3,124,1,131,1,83,0,41,2,78,122,25,110,111,32,98, - 117,105,108,116,45,105,110,32,109,111,100,117,108,101,32,110, - 97,109,101,100,32,41,4,114,141,0,0,0,114,145,0,0, - 0,114,70,0,0,0,114,140,0,0,0,41,2,114,15,0, - 0,0,114,82,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,218,18,95,98,117,105,108,116,105,110, - 95,102,114,111,109,95,110,97,109,101,86,4,0,0,115,8, - 0,0,0,0,1,10,1,8,1,12,1,114,195,0,0,0, - 99,2,0,0,0,0,0,0,0,10,0,0,0,5,0,0, - 0,67,0,0,0,115,166,0,0,0,124,1,97,0,124,0, - 97,1,116,2,116,1,131,1,125,2,116,1,106,3,160,4, - 161,0,68,0,93,72,92,2,125,3,125,4,116,5,124,4, - 124,2,131,2,114,26,124,3,116,1,106,6,107,6,114,60, - 116,7,125,5,110,18,116,0,160,8,124,3,161,1,114,26, - 116,9,125,5,110,2,113,26,116,10,124,4,124,5,131,2, - 125,6,116,11,124,6,124,4,131,2,1,0,113,26,116,1, - 106,3,116,12,25,0,125,7,100,1,68,0,93,46,125,8, - 124,8,116,1,106,3,107,7,114,138,116,13,124,8,131,1, - 125,9,110,10,116,1,106,3,124,8,25,0,125,9,116,14, - 124,7,124,8,124,9,131,3,1,0,113,114,100,2,83,0, - 41,3,122,250,83,101,116,117,112,32,105,109,112,111,114,116, - 108,105,98,32,98,121,32,105,109,112,111,114,116,105,110,103, - 32,110,101,101,100,101,100,32,98,117,105,108,116,45,105,110, - 32,109,111,100,117,108,101,115,32,97,110,100,32,105,110,106, - 101,99,116,105,110,103,32,116,104,101,109,10,32,32,32,32, - 105,110,116,111,32,116,104,101,32,103,108,111,98,97,108,32, - 110,97,109,101,115,112,97,99,101,46,10,10,32,32,32,32, - 65,115,32,115,121,115,32,105,115,32,110,101,101,100,101,100, - 32,102,111,114,32,115,121,115,46,109,111,100,117,108,101,115, - 32,97,99,99,101,115,115,32,97,110,100,32,95,105,109,112, - 32,105,115,32,110,101,101,100,101,100,32,116,111,32,108,111, - 97,100,32,98,117,105,108,116,45,105,110,10,32,32,32,32, - 109,111,100,117,108,101,115,44,32,116,104,111,115,101,32,116, - 119,111,32,109,111,100,117,108,101,115,32,109,117,115,116,32, - 98,101,32,101,120,112,108,105,99,105,116,108,121,32,112,97, - 115,115,101,100,32,105,110,46,10,10,32,32,32,32,41,3, - 114,20,0,0,0,114,166,0,0,0,114,56,0,0,0,78, - 41,15,114,49,0,0,0,114,14,0,0,0,114,13,0,0, - 0,114,79,0,0,0,218,5,105,116,101,109,115,114,170,0, - 0,0,114,69,0,0,0,114,141,0,0,0,114,75,0,0, - 0,114,151,0,0,0,114,128,0,0,0,114,133,0,0,0, - 114,1,0,0,0,114,195,0,0,0,114,5,0,0,0,41, - 10,218,10,115,121,115,95,109,111,100,117,108,101,218,11,95, - 105,109,112,95,109,111,100,117,108,101,90,11,109,111,100,117, - 108,101,95,116,121,112,101,114,15,0,0,0,114,83,0,0, - 0,114,93,0,0,0,114,82,0,0,0,90,11,115,101,108, - 102,95,109,111,100,117,108,101,90,12,98,117,105,108,116,105, - 110,95,110,97,109,101,90,14,98,117,105,108,116,105,110,95, - 109,111,100,117,108,101,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,6,95,115,101,116,117,112,93,4,0, - 0,115,36,0,0,0,0,9,4,1,4,3,8,1,18,1, - 10,1,10,1,6,1,10,1,6,2,2,1,10,1,12,3, - 10,1,8,1,10,1,10,2,10,1,114,199,0,0,0,99, - 2,0,0,0,0,0,0,0,2,0,0,0,3,0,0,0, - 67,0,0,0,115,38,0,0,0,116,0,124,0,124,1,131, - 2,1,0,116,1,106,2,160,3,116,4,161,1,1,0,116, - 1,106,2,160,3,116,5,161,1,1,0,100,1,83,0,41, - 2,122,48,73,110,115,116,97,108,108,32,105,109,112,111,114, - 116,101,114,115,32,102,111,114,32,98,117,105,108,116,105,110, - 32,97,110,100,32,102,114,111,122,101,110,32,109,111,100,117, - 108,101,115,78,41,6,114,199,0,0,0,114,14,0,0,0, - 114,165,0,0,0,114,109,0,0,0,114,141,0,0,0,114, - 151,0,0,0,41,2,114,197,0,0,0,114,198,0,0,0, + 0,116,0,124,0,124,1,131,2,1,0,116,1,106,2,160, + 3,116,4,161,1,1,0,116,1,106,2,160,3,116,5,161, + 1,1,0,100,1,83,0,41,2,122,48,73,110,115,116,97, + 108,108,32,105,109,112,111,114,116,101,114,115,32,102,111,114, + 32,98,117,105,108,116,105,110,32,97,110,100,32,102,114,111, + 122,101,110,32,109,111,100,117,108,101,115,78,41,6,114,199, + 0,0,0,114,14,0,0,0,114,165,0,0,0,114,109,0, + 0,0,114,141,0,0,0,114,151,0,0,0,41,2,114,197, + 0,0,0,114,198,0,0,0,114,10,0,0,0,114,10,0, + 0,0,114,11,0,0,0,218,8,95,105,110,115,116,97,108, + 108,128,4,0,0,115,6,0,0,0,0,2,10,2,12,1, + 114,200,0,0,0,99,0,0,0,0,0,0,0,0,1,0, + 0,0,4,0,0,0,67,0,0,0,115,32,0,0,0,100, + 1,100,2,108,0,125,0,124,0,97,1,124,0,160,2,116, + 3,106,4,116,5,25,0,161,1,1,0,100,2,83,0,41, + 3,122,57,73,110,115,116,97,108,108,32,105,109,112,111,114, + 116,101,114,115,32,116,104,97,116,32,114,101,113,117,105,114, + 101,32,101,120,116,101,114,110,97,108,32,102,105,108,101,115, + 121,115,116,101,109,32,97,99,99,101,115,115,114,19,0,0, + 0,78,41,6,218,26,95,102,114,111,122,101,110,95,105,109, + 112,111,114,116,108,105,98,95,101,120,116,101,114,110,97,108, + 114,115,0,0,0,114,200,0,0,0,114,14,0,0,0,114, + 79,0,0,0,114,1,0,0,0,41,1,114,201,0,0,0, 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 8,95,105,110,115,116,97,108,108,128,4,0,0,115,6,0, - 0,0,0,2,10,2,12,1,114,200,0,0,0,99,0,0, - 0,0,0,0,0,0,1,0,0,0,4,0,0,0,67,0, - 0,0,115,32,0,0,0,100,1,100,2,108,0,125,0,124, - 0,97,1,124,0,160,2,116,3,106,4,116,5,25,0,161, - 1,1,0,100,2,83,0,41,3,122,57,73,110,115,116,97, - 108,108,32,105,109,112,111,114,116,101,114,115,32,116,104,97, - 116,32,114,101,113,117,105,114,101,32,101,120,116,101,114,110, - 97,108,32,102,105,108,101,115,121,115,116,101,109,32,97,99, - 99,101,115,115,114,19,0,0,0,78,41,6,218,26,95,102, - 114,111,122,101,110,95,105,109,112,111,114,116,108,105,98,95, - 101,120,116,101,114,110,97,108,114,115,0,0,0,114,200,0, - 0,0,114,14,0,0,0,114,79,0,0,0,114,1,0,0, - 0,41,1,114,201,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,218,27,95,105,110,115,116,97,108, - 108,95,101,120,116,101,114,110,97,108,95,105,109,112,111,114, - 116,101,114,115,136,4,0,0,115,6,0,0,0,0,3,8, - 1,4,1,114,202,0,0,0,41,2,78,78,41,1,78,41, - 2,78,114,19,0,0,0,41,4,78,78,114,10,0,0,0, - 114,19,0,0,0,41,51,114,3,0,0,0,114,115,0,0, - 0,114,12,0,0,0,114,16,0,0,0,114,51,0,0,0, - 114,29,0,0,0,114,36,0,0,0,114,17,0,0,0,114, - 18,0,0,0,114,41,0,0,0,114,42,0,0,0,114,45, - 0,0,0,114,57,0,0,0,114,59,0,0,0,114,68,0, - 0,0,114,74,0,0,0,114,77,0,0,0,114,84,0,0, - 0,114,95,0,0,0,114,96,0,0,0,114,102,0,0,0, - 114,78,0,0,0,114,128,0,0,0,114,133,0,0,0,114, - 136,0,0,0,114,91,0,0,0,114,80,0,0,0,114,139, - 0,0,0,114,140,0,0,0,114,81,0,0,0,114,141,0, - 0,0,114,151,0,0,0,114,156,0,0,0,114,162,0,0, - 0,114,164,0,0,0,114,169,0,0,0,114,173,0,0,0, - 90,15,95,69,82,82,95,77,83,71,95,80,82,69,70,73, - 88,114,175,0,0,0,114,178,0,0,0,218,6,111,98,106, - 101,99,116,114,179,0,0,0,114,180,0,0,0,114,181,0, - 0,0,114,185,0,0,0,114,191,0,0,0,114,194,0,0, - 0,114,195,0,0,0,114,199,0,0,0,114,200,0,0,0, - 114,202,0,0,0,114,10,0,0,0,114,10,0,0,0,114, - 10,0,0,0,114,11,0,0,0,218,8,60,109,111,100,117, - 108,101,62,8,0,0,0,115,96,0,0,0,4,17,4,2, - 8,8,8,8,4,2,4,3,16,4,14,68,14,21,14,16, - 8,37,8,17,8,11,14,8,8,11,8,12,8,16,8,36, - 14,27,14,101,16,26,10,45,14,72,8,17,8,17,8,24, - 8,29,8,23,8,15,14,73,14,77,14,13,8,9,8,9, - 10,47,8,16,4,1,8,2,8,27,6,3,8,16,10,15, - 14,37,8,27,10,37,8,7,8,35,8,8, + 27,95,105,110,115,116,97,108,108,95,101,120,116,101,114,110, + 97,108,95,105,109,112,111,114,116,101,114,115,136,4,0,0, + 115,6,0,0,0,0,3,8,1,4,1,114,202,0,0,0, + 41,2,78,78,41,1,78,41,2,78,114,19,0,0,0,41, + 4,78,78,114,10,0,0,0,114,19,0,0,0,41,51,114, + 3,0,0,0,114,115,0,0,0,114,12,0,0,0,114,16, + 0,0,0,114,51,0,0,0,114,29,0,0,0,114,36,0, + 0,0,114,17,0,0,0,114,18,0,0,0,114,41,0,0, + 0,114,42,0,0,0,114,45,0,0,0,114,57,0,0,0, + 114,59,0,0,0,114,68,0,0,0,114,74,0,0,0,114, + 77,0,0,0,114,84,0,0,0,114,95,0,0,0,114,96, + 0,0,0,114,102,0,0,0,114,78,0,0,0,114,128,0, + 0,0,114,133,0,0,0,114,136,0,0,0,114,91,0,0, + 0,114,80,0,0,0,114,139,0,0,0,114,140,0,0,0, + 114,81,0,0,0,114,141,0,0,0,114,151,0,0,0,114, + 156,0,0,0,114,162,0,0,0,114,164,0,0,0,114,169, + 0,0,0,114,173,0,0,0,90,15,95,69,82,82,95,77, + 83,71,95,80,82,69,70,73,88,114,175,0,0,0,114,178, + 0,0,0,218,6,111,98,106,101,99,116,114,179,0,0,0, + 114,180,0,0,0,114,181,0,0,0,114,185,0,0,0,114, + 191,0,0,0,114,194,0,0,0,114,195,0,0,0,114,199, + 0,0,0,114,200,0,0,0,114,202,0,0,0,114,10,0, + 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, + 0,218,8,60,109,111,100,117,108,101,62,8,0,0,0,115, + 96,0,0,0,4,17,4,2,8,8,8,8,4,2,4,3, + 16,4,14,68,14,21,14,16,8,37,8,17,8,11,14,8, + 8,11,8,12,8,16,8,36,14,27,14,101,16,26,10,45, + 14,72,8,17,8,17,8,24,8,29,8,23,8,15,14,73, + 14,77,14,13,8,9,8,9,10,47,8,16,4,1,8,2, + 8,27,6,3,8,16,10,15,14,37,8,27,10,37,8,7, + 8,35,8,8, }; diff --git a/Python/importlib_external.h b/Python/importlib_external.h index 8674b2193a0e..9a813c963e03 100644 --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -1,999 +1,1003 @@ /* Auto-generated by Programs/_freeze_importlib.c */ const unsigned char _Py_M__importlib_external[] = { 99,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0, - 0,64,0,0,0,115,24,2,0,0,100,0,90,0,100,1, + 0,64,0,0,0,115,44,2,0,0,100,0,90,0,100,1, 90,1,100,2,90,2,101,2,101,1,23,0,90,3,100,3, 100,4,132,0,90,4,100,5,100,6,132,0,90,5,100,7, 100,8,132,0,90,6,100,9,100,10,132,0,90,7,100,11, 100,12,132,0,90,8,100,13,100,14,132,0,90,9,100,15, 100,16,132,0,90,10,100,17,100,18,132,0,90,11,100,19, - 100,20,132,0,90,12,100,21,100,22,132,0,90,13,100,99, - 100,24,100,25,132,1,90,14,101,15,101,14,106,16,131,1, - 90,17,100,26,160,18,100,27,100,28,161,2,100,29,23,0, - 90,19,101,20,160,21,101,19,100,28,161,2,90,22,100,30, - 90,23,100,31,90,24,100,32,103,1,90,25,100,33,103,1, - 90,26,101,26,4,0,90,27,90,28,100,100,100,34,100,35, - 156,1,100,36,100,37,132,3,90,29,100,38,100,39,132,0, - 90,30,100,40,100,41,132,0,90,31,100,42,100,43,132,0, - 90,32,100,44,100,45,132,0,90,33,100,46,100,47,132,0, - 90,34,100,48,100,49,132,0,90,35,100,50,100,51,132,0, - 90,36,100,52,100,53,132,0,90,37,100,54,100,55,132,0, - 90,38,100,101,100,56,100,57,132,1,90,39,100,102,100,59, - 100,60,132,1,90,40,100,103,100,62,100,63,132,1,90,41, - 100,64,100,65,132,0,90,42,101,43,131,0,90,44,100,104, - 100,34,101,44,100,66,156,2,100,67,100,68,132,3,90,45, - 71,0,100,69,100,70,132,0,100,70,131,2,90,46,71,0, - 100,71,100,72,132,0,100,72,131,2,90,47,71,0,100,73, - 100,74,132,0,100,74,101,47,131,3,90,48,71,0,100,75, - 100,76,132,0,100,76,131,2,90,49,71,0,100,77,100,78, - 132,0,100,78,101,49,101,48,131,4,90,50,71,0,100,79, - 100,80,132,0,100,80,101,49,101,47,131,4,90,51,103,0, - 90,52,71,0,100,81,100,82,132,0,100,82,101,49,101,47, - 131,4,90,53,71,0,100,83,100,84,132,0,100,84,131,2, - 90,54,71,0,100,85,100,86,132,0,100,86,131,2,90,55, - 71,0,100,87,100,88,132,0,100,88,131,2,90,56,71,0, - 100,89,100,90,132,0,100,90,131,2,90,57,100,105,100,91, - 100,92,132,1,90,58,100,93,100,94,132,0,90,59,100,95, - 100,96,132,0,90,60,100,97,100,98,132,0,90,61,100,34, - 83,0,41,106,97,94,1,0,0,67,111,114,101,32,105,109, - 112,108,101,109,101,110,116,97,116,105,111,110,32,111,102,32, - 112,97,116,104,45,98,97,115,101,100,32,105,109,112,111,114, - 116,46,10,10,84,104,105,115,32,109,111,100,117,108,101,32, - 105,115,32,78,79,84,32,109,101,97,110,116,32,116,111,32, - 98,101,32,100,105,114,101,99,116,108,121,32,105,109,112,111, - 114,116,101,100,33,32,73,116,32,104,97,115,32,98,101,101, - 110,32,100,101,115,105,103,110,101,100,32,115,117,99,104,10, - 116,104,97,116,32,105,116,32,99,97,110,32,98,101,32,98, - 111,111,116,115,116,114,97,112,112,101,100,32,105,110,116,111, - 32,80,121,116,104,111,110,32,97,115,32,116,104,101,32,105, - 109,112,108,101,109,101,110,116,97,116,105,111,110,32,111,102, - 32,105,109,112,111,114,116,46,32,65,115,10,115,117,99,104, - 32,105,116,32,114,101,113,117,105,114,101,115,32,116,104,101, - 32,105,110,106,101,99,116,105,111,110,32,111,102,32,115,112, - 101,99,105,102,105,99,32,109,111,100,117,108,101,115,32,97, - 110,100,32,97,116,116,114,105,98,117,116,101,115,32,105,110, - 32,111,114,100,101,114,32,116,111,10,119,111,114,107,46,32, - 79,110,101,32,115,104,111,117,108,100,32,117,115,101,32,105, - 109,112,111,114,116,108,105,98,32,97,115,32,116,104,101,32, - 112,117,98,108,105,99,45,102,97,99,105,110,103,32,118,101, - 114,115,105,111,110,32,111,102,32,116,104,105,115,32,109,111, - 100,117,108,101,46,10,10,41,1,218,3,119,105,110,41,2, - 90,6,99,121,103,119,105,110,90,6,100,97,114,119,105,110, - 99,0,0,0,0,0,0,0,0,1,0,0,0,3,0,0, - 0,3,0,0,0,115,60,0,0,0,116,0,106,1,160,2, - 116,3,161,1,114,48,116,0,106,1,160,2,116,4,161,1, - 114,30,100,1,137,0,110,4,100,2,137,0,135,0,102,1, - 100,3,100,4,132,8,125,0,110,8,100,5,100,4,132,0, - 125,0,124,0,83,0,41,6,78,90,12,80,89,84,72,79, - 78,67,65,83,69,79,75,115,12,0,0,0,80,89,84,72, - 79,78,67,65,83,69,79,75,99,0,0,0,0,0,0,0, - 0,0,0,0,0,2,0,0,0,19,0,0,0,115,10,0, - 0,0,136,0,116,0,106,1,107,6,83,0,41,1,122,53, + 100,20,132,0,90,12,100,21,100,22,132,0,90,13,100,23, + 102,1,100,24,100,25,132,1,90,14,101,15,101,14,106,16, + 131,1,90,17,100,26,160,18,100,27,100,28,161,2,100,29, + 23,0,90,19,101,20,160,21,101,19,100,28,161,2,90,22, + 100,30,90,23,100,31,90,24,100,32,103,1,90,25,100,33, + 103,1,90,26,101,26,4,0,90,27,90,28,100,34,102,1, + 100,34,100,35,156,1,100,36,100,37,132,3,90,29,100,38, + 100,39,132,0,90,30,100,40,100,41,132,0,90,31,100,42, + 100,43,132,0,90,32,100,44,100,45,132,0,90,33,100,46, + 100,47,132,0,90,34,100,48,100,49,132,0,90,35,100,50, + 100,51,132,0,90,36,100,52,100,53,132,0,90,37,100,54, + 100,55,132,0,90,38,100,34,100,34,100,34,102,3,100,56, + 100,57,132,1,90,39,100,58,100,58,102,2,100,59,100,60, + 132,1,90,40,100,61,102,1,100,62,100,63,132,1,90,41, + 100,64,100,65,132,0,90,42,101,43,131,0,90,44,100,34, + 102,1,100,34,101,44,100,66,156,2,100,67,100,68,132,3, + 90,45,71,0,100,69,100,70,132,0,100,70,131,2,90,46, + 71,0,100,71,100,72,132,0,100,72,131,2,90,47,71,0, + 100,73,100,74,132,0,100,74,101,47,131,3,90,48,71,0, + 100,75,100,76,132,0,100,76,131,2,90,49,71,0,100,77, + 100,78,132,0,100,78,101,49,101,48,131,4,90,50,71,0, + 100,79,100,80,132,0,100,80,101,49,101,47,131,4,90,51, + 103,0,90,52,71,0,100,81,100,82,132,0,100,82,101,49, + 101,47,131,4,90,53,71,0,100,83,100,84,132,0,100,84, + 131,2,90,54,71,0,100,85,100,86,132,0,100,86,131,2, + 90,55,71,0,100,87,100,88,132,0,100,88,131,2,90,56, + 71,0,100,89,100,90,132,0,100,90,131,2,90,57,100,34, + 102,1,100,91,100,92,132,1,90,58,100,93,100,94,132,0, + 90,59,100,95,100,96,132,0,90,60,100,97,100,98,132,0, + 90,61,100,34,83,0,41,99,97,94,1,0,0,67,111,114, + 101,32,105,109,112,108,101,109,101,110,116,97,116,105,111,110, + 32,111,102,32,112,97,116,104,45,98,97,115,101,100,32,105, + 109,112,111,114,116,46,10,10,84,104,105,115,32,109,111,100, + 117,108,101,32,105,115,32,78,79,84,32,109,101,97,110,116, + 32,116,111,32,98,101,32,100,105,114,101,99,116,108,121,32, + 105,109,112,111,114,116,101,100,33,32,73,116,32,104,97,115, + 32,98,101,101,110,32,100,101,115,105,103,110,101,100,32,115, + 117,99,104,10,116,104,97,116,32,105,116,32,99,97,110,32, + 98,101,32,98,111,111,116,115,116,114,97,112,112,101,100,32, + 105,110,116,111,32,80,121,116,104,111,110,32,97,115,32,116, + 104,101,32,105,109,112,108,101,109,101,110,116,97,116,105,111, + 110,32,111,102,32,105,109,112,111,114,116,46,32,65,115,10, + 115,117,99,104,32,105,116,32,114,101,113,117,105,114,101,115, + 32,116,104,101,32,105,110,106,101,99,116,105,111,110,32,111, + 102,32,115,112,101,99,105,102,105,99,32,109,111,100,117,108, + 101,115,32,97,110,100,32,97,116,116,114,105,98,117,116,101, + 115,32,105,110,32,111,114,100,101,114,32,116,111,10,119,111, + 114,107,46,32,79,110,101,32,115,104,111,117,108,100,32,117, + 115,101,32,105,109,112,111,114,116,108,105,98,32,97,115,32, + 116,104,101,32,112,117,98,108,105,99,45,102,97,99,105,110, + 103,32,118,101,114,115,105,111,110,32,111,102,32,116,104,105, + 115,32,109,111,100,117,108,101,46,10,10,41,1,218,3,119, + 105,110,41,2,90,6,99,121,103,119,105,110,90,6,100,97, + 114,119,105,110,99,0,0,0,0,0,0,0,0,1,0,0, + 0,3,0,0,0,3,0,0,0,115,60,0,0,0,116,0, + 106,1,160,2,116,3,161,1,114,48,116,0,106,1,160,2, + 116,4,161,1,114,30,100,1,137,0,110,4,100,2,137,0, + 135,0,102,1,100,3,100,4,132,8,125,0,110,8,100,5, + 100,4,132,0,125,0,124,0,83,0,41,6,78,90,12,80, + 89,84,72,79,78,67,65,83,69,79,75,115,12,0,0,0, + 80,89,84,72,79,78,67,65,83,69,79,75,99,0,0,0, + 0,0,0,0,0,0,0,0,0,2,0,0,0,19,0,0, + 0,115,10,0,0,0,136,0,116,0,106,1,107,6,83,0, + 41,1,122,53,84,114,117,101,32,105,102,32,102,105,108,101, + 110,97,109,101,115,32,109,117,115,116,32,98,101,32,99,104, + 101,99,107,101,100,32,99,97,115,101,45,105,110,115,101,110, + 115,105,116,105,118,101,108,121,46,41,2,218,3,95,111,115, + 90,7,101,110,118,105,114,111,110,169,0,41,1,218,3,107, + 101,121,114,2,0,0,0,250,38,60,102,114,111,122,101,110, + 32,105,109,112,111,114,116,108,105,98,46,95,98,111,111,116, + 115,116,114,97,112,95,101,120,116,101,114,110,97,108,62,218, + 11,95,114,101,108,97,120,95,99,97,115,101,36,0,0,0, + 115,2,0,0,0,0,2,122,37,95,109,97,107,101,95,114, + 101,108,97,120,95,99,97,115,101,46,60,108,111,99,97,108, + 115,62,46,95,114,101,108,97,120,95,99,97,115,101,99,0, + 0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,83, + 0,0,0,115,4,0,0,0,100,1,83,0,41,2,122,53, 84,114,117,101,32,105,102,32,102,105,108,101,110,97,109,101, 115,32,109,117,115,116,32,98,101,32,99,104,101,99,107,101, 100,32,99,97,115,101,45,105,110,115,101,110,115,105,116,105, - 118,101,108,121,46,41,2,218,3,95,111,115,90,7,101,110, - 118,105,114,111,110,169,0,41,1,218,3,107,101,121,114,2, - 0,0,0,250,38,60,102,114,111,122,101,110,32,105,109,112, - 111,114,116,108,105,98,46,95,98,111,111,116,115,116,114,97, - 112,95,101,120,116,101,114,110,97,108,62,218,11,95,114,101, - 108,97,120,95,99,97,115,101,36,0,0,0,115,2,0,0, - 0,0,2,122,37,95,109,97,107,101,95,114,101,108,97,120, - 95,99,97,115,101,46,60,108,111,99,97,108,115,62,46,95, - 114,101,108,97,120,95,99,97,115,101,99,0,0,0,0,0, - 0,0,0,0,0,0,0,1,0,0,0,83,0,0,0,115, - 4,0,0,0,100,1,83,0,41,2,122,53,84,114,117,101, - 32,105,102,32,102,105,108,101,110,97,109,101,115,32,109,117, - 115,116,32,98,101,32,99,104,101,99,107,101,100,32,99,97, - 115,101,45,105,110,115,101,110,115,105,116,105,118,101,108,121, - 46,70,114,2,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,114,5,0,0,0, - 40,0,0,0,115,2,0,0,0,0,2,41,5,218,3,115, - 121,115,218,8,112,108,97,116,102,111,114,109,218,10,115,116, - 97,114,116,115,119,105,116,104,218,27,95,67,65,83,69,95, - 73,78,83,69,78,83,73,84,73,86,69,95,80,76,65,84, - 70,79,82,77,83,218,35,95,67,65,83,69,95,73,78,83, - 69,78,83,73,84,73,86,69,95,80,76,65,84,70,79,82, - 77,83,95,83,84,82,95,75,69,89,41,1,114,5,0,0, - 0,114,2,0,0,0,41,1,114,3,0,0,0,114,4,0, - 0,0,218,16,95,109,97,107,101,95,114,101,108,97,120,95, - 99,97,115,101,29,0,0,0,115,14,0,0,0,0,1,12, - 1,12,1,6,2,4,2,14,4,8,3,114,11,0,0,0, - 99,1,0,0,0,0,0,0,0,1,0,0,0,4,0,0, - 0,67,0,0,0,115,20,0,0,0,116,0,124,0,131,1, - 100,1,64,0,160,1,100,2,100,3,161,2,83,0,41,4, - 122,42,67,111,110,118,101,114,116,32,97,32,51,50,45,98, - 105,116,32,105,110,116,101,103,101,114,32,116,111,32,108,105, - 116,116,108,101,45,101,110,100,105,97,110,46,108,3,0,0, - 0,255,127,255,127,3,0,233,4,0,0,0,218,6,108,105, - 116,116,108,101,41,2,218,3,105,110,116,218,8,116,111,95, - 98,121,116,101,115,41,1,218,1,120,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,218,7,95,119,95,108,111, - 110,103,46,0,0,0,115,2,0,0,0,0,2,114,17,0, - 0,0,99,1,0,0,0,0,0,0,0,1,0,0,0,4, - 0,0,0,67,0,0,0,115,12,0,0,0,116,0,160,1, - 124,0,100,1,161,2,83,0,41,2,122,47,67,111,110,118, - 101,114,116,32,52,32,98,121,116,101,115,32,105,110,32,108, - 105,116,116,108,101,45,101,110,100,105,97,110,32,116,111,32, - 97,110,32,105,110,116,101,103,101,114,46,114,13,0,0,0, - 41,2,114,14,0,0,0,218,10,102,114,111,109,95,98,121, - 116,101,115,41,1,90,9,105,110,116,95,98,121,116,101,115, - 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 7,95,114,95,108,111,110,103,51,0,0,0,115,2,0,0, - 0,0,2,114,19,0,0,0,99,0,0,0,0,0,0,0, - 0,1,0,0,0,4,0,0,0,71,0,0,0,115,20,0, - 0,0,116,0,160,1,100,1,100,2,132,0,124,0,68,0, - 131,1,161,1,83,0,41,3,122,31,82,101,112,108,97,99, - 101,109,101,110,116,32,102,111,114,32,111,115,46,112,97,116, - 104,46,106,111,105,110,40,41,46,99,1,0,0,0,0,0, - 0,0,2,0,0,0,5,0,0,0,83,0,0,0,115,26, - 0,0,0,103,0,124,0,93,18,125,1,124,1,114,4,124, - 1,160,0,116,1,161,1,145,2,113,4,83,0,114,2,0, - 0,0,41,2,218,6,114,115,116,114,105,112,218,15,112,97, - 116,104,95,115,101,112,97,114,97,116,111,114,115,41,2,218, - 2,46,48,218,4,112,97,114,116,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,218,10,60,108,105,115,116,99, - 111,109,112,62,58,0,0,0,115,2,0,0,0,6,1,122, - 30,95,112,97,116,104,95,106,111,105,110,46,60,108,111,99, - 97,108,115,62,46,60,108,105,115,116,99,111,109,112,62,41, - 2,218,8,112,97,116,104,95,115,101,112,218,4,106,111,105, - 110,41,1,218,10,112,97,116,104,95,112,97,114,116,115,114, - 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,10, - 95,112,97,116,104,95,106,111,105,110,56,0,0,0,115,4, - 0,0,0,0,2,10,1,114,28,0,0,0,99,1,0,0, - 0,0,0,0,0,5,0,0,0,5,0,0,0,67,0,0, - 0,115,96,0,0,0,116,0,116,1,131,1,100,1,107,2, - 114,36,124,0,160,2,116,3,161,1,92,3,125,1,125,2, - 125,3,124,1,124,3,102,2,83,0,116,4,124,0,131,1, - 68,0,93,42,125,4,124,4,116,1,107,6,114,44,124,0, - 106,5,124,4,100,1,100,2,141,2,92,2,125,1,125,3, - 124,1,124,3,102,2,2,0,1,0,83,0,113,44,100,3, - 124,0,102,2,83,0,41,4,122,32,82,101,112,108,97,99, - 101,109,101,110,116,32,102,111,114,32,111,115,46,112,97,116, - 104,46,115,112,108,105,116,40,41,46,233,1,0,0,0,41, - 1,90,8,109,97,120,115,112,108,105,116,218,0,41,6,218, - 3,108,101,110,114,21,0,0,0,218,10,114,112,97,114,116, - 105,116,105,111,110,114,25,0,0,0,218,8,114,101,118,101, - 114,115,101,100,218,6,114,115,112,108,105,116,41,5,218,4, - 112,97,116,104,90,5,102,114,111,110,116,218,1,95,218,4, - 116,97,105,108,114,16,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,218,11,95,112,97,116,104,95, - 115,112,108,105,116,62,0,0,0,115,16,0,0,0,0,2, - 12,1,16,1,8,1,12,1,8,1,18,1,14,1,114,38, - 0,0,0,99,1,0,0,0,0,0,0,0,1,0,0,0, - 3,0,0,0,67,0,0,0,115,10,0,0,0,116,0,160, - 1,124,0,161,1,83,0,41,1,122,126,83,116,97,116,32, - 116,104,101,32,112,97,116,104,46,10,10,32,32,32,32,77, - 97,100,101,32,97,32,115,101,112,97,114,97,116,101,32,102, - 117,110,99,116,105,111,110,32,116,111,32,109,97,107,101,32, - 105,116,32,101,97,115,105,101,114,32,116,111,32,111,118,101, - 114,114,105,100,101,32,105,110,32,101,120,112,101,114,105,109, - 101,110,116,115,10,32,32,32,32,40,101,46,103,46,32,99, - 97,99,104,101,32,115,116,97,116,32,114,101,115,117,108,116, - 115,41,46,10,10,32,32,32,32,41,2,114,1,0,0,0, - 90,4,115,116,97,116,41,1,114,35,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,218,10,95,112, - 97,116,104,95,115,116,97,116,74,0,0,0,115,2,0,0, - 0,0,7,114,39,0,0,0,99,2,0,0,0,0,0,0, - 0,3,0,0,0,8,0,0,0,67,0,0,0,115,50,0, - 0,0,122,12,116,0,124,0,131,1,125,2,87,0,110,22, - 4,0,116,1,107,10,114,34,1,0,1,0,1,0,89,0, - 100,1,83,0,88,0,124,2,106,2,100,2,64,0,124,1, - 107,2,83,0,41,3,122,49,84,101,115,116,32,119,104,101, - 116,104,101,114,32,116,104,101,32,112,97,116,104,32,105,115, - 32,116,104,101,32,115,112,101,99,105,102,105,101,100,32,109, - 111,100,101,32,116,121,112,101,46,70,105,0,240,0,0,41, - 3,114,39,0,0,0,218,7,79,83,69,114,114,111,114,218, - 7,115,116,95,109,111,100,101,41,3,114,35,0,0,0,218, - 4,109,111,100,101,90,9,115,116,97,116,95,105,110,102,111, - 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 18,95,112,97,116,104,95,105,115,95,109,111,100,101,95,116, - 121,112,101,84,0,0,0,115,10,0,0,0,0,2,2,1, - 12,1,14,1,8,1,114,43,0,0,0,99,1,0,0,0, - 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, - 115,10,0,0,0,116,0,124,0,100,1,131,2,83,0,41, - 2,122,31,82,101,112,108,97,99,101,109,101,110,116,32,102, - 111,114,32,111,115,46,112,97,116,104,46,105,115,102,105,108, - 101,46,105,0,128,0,0,41,1,114,43,0,0,0,41,1, - 114,35,0,0,0,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,218,12,95,112,97,116,104,95,105,115,102,105, - 108,101,93,0,0,0,115,2,0,0,0,0,2,114,44,0, - 0,0,99,1,0,0,0,0,0,0,0,1,0,0,0,3, - 0,0,0,67,0,0,0,115,22,0,0,0,124,0,115,12, - 116,0,160,1,161,0,125,0,116,2,124,0,100,1,131,2, - 83,0,41,2,122,30,82,101,112,108,97,99,101,109,101,110, + 118,101,108,121,46,70,114,2,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,114, + 5,0,0,0,40,0,0,0,115,2,0,0,0,0,2,41, + 5,218,3,115,121,115,218,8,112,108,97,116,102,111,114,109, + 218,10,115,116,97,114,116,115,119,105,116,104,218,27,95,67, + 65,83,69,95,73,78,83,69,78,83,73,84,73,86,69,95, + 80,76,65,84,70,79,82,77,83,218,35,95,67,65,83,69, + 95,73,78,83,69,78,83,73,84,73,86,69,95,80,76,65, + 84,70,79,82,77,83,95,83,84,82,95,75,69,89,41,1, + 114,5,0,0,0,114,2,0,0,0,41,1,114,3,0,0, + 0,114,4,0,0,0,218,16,95,109,97,107,101,95,114,101, + 108,97,120,95,99,97,115,101,29,0,0,0,115,14,0,0, + 0,0,1,12,1,12,1,6,2,4,2,14,4,8,3,114, + 11,0,0,0,99,1,0,0,0,0,0,0,0,1,0,0, + 0,4,0,0,0,67,0,0,0,115,20,0,0,0,116,0, + 124,0,131,1,100,1,64,0,160,1,100,2,100,3,161,2, + 83,0,41,4,122,42,67,111,110,118,101,114,116,32,97,32, + 51,50,45,98,105,116,32,105,110,116,101,103,101,114,32,116, + 111,32,108,105,116,116,108,101,45,101,110,100,105,97,110,46, + 108,3,0,0,0,255,127,255,127,3,0,233,4,0,0,0, + 218,6,108,105,116,116,108,101,41,2,218,3,105,110,116,218, + 8,116,111,95,98,121,116,101,115,41,1,218,1,120,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,218,7,95, + 119,95,108,111,110,103,46,0,0,0,115,2,0,0,0,0, + 2,114,17,0,0,0,99,1,0,0,0,0,0,0,0,1, + 0,0,0,4,0,0,0,67,0,0,0,115,12,0,0,0, + 116,0,160,1,124,0,100,1,161,2,83,0,41,2,122,47, + 67,111,110,118,101,114,116,32,52,32,98,121,116,101,115,32, + 105,110,32,108,105,116,116,108,101,45,101,110,100,105,97,110, + 32,116,111,32,97,110,32,105,110,116,101,103,101,114,46,114, + 13,0,0,0,41,2,114,14,0,0,0,218,10,102,114,111, + 109,95,98,121,116,101,115,41,1,90,9,105,110,116,95,98, + 121,116,101,115,114,2,0,0,0,114,2,0,0,0,114,4, + 0,0,0,218,7,95,114,95,108,111,110,103,51,0,0,0, + 115,2,0,0,0,0,2,114,19,0,0,0,99,0,0,0, + 0,0,0,0,0,1,0,0,0,4,0,0,0,71,0,0, + 0,115,20,0,0,0,116,0,160,1,100,1,100,2,132,0, + 124,0,68,0,131,1,161,1,83,0,41,3,122,31,82,101, + 112,108,97,99,101,109,101,110,116,32,102,111,114,32,111,115, + 46,112,97,116,104,46,106,111,105,110,40,41,46,99,1,0, + 0,0,0,0,0,0,2,0,0,0,5,0,0,0,83,0, + 0,0,115,26,0,0,0,103,0,124,0,93,18,125,1,124, + 1,114,22,124,1,160,0,116,1,161,1,145,2,113,4,83, + 0,114,2,0,0,0,41,2,218,6,114,115,116,114,105,112, + 218,15,112,97,116,104,95,115,101,112,97,114,97,116,111,114, + 115,41,2,218,2,46,48,218,4,112,97,114,116,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,218,10,60,108, + 105,115,116,99,111,109,112,62,58,0,0,0,115,6,0,0, + 0,6,1,2,0,4,255,122,30,95,112,97,116,104,95,106, + 111,105,110,46,60,108,111,99,97,108,115,62,46,60,108,105, + 115,116,99,111,109,112,62,41,2,218,8,112,97,116,104,95, + 115,101,112,218,4,106,111,105,110,41,1,218,10,112,97,116, + 104,95,112,97,114,116,115,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,10,95,112,97,116,104,95,106,111, + 105,110,56,0,0,0,115,6,0,0,0,0,2,10,1,2, + 255,114,28,0,0,0,99,1,0,0,0,0,0,0,0,5, + 0,0,0,5,0,0,0,67,0,0,0,115,96,0,0,0, + 116,0,116,1,131,1,100,1,107,2,114,36,124,0,160,2, + 116,3,161,1,92,3,125,1,125,2,125,3,124,1,124,3, + 102,2,83,0,116,4,124,0,131,1,68,0,93,42,125,4, + 124,4,116,1,107,6,114,44,124,0,106,5,124,4,100,1, + 100,2,141,2,92,2,125,1,125,3,124,1,124,3,102,2, + 2,0,1,0,83,0,113,44,100,3,124,0,102,2,83,0, + 41,4,122,32,82,101,112,108,97,99,101,109,101,110,116,32, + 102,111,114,32,111,115,46,112,97,116,104,46,115,112,108,105, + 116,40,41,46,233,1,0,0,0,41,1,90,8,109,97,120, + 115,112,108,105,116,218,0,41,6,218,3,108,101,110,114,21, + 0,0,0,218,10,114,112,97,114,116,105,116,105,111,110,114, + 25,0,0,0,218,8,114,101,118,101,114,115,101,100,218,6, + 114,115,112,108,105,116,41,5,218,4,112,97,116,104,90,5, + 102,114,111,110,116,218,1,95,218,4,116,97,105,108,114,16, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,218,11,95,112,97,116,104,95,115,112,108,105,116,62, + 0,0,0,115,16,0,0,0,0,2,12,1,16,1,8,1, + 12,1,8,1,18,1,14,1,114,38,0,0,0,99,1,0, + 0,0,0,0,0,0,1,0,0,0,3,0,0,0,67,0, + 0,0,115,10,0,0,0,116,0,160,1,124,0,161,1,83, + 0,41,1,122,126,83,116,97,116,32,116,104,101,32,112,97, + 116,104,46,10,10,32,32,32,32,77,97,100,101,32,97,32, + 115,101,112,97,114,97,116,101,32,102,117,110,99,116,105,111, + 110,32,116,111,32,109,97,107,101,32,105,116,32,101,97,115, + 105,101,114,32,116,111,32,111,118,101,114,114,105,100,101,32, + 105,110,32,101,120,112,101,114,105,109,101,110,116,115,10,32, + 32,32,32,40,101,46,103,46,32,99,97,99,104,101,32,115, + 116,97,116,32,114,101,115,117,108,116,115,41,46,10,10,32, + 32,32,32,41,2,114,1,0,0,0,90,4,115,116,97,116, + 41,1,114,35,0,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,10,95,112,97,116,104,95,115,116, + 97,116,74,0,0,0,115,2,0,0,0,0,7,114,39,0, + 0,0,99,2,0,0,0,0,0,0,0,3,0,0,0,8, + 0,0,0,67,0,0,0,115,50,0,0,0,122,12,116,0, + 124,0,131,1,125,2,87,0,110,22,4,0,116,1,107,10, + 114,34,1,0,1,0,1,0,89,0,100,1,83,0,88,0, + 124,2,106,2,100,2,64,0,124,1,107,2,83,0,41,3, + 122,49,84,101,115,116,32,119,104,101,116,104,101,114,32,116, + 104,101,32,112,97,116,104,32,105,115,32,116,104,101,32,115, + 112,101,99,105,102,105,101,100,32,109,111,100,101,32,116,121, + 112,101,46,70,105,0,240,0,0,41,3,114,39,0,0,0, + 218,7,79,83,69,114,114,111,114,218,7,115,116,95,109,111, + 100,101,41,3,114,35,0,0,0,218,4,109,111,100,101,90, + 9,115,116,97,116,95,105,110,102,111,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,218,18,95,112,97,116,104, + 95,105,115,95,109,111,100,101,95,116,121,112,101,84,0,0, + 0,115,10,0,0,0,0,2,2,1,12,1,14,1,8,1, + 114,43,0,0,0,99,1,0,0,0,0,0,0,0,1,0, + 0,0,3,0,0,0,67,0,0,0,115,10,0,0,0,116, + 0,124,0,100,1,131,2,83,0,41,2,122,31,82,101,112, + 108,97,99,101,109,101,110,116,32,102,111,114,32,111,115,46, + 112,97,116,104,46,105,115,102,105,108,101,46,105,0,128,0, + 0,41,1,114,43,0,0,0,41,1,114,35,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,12, + 95,112,97,116,104,95,105,115,102,105,108,101,93,0,0,0, + 115,2,0,0,0,0,2,114,44,0,0,0,99,1,0,0, + 0,0,0,0,0,1,0,0,0,3,0,0,0,67,0,0, + 0,115,22,0,0,0,124,0,115,12,116,0,160,1,161,0, + 125,0,116,2,124,0,100,1,131,2,83,0,41,2,122,30, + 82,101,112,108,97,99,101,109,101,110,116,32,102,111,114,32, + 111,115,46,112,97,116,104,46,105,115,100,105,114,46,105,0, + 64,0,0,41,3,114,1,0,0,0,218,6,103,101,116,99, + 119,100,114,43,0,0,0,41,1,114,35,0,0,0,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,218,11,95, + 112,97,116,104,95,105,115,100,105,114,98,0,0,0,115,6, + 0,0,0,0,2,4,1,8,1,114,46,0,0,0,99,1, + 0,0,0,0,0,0,0,1,0,0,0,3,0,0,0,67, + 0,0,0,115,26,0,0,0,124,0,160,0,116,1,161,1, + 112,24,124,0,100,1,100,2,133,2,25,0,116,2,107,6, + 83,0,41,3,122,142,82,101,112,108,97,99,101,109,101,110, 116,32,102,111,114,32,111,115,46,112,97,116,104,46,105,115, - 100,105,114,46,105,0,64,0,0,41,3,114,1,0,0,0, - 218,6,103,101,116,99,119,100,114,43,0,0,0,41,1,114, - 35,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, - 0,0,0,218,11,95,112,97,116,104,95,105,115,100,105,114, - 98,0,0,0,115,6,0,0,0,0,2,4,1,8,1,114, - 46,0,0,0,99,1,0,0,0,0,0,0,0,1,0,0, - 0,3,0,0,0,67,0,0,0,115,26,0,0,0,124,0, - 160,0,116,1,161,1,112,24,124,0,100,1,100,2,133,2, - 25,0,116,2,107,6,83,0,41,3,122,142,82,101,112,108, - 97,99,101,109,101,110,116,32,102,111,114,32,111,115,46,112, - 97,116,104,46,105,115,97,98,115,46,10,10,32,32,32,32, - 67,111,110,115,105,100,101,114,115,32,97,32,87,105,110,100, - 111,119,115,32,100,114,105,118,101,45,114,101,108,97,116,105, - 118,101,32,112,97,116,104,32,40,110,111,32,100,114,105,118, - 101,44,32,98,117,116,32,115,116,97,114,116,115,32,119,105, - 116,104,32,115,108,97,115,104,41,32,116,111,10,32,32,32, - 32,115,116,105,108,108,32,98,101,32,34,97,98,115,111,108, - 117,116,101,34,46,10,32,32,32,32,114,29,0,0,0,233, - 3,0,0,0,41,3,114,8,0,0,0,114,21,0,0,0, - 218,20,95,112,97,116,104,115,101,112,115,95,119,105,116,104, - 95,99,111,108,111,110,41,1,114,35,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,218,11,95,112, - 97,116,104,95,105,115,97,98,115,105,0,0,0,115,2,0, - 0,0,0,6,114,49,0,0,0,233,182,1,0,0,99,3, - 0,0,0,0,0,0,0,6,0,0,0,11,0,0,0,67, - 0,0,0,115,162,0,0,0,100,1,160,0,124,0,116,1, - 124,0,131,1,161,2,125,3,116,2,160,3,124,3,116,2, - 106,4,116,2,106,5,66,0,116,2,106,6,66,0,124,2, - 100,2,64,0,161,3,125,4,122,50,116,7,160,8,124,4, - 100,3,161,2,143,16,125,5,124,5,160,9,124,1,161,1, - 1,0,87,0,53,0,81,0,82,0,88,0,116,2,160,10, - 124,3,124,0,161,2,1,0,87,0,110,58,4,0,116,11, - 107,10,114,156,1,0,1,0,1,0,122,14,116,2,160,12, - 124,3,161,1,1,0,87,0,110,20,4,0,116,11,107,10, - 114,148,1,0,1,0,1,0,89,0,110,2,88,0,130,0, - 89,0,110,2,88,0,100,4,83,0,41,5,122,162,66,101, - 115,116,45,101,102,102,111,114,116,32,102,117,110,99,116,105, - 111,110,32,116,111,32,119,114,105,116,101,32,100,97,116,97, - 32,116,111,32,97,32,112,97,116,104,32,97,116,111,109,105, - 99,97,108,108,121,46,10,32,32,32,32,66,101,32,112,114, - 101,112,97,114,101,100,32,116,111,32,104,97,110,100,108,101, - 32,97,32,70,105,108,101,69,120,105,115,116,115,69,114,114, - 111,114,32,105,102,32,99,111,110,99,117,114,114,101,110,116, - 32,119,114,105,116,105,110,103,32,111,102,32,116,104,101,10, - 32,32,32,32,116,101,109,112,111,114,97,114,121,32,102,105, - 108,101,32,105,115,32,97,116,116,101,109,112,116,101,100,46, - 122,5,123,125,46,123,125,105,182,1,0,0,90,2,119,98, - 78,41,13,218,6,102,111,114,109,97,116,218,2,105,100,114, - 1,0,0,0,90,4,111,112,101,110,90,6,79,95,69,88, - 67,76,90,7,79,95,67,82,69,65,84,90,8,79,95,87, - 82,79,78,76,89,218,3,95,105,111,218,6,70,105,108,101, - 73,79,218,5,119,114,105,116,101,218,7,114,101,112,108,97, - 99,101,114,40,0,0,0,90,6,117,110,108,105,110,107,41, - 6,114,35,0,0,0,218,4,100,97,116,97,114,42,0,0, - 0,90,8,112,97,116,104,95,116,109,112,90,2,102,100,218, - 4,102,105,108,101,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,218,13,95,119,114,105,116,101,95,97,116,111, - 109,105,99,114,0,0,0,115,26,0,0,0,0,5,16,1, - 6,1,26,1,2,3,14,1,20,1,16,1,14,1,2,1, - 14,1,14,1,6,1,114,59,0,0,0,105,73,13,0,0, - 233,2,0,0,0,114,13,0,0,0,115,2,0,0,0,13, - 10,90,11,95,95,112,121,99,97,99,104,101,95,95,122,4, - 111,112,116,45,122,3,46,112,121,122,4,46,112,121,99,78, - 41,1,218,12,111,112,116,105,109,105,122,97,116,105,111,110, - 99,2,0,0,0,1,0,0,0,12,0,0,0,5,0,0, - 0,67,0,0,0,115,88,1,0,0,124,1,100,1,107,9, - 114,52,116,0,160,1,100,2,116,2,161,2,1,0,124,2, - 100,1,107,9,114,40,100,3,125,3,116,3,124,3,131,1, - 130,1,124,1,114,48,100,4,110,2,100,5,125,2,116,4, - 160,5,124,0,161,1,125,0,116,6,124,0,131,1,92,2, - 125,4,125,5,124,5,160,7,100,6,161,1,92,3,125,6, - 125,7,125,8,116,8,106,9,106,10,125,9,124,9,100,1, - 107,8,114,114,116,11,100,7,131,1,130,1,100,4,160,12, - 124,6,114,126,124,6,110,2,124,8,124,7,124,9,103,3, - 161,1,125,10,124,2,100,1,107,8,114,172,116,8,106,13, - 106,14,100,8,107,2,114,164,100,4,125,2,110,8,116,8, - 106,13,106,14,125,2,116,15,124,2,131,1,125,2,124,2, - 100,4,107,3,114,224,124,2,160,16,161,0,115,210,116,17, - 100,9,160,18,124,2,161,1,131,1,130,1,100,10,160,18, - 124,10,116,19,124,2,161,3,125,10,124,10,116,20,100,8, - 25,0,23,0,125,11,116,8,106,21,100,1,107,9,144,1, - 114,76,116,22,124,4,131,1,144,1,115,16,116,23,116,4, - 160,24,161,0,124,4,131,2,125,4,124,4,100,5,25,0, - 100,11,107,2,144,1,114,56,124,4,100,8,25,0,116,25, - 107,7,144,1,114,56,124,4,100,12,100,1,133,2,25,0, - 125,4,116,23,116,8,106,21,124,4,160,26,116,25,161,1, - 124,11,131,3,83,0,116,23,124,4,116,27,124,11,131,3, - 83,0,41,13,97,254,2,0,0,71,105,118,101,110,32,116, - 104,101,32,112,97,116,104,32,116,111,32,97,32,46,112,121, - 32,102,105,108,101,44,32,114,101,116,117,114,110,32,116,104, - 101,32,112,97,116,104,32,116,111,32,105,116,115,32,46,112, - 121,99,32,102,105,108,101,46,10,10,32,32,32,32,84,104, - 101,32,46,112,121,32,102,105,108,101,32,100,111,101,115,32, - 110,111,116,32,110,101,101,100,32,116,111,32,101,120,105,115, - 116,59,32,116,104,105,115,32,115,105,109,112,108,121,32,114, - 101,116,117,114,110,115,32,116,104,101,32,112,97,116,104,32, - 116,111,32,116,104,101,10,32,32,32,32,46,112,121,99,32, - 102,105,108,101,32,99,97,108,99,117,108,97,116,101,100,32, - 97,115,32,105,102,32,116,104,101,32,46,112,121,32,102,105, - 108,101,32,119,101,114,101,32,105,109,112,111,114,116,101,100, - 46,10,10,32,32,32,32,84,104,101,32,39,111,112,116,105, - 109,105,122,97,116,105,111,110,39,32,112,97,114,97,109,101, - 116,101,114,32,99,111,110,116,114,111,108,115,32,116,104,101, - 32,112,114,101,115,117,109,101,100,32,111,112,116,105,109,105, - 122,97,116,105,111,110,32,108,101,118,101,108,32,111,102,10, - 32,32,32,32,116,104,101,32,98,121,116,101,99,111,100,101, - 32,102,105,108,101,46,32,73,102,32,39,111,112,116,105,109, - 105,122,97,116,105,111,110,39,32,105,115,32,110,111,116,32, - 78,111,110,101,44,32,116,104,101,32,115,116,114,105,110,103, - 32,114,101,112,114,101,115,101,110,116,97,116,105,111,110,10, - 32,32,32,32,111,102,32,116,104,101,32,97,114,103,117,109, - 101,110,116,32,105,115,32,116,97,107,101,110,32,97,110,100, - 32,118,101,114,105,102,105,101,100,32,116,111,32,98,101,32, - 97,108,112,104,97,110,117,109,101,114,105,99,32,40,101,108, - 115,101,32,86,97,108,117,101,69,114,114,111,114,10,32,32, - 32,32,105,115,32,114,97,105,115,101,100,41,46,10,10,32, - 32,32,32,84,104,101,32,100,101,98,117,103,95,111,118,101, - 114,114,105,100,101,32,112,97,114,97,109,101,116,101,114,32, - 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,73, - 102,32,100,101,98,117,103,95,111,118,101,114,114,105,100,101, - 32,105,115,32,110,111,116,32,78,111,110,101,44,10,32,32, - 32,32,97,32,84,114,117,101,32,118,97,108,117,101,32,105, - 115,32,116,104,101,32,115,97,109,101,32,97,115,32,115,101, - 116,116,105,110,103,32,39,111,112,116,105,109,105,122,97,116, - 105,111,110,39,32,116,111,32,116,104,101,32,101,109,112,116, - 121,32,115,116,114,105,110,103,10,32,32,32,32,119,104,105, - 108,101,32,97,32,70,97,108,115,101,32,118,97,108,117,101, - 32,105,115,32,101,113,117,105,118,97,108,101,110,116,32,116, - 111,32,115,101,116,116,105,110,103,32,39,111,112,116,105,109, - 105,122,97,116,105,111,110,39,32,116,111,32,39,49,39,46, - 10,10,32,32,32,32,73,102,32,115,121,115,46,105,109,112, - 108,101,109,101,110,116,97,116,105,111,110,46,99,97,99,104, - 101,95,116,97,103,32,105,115,32,78,111,110,101,32,116,104, - 101,110,32,78,111,116,73,109,112,108,101,109,101,110,116,101, - 100,69,114,114,111,114,32,105,115,32,114,97,105,115,101,100, - 46,10,10,32,32,32,32,78,122,70,116,104,101,32,100,101, - 98,117,103,95,111,118,101,114,114,105,100,101,32,112,97,114, - 97,109,101,116,101,114,32,105,115,32,100,101,112,114,101,99, - 97,116,101,100,59,32,117,115,101,32,39,111,112,116,105,109, - 105,122,97,116,105,111,110,39,32,105,110,115,116,101,97,100, - 122,50,100,101,98,117,103,95,111,118,101,114,114,105,100,101, - 32,111,114,32,111,112,116,105,109,105,122,97,116,105,111,110, - 32,109,117,115,116,32,98,101,32,115,101,116,32,116,111,32, - 78,111,110,101,114,30,0,0,0,114,29,0,0,0,218,1, - 46,122,36,115,121,115,46,105,109,112,108,101,109,101,110,116, - 97,116,105,111,110,46,99,97,99,104,101,95,116,97,103,32, - 105,115,32,78,111,110,101,233,0,0,0,0,122,24,123,33, - 114,125,32,105,115,32,110,111,116,32,97,108,112,104,97,110, - 117,109,101,114,105,99,122,7,123,125,46,123,125,123,125,250, - 1,58,114,60,0,0,0,41,28,218,9,95,119,97,114,110, - 105,110,103,115,218,4,119,97,114,110,218,18,68,101,112,114, - 101,99,97,116,105,111,110,87,97,114,110,105,110,103,218,9, - 84,121,112,101,69,114,114,111,114,114,1,0,0,0,218,6, - 102,115,112,97,116,104,114,38,0,0,0,114,32,0,0,0, - 114,6,0,0,0,218,14,105,109,112,108,101,109,101,110,116, - 97,116,105,111,110,218,9,99,97,99,104,101,95,116,97,103, - 218,19,78,111,116,73,109,112,108,101,109,101,110,116,101,100, - 69,114,114,111,114,114,26,0,0,0,218,5,102,108,97,103, - 115,218,8,111,112,116,105,109,105,122,101,218,3,115,116,114, - 218,7,105,115,97,108,110,117,109,218,10,86,97,108,117,101, - 69,114,114,111,114,114,51,0,0,0,218,4,95,79,80,84, - 218,17,66,89,84,69,67,79,68,69,95,83,85,70,70,73, - 88,69,83,218,14,112,121,99,97,99,104,101,95,112,114,101, - 102,105,120,114,49,0,0,0,114,28,0,0,0,114,45,0, - 0,0,114,21,0,0,0,218,6,108,115,116,114,105,112,218, - 8,95,80,89,67,65,67,72,69,41,12,114,35,0,0,0, - 90,14,100,101,98,117,103,95,111,118,101,114,114,105,100,101, - 114,61,0,0,0,218,7,109,101,115,115,97,103,101,218,4, - 104,101,97,100,114,37,0,0,0,90,4,98,97,115,101,218, - 3,115,101,112,218,4,114,101,115,116,90,3,116,97,103,90, - 15,97,108,109,111,115,116,95,102,105,108,101,110,97,109,101, - 218,8,102,105,108,101,110,97,109,101,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,218,17,99,97,99,104,101, - 95,102,114,111,109,95,115,111,117,114,99,101,26,1,0,0, - 115,68,0,0,0,0,18,8,1,6,1,6,1,8,1,4, + 97,98,115,46,10,10,32,32,32,32,67,111,110,115,105,100, + 101,114,115,32,97,32,87,105,110,100,111,119,115,32,100,114, + 105,118,101,45,114,101,108,97,116,105,118,101,32,112,97,116, + 104,32,40,110,111,32,100,114,105,118,101,44,32,98,117,116, + 32,115,116,97,114,116,115,32,119,105,116,104,32,115,108,97, + 115,104,41,32,116,111,10,32,32,32,32,115,116,105,108,108, + 32,98,101,32,34,97,98,115,111,108,117,116,101,34,46,10, + 32,32,32,32,114,29,0,0,0,233,3,0,0,0,41,3, + 114,8,0,0,0,114,21,0,0,0,218,20,95,112,97,116, + 104,115,101,112,115,95,119,105,116,104,95,99,111,108,111,110, + 41,1,114,35,0,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,11,95,112,97,116,104,95,105,115, + 97,98,115,105,0,0,0,115,2,0,0,0,0,6,114,49, + 0,0,0,105,182,1,0,0,99,3,0,0,0,0,0,0, + 0,6,0,0,0,11,0,0,0,67,0,0,0,115,162,0, + 0,0,100,1,160,0,124,0,116,1,124,0,131,1,161,2, + 125,3,116,2,160,3,124,3,116,2,106,4,116,2,106,5, + 66,0,116,2,106,6,66,0,124,2,100,2,64,0,161,3, + 125,4,122,50,116,7,160,8,124,4,100,3,161,2,143,16, + 125,5,124,5,160,9,124,1,161,1,1,0,87,0,53,0, + 81,0,82,0,88,0,116,2,160,10,124,3,124,0,161,2, + 1,0,87,0,110,58,4,0,116,11,107,10,114,156,1,0, + 1,0,1,0,122,14,116,2,160,12,124,3,161,1,1,0, + 87,0,110,20,4,0,116,11,107,10,114,148,1,0,1,0, + 1,0,89,0,110,2,88,0,130,0,89,0,110,2,88,0, + 100,4,83,0,41,5,122,162,66,101,115,116,45,101,102,102, + 111,114,116,32,102,117,110,99,116,105,111,110,32,116,111,32, + 119,114,105,116,101,32,100,97,116,97,32,116,111,32,97,32, + 112,97,116,104,32,97,116,111,109,105,99,97,108,108,121,46, + 10,32,32,32,32,66,101,32,112,114,101,112,97,114,101,100, + 32,116,111,32,104,97,110,100,108,101,32,97,32,70,105,108, + 101,69,120,105,115,116,115,69,114,114,111,114,32,105,102,32, + 99,111,110,99,117,114,114,101,110,116,32,119,114,105,116,105, + 110,103,32,111,102,32,116,104,101,10,32,32,32,32,116,101, + 109,112,111,114,97,114,121,32,102,105,108,101,32,105,115,32, + 97,116,116,101,109,112,116,101,100,46,122,5,123,125,46,123, + 125,105,182,1,0,0,90,2,119,98,78,41,13,218,6,102, + 111,114,109,97,116,218,2,105,100,114,1,0,0,0,90,4, + 111,112,101,110,90,6,79,95,69,88,67,76,90,7,79,95, + 67,82,69,65,84,90,8,79,95,87,82,79,78,76,89,218, + 3,95,105,111,218,6,70,105,108,101,73,79,218,5,119,114, + 105,116,101,218,7,114,101,112,108,97,99,101,114,40,0,0, + 0,90,6,117,110,108,105,110,107,41,6,114,35,0,0,0, + 218,4,100,97,116,97,114,42,0,0,0,90,8,112,97,116, + 104,95,116,109,112,90,2,102,100,218,4,102,105,108,101,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,13, + 95,119,114,105,116,101,95,97,116,111,109,105,99,114,0,0, + 0,115,30,0,0,0,0,5,16,1,6,1,16,0,6,255, + 4,2,2,3,14,1,20,1,16,1,14,1,2,1,14,1, + 14,1,6,1,114,58,0,0,0,105,73,13,0,0,233,2, + 0,0,0,114,13,0,0,0,115,2,0,0,0,13,10,90, + 11,95,95,112,121,99,97,99,104,101,95,95,122,4,111,112, + 116,45,122,3,46,112,121,122,4,46,112,121,99,78,41,1, + 218,12,111,112,116,105,109,105,122,97,116,105,111,110,99,2, + 0,0,0,1,0,0,0,12,0,0,0,5,0,0,0,67, + 0,0,0,115,88,1,0,0,124,1,100,1,107,9,114,52, + 116,0,160,1,100,2,116,2,161,2,1,0,124,2,100,1, + 107,9,114,40,100,3,125,3,116,3,124,3,131,1,130,1, + 124,1,114,48,100,4,110,2,100,5,125,2,116,4,160,5, + 124,0,161,1,125,0,116,6,124,0,131,1,92,2,125,4, + 125,5,124,5,160,7,100,6,161,1,92,3,125,6,125,7, + 125,8,116,8,106,9,106,10,125,9,124,9,100,1,107,8, + 114,114,116,11,100,7,131,1,130,1,100,4,160,12,124,6, + 114,126,124,6,110,2,124,8,124,7,124,9,103,3,161,1, + 125,10,124,2,100,1,107,8,114,172,116,8,106,13,106,14, + 100,8,107,2,114,164,100,4,125,2,110,8,116,8,106,13, + 106,14,125,2,116,15,124,2,131,1,125,2,124,2,100,4, + 107,3,114,224,124,2,160,16,161,0,115,210,116,17,100,9, + 160,18,124,2,161,1,131,1,130,1,100,10,160,18,124,10, + 116,19,124,2,161,3,125,10,124,10,116,20,100,8,25,0, + 23,0,125,11,116,8,106,21,100,1,107,9,144,1,114,76, + 116,22,124,4,131,1,144,1,115,16,116,23,116,4,160,24, + 161,0,124,4,131,2,125,4,124,4,100,5,25,0,100,11, + 107,2,144,1,114,56,124,4,100,8,25,0,116,25,107,7, + 144,1,114,56,124,4,100,12,100,1,133,2,25,0,125,4, + 116,23,116,8,106,21,124,4,160,26,116,25,161,1,124,11, + 131,3,83,0,116,23,124,4,116,27,124,11,131,3,83,0, + 41,13,97,254,2,0,0,71,105,118,101,110,32,116,104,101, + 32,112,97,116,104,32,116,111,32,97,32,46,112,121,32,102, + 105,108,101,44,32,114,101,116,117,114,110,32,116,104,101,32, + 112,97,116,104,32,116,111,32,105,116,115,32,46,112,121,99, + 32,102,105,108,101,46,10,10,32,32,32,32,84,104,101,32, + 46,112,121,32,102,105,108,101,32,100,111,101,115,32,110,111, + 116,32,110,101,101,100,32,116,111,32,101,120,105,115,116,59, + 32,116,104,105,115,32,115,105,109,112,108,121,32,114,101,116, + 117,114,110,115,32,116,104,101,32,112,97,116,104,32,116,111, + 32,116,104,101,10,32,32,32,32,46,112,121,99,32,102,105, + 108,101,32,99,97,108,99,117,108,97,116,101,100,32,97,115, + 32,105,102,32,116,104,101,32,46,112,121,32,102,105,108,101, + 32,119,101,114,101,32,105,109,112,111,114,116,101,100,46,10, + 10,32,32,32,32,84,104,101,32,39,111,112,116,105,109,105, + 122,97,116,105,111,110,39,32,112,97,114,97,109,101,116,101, + 114,32,99,111,110,116,114,111,108,115,32,116,104,101,32,112, + 114,101,115,117,109,101,100,32,111,112,116,105,109,105,122,97, + 116,105,111,110,32,108,101,118,101,108,32,111,102,10,32,32, + 32,32,116,104,101,32,98,121,116,101,99,111,100,101,32,102, + 105,108,101,46,32,73,102,32,39,111,112,116,105,109,105,122, + 97,116,105,111,110,39,32,105,115,32,110,111,116,32,78,111, + 110,101,44,32,116,104,101,32,115,116,114,105,110,103,32,114, + 101,112,114,101,115,101,110,116,97,116,105,111,110,10,32,32, + 32,32,111,102,32,116,104,101,32,97,114,103,117,109,101,110, + 116,32,105,115,32,116,97,107,101,110,32,97,110,100,32,118, + 101,114,105,102,105,101,100,32,116,111,32,98,101,32,97,108, + 112,104,97,110,117,109,101,114,105,99,32,40,101,108,115,101, + 32,86,97,108,117,101,69,114,114,111,114,10,32,32,32,32, + 105,115,32,114,97,105,115,101,100,41,46,10,10,32,32,32, + 32,84,104,101,32,100,101,98,117,103,95,111,118,101,114,114, + 105,100,101,32,112,97,114,97,109,101,116,101,114,32,105,115, + 32,100,101,112,114,101,99,97,116,101,100,46,32,73,102,32, + 100,101,98,117,103,95,111,118,101,114,114,105,100,101,32,105, + 115,32,110,111,116,32,78,111,110,101,44,10,32,32,32,32, + 97,32,84,114,117,101,32,118,97,108,117,101,32,105,115,32, + 116,104,101,32,115,97,109,101,32,97,115,32,115,101,116,116, + 105,110,103,32,39,111,112,116,105,109,105,122,97,116,105,111, + 110,39,32,116,111,32,116,104,101,32,101,109,112,116,121,32, + 115,116,114,105,110,103,10,32,32,32,32,119,104,105,108,101, + 32,97,32,70,97,108,115,101,32,118,97,108,117,101,32,105, + 115,32,101,113,117,105,118,97,108,101,110,116,32,116,111,32, + 115,101,116,116,105,110,103,32,39,111,112,116,105,109,105,122, + 97,116,105,111,110,39,32,116,111,32,39,49,39,46,10,10, + 32,32,32,32,73,102,32,115,121,115,46,105,109,112,108,101, + 109,101,110,116,97,116,105,111,110,46,99,97,99,104,101,95, + 116,97,103,32,105,115,32,78,111,110,101,32,116,104,101,110, + 32,78,111,116,73,109,112,108,101,109,101,110,116,101,100,69, + 114,114,111,114,32,105,115,32,114,97,105,115,101,100,46,10, + 10,32,32,32,32,78,122,70,116,104,101,32,100,101,98,117, + 103,95,111,118,101,114,114,105,100,101,32,112,97,114,97,109, + 101,116,101,114,32,105,115,32,100,101,112,114,101,99,97,116, + 101,100,59,32,117,115,101,32,39,111,112,116,105,109,105,122, + 97,116,105,111,110,39,32,105,110,115,116,101,97,100,122,50, + 100,101,98,117,103,95,111,118,101,114,114,105,100,101,32,111, + 114,32,111,112,116,105,109,105,122,97,116,105,111,110,32,109, + 117,115,116,32,98,101,32,115,101,116,32,116,111,32,78,111, + 110,101,114,30,0,0,0,114,29,0,0,0,218,1,46,122, + 36,115,121,115,46,105,109,112,108,101,109,101,110,116,97,116, + 105,111,110,46,99,97,99,104,101,95,116,97,103,32,105,115, + 32,78,111,110,101,233,0,0,0,0,122,24,123,33,114,125, + 32,105,115,32,110,111,116,32,97,108,112,104,97,110,117,109, + 101,114,105,99,122,7,123,125,46,123,125,123,125,250,1,58, + 114,59,0,0,0,41,28,218,9,95,119,97,114,110,105,110, + 103,115,218,4,119,97,114,110,218,18,68,101,112,114,101,99, + 97,116,105,111,110,87,97,114,110,105,110,103,218,9,84,121, + 112,101,69,114,114,111,114,114,1,0,0,0,218,6,102,115, + 112,97,116,104,114,38,0,0,0,114,32,0,0,0,114,6, + 0,0,0,218,14,105,109,112,108,101,109,101,110,116,97,116, + 105,111,110,218,9,99,97,99,104,101,95,116,97,103,218,19, + 78,111,116,73,109,112,108,101,109,101,110,116,101,100,69,114, + 114,111,114,114,26,0,0,0,218,5,102,108,97,103,115,218, + 8,111,112,116,105,109,105,122,101,218,3,115,116,114,218,7, + 105,115,97,108,110,117,109,218,10,86,97,108,117,101,69,114, + 114,111,114,114,50,0,0,0,218,4,95,79,80,84,218,17, + 66,89,84,69,67,79,68,69,95,83,85,70,70,73,88,69, + 83,218,14,112,121,99,97,99,104,101,95,112,114,101,102,105, + 120,114,49,0,0,0,114,28,0,0,0,114,45,0,0,0, + 114,21,0,0,0,218,6,108,115,116,114,105,112,218,8,95, + 80,89,67,65,67,72,69,41,12,114,35,0,0,0,90,14, + 100,101,98,117,103,95,111,118,101,114,114,105,100,101,114,60, + 0,0,0,218,7,109,101,115,115,97,103,101,218,4,104,101, + 97,100,114,37,0,0,0,90,4,98,97,115,101,218,3,115, + 101,112,218,4,114,101,115,116,90,3,116,97,103,90,15,97, + 108,109,111,115,116,95,102,105,108,101,110,97,109,101,218,8, + 102,105,108,101,110,97,109,101,114,2,0,0,0,114,2,0, + 0,0,114,4,0,0,0,218,17,99,97,99,104,101,95,102, + 114,111,109,95,115,111,117,114,99,101,26,1,0,0,115,72, + 0,0,0,0,18,8,1,6,1,2,255,4,2,8,1,4, 1,8,1,12,1,10,1,12,1,16,1,8,1,8,1,8, 1,24,1,8,1,12,1,6,2,8,1,8,1,8,1,8, 1,14,1,14,1,12,1,12,9,10,1,14,5,28,1,12, - 4,2,1,4,1,8,1,6,2,114,88,0,0,0,99,1, - 0,0,0,0,0,0,0,10,0,0,0,5,0,0,0,67, - 0,0,0,115,46,1,0,0,116,0,106,1,106,2,100,1, - 107,8,114,20,116,3,100,2,131,1,130,1,116,4,160,5, - 124,0,161,1,125,0,116,6,124,0,131,1,92,2,125,1, - 125,2,100,3,125,3,116,0,106,7,100,1,107,9,114,102, - 116,0,106,7,160,8,116,9,161,1,125,4,124,1,160,10, - 124,4,116,11,23,0,161,1,114,102,124,1,116,12,124,4, - 131,1,100,1,133,2,25,0,125,1,100,4,125,3,124,3, - 115,144,116,6,124,1,131,1,92,2,125,1,125,5,124,5, - 116,13,107,3,114,144,116,14,116,13,155,0,100,5,124,0, - 155,2,157,3,131,1,130,1,124,2,160,15,100,6,161,1, - 125,6,124,6,100,7,107,7,114,178,116,14,100,8,124,2, - 155,2,157,2,131,1,130,1,110,92,124,6,100,9,107,2, - 144,1,114,14,124,2,160,16,100,6,100,10,161,2,100,11, - 25,0,125,7,124,7,160,10,116,17,161,1,115,228,116,14, - 100,12,116,17,155,2,157,2,131,1,130,1,124,7,116,12, - 116,17,131,1,100,1,133,2,25,0,125,8,124,8,160,18, - 161,0,144,1,115,14,116,14,100,13,124,7,155,2,100,14, - 157,3,131,1,130,1,124,2,160,19,100,6,161,1,100,15, - 25,0,125,9,116,20,124,1,124,9,116,21,100,15,25,0, - 23,0,131,2,83,0,41,16,97,110,1,0,0,71,105,118, - 101,110,32,116,104,101,32,112,97,116,104,32,116,111,32,97, - 32,46,112,121,99,46,32,102,105,108,101,44,32,114,101,116, - 117,114,110,32,116,104,101,32,112,97,116,104,32,116,111,32, - 105,116,115,32,46,112,121,32,102,105,108,101,46,10,10,32, - 32,32,32,84,104,101,32,46,112,121,99,32,102,105,108,101, - 32,100,111,101,115,32,110,111,116,32,110,101,101,100,32,116, - 111,32,101,120,105,115,116,59,32,116,104,105,115,32,115,105, - 109,112,108,121,32,114,101,116,117,114,110,115,32,116,104,101, - 32,112,97,116,104,32,116,111,10,32,32,32,32,116,104,101, - 32,46,112,121,32,102,105,108,101,32,99,97,108,99,117,108, - 97,116,101,100,32,116,111,32,99,111,114,114,101,115,112,111, - 110,100,32,116,111,32,116,104,101,32,46,112,121,99,32,102, - 105,108,101,46,32,32,73,102,32,112,97,116,104,32,100,111, - 101,115,10,32,32,32,32,110,111,116,32,99,111,110,102,111, - 114,109,32,116,111,32,80,69,80,32,51,49,52,55,47,52, - 56,56,32,102,111,114,109,97,116,44,32,86,97,108,117,101, - 69,114,114,111,114,32,119,105,108,108,32,98,101,32,114,97, - 105,115,101,100,46,32,73,102,10,32,32,32,32,115,121,115, - 46,105,109,112,108,101,109,101,110,116,97,116,105,111,110,46, - 99,97,99,104,101,95,116,97,103,32,105,115,32,78,111,110, - 101,32,116,104,101,110,32,78,111,116,73,109,112,108,101,109, - 101,110,116,101,100,69,114,114,111,114,32,105,115,32,114,97, - 105,115,101,100,46,10,10,32,32,32,32,78,122,36,115,121, - 115,46,105,109,112,108,101,109,101,110,116,97,116,105,111,110, - 46,99,97,99,104,101,95,116,97,103,32,105,115,32,78,111, - 110,101,70,84,122,31,32,110,111,116,32,98,111,116,116,111, - 109,45,108,101,118,101,108,32,100,105,114,101,99,116,111,114, - 121,32,105,110,32,114,62,0,0,0,62,2,0,0,0,114, - 60,0,0,0,114,47,0,0,0,122,29,101,120,112,101,99, - 116,101,100,32,111,110,108,121,32,50,32,111,114,32,51,32, - 100,111,116,115,32,105,110,32,114,47,0,0,0,114,60,0, - 0,0,233,254,255,255,255,122,53,111,112,116,105,109,105,122, - 97,116,105,111,110,32,112,111,114,116,105,111,110,32,111,102, - 32,102,105,108,101,110,97,109,101,32,100,111,101,115,32,110, - 111,116,32,115,116,97,114,116,32,119,105,116,104,32,122,19, - 111,112,116,105,109,105,122,97,116,105,111,110,32,108,101,118, - 101,108,32,122,29,32,105,115,32,110,111,116,32,97,110,32, - 97,108,112,104,97,110,117,109,101,114,105,99,32,118,97,108, - 117,101,114,63,0,0,0,41,22,114,6,0,0,0,114,70, - 0,0,0,114,71,0,0,0,114,72,0,0,0,114,1,0, - 0,0,114,69,0,0,0,114,38,0,0,0,114,80,0,0, - 0,114,20,0,0,0,114,21,0,0,0,114,8,0,0,0, - 114,25,0,0,0,114,31,0,0,0,114,82,0,0,0,114, - 77,0,0,0,218,5,99,111,117,110,116,114,34,0,0,0, - 114,78,0,0,0,114,76,0,0,0,218,9,112,97,114,116, - 105,116,105,111,110,114,28,0,0,0,218,15,83,79,85,82, - 67,69,95,83,85,70,70,73,88,69,83,41,10,114,35,0, - 0,0,114,84,0,0,0,90,16,112,121,99,97,99,104,101, - 95,102,105,108,101,110,97,109,101,90,23,102,111,117,110,100, - 95,105,110,95,112,121,99,97,99,104,101,95,112,114,101,102, - 105,120,90,13,115,116,114,105,112,112,101,100,95,112,97,116, - 104,90,7,112,121,99,97,99,104,101,90,9,100,111,116,95, - 99,111,117,110,116,114,61,0,0,0,90,9,111,112,116,95, - 108,101,118,101,108,90,13,98,97,115,101,95,102,105,108,101, - 110,97,109,101,114,2,0,0,0,114,2,0,0,0,114,4, - 0,0,0,218,17,115,111,117,114,99,101,95,102,114,111,109, - 95,99,97,99,104,101,97,1,0,0,115,52,0,0,0,0, - 9,12,1,8,1,10,1,12,1,4,1,10,1,12,1,14, - 1,16,1,4,1,4,1,12,1,8,1,18,2,10,1,8, - 1,16,1,10,1,16,1,10,1,14,2,16,1,10,1,16, - 2,14,1,114,93,0,0,0,99,1,0,0,0,0,0,0, - 0,5,0,0,0,9,0,0,0,67,0,0,0,115,126,0, - 0,0,116,0,124,0,131,1,100,1,107,2,114,16,100,2, - 83,0,124,0,160,1,100,3,161,1,92,3,125,1,125,2, - 125,3,124,1,114,56,124,3,160,2,161,0,100,4,100,5, - 133,2,25,0,100,6,107,3,114,60,124,0,83,0,122,12, - 116,3,124,0,131,1,125,4,87,0,110,36,4,0,116,4, - 116,5,102,2,107,10,114,108,1,0,1,0,1,0,124,0, - 100,2,100,5,133,2,25,0,125,4,89,0,110,2,88,0, - 116,6,124,4,131,1,114,122,124,4,83,0,124,0,83,0, - 41,7,122,188,67,111,110,118,101,114,116,32,97,32,98,121, - 116,101,99,111,100,101,32,102,105,108,101,32,112,97,116,104, - 32,116,111,32,97,32,115,111,117,114,99,101,32,112,97,116, - 104,32,40,105,102,32,112,111,115,115,105,98,108,101,41,46, - 10,10,32,32,32,32,84,104,105,115,32,102,117,110,99,116, - 105,111,110,32,101,120,105,115,116,115,32,112,117,114,101,108, - 121,32,102,111,114,32,98,97,99,107,119,97,114,100,115,45, - 99,111,109,112,97,116,105,98,105,108,105,116,121,32,102,111, - 114,10,32,32,32,32,80,121,73,109,112,111,114,116,95,69, - 120,101,99,67,111,100,101,77,111,100,117,108,101,87,105,116, - 104,70,105,108,101,110,97,109,101,115,40,41,32,105,110,32, - 116,104,101,32,67,32,65,80,73,46,10,10,32,32,32,32, - 114,63,0,0,0,78,114,62,0,0,0,233,253,255,255,255, - 233,255,255,255,255,90,2,112,121,41,7,114,31,0,0,0, - 114,32,0,0,0,218,5,108,111,119,101,114,114,93,0,0, - 0,114,72,0,0,0,114,77,0,0,0,114,44,0,0,0, - 41,5,218,13,98,121,116,101,99,111,100,101,95,112,97,116, - 104,114,86,0,0,0,114,36,0,0,0,90,9,101,120,116, - 101,110,115,105,111,110,218,11,115,111,117,114,99,101,95,112, - 97,116,104,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,218,15,95,103,101,116,95,115,111,117,114,99,101,102, - 105,108,101,137,1,0,0,115,20,0,0,0,0,7,12,1, - 4,1,16,1,24,1,4,1,2,1,12,1,18,1,18,1, - 114,99,0,0,0,99,1,0,0,0,0,0,0,0,1,0, - 0,0,8,0,0,0,67,0,0,0,115,74,0,0,0,124, - 0,160,0,116,1,116,2,131,1,161,1,114,48,122,10,116, - 3,124,0,131,1,87,0,83,0,4,0,116,4,107,10,114, - 44,1,0,1,0,1,0,89,0,113,70,88,0,110,22,124, - 0,160,0,116,1,116,5,131,1,161,1,114,66,124,0,83, - 0,100,0,83,0,100,0,83,0,41,1,78,41,6,218,8, - 101,110,100,115,119,105,116,104,218,5,116,117,112,108,101,114, - 92,0,0,0,114,88,0,0,0,114,72,0,0,0,114,79, - 0,0,0,41,1,114,87,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,218,11,95,103,101,116,95, - 99,97,99,104,101,100,156,1,0,0,115,16,0,0,0,0, - 1,14,1,2,1,10,1,14,1,8,1,14,1,4,2,114, - 102,0,0,0,99,1,0,0,0,0,0,0,0,2,0,0, - 0,8,0,0,0,67,0,0,0,115,52,0,0,0,122,14, - 116,0,124,0,131,1,106,1,125,1,87,0,110,24,4,0, - 116,2,107,10,114,38,1,0,1,0,1,0,100,1,125,1, - 89,0,110,2,88,0,124,1,100,2,79,0,125,1,124,1, - 83,0,41,3,122,51,67,97,108,99,117,108,97,116,101,32, - 116,104,101,32,109,111,100,101,32,112,101,114,109,105,115,115, - 105,111,110,115,32,102,111,114,32,97,32,98,121,116,101,99, - 111,100,101,32,102,105,108,101,46,105,182,1,0,0,233,128, - 0,0,0,41,3,114,39,0,0,0,114,41,0,0,0,114, - 40,0,0,0,41,2,114,35,0,0,0,114,42,0,0,0, - 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 10,95,99,97,108,99,95,109,111,100,101,168,1,0,0,115, - 12,0,0,0,0,2,2,1,14,1,14,1,10,3,8,1, - 114,104,0,0,0,99,1,0,0,0,0,0,0,0,3,0, - 0,0,8,0,0,0,3,0,0,0,115,68,0,0,0,100, - 6,135,0,102,1,100,2,100,3,132,9,125,1,122,10,116, - 0,106,1,125,2,87,0,110,28,4,0,116,2,107,10,114, - 52,1,0,1,0,1,0,100,4,100,5,132,0,125,2,89, - 0,110,2,88,0,124,2,124,1,136,0,131,2,1,0,124, - 1,83,0,41,7,122,252,68,101,99,111,114,97,116,111,114, - 32,116,111,32,118,101,114,105,102,121,32,116,104,97,116,32, - 116,104,101,32,109,111,100,117,108,101,32,98,101,105,110,103, - 32,114,101,113,117,101,115,116,101,100,32,109,97,116,99,104, - 101,115,32,116,104,101,32,111,110,101,32,116,104,101,10,32, - 32,32,32,108,111,97,100,101,114,32,99,97,110,32,104,97, - 110,100,108,101,46,10,10,32,32,32,32,84,104,101,32,102, - 105,114,115,116,32,97,114,103,117,109,101,110,116,32,40,115, - 101,108,102,41,32,109,117,115,116,32,100,101,102,105,110,101, - 32,95,110,97,109,101,32,119,104,105,99,104,32,116,104,101, - 32,115,101,99,111,110,100,32,97,114,103,117,109,101,110,116, - 32,105,115,10,32,32,32,32,99,111,109,112,97,114,101,100, - 32,97,103,97,105,110,115,116,46,32,73,102,32,116,104,101, - 32,99,111,109,112,97,114,105,115,111,110,32,102,97,105,108, - 115,32,116,104,101,110,32,73,109,112,111,114,116,69,114,114, - 111,114,32,105,115,32,114,97,105,115,101,100,46,10,10,32, - 32,32,32,78,99,2,0,0,0,0,0,0,0,4,0,0, - 0,4,0,0,0,31,0,0,0,115,66,0,0,0,124,1, - 100,0,107,8,114,16,124,0,106,0,125,1,110,32,124,0, - 106,0,124,1,107,3,114,48,116,1,100,1,124,0,106,0, - 124,1,102,2,22,0,124,1,100,2,141,2,130,1,136,0, - 124,0,124,1,102,2,124,2,158,2,124,3,142,1,83,0, - 41,3,78,122,30,108,111,97,100,101,114,32,102,111,114,32, - 37,115,32,99,97,110,110,111,116,32,104,97,110,100,108,101, - 32,37,115,41,1,218,4,110,97,109,101,41,2,114,105,0, - 0,0,218,11,73,109,112,111,114,116,69,114,114,111,114,41, - 4,218,4,115,101,108,102,114,105,0,0,0,218,4,97,114, - 103,115,90,6,107,119,97,114,103,115,41,1,218,6,109,101, - 116,104,111,100,114,2,0,0,0,114,4,0,0,0,218,19, - 95,99,104,101,99,107,95,110,97,109,101,95,119,114,97,112, - 112,101,114,188,1,0,0,115,12,0,0,0,0,1,8,1, - 8,1,10,1,4,1,18,1,122,40,95,99,104,101,99,107, - 95,110,97,109,101,46,60,108,111,99,97,108,115,62,46,95, - 99,104,101,99,107,95,110,97,109,101,95,119,114,97,112,112, - 101,114,99,2,0,0,0,0,0,0,0,3,0,0,0,7, - 0,0,0,83,0,0,0,115,56,0,0,0,100,1,68,0, - 93,32,125,2,116,0,124,1,124,2,131,2,114,4,116,1, - 124,0,124,2,116,2,124,1,124,2,131,2,131,3,1,0, - 113,4,124,0,106,3,160,4,124,1,106,3,161,1,1,0, - 100,0,83,0,41,2,78,41,4,218,10,95,95,109,111,100, - 117,108,101,95,95,218,8,95,95,110,97,109,101,95,95,218, - 12,95,95,113,117,97,108,110,97,109,101,95,95,218,7,95, - 95,100,111,99,95,95,41,5,218,7,104,97,115,97,116,116, - 114,218,7,115,101,116,97,116,116,114,218,7,103,101,116,97, - 116,116,114,218,8,95,95,100,105,99,116,95,95,218,6,117, - 112,100,97,116,101,41,3,90,3,110,101,119,90,3,111,108, - 100,114,56,0,0,0,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,218,5,95,119,114,97,112,199,1,0,0, - 115,8,0,0,0,0,1,8,1,10,1,20,1,122,26,95, - 99,104,101,99,107,95,110,97,109,101,46,60,108,111,99,97, - 108,115,62,46,95,119,114,97,112,41,1,78,41,3,218,10, - 95,98,111,111,116,115,116,114,97,112,114,120,0,0,0,218, - 9,78,97,109,101,69,114,114,111,114,41,3,114,109,0,0, - 0,114,110,0,0,0,114,120,0,0,0,114,2,0,0,0, - 41,1,114,109,0,0,0,114,4,0,0,0,218,11,95,99, - 104,101,99,107,95,110,97,109,101,180,1,0,0,115,14,0, - 0,0,0,8,14,7,2,1,10,1,14,2,14,5,10,1, - 114,123,0,0,0,99,2,0,0,0,0,0,0,0,5,0, - 0,0,6,0,0,0,67,0,0,0,115,60,0,0,0,124, - 0,160,0,124,1,161,1,92,2,125,2,125,3,124,2,100, - 1,107,8,114,56,116,1,124,3,131,1,114,56,100,2,125, - 4,116,2,160,3,124,4,160,4,124,3,100,3,25,0,161, - 1,116,5,161,2,1,0,124,2,83,0,41,4,122,155,84, - 114,121,32,116,111,32,102,105,110,100,32,97,32,108,111,97, - 100,101,114,32,102,111,114,32,116,104,101,32,115,112,101,99, - 105,102,105,101,100,32,109,111,100,117,108,101,32,98,121,32, - 100,101,108,101,103,97,116,105,110,103,32,116,111,10,32,32, - 32,32,115,101,108,102,46,102,105,110,100,95,108,111,97,100, - 101,114,40,41,46,10,10,32,32,32,32,84,104,105,115,32, - 109,101,116,104,111,100,32,105,115,32,100,101,112,114,101,99, - 97,116,101,100,32,105,110,32,102,97,118,111,114,32,111,102, - 32,102,105,110,100,101,114,46,102,105,110,100,95,115,112,101, - 99,40,41,46,10,10,32,32,32,32,78,122,44,78,111,116, - 32,105,109,112,111,114,116,105,110,103,32,100,105,114,101,99, - 116,111,114,121,32,123,125,58,32,109,105,115,115,105,110,103, - 32,95,95,105,110,105,116,95,95,114,63,0,0,0,41,6, - 218,11,102,105,110,100,95,108,111,97,100,101,114,114,31,0, - 0,0,114,65,0,0,0,114,66,0,0,0,114,51,0,0, - 0,218,13,73,109,112,111,114,116,87,97,114,110,105,110,103, - 41,5,114,107,0,0,0,218,8,102,117,108,108,110,97,109, - 101,218,6,108,111,97,100,101,114,218,8,112,111,114,116,105, - 111,110,115,218,3,109,115,103,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,218,17,95,102,105,110,100,95,109, - 111,100,117,108,101,95,115,104,105,109,208,1,0,0,115,10, - 0,0,0,0,10,14,1,16,1,4,1,22,1,114,130,0, - 0,0,99,3,0,0,0,0,0,0,0,6,0,0,0,4, - 0,0,0,67,0,0,0,115,158,0,0,0,124,0,100,1, - 100,2,133,2,25,0,125,3,124,3,116,0,107,3,114,60, - 100,3,124,1,155,2,100,4,124,3,155,2,157,4,125,4, - 116,1,160,2,100,5,124,4,161,2,1,0,116,3,124,4, - 102,1,124,2,142,1,130,1,116,4,124,0,131,1,100,6, - 107,0,114,102,100,7,124,1,155,2,157,2,125,4,116,1, - 160,2,100,5,124,4,161,2,1,0,116,5,124,4,131,1, - 130,1,116,6,124,0,100,2,100,8,133,2,25,0,131,1, - 125,5,124,5,100,9,64,0,114,154,100,10,124,5,155,2, - 100,11,124,1,155,2,157,4,125,4,116,3,124,4,102,1, - 124,2,142,1,130,1,124,5,83,0,41,12,97,84,2,0, - 0,80,101,114,102,111,114,109,32,98,97,115,105,99,32,118, - 97,108,105,100,105,116,121,32,99,104,101,99,107,105,110,103, - 32,111,102,32,97,32,112,121,99,32,104,101,97,100,101,114, - 32,97,110,100,32,114,101,116,117,114,110,32,116,104,101,32, - 102,108,97,103,115,32,102,105,101,108,100,44,10,32,32,32, - 32,119,104,105,99,104,32,100,101,116,101,114,109,105,110,101, - 115,32,104,111,119,32,116,104,101,32,112,121,99,32,115,104, - 111,117,108,100,32,98,101,32,102,117,114,116,104,101,114,32, - 118,97,108,105,100,97,116,101,100,32,97,103,97,105,110,115, - 116,32,116,104,101,32,115,111,117,114,99,101,46,10,10,32, - 32,32,32,42,100,97,116,97,42,32,105,115,32,116,104,101, - 32,99,111,110,116,101,110,116,115,32,111,102,32,116,104,101, - 32,112,121,99,32,102,105,108,101,46,32,40,79,110,108,121, - 32,116,104,101,32,102,105,114,115,116,32,49,54,32,98,121, - 116,101,115,32,97,114,101,10,32,32,32,32,114,101,113,117, - 105,114,101,100,44,32,116,104,111,117,103,104,46,41,10,10, - 32,32,32,32,42,110,97,109,101,42,32,105,115,32,116,104, - 101,32,110,97,109,101,32,111,102,32,116,104,101,32,109,111, - 100,117,108,101,32,98,101,105,110,103,32,105,109,112,111,114, - 116,101,100,46,32,73,116,32,105,115,32,117,115,101,100,32, - 102,111,114,32,108,111,103,103,105,110,103,46,10,10,32,32, - 32,32,42,101,120,99,95,100,101,116,97,105,108,115,42,32, - 105,115,32,97,32,100,105,99,116,105,111,110,97,114,121,32, - 112,97,115,115,101,100,32,116,111,32,73,109,112,111,114,116, - 69,114,114,111,114,32,105,102,32,105,116,32,114,97,105,115, - 101,100,32,102,111,114,10,32,32,32,32,105,109,112,114,111, - 118,101,100,32,100,101,98,117,103,103,105,110,103,46,10,10, - 32,32,32,32,73,109,112,111,114,116,69,114,114,111,114,32, - 105,115,32,114,97,105,115,101,100,32,119,104,101,110,32,116, - 104,101,32,109,97,103,105,99,32,110,117,109,98,101,114,32, - 105,115,32,105,110,99,111,114,114,101,99,116,32,111,114,32, - 119,104,101,110,32,116,104,101,32,102,108,97,103,115,10,32, - 32,32,32,102,105,101,108,100,32,105,115,32,105,110,118,97, - 108,105,100,46,32,69,79,70,69,114,114,111,114,32,105,115, - 32,114,97,105,115,101,100,32,119,104,101,110,32,116,104,101, - 32,100,97,116,97,32,105,115,32,102,111,117,110,100,32,116, - 111,32,98,101,32,116,114,117,110,99,97,116,101,100,46,10, - 10,32,32,32,32,78,114,12,0,0,0,122,20,98,97,100, - 32,109,97,103,105,99,32,110,117,109,98,101,114,32,105,110, - 32,122,2,58,32,122,2,123,125,233,16,0,0,0,122,40, - 114,101,97,99,104,101,100,32,69,79,70,32,119,104,105,108, - 101,32,114,101,97,100,105,110,103,32,112,121,99,32,104,101, - 97,100,101,114,32,111,102,32,233,8,0,0,0,233,252,255, - 255,255,122,14,105,110,118,97,108,105,100,32,102,108,97,103, - 115,32,122,4,32,105,110,32,41,7,218,12,77,65,71,73, - 67,95,78,85,77,66,69,82,114,121,0,0,0,218,16,95, - 118,101,114,98,111,115,101,95,109,101,115,115,97,103,101,114, - 106,0,0,0,114,31,0,0,0,218,8,69,79,70,69,114, - 114,111,114,114,19,0,0,0,41,6,114,57,0,0,0,114, - 105,0,0,0,218,11,101,120,99,95,100,101,116,97,105,108, - 115,90,5,109,97,103,105,99,114,83,0,0,0,114,73,0, + 4,2,1,4,1,8,1,2,253,4,5,114,87,0,0,0, + 99,1,0,0,0,0,0,0,0,10,0,0,0,5,0,0, + 0,67,0,0,0,115,46,1,0,0,116,0,106,1,106,2, + 100,1,107,8,114,20,116,3,100,2,131,1,130,1,116,4, + 160,5,124,0,161,1,125,0,116,6,124,0,131,1,92,2, + 125,1,125,2,100,3,125,3,116,0,106,7,100,1,107,9, + 114,102,116,0,106,7,160,8,116,9,161,1,125,4,124,1, + 160,10,124,4,116,11,23,0,161,1,114,102,124,1,116,12, + 124,4,131,1,100,1,133,2,25,0,125,1,100,4,125,3, + 124,3,115,144,116,6,124,1,131,1,92,2,125,1,125,5, + 124,5,116,13,107,3,114,144,116,14,116,13,155,0,100,5, + 124,0,155,2,157,3,131,1,130,1,124,2,160,15,100,6, + 161,1,125,6,124,6,100,7,107,7,114,178,116,14,100,8, + 124,2,155,2,157,2,131,1,130,1,110,92,124,6,100,9, + 107,2,144,1,114,14,124,2,160,16,100,6,100,10,161,2, + 100,11,25,0,125,7,124,7,160,10,116,17,161,1,115,228, + 116,14,100,12,116,17,155,2,157,2,131,1,130,1,124,7, + 116,12,116,17,131,1,100,1,133,2,25,0,125,8,124,8, + 160,18,161,0,144,1,115,14,116,14,100,13,124,7,155,2, + 100,14,157,3,131,1,130,1,124,2,160,19,100,6,161,1, + 100,15,25,0,125,9,116,20,124,1,124,9,116,21,100,15, + 25,0,23,0,131,2,83,0,41,16,97,110,1,0,0,71, + 105,118,101,110,32,116,104,101,32,112,97,116,104,32,116,111, + 32,97,32,46,112,121,99,46,32,102,105,108,101,44,32,114, + 101,116,117,114,110,32,116,104,101,32,112,97,116,104,32,116, + 111,32,105,116,115,32,46,112,121,32,102,105,108,101,46,10, + 10,32,32,32,32,84,104,101,32,46,112,121,99,32,102,105, + 108,101,32,100,111,101,115,32,110,111,116,32,110,101,101,100, + 32,116,111,32,101,120,105,115,116,59,32,116,104,105,115,32, + 115,105,109,112,108,121,32,114,101,116,117,114,110,115,32,116, + 104,101,32,112,97,116,104,32,116,111,10,32,32,32,32,116, + 104,101,32,46,112,121,32,102,105,108,101,32,99,97,108,99, + 117,108,97,116,101,100,32,116,111,32,99,111,114,114,101,115, + 112,111,110,100,32,116,111,32,116,104,101,32,46,112,121,99, + 32,102,105,108,101,46,32,32,73,102,32,112,97,116,104,32, + 100,111,101,115,10,32,32,32,32,110,111,116,32,99,111,110, + 102,111,114,109,32,116,111,32,80,69,80,32,51,49,52,55, + 47,52,56,56,32,102,111,114,109,97,116,44,32,86,97,108, + 117,101,69,114,114,111,114,32,119,105,108,108,32,98,101,32, + 114,97,105,115,101,100,46,32,73,102,10,32,32,32,32,115, + 121,115,46,105,109,112,108,101,109,101,110,116,97,116,105,111, + 110,46,99,97,99,104,101,95,116,97,103,32,105,115,32,78, + 111,110,101,32,116,104,101,110,32,78,111,116,73,109,112,108, + 101,109,101,110,116,101,100,69,114,114,111,114,32,105,115,32, + 114,97,105,115,101,100,46,10,10,32,32,32,32,78,122,36, + 115,121,115,46,105,109,112,108,101,109,101,110,116,97,116,105, + 111,110,46,99,97,99,104,101,95,116,97,103,32,105,115,32, + 78,111,110,101,70,84,122,31,32,110,111,116,32,98,111,116, + 116,111,109,45,108,101,118,101,108,32,100,105,114,101,99,116, + 111,114,121,32,105,110,32,114,61,0,0,0,62,2,0,0, + 0,114,59,0,0,0,114,47,0,0,0,122,29,101,120,112, + 101,99,116,101,100,32,111,110,108,121,32,50,32,111,114,32, + 51,32,100,111,116,115,32,105,110,32,114,47,0,0,0,114, + 59,0,0,0,233,254,255,255,255,122,53,111,112,116,105,109, + 105,122,97,116,105,111,110,32,112,111,114,116,105,111,110,32, + 111,102,32,102,105,108,101,110,97,109,101,32,100,111,101,115, + 32,110,111,116,32,115,116,97,114,116,32,119,105,116,104,32, + 122,19,111,112,116,105,109,105,122,97,116,105,111,110,32,108, + 101,118,101,108,32,122,29,32,105,115,32,110,111,116,32,97, + 110,32,97,108,112,104,97,110,117,109,101,114,105,99,32,118, + 97,108,117,101,114,62,0,0,0,41,22,114,6,0,0,0, + 114,69,0,0,0,114,70,0,0,0,114,71,0,0,0,114, + 1,0,0,0,114,68,0,0,0,114,38,0,0,0,114,79, + 0,0,0,114,20,0,0,0,114,21,0,0,0,114,8,0, + 0,0,114,25,0,0,0,114,31,0,0,0,114,81,0,0, + 0,114,76,0,0,0,218,5,99,111,117,110,116,114,34,0, + 0,0,114,77,0,0,0,114,75,0,0,0,218,9,112,97, + 114,116,105,116,105,111,110,114,28,0,0,0,218,15,83,79, + 85,82,67,69,95,83,85,70,70,73,88,69,83,41,10,114, + 35,0,0,0,114,83,0,0,0,90,16,112,121,99,97,99, + 104,101,95,102,105,108,101,110,97,109,101,90,23,102,111,117, + 110,100,95,105,110,95,112,121,99,97,99,104,101,95,112,114, + 101,102,105,120,90,13,115,116,114,105,112,112,101,100,95,112, + 97,116,104,90,7,112,121,99,97,99,104,101,90,9,100,111, + 116,95,99,111,117,110,116,114,60,0,0,0,90,9,111,112, + 116,95,108,101,118,101,108,90,13,98,97,115,101,95,102,105, + 108,101,110,97,109,101,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,218,17,115,111,117,114,99,101,95,102,114, + 111,109,95,99,97,99,104,101,97,1,0,0,115,52,0,0, + 0,0,9,12,1,8,1,10,1,12,1,4,1,10,1,12, + 1,14,1,16,1,4,1,4,1,12,1,8,1,18,2,10, + 1,8,1,16,1,10,1,16,1,10,1,14,2,16,1,10, + 1,16,2,14,1,114,92,0,0,0,99,1,0,0,0,0, + 0,0,0,5,0,0,0,9,0,0,0,67,0,0,0,115, + 126,0,0,0,116,0,124,0,131,1,100,1,107,2,114,16, + 100,2,83,0,124,0,160,1,100,3,161,1,92,3,125,1, + 125,2,125,3,124,1,114,56,124,3,160,2,161,0,100,4, + 100,5,133,2,25,0,100,6,107,3,114,60,124,0,83,0, + 122,12,116,3,124,0,131,1,125,4,87,0,110,36,4,0, + 116,4,116,5,102,2,107,10,114,108,1,0,1,0,1,0, + 124,0,100,2,100,5,133,2,25,0,125,4,89,0,110,2, + 88,0,116,6,124,4,131,1,114,122,124,4,83,0,124,0, + 83,0,41,7,122,188,67,111,110,118,101,114,116,32,97,32, + 98,121,116,101,99,111,100,101,32,102,105,108,101,32,112,97, + 116,104,32,116,111,32,97,32,115,111,117,114,99,101,32,112, + 97,116,104,32,40,105,102,32,112,111,115,115,105,98,108,101, + 41,46,10,10,32,32,32,32,84,104,105,115,32,102,117,110, + 99,116,105,111,110,32,101,120,105,115,116,115,32,112,117,114, + 101,108,121,32,102,111,114,32,98,97,99,107,119,97,114,100, + 115,45,99,111,109,112,97,116,105,98,105,108,105,116,121,32, + 102,111,114,10,32,32,32,32,80,121,73,109,112,111,114,116, + 95,69,120,101,99,67,111,100,101,77,111,100,117,108,101,87, + 105,116,104,70,105,108,101,110,97,109,101,115,40,41,32,105, + 110,32,116,104,101,32,67,32,65,80,73,46,10,10,32,32, + 32,32,114,62,0,0,0,78,114,61,0,0,0,233,253,255, + 255,255,233,255,255,255,255,90,2,112,121,41,7,114,31,0, + 0,0,114,32,0,0,0,218,5,108,111,119,101,114,114,92, + 0,0,0,114,71,0,0,0,114,76,0,0,0,114,44,0, + 0,0,41,5,218,13,98,121,116,101,99,111,100,101,95,112, + 97,116,104,114,85,0,0,0,114,36,0,0,0,90,9,101, + 120,116,101,110,115,105,111,110,218,11,115,111,117,114,99,101, + 95,112,97,116,104,114,2,0,0,0,114,2,0,0,0,114, + 4,0,0,0,218,15,95,103,101,116,95,115,111,117,114,99, + 101,102,105,108,101,137,1,0,0,115,20,0,0,0,0,7, + 12,1,4,1,16,1,24,1,4,1,2,1,12,1,18,1, + 18,1,114,98,0,0,0,99,1,0,0,0,0,0,0,0, + 1,0,0,0,8,0,0,0,67,0,0,0,115,74,0,0, + 0,124,0,160,0,116,1,116,2,131,1,161,1,114,48,122, + 10,116,3,124,0,131,1,87,0,83,0,4,0,116,4,107, + 10,114,44,1,0,1,0,1,0,89,0,113,70,88,0,110, + 22,124,0,160,0,116,1,116,5,131,1,161,1,114,66,124, + 0,83,0,100,0,83,0,100,0,83,0,41,1,78,41,6, + 218,8,101,110,100,115,119,105,116,104,218,5,116,117,112,108, + 101,114,91,0,0,0,114,87,0,0,0,114,71,0,0,0, + 114,78,0,0,0,41,1,114,86,0,0,0,114,2,0,0, + 0,114,2,0,0,0,114,4,0,0,0,218,11,95,103,101, + 116,95,99,97,99,104,101,100,156,1,0,0,115,16,0,0, + 0,0,1,14,1,2,1,10,1,14,1,8,1,14,1,4, + 2,114,101,0,0,0,99,1,0,0,0,0,0,0,0,2, + 0,0,0,8,0,0,0,67,0,0,0,115,52,0,0,0, + 122,14,116,0,124,0,131,1,106,1,125,1,87,0,110,24, + 4,0,116,2,107,10,114,38,1,0,1,0,1,0,100,1, + 125,1,89,0,110,2,88,0,124,1,100,2,79,0,125,1, + 124,1,83,0,41,3,122,51,67,97,108,99,117,108,97,116, + 101,32,116,104,101,32,109,111,100,101,32,112,101,114,109,105, + 115,115,105,111,110,115,32,102,111,114,32,97,32,98,121,116, + 101,99,111,100,101,32,102,105,108,101,46,105,182,1,0,0, + 233,128,0,0,0,41,3,114,39,0,0,0,114,41,0,0, + 0,114,40,0,0,0,41,2,114,35,0,0,0,114,42,0, 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, - 0,218,13,95,99,108,97,115,115,105,102,121,95,112,121,99, - 225,1,0,0,115,28,0,0,0,0,16,12,1,8,1,16, - 1,12,1,12,1,12,1,10,1,12,1,8,1,16,2,8, - 1,16,1,12,1,114,138,0,0,0,99,5,0,0,0,0, - 0,0,0,6,0,0,0,4,0,0,0,67,0,0,0,115, - 112,0,0,0,116,0,124,0,100,1,100,2,133,2,25,0, - 131,1,124,1,100,3,64,0,107,3,114,58,100,4,124,3, - 155,2,157,2,125,5,116,1,160,2,100,5,124,5,161,2, - 1,0,116,3,124,5,102,1,124,4,142,1,130,1,124,2, - 100,6,107,9,114,108,116,0,124,0,100,2,100,7,133,2, - 25,0,131,1,124,2,100,3,64,0,107,3,114,108,116,3, - 100,4,124,3,155,2,157,2,102,1,124,4,142,1,130,1, - 100,6,83,0,41,8,97,7,2,0,0,86,97,108,105,100, - 97,116,101,32,97,32,112,121,99,32,97,103,97,105,110,115, - 116,32,116,104,101,32,115,111,117,114,99,101,32,108,97,115, - 116,45,109,111,100,105,102,105,101,100,32,116,105,109,101,46, - 10,10,32,32,32,32,42,100,97,116,97,42,32,105,115,32, - 116,104,101,32,99,111,110,116,101,110,116,115,32,111,102,32, - 116,104,101,32,112,121,99,32,102,105,108,101,46,32,40,79, - 110,108,121,32,116,104,101,32,102,105,114,115,116,32,49,54, - 32,98,121,116,101,115,32,97,114,101,10,32,32,32,32,114, - 101,113,117,105,114,101,100,46,41,10,10,32,32,32,32,42, - 115,111,117,114,99,101,95,109,116,105,109,101,42,32,105,115, - 32,116,104,101,32,108,97,115,116,32,109,111,100,105,102,105, - 101,100,32,116,105,109,101,115,116,97,109,112,32,111,102,32, - 116,104,101,32,115,111,117,114,99,101,32,102,105,108,101,46, - 10,10,32,32,32,32,42,115,111,117,114,99,101,95,115,105, - 122,101,42,32,105,115,32,78,111,110,101,32,111,114,32,116, - 104,101,32,115,105,122,101,32,111,102,32,116,104,101,32,115, - 111,117,114,99,101,32,102,105,108,101,32,105,110,32,98,121, - 116,101,115,46,10,10,32,32,32,32,42,110,97,109,101,42, - 32,105,115,32,116,104,101,32,110,97,109,101,32,111,102,32, - 116,104,101,32,109,111,100,117,108,101,32,98,101,105,110,103, - 32,105,109,112,111,114,116,101,100,46,32,73,116,32,105,115, - 32,117,115,101,100,32,102,111,114,32,108,111,103,103,105,110, - 103,46,10,10,32,32,32,32,42,101,120,99,95,100,101,116, - 97,105,108,115,42,32,105,115,32,97,32,100,105,99,116,105, - 111,110,97,114,121,32,112,97,115,115,101,100,32,116,111,32, - 73,109,112,111,114,116,69,114,114,111,114,32,105,102,32,105, - 116,32,114,97,105,115,101,100,32,102,111,114,10,32,32,32, - 32,105,109,112,114,111,118,101,100,32,100,101,98,117,103,103, - 105,110,103,46,10,10,32,32,32,32,65,110,32,73,109,112, - 111,114,116,69,114,114,111,114,32,105,115,32,114,97,105,115, - 101,100,32,105,102,32,116,104,101,32,98,121,116,101,99,111, - 100,101,32,105,115,32,115,116,97,108,101,46,10,10,32,32, - 32,32,114,132,0,0,0,233,12,0,0,0,108,3,0,0, - 0,255,127,255,127,3,0,122,22,98,121,116,101,99,111,100, - 101,32,105,115,32,115,116,97,108,101,32,102,111,114,32,122, - 2,123,125,78,114,131,0,0,0,41,4,114,19,0,0,0, - 114,121,0,0,0,114,135,0,0,0,114,106,0,0,0,41, - 6,114,57,0,0,0,218,12,115,111,117,114,99,101,95,109, - 116,105,109,101,218,11,115,111,117,114,99,101,95,115,105,122, - 101,114,105,0,0,0,114,137,0,0,0,114,83,0,0,0, + 0,218,10,95,99,97,108,99,95,109,111,100,101,168,1,0, + 0,115,12,0,0,0,0,2,2,1,14,1,14,1,10,3, + 8,1,114,103,0,0,0,99,1,0,0,0,0,0,0,0, + 3,0,0,0,8,0,0,0,3,0,0,0,115,68,0,0, + 0,100,6,135,0,102,1,100,2,100,3,132,9,125,1,122, + 10,116,0,106,1,125,2,87,0,110,28,4,0,116,2,107, + 10,114,52,1,0,1,0,1,0,100,4,100,5,132,0,125, + 2,89,0,110,2,88,0,124,2,124,1,136,0,131,2,1, + 0,124,1,83,0,41,7,122,252,68,101,99,111,114,97,116, + 111,114,32,116,111,32,118,101,114,105,102,121,32,116,104,97, + 116,32,116,104,101,32,109,111,100,117,108,101,32,98,101,105, + 110,103,32,114,101,113,117,101,115,116,101,100,32,109,97,116, + 99,104,101,115,32,116,104,101,32,111,110,101,32,116,104,101, + 10,32,32,32,32,108,111,97,100,101,114,32,99,97,110,32, + 104,97,110,100,108,101,46,10,10,32,32,32,32,84,104,101, + 32,102,105,114,115,116,32,97,114,103,117,109,101,110,116,32, + 40,115,101,108,102,41,32,109,117,115,116,32,100,101,102,105, + 110,101,32,95,110,97,109,101,32,119,104,105,99,104,32,116, + 104,101,32,115,101,99,111,110,100,32,97,114,103,117,109,101, + 110,116,32,105,115,10,32,32,32,32,99,111,109,112,97,114, + 101,100,32,97,103,97,105,110,115,116,46,32,73,102,32,116, + 104,101,32,99,111,109,112,97,114,105,115,111,110,32,102,97, + 105,108,115,32,116,104,101,110,32,73,109,112,111,114,116,69, + 114,114,111,114,32,105,115,32,114,97,105,115,101,100,46,10, + 10,32,32,32,32,78,99,2,0,0,0,0,0,0,0,4, + 0,0,0,4,0,0,0,31,0,0,0,115,66,0,0,0, + 124,1,100,0,107,8,114,16,124,0,106,0,125,1,110,32, + 124,0,106,0,124,1,107,3,114,48,116,1,100,1,124,0, + 106,0,124,1,102,2,22,0,124,1,100,2,141,2,130,1, + 136,0,124,0,124,1,102,2,124,2,158,2,124,3,142,1, + 83,0,41,3,78,122,30,108,111,97,100,101,114,32,102,111, + 114,32,37,115,32,99,97,110,110,111,116,32,104,97,110,100, + 108,101,32,37,115,41,1,218,4,110,97,109,101,41,2,114, + 104,0,0,0,218,11,73,109,112,111,114,116,69,114,114,111, + 114,41,4,218,4,115,101,108,102,114,104,0,0,0,218,4, + 97,114,103,115,90,6,107,119,97,114,103,115,41,1,218,6, + 109,101,116,104,111,100,114,2,0,0,0,114,4,0,0,0, + 218,19,95,99,104,101,99,107,95,110,97,109,101,95,119,114, + 97,112,112,101,114,188,1,0,0,115,18,0,0,0,0,1, + 8,1,8,1,10,1,4,1,8,255,2,1,2,255,6,2, + 122,40,95,99,104,101,99,107,95,110,97,109,101,46,60,108, + 111,99,97,108,115,62,46,95,99,104,101,99,107,95,110,97, + 109,101,95,119,114,97,112,112,101,114,99,2,0,0,0,0, + 0,0,0,3,0,0,0,7,0,0,0,83,0,0,0,115, + 56,0,0,0,100,1,68,0,93,32,125,2,116,0,124,1, + 124,2,131,2,114,4,116,1,124,0,124,2,116,2,124,1, + 124,2,131,2,131,3,1,0,113,4,124,0,106,3,160,4, + 124,1,106,3,161,1,1,0,100,0,83,0,41,2,78,41, + 4,218,10,95,95,109,111,100,117,108,101,95,95,218,8,95, + 95,110,97,109,101,95,95,218,12,95,95,113,117,97,108,110, + 97,109,101,95,95,218,7,95,95,100,111,99,95,95,41,5, + 218,7,104,97,115,97,116,116,114,218,7,115,101,116,97,116, + 116,114,218,7,103,101,116,97,116,116,114,218,8,95,95,100, + 105,99,116,95,95,218,6,117,112,100,97,116,101,41,3,90, + 3,110,101,119,90,3,111,108,100,114,55,0,0,0,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,218,5,95, + 119,114,97,112,199,1,0,0,115,8,0,0,0,0,1,8, + 1,10,1,20,1,122,26,95,99,104,101,99,107,95,110,97, + 109,101,46,60,108,111,99,97,108,115,62,46,95,119,114,97, + 112,41,1,78,41,3,218,10,95,98,111,111,116,115,116,114, + 97,112,114,119,0,0,0,218,9,78,97,109,101,69,114,114, + 111,114,41,3,114,108,0,0,0,114,109,0,0,0,114,119, + 0,0,0,114,2,0,0,0,41,1,114,108,0,0,0,114, + 4,0,0,0,218,11,95,99,104,101,99,107,95,110,97,109, + 101,180,1,0,0,115,14,0,0,0,0,8,14,7,2,1, + 10,1,14,2,14,5,10,1,114,122,0,0,0,99,2,0, + 0,0,0,0,0,0,5,0,0,0,6,0,0,0,67,0, + 0,0,115,60,0,0,0,124,0,160,0,124,1,161,1,92, + 2,125,2,125,3,124,2,100,1,107,8,114,56,116,1,124, + 3,131,1,114,56,100,2,125,4,116,2,160,3,124,4,160, + 4,124,3,100,3,25,0,161,1,116,5,161,2,1,0,124, + 2,83,0,41,4,122,155,84,114,121,32,116,111,32,102,105, + 110,100,32,97,32,108,111,97,100,101,114,32,102,111,114,32, + 116,104,101,32,115,112,101,99,105,102,105,101,100,32,109,111, + 100,117,108,101,32,98,121,32,100,101,108,101,103,97,116,105, + 110,103,32,116,111,10,32,32,32,32,115,101,108,102,46,102, + 105,110,100,95,108,111,97,100,101,114,40,41,46,10,10,32, + 32,32,32,84,104,105,115,32,109,101,116,104,111,100,32,105, + 115,32,100,101,112,114,101,99,97,116,101,100,32,105,110,32, + 102,97,118,111,114,32,111,102,32,102,105,110,100,101,114,46, + 102,105,110,100,95,115,112,101,99,40,41,46,10,10,32,32, + 32,32,78,122,44,78,111,116,32,105,109,112,111,114,116,105, + 110,103,32,100,105,114,101,99,116,111,114,121,32,123,125,58, + 32,109,105,115,115,105,110,103,32,95,95,105,110,105,116,95, + 95,114,62,0,0,0,41,6,218,11,102,105,110,100,95,108, + 111,97,100,101,114,114,31,0,0,0,114,64,0,0,0,114, + 65,0,0,0,114,50,0,0,0,218,13,73,109,112,111,114, + 116,87,97,114,110,105,110,103,41,5,114,106,0,0,0,218, + 8,102,117,108,108,110,97,109,101,218,6,108,111,97,100,101, + 114,218,8,112,111,114,116,105,111,110,115,218,3,109,115,103, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 23,95,118,97,108,105,100,97,116,101,95,116,105,109,101,115, - 116,97,109,112,95,112,121,99,2,2,0,0,115,14,0,0, - 0,0,19,24,1,10,1,12,1,12,1,8,1,24,1,114, - 142,0,0,0,99,4,0,0,0,0,0,0,0,4,0,0, - 0,3,0,0,0,67,0,0,0,115,38,0,0,0,124,0, - 100,1,100,2,133,2,25,0,124,1,107,3,114,34,116,0, - 100,3,124,2,155,2,157,2,102,1,124,3,142,1,130,1, - 100,4,83,0,41,5,97,243,1,0,0,86,97,108,105,100, - 97,116,101,32,97,32,104,97,115,104,45,98,97,115,101,100, - 32,112,121,99,32,98,121,32,99,104,101,99,107,105,110,103, - 32,116,104,101,32,114,101,97,108,32,115,111,117,114,99,101, - 32,104,97,115,104,32,97,103,97,105,110,115,116,32,116,104, - 101,32,111,110,101,32,105,110,10,32,32,32,32,116,104,101, - 32,112,121,99,32,104,101,97,100,101,114,46,10,10,32,32, - 32,32,42,100,97,116,97,42,32,105,115,32,116,104,101,32, - 99,111,110,116,101,110,116,115,32,111,102,32,116,104,101,32, - 112,121,99,32,102,105,108,101,46,32,40,79,110,108,121,32, - 116,104,101,32,102,105,114,115,116,32,49,54,32,98,121,116, - 101,115,32,97,114,101,10,32,32,32,32,114,101,113,117,105, - 114,101,100,46,41,10,10,32,32,32,32,42,115,111,117,114, - 99,101,95,104,97,115,104,42,32,105,115,32,116,104,101,32, - 105,109,112,111,114,116,108,105,98,46,117,116,105,108,46,115, - 111,117,114,99,101,95,104,97,115,104,40,41,32,111,102,32, - 116,104,101,32,115,111,117,114,99,101,32,102,105,108,101,46, - 10,10,32,32,32,32,42,110,97,109,101,42,32,105,115,32, - 116,104,101,32,110,97,109,101,32,111,102,32,116,104,101,32, - 109,111,100,117,108,101,32,98,101,105,110,103,32,105,109,112, - 111,114,116,101,100,46,32,73,116,32,105,115,32,117,115,101, - 100,32,102,111,114,32,108,111,103,103,105,110,103,46,10,10, - 32,32,32,32,42,101,120,99,95,100,101,116,97,105,108,115, - 42,32,105,115,32,97,32,100,105,99,116,105,111,110,97,114, - 121,32,112,97,115,115,101,100,32,116,111,32,73,109,112,111, - 114,116,69,114,114,111,114,32,105,102,32,105,116,32,114,97, - 105,115,101,100,32,102,111,114,10,32,32,32,32,105,109,112, - 114,111,118,101,100,32,100,101,98,117,103,103,105,110,103,46, - 10,10,32,32,32,32,65,110,32,73,109,112,111,114,116,69, - 114,114,111,114,32,105,115,32,114,97,105,115,101,100,32,105, - 102,32,116,104,101,32,98,121,116,101,99,111,100,101,32,105, - 115,32,115,116,97,108,101,46,10,10,32,32,32,32,114,132, - 0,0,0,114,131,0,0,0,122,46,104,97,115,104,32,105, - 110,32,98,121,116,101,99,111,100,101,32,100,111,101,115,110, - 39,116,32,109,97,116,99,104,32,104,97,115,104,32,111,102, - 32,115,111,117,114,99,101,32,78,41,1,114,106,0,0,0, - 41,4,114,57,0,0,0,218,11,115,111,117,114,99,101,95, - 104,97,115,104,114,105,0,0,0,114,137,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,218,18,95, - 118,97,108,105,100,97,116,101,95,104,97,115,104,95,112,121, - 99,30,2,0,0,115,8,0,0,0,0,17,16,1,2,1, - 10,1,114,144,0,0,0,99,4,0,0,0,0,0,0,0, - 5,0,0,0,5,0,0,0,67,0,0,0,115,80,0,0, - 0,116,0,160,1,124,0,161,1,125,4,116,2,124,4,116, - 3,131,2,114,56,116,4,160,5,100,1,124,2,161,2,1, - 0,124,3,100,2,107,9,114,52,116,6,160,7,124,4,124, - 3,161,2,1,0,124,4,83,0,116,8,100,3,160,9,124, + 17,95,102,105,110,100,95,109,111,100,117,108,101,95,115,104, + 105,109,208,1,0,0,115,10,0,0,0,0,10,14,1,16, + 1,4,1,22,1,114,129,0,0,0,99,3,0,0,0,0, + 0,0,0,6,0,0,0,4,0,0,0,67,0,0,0,115, + 158,0,0,0,124,0,100,1,100,2,133,2,25,0,125,3, + 124,3,116,0,107,3,114,60,100,3,124,1,155,2,100,4, + 124,3,155,2,157,4,125,4,116,1,160,2,100,5,124,4, + 161,2,1,0,116,3,124,4,102,1,124,2,142,1,130,1, + 116,4,124,0,131,1,100,6,107,0,114,102,100,7,124,1, + 155,2,157,2,125,4,116,1,160,2,100,5,124,4,161,2, + 1,0,116,5,124,4,131,1,130,1,116,6,124,0,100,2, + 100,8,133,2,25,0,131,1,125,5,124,5,100,9,64,0, + 114,154,100,10,124,5,155,2,100,11,124,1,155,2,157,4, + 125,4,116,3,124,4,102,1,124,2,142,1,130,1,124,5, + 83,0,41,12,97,84,2,0,0,80,101,114,102,111,114,109, + 32,98,97,115,105,99,32,118,97,108,105,100,105,116,121,32, + 99,104,101,99,107,105,110,103,32,111,102,32,97,32,112,121, + 99,32,104,101,97,100,101,114,32,97,110,100,32,114,101,116, + 117,114,110,32,116,104,101,32,102,108,97,103,115,32,102,105, + 101,108,100,44,10,32,32,32,32,119,104,105,99,104,32,100, + 101,116,101,114,109,105,110,101,115,32,104,111,119,32,116,104, + 101,32,112,121,99,32,115,104,111,117,108,100,32,98,101,32, + 102,117,114,116,104,101,114,32,118,97,108,105,100,97,116,101, + 100,32,97,103,97,105,110,115,116,32,116,104,101,32,115,111, + 117,114,99,101,46,10,10,32,32,32,32,42,100,97,116,97, + 42,32,105,115,32,116,104,101,32,99,111,110,116,101,110,116, + 115,32,111,102,32,116,104,101,32,112,121,99,32,102,105,108, + 101,46,32,40,79,110,108,121,32,116,104,101,32,102,105,114, + 115,116,32,49,54,32,98,121,116,101,115,32,97,114,101,10, + 32,32,32,32,114,101,113,117,105,114,101,100,44,32,116,104, + 111,117,103,104,46,41,10,10,32,32,32,32,42,110,97,109, + 101,42,32,105,115,32,116,104,101,32,110,97,109,101,32,111, + 102,32,116,104,101,32,109,111,100,117,108,101,32,98,101,105, + 110,103,32,105,109,112,111,114,116,101,100,46,32,73,116,32, + 105,115,32,117,115,101,100,32,102,111,114,32,108,111,103,103, + 105,110,103,46,10,10,32,32,32,32,42,101,120,99,95,100, + 101,116,97,105,108,115,42,32,105,115,32,97,32,100,105,99, + 116,105,111,110,97,114,121,32,112,97,115,115,101,100,32,116, + 111,32,73,109,112,111,114,116,69,114,114,111,114,32,105,102, + 32,105,116,32,114,97,105,115,101,100,32,102,111,114,10,32, + 32,32,32,105,109,112,114,111,118,101,100,32,100,101,98,117, + 103,103,105,110,103,46,10,10,32,32,32,32,73,109,112,111, + 114,116,69,114,114,111,114,32,105,115,32,114,97,105,115,101, + 100,32,119,104,101,110,32,116,104,101,32,109,97,103,105,99, + 32,110,117,109,98,101,114,32,105,115,32,105,110,99,111,114, + 114,101,99,116,32,111,114,32,119,104,101,110,32,116,104,101, + 32,102,108,97,103,115,10,32,32,32,32,102,105,101,108,100, + 32,105,115,32,105,110,118,97,108,105,100,46,32,69,79,70, + 69,114,114,111,114,32,105,115,32,114,97,105,115,101,100,32, + 119,104,101,110,32,116,104,101,32,100,97,116,97,32,105,115, + 32,102,111,117,110,100,32,116,111,32,98,101,32,116,114,117, + 110,99,97,116,101,100,46,10,10,32,32,32,32,78,114,12, + 0,0,0,122,20,98,97,100,32,109,97,103,105,99,32,110, + 117,109,98,101,114,32,105,110,32,122,2,58,32,122,2,123, + 125,233,16,0,0,0,122,40,114,101,97,99,104,101,100,32, + 69,79,70,32,119,104,105,108,101,32,114,101,97,100,105,110, + 103,32,112,121,99,32,104,101,97,100,101,114,32,111,102,32, + 233,8,0,0,0,233,252,255,255,255,122,14,105,110,118,97, + 108,105,100,32,102,108,97,103,115,32,122,4,32,105,110,32, + 41,7,218,12,77,65,71,73,67,95,78,85,77,66,69,82, + 114,120,0,0,0,218,16,95,118,101,114,98,111,115,101,95, + 109,101,115,115,97,103,101,114,105,0,0,0,114,31,0,0, + 0,218,8,69,79,70,69,114,114,111,114,114,19,0,0,0, + 41,6,114,56,0,0,0,114,104,0,0,0,218,11,101,120, + 99,95,100,101,116,97,105,108,115,90,5,109,97,103,105,99, + 114,82,0,0,0,114,72,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,218,13,95,99,108,97,115, + 115,105,102,121,95,112,121,99,225,1,0,0,115,28,0,0, + 0,0,16,12,1,8,1,16,1,12,1,12,1,12,1,10, + 1,12,1,8,1,16,2,8,1,16,1,12,1,114,137,0, + 0,0,99,5,0,0,0,0,0,0,0,6,0,0,0,4, + 0,0,0,67,0,0,0,115,112,0,0,0,116,0,124,0, + 100,1,100,2,133,2,25,0,131,1,124,1,100,3,64,0, + 107,3,114,58,100,4,124,3,155,2,157,2,125,5,116,1, + 160,2,100,5,124,5,161,2,1,0,116,3,124,5,102,1, + 124,4,142,1,130,1,124,2,100,6,107,9,114,108,116,0, + 124,0,100,2,100,7,133,2,25,0,131,1,124,2,100,3, + 64,0,107,3,114,108,116,3,100,4,124,3,155,2,157,2, + 102,1,124,4,142,1,130,1,100,6,83,0,41,8,97,7, + 2,0,0,86,97,108,105,100,97,116,101,32,97,32,112,121, + 99,32,97,103,97,105,110,115,116,32,116,104,101,32,115,111, + 117,114,99,101,32,108,97,115,116,45,109,111,100,105,102,105, + 101,100,32,116,105,109,101,46,10,10,32,32,32,32,42,100, + 97,116,97,42,32,105,115,32,116,104,101,32,99,111,110,116, + 101,110,116,115,32,111,102,32,116,104,101,32,112,121,99,32, + 102,105,108,101,46,32,40,79,110,108,121,32,116,104,101,32, + 102,105,114,115,116,32,49,54,32,98,121,116,101,115,32,97, + 114,101,10,32,32,32,32,114,101,113,117,105,114,101,100,46, + 41,10,10,32,32,32,32,42,115,111,117,114,99,101,95,109, + 116,105,109,101,42,32,105,115,32,116,104,101,32,108,97,115, + 116,32,109,111,100,105,102,105,101,100,32,116,105,109,101,115, + 116,97,109,112,32,111,102,32,116,104,101,32,115,111,117,114, + 99,101,32,102,105,108,101,46,10,10,32,32,32,32,42,115, + 111,117,114,99,101,95,115,105,122,101,42,32,105,115,32,78, + 111,110,101,32,111,114,32,116,104,101,32,115,105,122,101,32, + 111,102,32,116,104,101,32,115,111,117,114,99,101,32,102,105, + 108,101,32,105,110,32,98,121,116,101,115,46,10,10,32,32, + 32,32,42,110,97,109,101,42,32,105,115,32,116,104,101,32, + 110,97,109,101,32,111,102,32,116,104,101,32,109,111,100,117, + 108,101,32,98,101,105,110,103,32,105,109,112,111,114,116,101, + 100,46,32,73,116,32,105,115,32,117,115,101,100,32,102,111, + 114,32,108,111,103,103,105,110,103,46,10,10,32,32,32,32, + 42,101,120,99,95,100,101,116,97,105,108,115,42,32,105,115, + 32,97,32,100,105,99,116,105,111,110,97,114,121,32,112,97, + 115,115,101,100,32,116,111,32,73,109,112,111,114,116,69,114, + 114,111,114,32,105,102,32,105,116,32,114,97,105,115,101,100, + 32,102,111,114,10,32,32,32,32,105,109,112,114,111,118,101, + 100,32,100,101,98,117,103,103,105,110,103,46,10,10,32,32, + 32,32,65,110,32,73,109,112,111,114,116,69,114,114,111,114, + 32,105,115,32,114,97,105,115,101,100,32,105,102,32,116,104, + 101,32,98,121,116,101,99,111,100,101,32,105,115,32,115,116, + 97,108,101,46,10,10,32,32,32,32,114,131,0,0,0,233, + 12,0,0,0,108,3,0,0,0,255,127,255,127,3,0,122, + 22,98,121,116,101,99,111,100,101,32,105,115,32,115,116,97, + 108,101,32,102,111,114,32,122,2,123,125,78,114,130,0,0, + 0,41,4,114,19,0,0,0,114,120,0,0,0,114,134,0, + 0,0,114,105,0,0,0,41,6,114,56,0,0,0,218,12, + 115,111,117,114,99,101,95,109,116,105,109,101,218,11,115,111, + 117,114,99,101,95,115,105,122,101,114,104,0,0,0,114,136, + 0,0,0,114,82,0,0,0,114,2,0,0,0,114,2,0, + 0,0,114,4,0,0,0,218,23,95,118,97,108,105,100,97, + 116,101,95,116,105,109,101,115,116,97,109,112,95,112,121,99, + 2,2,0,0,115,16,0,0,0,0,19,24,1,10,1,12, + 1,12,1,8,1,22,255,2,2,114,141,0,0,0,99,4, + 0,0,0,0,0,0,0,4,0,0,0,3,0,0,0,67, + 0,0,0,115,38,0,0,0,124,0,100,1,100,2,133,2, + 25,0,124,1,107,3,114,34,116,0,100,3,124,2,155,2, + 157,2,102,1,124,3,142,1,130,1,100,4,83,0,41,5, + 97,243,1,0,0,86,97,108,105,100,97,116,101,32,97,32, + 104,97,115,104,45,98,97,115,101,100,32,112,121,99,32,98, + 121,32,99,104,101,99,107,105,110,103,32,116,104,101,32,114, + 101,97,108,32,115,111,117,114,99,101,32,104,97,115,104,32, + 97,103,97,105,110,115,116,32,116,104,101,32,111,110,101,32, + 105,110,10,32,32,32,32,116,104,101,32,112,121,99,32,104, + 101,97,100,101,114,46,10,10,32,32,32,32,42,100,97,116, + 97,42,32,105,115,32,116,104,101,32,99,111,110,116,101,110, + 116,115,32,111,102,32,116,104,101,32,112,121,99,32,102,105, + 108,101,46,32,40,79,110,108,121,32,116,104,101,32,102,105, + 114,115,116,32,49,54,32,98,121,116,101,115,32,97,114,101, + 10,32,32,32,32,114,101,113,117,105,114,101,100,46,41,10, + 10,32,32,32,32,42,115,111,117,114,99,101,95,104,97,115, + 104,42,32,105,115,32,116,104,101,32,105,109,112,111,114,116, + 108,105,98,46,117,116,105,108,46,115,111,117,114,99,101,95, + 104,97,115,104,40,41,32,111,102,32,116,104,101,32,115,111, + 117,114,99,101,32,102,105,108,101,46,10,10,32,32,32,32, + 42,110,97,109,101,42,32,105,115,32,116,104,101,32,110,97, + 109,101,32,111,102,32,116,104,101,32,109,111,100,117,108,101, + 32,98,101,105,110,103,32,105,109,112,111,114,116,101,100,46, + 32,73,116,32,105,115,32,117,115,101,100,32,102,111,114,32, + 108,111,103,103,105,110,103,46,10,10,32,32,32,32,42,101, + 120,99,95,100,101,116,97,105,108,115,42,32,105,115,32,97, + 32,100,105,99,116,105,111,110,97,114,121,32,112,97,115,115, + 101,100,32,116,111,32,73,109,112,111,114,116,69,114,114,111, + 114,32,105,102,32,105,116,32,114,97,105,115,101,100,32,102, + 111,114,10,32,32,32,32,105,109,112,114,111,118,101,100,32, + 100,101,98,117,103,103,105,110,103,46,10,10,32,32,32,32, + 65,110,32,73,109,112,111,114,116,69,114,114,111,114,32,105, + 115,32,114,97,105,115,101,100,32,105,102,32,116,104,101,32, + 98,121,116,101,99,111,100,101,32,105,115,32,115,116,97,108, + 101,46,10,10,32,32,32,32,114,131,0,0,0,114,130,0, + 0,0,122,46,104,97,115,104,32,105,110,32,98,121,116,101, + 99,111,100,101,32,100,111,101,115,110,39,116,32,109,97,116, + 99,104,32,104,97,115,104,32,111,102,32,115,111,117,114,99, + 101,32,78,41,1,114,105,0,0,0,41,4,114,56,0,0, + 0,218,11,115,111,117,114,99,101,95,104,97,115,104,114,104, + 0,0,0,114,136,0,0,0,114,2,0,0,0,114,2,0, + 0,0,114,4,0,0,0,218,18,95,118,97,108,105,100,97, + 116,101,95,104,97,115,104,95,112,121,99,30,2,0,0,115, + 12,0,0,0,0,17,16,1,2,1,8,255,2,2,2,254, + 114,143,0,0,0,99,4,0,0,0,0,0,0,0,5,0, + 0,0,5,0,0,0,67,0,0,0,115,82,0,0,0,116, + 0,160,1,124,0,161,1,125,4,116,2,124,4,116,3,131, + 2,114,58,116,4,160,5,100,1,124,2,161,2,1,0,124, + 3,100,2,107,9,114,52,116,6,160,7,124,4,124,3,161, + 2,1,0,124,4,83,0,110,20,116,8,100,3,160,9,124, 2,161,1,124,1,124,2,100,4,141,3,130,1,100,2,83, 0,41,5,122,35,67,111,109,112,105,108,101,32,98,121,116, 101,99,111,100,101,32,97,115,32,102,111,117,110,100,32,105, 110,32,97,32,112,121,99,46,122,21,99,111,100,101,32,111, 98,106,101,99,116,32,102,114,111,109,32,123,33,114,125,78, 122,23,78,111,110,45,99,111,100,101,32,111,98,106,101,99, - 116,32,105,110,32,123,33,114,125,41,2,114,105,0,0,0, + 116,32,105,110,32,123,33,114,125,41,2,114,104,0,0,0, 114,35,0,0,0,41,10,218,7,109,97,114,115,104,97,108, 90,5,108,111,97,100,115,218,10,105,115,105,110,115,116,97, 110,99,101,218,10,95,99,111,100,101,95,116,121,112,101,114, - 121,0,0,0,114,135,0,0,0,218,4,95,105,109,112,90, + 120,0,0,0,114,134,0,0,0,218,4,95,105,109,112,90, 16,95,102,105,120,95,99,111,95,102,105,108,101,110,97,109, - 101,114,106,0,0,0,114,51,0,0,0,41,5,114,57,0, - 0,0,114,105,0,0,0,114,97,0,0,0,114,98,0,0, + 101,114,105,0,0,0,114,50,0,0,0,41,5,114,56,0, + 0,0,114,104,0,0,0,114,96,0,0,0,114,97,0,0, 0,218,4,99,111,100,101,114,2,0,0,0,114,2,0,0, 0,114,4,0,0,0,218,17,95,99,111,109,112,105,108,101, - 95,98,121,116,101,99,111,100,101,54,2,0,0,115,16,0, - 0,0,0,2,10,1,10,1,12,1,8,1,12,1,4,2, - 10,1,114,150,0,0,0,114,63,0,0,0,99,3,0,0, - 0,0,0,0,0,4,0,0,0,5,0,0,0,67,0,0, - 0,115,70,0,0,0,116,0,116,1,131,1,125,3,124,3, - 160,2,116,3,100,1,131,1,161,1,1,0,124,3,160,2, - 116,3,124,1,131,1,161,1,1,0,124,3,160,2,116,3, - 124,2,131,1,161,1,1,0,124,3,160,2,116,4,160,5, - 124,0,161,1,161,1,1,0,124,3,83,0,41,2,122,43, - 80,114,111,100,117,99,101,32,116,104,101,32,100,97,116,97, - 32,102,111,114,32,97,32,116,105,109,101,115,116,97,109,112, - 45,98,97,115,101,100,32,112,121,99,46,114,63,0,0,0, - 41,6,218,9,98,121,116,101,97,114,114,97,121,114,134,0, - 0,0,218,6,101,120,116,101,110,100,114,17,0,0,0,114, - 145,0,0,0,218,5,100,117,109,112,115,41,4,114,149,0, - 0,0,218,5,109,116,105,109,101,114,141,0,0,0,114,57, - 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,218,22,95,99,111,100,101,95,116,111,95,116,105,109, - 101,115,116,97,109,112,95,112,121,99,67,2,0,0,115,12, - 0,0,0,0,2,8,1,14,1,14,1,14,1,16,1,114, - 155,0,0,0,84,99,3,0,0,0,0,0,0,0,5,0, - 0,0,5,0,0,0,67,0,0,0,115,80,0,0,0,116, - 0,116,1,131,1,125,3,100,1,124,2,100,1,62,0,66, - 0,125,4,124,3,160,2,116,3,124,4,131,1,161,1,1, - 0,116,4,124,1,131,1,100,2,107,2,115,50,116,5,130, - 1,124,3,160,2,124,1,161,1,1,0,124,3,160,2,116, - 6,160,7,124,0,161,1,161,1,1,0,124,3,83,0,41, - 3,122,38,80,114,111,100,117,99,101,32,116,104,101,32,100, - 97,116,97,32,102,111,114,32,97,32,104,97,115,104,45,98, - 97,115,101,100,32,112,121,99,46,114,29,0,0,0,114,132, - 0,0,0,41,8,114,151,0,0,0,114,134,0,0,0,114, - 152,0,0,0,114,17,0,0,0,114,31,0,0,0,218,14, - 65,115,115,101,114,116,105,111,110,69,114,114,111,114,114,145, - 0,0,0,114,153,0,0,0,41,5,114,149,0,0,0,114, - 143,0,0,0,90,7,99,104,101,99,107,101,100,114,57,0, - 0,0,114,73,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,218,17,95,99,111,100,101,95,116,111, - 95,104,97,115,104,95,112,121,99,77,2,0,0,115,14,0, - 0,0,0,2,8,1,12,1,14,1,16,1,10,1,16,1, - 114,157,0,0,0,99,1,0,0,0,0,0,0,0,5,0, - 0,0,6,0,0,0,67,0,0,0,115,62,0,0,0,100, - 1,100,2,108,0,125,1,116,1,160,2,124,0,161,1,106, - 3,125,2,124,1,160,4,124,2,161,1,125,3,116,1,160, - 5,100,2,100,3,161,2,125,4,124,4,160,6,124,0,160, - 6,124,3,100,1,25,0,161,1,161,1,83,0,41,4,122, - 121,68,101,99,111,100,101,32,98,121,116,101,115,32,114,101, - 112,114,101,115,101,110,116,105,110,103,32,115,111,117,114,99, - 101,32,99,111,100,101,32,97,110,100,32,114,101,116,117,114, - 110,32,116,104,101,32,115,116,114,105,110,103,46,10,10,32, - 32,32,32,85,110,105,118,101,114,115,97,108,32,110,101,119, - 108,105,110,101,32,115,117,112,112,111,114,116,32,105,115,32, - 117,115,101,100,32,105,110,32,116,104,101,32,100,101,99,111, - 100,105,110,103,46,10,32,32,32,32,114,63,0,0,0,78, - 84,41,7,218,8,116,111,107,101,110,105,122,101,114,53,0, - 0,0,90,7,66,121,116,101,115,73,79,90,8,114,101,97, - 100,108,105,110,101,90,15,100,101,116,101,99,116,95,101,110, - 99,111,100,105,110,103,90,25,73,110,99,114,101,109,101,110, - 116,97,108,78,101,119,108,105,110,101,68,101,99,111,100,101, - 114,218,6,100,101,99,111,100,101,41,5,218,12,115,111,117, - 114,99,101,95,98,121,116,101,115,114,158,0,0,0,90,21, - 115,111,117,114,99,101,95,98,121,116,101,115,95,114,101,97, - 100,108,105,110,101,218,8,101,110,99,111,100,105,110,103,90, - 15,110,101,119,108,105,110,101,95,100,101,99,111,100,101,114, + 95,98,121,116,101,99,111,100,101,54,2,0,0,115,20,0, + 0,0,0,2,10,1,10,1,12,1,8,1,12,1,6,2, + 10,1,2,0,2,255,114,149,0,0,0,114,62,0,0,0, + 99,3,0,0,0,0,0,0,0,4,0,0,0,5,0,0, + 0,67,0,0,0,115,70,0,0,0,116,0,116,1,131,1, + 125,3,124,3,160,2,116,3,100,1,131,1,161,1,1,0, + 124,3,160,2,116,3,124,1,131,1,161,1,1,0,124,3, + 160,2,116,3,124,2,131,1,161,1,1,0,124,3,160,2, + 116,4,160,5,124,0,161,1,161,1,1,0,124,3,83,0, + 41,2,122,43,80,114,111,100,117,99,101,32,116,104,101,32, + 100,97,116,97,32,102,111,114,32,97,32,116,105,109,101,115, + 116,97,109,112,45,98,97,115,101,100,32,112,121,99,46,114, + 62,0,0,0,41,6,218,9,98,121,116,101,97,114,114,97, + 121,114,133,0,0,0,218,6,101,120,116,101,110,100,114,17, + 0,0,0,114,144,0,0,0,218,5,100,117,109,112,115,41, + 4,114,148,0,0,0,218,5,109,116,105,109,101,114,140,0, + 0,0,114,56,0,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,22,95,99,111,100,101,95,116,111, + 95,116,105,109,101,115,116,97,109,112,95,112,121,99,67,2, + 0,0,115,12,0,0,0,0,2,8,1,14,1,14,1,14, + 1,16,1,114,154,0,0,0,84,99,3,0,0,0,0,0, + 0,0,5,0,0,0,5,0,0,0,67,0,0,0,115,80, + 0,0,0,116,0,116,1,131,1,125,3,100,1,124,2,100, + 1,62,0,66,0,125,4,124,3,160,2,116,3,124,4,131, + 1,161,1,1,0,116,4,124,1,131,1,100,2,107,2,115, + 50,116,5,130,1,124,3,160,2,124,1,161,1,1,0,124, + 3,160,2,116,6,160,7,124,0,161,1,161,1,1,0,124, + 3,83,0,41,3,122,38,80,114,111,100,117,99,101,32,116, + 104,101,32,100,97,116,97,32,102,111,114,32,97,32,104,97, + 115,104,45,98,97,115,101,100,32,112,121,99,46,114,29,0, + 0,0,114,131,0,0,0,41,8,114,150,0,0,0,114,133, + 0,0,0,114,151,0,0,0,114,17,0,0,0,114,31,0, + 0,0,218,14,65,115,115,101,114,116,105,111,110,69,114,114, + 111,114,114,144,0,0,0,114,152,0,0,0,41,5,114,148, + 0,0,0,114,142,0,0,0,90,7,99,104,101,99,107,101, + 100,114,56,0,0,0,114,72,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,218,17,95,99,111,100, + 101,95,116,111,95,104,97,115,104,95,112,121,99,77,2,0, + 0,115,14,0,0,0,0,2,8,1,12,1,14,1,16,1, + 10,1,16,1,114,156,0,0,0,99,1,0,0,0,0,0, + 0,0,5,0,0,0,6,0,0,0,67,0,0,0,115,62, + 0,0,0,100,1,100,2,108,0,125,1,116,1,160,2,124, + 0,161,1,106,3,125,2,124,1,160,4,124,2,161,1,125, + 3,116,1,160,5,100,2,100,3,161,2,125,4,124,4,160, + 6,124,0,160,6,124,3,100,1,25,0,161,1,161,1,83, + 0,41,4,122,121,68,101,99,111,100,101,32,98,121,116,101, + 115,32,114,101,112,114,101,115,101,110,116,105,110,103,32,115, + 111,117,114,99,101,32,99,111,100,101,32,97,110,100,32,114, + 101,116,117,114,110,32,116,104,101,32,115,116,114,105,110,103, + 46,10,10,32,32,32,32,85,110,105,118,101,114,115,97,108, + 32,110,101,119,108,105,110,101,32,115,117,112,112,111,114,116, + 32,105,115,32,117,115,101,100,32,105,110,32,116,104,101,32, + 100,101,99,111,100,105,110,103,46,10,32,32,32,32,114,62, + 0,0,0,78,84,41,7,218,8,116,111,107,101,110,105,122, + 101,114,52,0,0,0,90,7,66,121,116,101,115,73,79,90, + 8,114,101,97,100,108,105,110,101,90,15,100,101,116,101,99, + 116,95,101,110,99,111,100,105,110,103,90,25,73,110,99,114, + 101,109,101,110,116,97,108,78,101,119,108,105,110,101,68,101, + 99,111,100,101,114,218,6,100,101,99,111,100,101,41,5,218, + 12,115,111,117,114,99,101,95,98,121,116,101,115,114,157,0, + 0,0,90,21,115,111,117,114,99,101,95,98,121,116,101,115, + 95,114,101,97,100,108,105,110,101,218,8,101,110,99,111,100, + 105,110,103,90,15,110,101,119,108,105,110,101,95,100,101,99, + 111,100,101,114,114,2,0,0,0,114,2,0,0,0,114,4, + 0,0,0,218,13,100,101,99,111,100,101,95,115,111,117,114, + 99,101,88,2,0,0,115,10,0,0,0,0,5,8,1,12, + 1,10,1,12,1,114,161,0,0,0,41,2,114,126,0,0, + 0,218,26,115,117,98,109,111,100,117,108,101,95,115,101,97, + 114,99,104,95,108,111,99,97,116,105,111,110,115,99,2,0, + 0,0,2,0,0,0,9,0,0,0,8,0,0,0,67,0, + 0,0,115,16,1,0,0,124,1,100,1,107,8,114,60,100, + 2,125,1,116,0,124,2,100,3,131,2,114,70,122,14,124, + 2,160,1,124,0,161,1,125,1,87,0,113,70,4,0,116, + 2,107,10,114,56,1,0,1,0,1,0,89,0,113,70,88, + 0,110,10,116,3,160,4,124,1,161,1,125,1,116,5,106, + 6,124,0,124,2,124,1,100,4,141,3,125,4,100,5,124, + 4,95,7,124,2,100,1,107,8,114,154,116,8,131,0,68, + 0,93,42,92,2,125,5,125,6,124,1,160,9,116,10,124, + 6,131,1,161,1,114,106,124,5,124,0,124,1,131,2,125, + 2,124,2,124,4,95,11,1,0,113,154,113,106,100,1,83, + 0,124,3,116,12,107,8,114,220,116,0,124,2,100,6,131, + 2,114,226,122,14,124,2,160,13,124,0,161,1,125,7,87, + 0,110,20,4,0,116,2,107,10,114,206,1,0,1,0,1, + 0,89,0,113,226,88,0,124,7,114,226,103,0,124,4,95, + 14,110,6,124,3,124,4,95,14,124,4,106,14,103,0,107, + 2,144,1,114,12,124,1,144,1,114,12,116,15,124,1,131, + 1,100,7,25,0,125,8,124,4,106,14,160,16,124,8,161, + 1,1,0,124,4,83,0,41,8,97,61,1,0,0,82,101, + 116,117,114,110,32,97,32,109,111,100,117,108,101,32,115,112, + 101,99,32,98,97,115,101,100,32,111,110,32,97,32,102,105, + 108,101,32,108,111,99,97,116,105,111,110,46,10,10,32,32, + 32,32,84,111,32,105,110,100,105,99,97,116,101,32,116,104, + 97,116,32,116,104,101,32,109,111,100,117,108,101,32,105,115, + 32,97,32,112,97,99,107,97,103,101,44,32,115,101,116,10, + 32,32,32,32,115,117,98,109,111,100,117,108,101,95,115,101, + 97,114,99,104,95,108,111,99,97,116,105,111,110,115,32,116, + 111,32,97,32,108,105,115,116,32,111,102,32,100,105,114,101, + 99,116,111,114,121,32,112,97,116,104,115,46,32,32,65,110, + 10,32,32,32,32,101,109,112,116,121,32,108,105,115,116,32, + 105,115,32,115,117,102,102,105,99,105,101,110,116,44,32,116, + 104,111,117,103,104,32,105,116,115,32,110,111,116,32,111,116, + 104,101,114,119,105,115,101,32,117,115,101,102,117,108,32,116, + 111,32,116,104,101,10,32,32,32,32,105,109,112,111,114,116, + 32,115,121,115,116,101,109,46,10,10,32,32,32,32,84,104, + 101,32,108,111,97,100,101,114,32,109,117,115,116,32,116,97, + 107,101,32,97,32,115,112,101,99,32,97,115,32,105,116,115, + 32,111,110,108,121,32,95,95,105,110,105,116,95,95,40,41, + 32,97,114,103,46,10,10,32,32,32,32,78,122,9,60,117, + 110,107,110,111,119,110,62,218,12,103,101,116,95,102,105,108, + 101,110,97,109,101,41,1,218,6,111,114,105,103,105,110,84, + 218,10,105,115,95,112,97,99,107,97,103,101,114,62,0,0, + 0,41,17,114,114,0,0,0,114,163,0,0,0,114,105,0, + 0,0,114,1,0,0,0,114,68,0,0,0,114,120,0,0, + 0,218,10,77,111,100,117,108,101,83,112,101,99,90,13,95, + 115,101,116,95,102,105,108,101,97,116,116,114,218,27,95,103, + 101,116,95,115,117,112,112,111,114,116,101,100,95,102,105,108, + 101,95,108,111,97,100,101,114,115,114,99,0,0,0,114,100, + 0,0,0,114,126,0,0,0,218,9,95,80,79,80,85,76, + 65,84,69,114,165,0,0,0,114,162,0,0,0,114,38,0, + 0,0,218,6,97,112,112,101,110,100,41,9,114,104,0,0, + 0,90,8,108,111,99,97,116,105,111,110,114,126,0,0,0, + 114,162,0,0,0,218,4,115,112,101,99,218,12,108,111,97, + 100,101,114,95,99,108,97,115,115,218,8,115,117,102,102,105, + 120,101,115,114,165,0,0,0,90,7,100,105,114,110,97,109, + 101,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 218,23,115,112,101,99,95,102,114,111,109,95,102,105,108,101, + 95,108,111,99,97,116,105,111,110,105,2,0,0,115,62,0, + 0,0,0,12,8,4,4,1,10,2,2,1,14,1,14,1, + 8,2,10,8,16,1,6,3,8,1,14,1,14,1,10,1, + 6,1,6,2,4,3,8,2,10,1,2,1,14,1,14,1, + 6,2,4,1,8,2,6,1,12,1,6,1,12,1,12,2, + 114,173,0,0,0,99,0,0,0,0,0,0,0,0,0,0, + 0,0,4,0,0,0,64,0,0,0,115,86,0,0,0,101, + 0,90,1,100,0,90,2,100,1,90,3,100,2,90,4,100, + 3,90,5,100,4,90,6,101,7,100,5,100,6,132,0,131, + 1,90,8,101,7,100,7,100,8,132,0,131,1,90,9,101, + 7,100,9,100,9,102,2,100,10,100,11,132,1,131,1,90, + 10,101,7,100,9,102,1,100,12,100,13,132,1,131,1,90, + 11,100,9,83,0,41,14,218,21,87,105,110,100,111,119,115, + 82,101,103,105,115,116,114,121,70,105,110,100,101,114,122,62, + 77,101,116,97,32,112,97,116,104,32,102,105,110,100,101,114, + 32,102,111,114,32,109,111,100,117,108,101,115,32,100,101,99, + 108,97,114,101,100,32,105,110,32,116,104,101,32,87,105,110, + 100,111,119,115,32,114,101,103,105,115,116,114,121,46,122,59, + 83,111,102,116,119,97,114,101,92,80,121,116,104,111,110,92, + 80,121,116,104,111,110,67,111,114,101,92,123,115,121,115,95, + 118,101,114,115,105,111,110,125,92,77,111,100,117,108,101,115, + 92,123,102,117,108,108,110,97,109,101,125,122,65,83,111,102, + 116,119,97,114,101,92,80,121,116,104,111,110,92,80,121,116, + 104,111,110,67,111,114,101,92,123,115,121,115,95,118,101,114, + 115,105,111,110,125,92,77,111,100,117,108,101,115,92,123,102, + 117,108,108,110,97,109,101,125,92,68,101,98,117,103,70,99, + 2,0,0,0,0,0,0,0,2,0,0,0,8,0,0,0, + 67,0,0,0,115,56,0,0,0,122,16,116,0,160,1,116, + 0,106,2,124,1,161,2,87,0,83,0,4,0,116,3,107, + 10,114,50,1,0,1,0,1,0,116,0,160,1,116,0,106, + 4,124,1,161,2,6,0,89,0,83,0,88,0,100,0,83, + 0,41,1,78,41,5,218,7,95,119,105,110,114,101,103,90, + 7,79,112,101,110,75,101,121,90,17,72,75,69,89,95,67, + 85,82,82,69,78,84,95,85,83,69,82,114,40,0,0,0, + 90,18,72,75,69,89,95,76,79,67,65,76,95,77,65,67, + 72,73,78,69,41,2,218,3,99,108,115,114,3,0,0,0, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 13,100,101,99,111,100,101,95,115,111,117,114,99,101,88,2, - 0,0,115,10,0,0,0,0,5,8,1,12,1,10,1,12, - 1,114,162,0,0,0,41,2,114,127,0,0,0,218,26,115, - 117,98,109,111,100,117,108,101,95,115,101,97,114,99,104,95, - 108,111,99,97,116,105,111,110,115,99,2,0,0,0,2,0, - 0,0,9,0,0,0,8,0,0,0,67,0,0,0,115,16, - 1,0,0,124,1,100,1,107,8,114,60,100,2,125,1,116, - 0,124,2,100,3,131,2,114,70,122,14,124,2,160,1,124, - 0,161,1,125,1,87,0,113,70,4,0,116,2,107,10,114, - 56,1,0,1,0,1,0,89,0,113,70,88,0,110,10,116, - 3,160,4,124,1,161,1,125,1,116,5,106,6,124,0,124, - 2,124,1,100,4,141,3,125,4,100,5,124,4,95,7,124, - 2,100,1,107,8,114,154,116,8,131,0,68,0,93,42,92, - 2,125,5,125,6,124,1,160,9,116,10,124,6,131,1,161, - 1,114,106,124,5,124,0,124,1,131,2,125,2,124,2,124, - 4,95,11,1,0,113,154,113,106,100,1,83,0,124,3,116, - 12,107,8,114,220,116,0,124,2,100,6,131,2,114,226,122, - 14,124,2,160,13,124,0,161,1,125,7,87,0,110,20,4, - 0,116,2,107,10,114,206,1,0,1,0,1,0,89,0,113, - 226,88,0,124,7,114,226,103,0,124,4,95,14,110,6,124, - 3,124,4,95,14,124,4,106,14,103,0,107,2,144,1,114, - 12,124,1,144,1,114,12,116,15,124,1,131,1,100,7,25, - 0,125,8,124,4,106,14,160,16,124,8,161,1,1,0,124, - 4,83,0,41,8,97,61,1,0,0,82,101,116,117,114,110, - 32,97,32,109,111,100,117,108,101,32,115,112,101,99,32,98, - 97,115,101,100,32,111,110,32,97,32,102,105,108,101,32,108, - 111,99,97,116,105,111,110,46,10,10,32,32,32,32,84,111, - 32,105,110,100,105,99,97,116,101,32,116,104,97,116,32,116, - 104,101,32,109,111,100,117,108,101,32,105,115,32,97,32,112, - 97,99,107,97,103,101,44,32,115,101,116,10,32,32,32,32, - 115,117,98,109,111,100,117,108,101,95,115,101,97,114,99,104, - 95,108,111,99,97,116,105,111,110,115,32,116,111,32,97,32, - 108,105,115,116,32,111,102,32,100,105,114,101,99,116,111,114, - 121,32,112,97,116,104,115,46,32,32,65,110,10,32,32,32, - 32,101,109,112,116,121,32,108,105,115,116,32,105,115,32,115, - 117,102,102,105,99,105,101,110,116,44,32,116,104,111,117,103, - 104,32,105,116,115,32,110,111,116,32,111,116,104,101,114,119, - 105,115,101,32,117,115,101,102,117,108,32,116,111,32,116,104, - 101,10,32,32,32,32,105,109,112,111,114,116,32,115,121,115, - 116,101,109,46,10,10,32,32,32,32,84,104,101,32,108,111, - 97,100,101,114,32,109,117,115,116,32,116,97,107,101,32,97, - 32,115,112,101,99,32,97,115,32,105,116,115,32,111,110,108, - 121,32,95,95,105,110,105,116,95,95,40,41,32,97,114,103, - 46,10,10,32,32,32,32,78,122,9,60,117,110,107,110,111, - 119,110,62,218,12,103,101,116,95,102,105,108,101,110,97,109, - 101,41,1,218,6,111,114,105,103,105,110,84,218,10,105,115, - 95,112,97,99,107,97,103,101,114,63,0,0,0,41,17,114, - 115,0,0,0,114,164,0,0,0,114,106,0,0,0,114,1, - 0,0,0,114,69,0,0,0,114,121,0,0,0,218,10,77, - 111,100,117,108,101,83,112,101,99,90,13,95,115,101,116,95, - 102,105,108,101,97,116,116,114,218,27,95,103,101,116,95,115, - 117,112,112,111,114,116,101,100,95,102,105,108,101,95,108,111, - 97,100,101,114,115,114,100,0,0,0,114,101,0,0,0,114, - 127,0,0,0,218,9,95,80,79,80,85,76,65,84,69,114, - 166,0,0,0,114,163,0,0,0,114,38,0,0,0,218,6, - 97,112,112,101,110,100,41,9,114,105,0,0,0,90,8,108, - 111,99,97,116,105,111,110,114,127,0,0,0,114,163,0,0, - 0,218,4,115,112,101,99,218,12,108,111,97,100,101,114,95, - 99,108,97,115,115,218,8,115,117,102,102,105,120,101,115,114, - 166,0,0,0,90,7,100,105,114,110,97,109,101,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,218,23,115,112, - 101,99,95,102,114,111,109,95,102,105,108,101,95,108,111,99, - 97,116,105,111,110,105,2,0,0,115,62,0,0,0,0,12, - 8,4,4,1,10,2,2,1,14,1,14,1,8,2,10,8, - 16,1,6,3,8,1,14,1,14,1,10,1,6,1,6,2, - 4,3,8,2,10,1,2,1,14,1,14,1,6,2,4,1, - 8,2,6,1,12,1,6,1,12,1,12,2,114,174,0,0, - 0,99,0,0,0,0,0,0,0,0,0,0,0,0,4,0, - 0,0,64,0,0,0,115,80,0,0,0,101,0,90,1,100, - 0,90,2,100,1,90,3,100,2,90,4,100,3,90,5,100, - 4,90,6,101,7,100,5,100,6,132,0,131,1,90,8,101, - 7,100,7,100,8,132,0,131,1,90,9,101,7,100,14,100, - 10,100,11,132,1,131,1,90,10,101,7,100,15,100,12,100, - 13,132,1,131,1,90,11,100,9,83,0,41,16,218,21,87, - 105,110,100,111,119,115,82,101,103,105,115,116,114,121,70,105, - 110,100,101,114,122,62,77,101,116,97,32,112,97,116,104,32, - 102,105,110,100,101,114,32,102,111,114,32,109,111,100,117,108, - 101,115,32,100,101,99,108,97,114,101,100,32,105,110,32,116, - 104,101,32,87,105,110,100,111,119,115,32,114,101,103,105,115, - 116,114,121,46,122,59,83,111,102,116,119,97,114,101,92,80, - 121,116,104,111,110,92,80,121,116,104,111,110,67,111,114,101, - 92,123,115,121,115,95,118,101,114,115,105,111,110,125,92,77, - 111,100,117,108,101,115,92,123,102,117,108,108,110,97,109,101, - 125,122,65,83,111,102,116,119,97,114,101,92,80,121,116,104, - 111,110,92,80,121,116,104,111,110,67,111,114,101,92,123,115, - 121,115,95,118,101,114,115,105,111,110,125,92,77,111,100,117, - 108,101,115,92,123,102,117,108,108,110,97,109,101,125,92,68, - 101,98,117,103,70,99,2,0,0,0,0,0,0,0,2,0, - 0,0,8,0,0,0,67,0,0,0,115,56,0,0,0,122, - 16,116,0,160,1,116,0,106,2,124,1,161,2,87,0,83, - 0,4,0,116,3,107,10,114,50,1,0,1,0,1,0,116, - 0,160,1,116,0,106,4,124,1,161,2,6,0,89,0,83, - 0,88,0,100,0,83,0,41,1,78,41,5,218,7,95,119, - 105,110,114,101,103,90,7,79,112,101,110,75,101,121,90,17, - 72,75,69,89,95,67,85,82,82,69,78,84,95,85,83,69, - 82,114,40,0,0,0,90,18,72,75,69,89,95,76,79,67, - 65,76,95,77,65,67,72,73,78,69,41,2,218,3,99,108, - 115,114,3,0,0,0,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,218,14,95,111,112,101,110,95,114,101,103, - 105,115,116,114,121,185,2,0,0,115,8,0,0,0,0,2, - 2,1,16,1,14,1,122,36,87,105,110,100,111,119,115,82, - 101,103,105,115,116,114,121,70,105,110,100,101,114,46,95,111, - 112,101,110,95,114,101,103,105,115,116,114,121,99,2,0,0, - 0,0,0,0,0,6,0,0,0,9,0,0,0,67,0,0, - 0,115,114,0,0,0,124,0,106,0,114,14,124,0,106,1, - 125,2,110,6,124,0,106,2,125,2,124,2,106,3,124,1, - 100,1,116,4,106,5,100,0,100,2,133,2,25,0,22,0, - 100,3,141,2,125,3,122,38,124,0,160,6,124,3,161,1, - 143,18,125,4,116,7,160,8,124,4,100,4,161,2,125,5, - 87,0,53,0,81,0,82,0,88,0,87,0,110,22,4,0, - 116,9,107,10,114,108,1,0,1,0,1,0,89,0,100,0, - 83,0,88,0,124,5,83,0,41,5,78,122,5,37,100,46, - 37,100,114,60,0,0,0,41,2,114,126,0,0,0,90,11, - 115,121,115,95,118,101,114,115,105,111,110,114,30,0,0,0, - 41,10,218,11,68,69,66,85,71,95,66,85,73,76,68,218, - 18,82,69,71,73,83,84,82,89,95,75,69,89,95,68,69, - 66,85,71,218,12,82,69,71,73,83,84,82,89,95,75,69, - 89,114,51,0,0,0,114,6,0,0,0,218,12,118,101,114, - 115,105,111,110,95,105,110,102,111,114,178,0,0,0,114,176, - 0,0,0,90,10,81,117,101,114,121,86,97,108,117,101,114, - 40,0,0,0,41,6,114,177,0,0,0,114,126,0,0,0, - 90,12,114,101,103,105,115,116,114,121,95,107,101,121,114,3, - 0,0,0,90,4,104,107,101,121,218,8,102,105,108,101,112, - 97,116,104,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,218,16,95,115,101,97,114,99,104,95,114,101,103,105, - 115,116,114,121,192,2,0,0,115,22,0,0,0,0,2,6, - 1,8,2,6,1,6,1,22,1,2,1,12,1,26,1,14, - 1,8,1,122,38,87,105,110,100,111,119,115,82,101,103,105, + 14,95,111,112,101,110,95,114,101,103,105,115,116,114,121,185, + 2,0,0,115,8,0,0,0,0,2,2,1,16,1,14,1, + 122,36,87,105,110,100,111,119,115,82,101,103,105,115,116,114, + 121,70,105,110,100,101,114,46,95,111,112,101,110,95,114,101, + 103,105,115,116,114,121,99,2,0,0,0,0,0,0,0,6, + 0,0,0,9,0,0,0,67,0,0,0,115,118,0,0,0, + 124,0,106,0,114,14,124,0,106,1,125,2,110,6,124,0, + 106,2,125,2,124,2,106,3,124,1,100,1,116,4,106,5, + 100,0,100,2,133,2,25,0,22,0,100,3,141,2,125,3, + 122,38,124,0,160,6,124,3,161,1,143,18,125,4,116,7, + 160,8,124,4,100,4,161,2,125,5,87,0,53,0,81,0, + 82,0,88,0,87,0,110,26,4,0,116,9,107,10,114,112, + 1,0,1,0,1,0,89,0,100,0,83,0,89,0,110,2, + 88,0,124,5,83,0,41,5,78,122,5,37,100,46,37,100, + 114,59,0,0,0,41,2,114,125,0,0,0,90,11,115,121, + 115,95,118,101,114,115,105,111,110,114,30,0,0,0,41,10, + 218,11,68,69,66,85,71,95,66,85,73,76,68,218,18,82, + 69,71,73,83,84,82,89,95,75,69,89,95,68,69,66,85, + 71,218,12,82,69,71,73,83,84,82,89,95,75,69,89,114, + 50,0,0,0,114,6,0,0,0,218,12,118,101,114,115,105, + 111,110,95,105,110,102,111,114,177,0,0,0,114,175,0,0, + 0,90,10,81,117,101,114,121,86,97,108,117,101,114,40,0, + 0,0,41,6,114,176,0,0,0,114,125,0,0,0,90,12, + 114,101,103,105,115,116,114,121,95,107,101,121,114,3,0,0, + 0,90,4,104,107,101,121,218,8,102,105,108,101,112,97,116, + 104,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 218,16,95,115,101,97,114,99,104,95,114,101,103,105,115,116, + 114,121,192,2,0,0,115,24,0,0,0,0,2,6,1,8, + 2,6,1,6,1,16,255,6,2,2,1,12,1,26,1,14, + 1,12,1,122,38,87,105,110,100,111,119,115,82,101,103,105, 115,116,114,121,70,105,110,100,101,114,46,95,115,101,97,114, 99,104,95,114,101,103,105,115,116,114,121,78,99,4,0,0, 0,0,0,0,0,8,0,0,0,8,0,0,0,67,0,0, @@ -1005,1658 +1009,1662 @@ const unsigned char _Py_M__importlib_external[] = { 160,4,116,5,124,6,131,1,161,1,114,64,116,6,106,7, 124,1,124,5,124,1,124,4,131,2,124,4,100,1,141,3, 125,7,124,7,2,0,1,0,83,0,113,64,100,0,83,0, - 41,2,78,41,1,114,165,0,0,0,41,8,114,184,0,0, - 0,114,39,0,0,0,114,40,0,0,0,114,168,0,0,0, - 114,100,0,0,0,114,101,0,0,0,114,121,0,0,0,218, + 41,2,78,41,1,114,164,0,0,0,41,8,114,183,0,0, + 0,114,39,0,0,0,114,40,0,0,0,114,167,0,0,0, + 114,99,0,0,0,114,100,0,0,0,114,120,0,0,0,218, 16,115,112,101,99,95,102,114,111,109,95,108,111,97,100,101, - 114,41,8,114,177,0,0,0,114,126,0,0,0,114,35,0, - 0,0,218,6,116,97,114,103,101,116,114,183,0,0,0,114, - 127,0,0,0,114,173,0,0,0,114,171,0,0,0,114,2, + 114,41,8,114,176,0,0,0,114,125,0,0,0,114,35,0, + 0,0,218,6,116,97,114,103,101,116,114,182,0,0,0,114, + 126,0,0,0,114,172,0,0,0,114,170,0,0,0,114,2, 0,0,0,114,2,0,0,0,114,4,0,0,0,218,9,102, - 105,110,100,95,115,112,101,99,207,2,0,0,115,26,0,0, + 105,110,100,95,115,112,101,99,207,2,0,0,115,28,0,0, 0,0,2,10,1,8,1,4,1,2,1,12,1,14,1,8, - 1,14,1,14,1,6,1,8,1,8,1,122,31,87,105,110, - 100,111,119,115,82,101,103,105,115,116,114,121,70,105,110,100, - 101,114,46,102,105,110,100,95,115,112,101,99,99,3,0,0, - 0,0,0,0,0,4,0,0,0,4,0,0,0,67,0,0, - 0,115,34,0,0,0,124,0,160,0,124,1,124,2,161,2, - 125,3,124,3,100,1,107,9,114,26,124,3,106,1,83,0, - 100,1,83,0,100,1,83,0,41,2,122,108,70,105,110,100, - 32,109,111,100,117,108,101,32,110,97,109,101,100,32,105,110, - 32,116,104,101,32,114,101,103,105,115,116,114,121,46,10,10, - 32,32,32,32,32,32,32,32,84,104,105,115,32,109,101,116, - 104,111,100,32,105,115,32,100,101,112,114,101,99,97,116,101, - 100,46,32,32,85,115,101,32,101,120,101,99,95,109,111,100, - 117,108,101,40,41,32,105,110,115,116,101,97,100,46,10,10, - 32,32,32,32,32,32,32,32,78,41,2,114,187,0,0,0, - 114,127,0,0,0,41,4,114,177,0,0,0,114,126,0,0, - 0,114,35,0,0,0,114,171,0,0,0,114,2,0,0,0, - 114,2,0,0,0,114,4,0,0,0,218,11,102,105,110,100, - 95,109,111,100,117,108,101,223,2,0,0,115,8,0,0,0, - 0,7,12,1,8,1,6,2,122,33,87,105,110,100,111,119, - 115,82,101,103,105,115,116,114,121,70,105,110,100,101,114,46, - 102,105,110,100,95,109,111,100,117,108,101,41,2,78,78,41, - 1,78,41,12,114,112,0,0,0,114,111,0,0,0,114,113, - 0,0,0,114,114,0,0,0,114,181,0,0,0,114,180,0, - 0,0,114,179,0,0,0,218,11,99,108,97,115,115,109,101, - 116,104,111,100,114,178,0,0,0,114,184,0,0,0,114,187, - 0,0,0,114,188,0,0,0,114,2,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,175,0,0, - 0,173,2,0,0,115,20,0,0,0,8,2,4,3,4,3, - 4,2,4,2,12,7,12,15,2,1,12,15,2,1,114,175, - 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, - 2,0,0,0,64,0,0,0,115,48,0,0,0,101,0,90, - 1,100,0,90,2,100,1,90,3,100,2,100,3,132,0,90, - 4,100,4,100,5,132,0,90,5,100,6,100,7,132,0,90, - 6,100,8,100,9,132,0,90,7,100,10,83,0,41,11,218, - 13,95,76,111,97,100,101,114,66,97,115,105,99,115,122,83, - 66,97,115,101,32,99,108,97,115,115,32,111,102,32,99,111, - 109,109,111,110,32,99,111,100,101,32,110,101,101,100,101,100, - 32,98,121,32,98,111,116,104,32,83,111,117,114,99,101,76, - 111,97,100,101,114,32,97,110,100,10,32,32,32,32,83,111, - 117,114,99,101,108,101,115,115,70,105,108,101,76,111,97,100, - 101,114,46,99,2,0,0,0,0,0,0,0,5,0,0,0, - 4,0,0,0,67,0,0,0,115,64,0,0,0,116,0,124, - 0,160,1,124,1,161,1,131,1,100,1,25,0,125,2,124, - 2,160,2,100,2,100,1,161,2,100,3,25,0,125,3,124, - 1,160,3,100,2,161,1,100,4,25,0,125,4,124,3,100, - 5,107,2,111,62,124,4,100,5,107,3,83,0,41,6,122, - 141,67,111,110,99,114,101,116,101,32,105,109,112,108,101,109, - 101,110,116,97,116,105,111,110,32,111,102,32,73,110,115,112, - 101,99,116,76,111,97,100,101,114,46,105,115,95,112,97,99, - 107,97,103,101,32,98,121,32,99,104,101,99,107,105,110,103, - 32,105,102,10,32,32,32,32,32,32,32,32,116,104,101,32, - 112,97,116,104,32,114,101,116,117,114,110,101,100,32,98,121, - 32,103,101,116,95,102,105,108,101,110,97,109,101,32,104,97, - 115,32,97,32,102,105,108,101,110,97,109,101,32,111,102,32, - 39,95,95,105,110,105,116,95,95,46,112,121,39,46,114,29, - 0,0,0,114,62,0,0,0,114,63,0,0,0,114,60,0, - 0,0,218,8,95,95,105,110,105,116,95,95,41,4,114,38, - 0,0,0,114,164,0,0,0,114,34,0,0,0,114,32,0, - 0,0,41,5,114,107,0,0,0,114,126,0,0,0,114,87, - 0,0,0,90,13,102,105,108,101,110,97,109,101,95,98,97, - 115,101,90,9,116,97,105,108,95,110,97,109,101,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,166,0,0, - 0,242,2,0,0,115,8,0,0,0,0,3,18,1,16,1, - 14,1,122,24,95,76,111,97,100,101,114,66,97,115,105,99, - 115,46,105,115,95,112,97,99,107,97,103,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, - 0,115,4,0,0,0,100,1,83,0,41,2,122,42,85,115, - 101,32,100,101,102,97,117,108,116,32,115,101,109,97,110,116, - 105,99,115,32,102,111,114,32,109,111,100,117,108,101,32,99, - 114,101,97,116,105,111,110,46,78,114,2,0,0,0,41,2, - 114,107,0,0,0,114,171,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,218,13,99,114,101,97,116, - 101,95,109,111,100,117,108,101,250,2,0,0,115,2,0,0, - 0,0,1,122,27,95,76,111,97,100,101,114,66,97,115,105, - 99,115,46,99,114,101,97,116,101,95,109,111,100,117,108,101, - 99,2,0,0,0,0,0,0,0,3,0,0,0,5,0,0, - 0,67,0,0,0,115,56,0,0,0,124,0,160,0,124,1, - 106,1,161,1,125,2,124,2,100,1,107,8,114,36,116,2, - 100,2,160,3,124,1,106,1,161,1,131,1,130,1,116,4, - 160,5,116,6,124,2,124,1,106,7,161,3,1,0,100,1, - 83,0,41,3,122,19,69,120,101,99,117,116,101,32,116,104, - 101,32,109,111,100,117,108,101,46,78,122,52,99,97,110,110, - 111,116,32,108,111,97,100,32,109,111,100,117,108,101,32,123, - 33,114,125,32,119,104,101,110,32,103,101,116,95,99,111,100, - 101,40,41,32,114,101,116,117,114,110,115,32,78,111,110,101, - 41,8,218,8,103,101,116,95,99,111,100,101,114,112,0,0, - 0,114,106,0,0,0,114,51,0,0,0,114,121,0,0,0, - 218,25,95,99,97,108,108,95,119,105,116,104,95,102,114,97, - 109,101,115,95,114,101,109,111,118,101,100,218,4,101,120,101, - 99,114,118,0,0,0,41,3,114,107,0,0,0,218,6,109, - 111,100,117,108,101,114,149,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,218,11,101,120,101,99,95, - 109,111,100,117,108,101,253,2,0,0,115,10,0,0,0,0, - 2,12,1,8,1,6,1,10,1,122,25,95,76,111,97,100, - 101,114,66,97,115,105,99,115,46,101,120,101,99,95,109,111, - 100,117,108,101,99,2,0,0,0,0,0,0,0,2,0,0, - 0,4,0,0,0,67,0,0,0,115,12,0,0,0,116,0, - 160,1,124,0,124,1,161,2,83,0,41,1,122,26,84,104, - 105,115,32,109,111,100,117,108,101,32,105,115,32,100,101,112, - 114,101,99,97,116,101,100,46,41,2,114,121,0,0,0,218, - 17,95,108,111,97,100,95,109,111,100,117,108,101,95,115,104, - 105,109,41,2,114,107,0,0,0,114,126,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,218,11,108, - 111,97,100,95,109,111,100,117,108,101,5,3,0,0,115,2, - 0,0,0,0,2,122,25,95,76,111,97,100,101,114,66,97, - 115,105,99,115,46,108,111,97,100,95,109,111,100,117,108,101, - 78,41,8,114,112,0,0,0,114,111,0,0,0,114,113,0, - 0,0,114,114,0,0,0,114,166,0,0,0,114,192,0,0, - 0,114,197,0,0,0,114,199,0,0,0,114,2,0,0,0, - 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,114, - 190,0,0,0,237,2,0,0,115,10,0,0,0,8,3,4, - 2,8,8,8,3,8,8,114,190,0,0,0,99,0,0,0, - 0,0,0,0,0,0,0,0,0,3,0,0,0,64,0,0, - 0,115,74,0,0,0,101,0,90,1,100,0,90,2,100,1, - 100,2,132,0,90,3,100,3,100,4,132,0,90,4,100,5, - 100,6,132,0,90,5,100,7,100,8,132,0,90,6,100,9, - 100,10,132,0,90,7,100,11,100,12,156,1,100,13,100,14, - 132,2,90,8,100,15,100,16,132,0,90,9,100,17,83,0, - 41,18,218,12,83,111,117,114,99,101,76,111,97,100,101,114, - 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, - 0,67,0,0,0,115,8,0,0,0,116,0,130,1,100,1, - 83,0,41,2,122,178,79,112,116,105,111,110,97,108,32,109, - 101,116,104,111,100,32,116,104,97,116,32,114,101,116,117,114, - 110,115,32,116,104,101,32,109,111,100,105,102,105,99,97,116, - 105,111,110,32,116,105,109,101,32,40,97,110,32,105,110,116, - 41,32,102,111,114,32,116,104,101,10,32,32,32,32,32,32, - 32,32,115,112,101,99,105,102,105,101,100,32,112,97,116,104, - 44,32,119,104,101,114,101,32,112,97,116,104,32,105,115,32, - 97,32,115,116,114,46,10,10,32,32,32,32,32,32,32,32, - 82,97,105,115,101,115,32,79,83,69,114,114,111,114,32,119, - 104,101,110,32,116,104,101,32,112,97,116,104,32,99,97,110, - 110,111,116,32,98,101,32,104,97,110,100,108,101,100,46,10, - 32,32,32,32,32,32,32,32,78,41,1,114,40,0,0,0, - 41,2,114,107,0,0,0,114,35,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,218,10,112,97,116, - 104,95,109,116,105,109,101,12,3,0,0,115,2,0,0,0, - 0,6,122,23,83,111,117,114,99,101,76,111,97,100,101,114, - 46,112,97,116,104,95,109,116,105,109,101,99,2,0,0,0, + 1,14,1,14,1,6,1,8,1,2,254,6,3,122,31,87, + 105,110,100,111,119,115,82,101,103,105,115,116,114,121,70,105, + 110,100,101,114,46,102,105,110,100,95,115,112,101,99,99,3, + 0,0,0,0,0,0,0,4,0,0,0,4,0,0,0,67, + 0,0,0,115,34,0,0,0,124,0,160,0,124,1,124,2, + 161,2,125,3,124,3,100,1,107,9,114,26,124,3,106,1, + 83,0,100,1,83,0,100,1,83,0,41,2,122,108,70,105, + 110,100,32,109,111,100,117,108,101,32,110,97,109,101,100,32, + 105,110,32,116,104,101,32,114,101,103,105,115,116,114,121,46, + 10,10,32,32,32,32,32,32,32,32,84,104,105,115,32,109, + 101,116,104,111,100,32,105,115,32,100,101,112,114,101,99,97, + 116,101,100,46,32,32,85,115,101,32,101,120,101,99,95,109, + 111,100,117,108,101,40,41,32,105,110,115,116,101,97,100,46, + 10,10,32,32,32,32,32,32,32,32,78,41,2,114,186,0, + 0,0,114,126,0,0,0,41,4,114,176,0,0,0,114,125, + 0,0,0,114,35,0,0,0,114,170,0,0,0,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,218,11,102,105, + 110,100,95,109,111,100,117,108,101,223,2,0,0,115,8,0, + 0,0,0,7,12,1,8,1,6,2,122,33,87,105,110,100, + 111,119,115,82,101,103,105,115,116,114,121,70,105,110,100,101, + 114,46,102,105,110,100,95,109,111,100,117,108,101,41,12,114, + 111,0,0,0,114,110,0,0,0,114,112,0,0,0,114,113, + 0,0,0,114,180,0,0,0,114,179,0,0,0,114,178,0, + 0,0,218,11,99,108,97,115,115,109,101,116,104,111,100,114, + 177,0,0,0,114,183,0,0,0,114,186,0,0,0,114,187, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,2,0, + 0,0,114,4,0,0,0,114,174,0,0,0,173,2,0,0, + 115,30,0,0,0,8,2,4,3,2,255,2,4,2,255,2, + 3,4,2,12,7,12,15,2,1,2,0,2,255,12,16,2, + 1,2,255,114,174,0,0,0,99,0,0,0,0,0,0,0, + 0,0,0,0,0,2,0,0,0,64,0,0,0,115,48,0, + 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, + 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, + 100,7,132,0,90,6,100,8,100,9,132,0,90,7,100,10, + 83,0,41,11,218,13,95,76,111,97,100,101,114,66,97,115, + 105,99,115,122,83,66,97,115,101,32,99,108,97,115,115,32, + 111,102,32,99,111,109,109,111,110,32,99,111,100,101,32,110, + 101,101,100,101,100,32,98,121,32,98,111,116,104,32,83,111, + 117,114,99,101,76,111,97,100,101,114,32,97,110,100,10,32, + 32,32,32,83,111,117,114,99,101,108,101,115,115,70,105,108, + 101,76,111,97,100,101,114,46,99,2,0,0,0,0,0,0, + 0,5,0,0,0,4,0,0,0,67,0,0,0,115,64,0, + 0,0,116,0,124,0,160,1,124,1,161,1,131,1,100,1, + 25,0,125,2,124,2,160,2,100,2,100,1,161,2,100,3, + 25,0,125,3,124,1,160,3,100,2,161,1,100,4,25,0, + 125,4,124,3,100,5,107,2,111,62,124,4,100,5,107,3, + 83,0,41,6,122,141,67,111,110,99,114,101,116,101,32,105, + 109,112,108,101,109,101,110,116,97,116,105,111,110,32,111,102, + 32,73,110,115,112,101,99,116,76,111,97,100,101,114,46,105, + 115,95,112,97,99,107,97,103,101,32,98,121,32,99,104,101, + 99,107,105,110,103,32,105,102,10,32,32,32,32,32,32,32, + 32,116,104,101,32,112,97,116,104,32,114,101,116,117,114,110, + 101,100,32,98,121,32,103,101,116,95,102,105,108,101,110,97, + 109,101,32,104,97,115,32,97,32,102,105,108,101,110,97,109, + 101,32,111,102,32,39,95,95,105,110,105,116,95,95,46,112, + 121,39,46,114,29,0,0,0,114,61,0,0,0,114,62,0, + 0,0,114,59,0,0,0,218,8,95,95,105,110,105,116,95, + 95,41,4,114,38,0,0,0,114,163,0,0,0,114,34,0, + 0,0,114,32,0,0,0,41,5,114,106,0,0,0,114,125, + 0,0,0,114,86,0,0,0,90,13,102,105,108,101,110,97, + 109,101,95,98,97,115,101,90,9,116,97,105,108,95,110,97, + 109,101,114,2,0,0,0,114,2,0,0,0,114,4,0,0, + 0,114,165,0,0,0,242,2,0,0,115,8,0,0,0,0, + 3,18,1,16,1,14,1,122,24,95,76,111,97,100,101,114, + 66,97,115,105,99,115,46,105,115,95,112,97,99,107,97,103, + 101,99,2,0,0,0,0,0,0,0,2,0,0,0,1,0, + 0,0,67,0,0,0,115,4,0,0,0,100,1,83,0,41, + 2,122,42,85,115,101,32,100,101,102,97,117,108,116,32,115, + 101,109,97,110,116,105,99,115,32,102,111,114,32,109,111,100, + 117,108,101,32,99,114,101,97,116,105,111,110,46,78,114,2, + 0,0,0,41,2,114,106,0,0,0,114,170,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,13, + 99,114,101,97,116,101,95,109,111,100,117,108,101,250,2,0, + 0,115,2,0,0,0,0,1,122,27,95,76,111,97,100,101, + 114,66,97,115,105,99,115,46,99,114,101,97,116,101,95,109, + 111,100,117,108,101,99,2,0,0,0,0,0,0,0,3,0, + 0,0,5,0,0,0,67,0,0,0,115,56,0,0,0,124, + 0,160,0,124,1,106,1,161,1,125,2,124,2,100,1,107, + 8,114,36,116,2,100,2,160,3,124,1,106,1,161,1,131, + 1,130,1,116,4,160,5,116,6,124,2,124,1,106,7,161, + 3,1,0,100,1,83,0,41,3,122,19,69,120,101,99,117, + 116,101,32,116,104,101,32,109,111,100,117,108,101,46,78,122, + 52,99,97,110,110,111,116,32,108,111,97,100,32,109,111,100, + 117,108,101,32,123,33,114,125,32,119,104,101,110,32,103,101, + 116,95,99,111,100,101,40,41,32,114,101,116,117,114,110,115, + 32,78,111,110,101,41,8,218,8,103,101,116,95,99,111,100, + 101,114,111,0,0,0,114,105,0,0,0,114,50,0,0,0, + 114,120,0,0,0,218,25,95,99,97,108,108,95,119,105,116, + 104,95,102,114,97,109,101,115,95,114,101,109,111,118,101,100, + 218,4,101,120,101,99,114,117,0,0,0,41,3,114,106,0, + 0,0,218,6,109,111,100,117,108,101,114,148,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,11, + 101,120,101,99,95,109,111,100,117,108,101,253,2,0,0,115, + 12,0,0,0,0,2,12,1,8,1,6,1,4,255,6,2, + 122,25,95,76,111,97,100,101,114,66,97,115,105,99,115,46, + 101,120,101,99,95,109,111,100,117,108,101,99,2,0,0,0, 0,0,0,0,2,0,0,0,4,0,0,0,67,0,0,0, - 115,14,0,0,0,100,1,124,0,160,0,124,1,161,1,105, - 1,83,0,41,2,97,170,1,0,0,79,112,116,105,111,110, - 97,108,32,109,101,116,104,111,100,32,114,101,116,117,114,110, - 105,110,103,32,97,32,109,101,116,97,100,97,116,97,32,100, - 105,99,116,32,102,111,114,32,116,104,101,32,115,112,101,99, - 105,102,105,101,100,32,112,97,116,104,10,32,32,32,32,32, - 32,32,32,116,111,32,98,121,32,116,104,101,32,112,97,116, - 104,32,40,115,116,114,41,46,10,32,32,32,32,32,32,32, - 32,80,111,115,115,105,98,108,101,32,107,101,121,115,58,10, - 32,32,32,32,32,32,32,32,45,32,39,109,116,105,109,101, - 39,32,40,109,97,110,100,97,116,111,114,121,41,32,105,115, - 32,116,104,101,32,110,117,109,101,114,105,99,32,116,105,109, - 101,115,116,97,109,112,32,111,102,32,108,97,115,116,32,115, - 111,117,114,99,101,10,32,32,32,32,32,32,32,32,32,32, - 99,111,100,101,32,109,111,100,105,102,105,99,97,116,105,111, - 110,59,10,32,32,32,32,32,32,32,32,45,32,39,115,105, - 122,101,39,32,40,111,112,116,105,111,110,97,108,41,32,105, - 115,32,116,104,101,32,115,105,122,101,32,105,110,32,98,121, - 116,101,115,32,111,102,32,116,104,101,32,115,111,117,114,99, - 101,32,99,111,100,101,46,10,10,32,32,32,32,32,32,32, - 32,73,109,112,108,101,109,101,110,116,105,110,103,32,116,104, - 105,115,32,109,101,116,104,111,100,32,97,108,108,111,119,115, - 32,116,104,101,32,108,111,97,100,101,114,32,116,111,32,114, - 101,97,100,32,98,121,116,101,99,111,100,101,32,102,105,108, - 101,115,46,10,32,32,32,32,32,32,32,32,82,97,105,115, - 101,115,32,79,83,69,114,114,111,114,32,119,104,101,110,32, - 116,104,101,32,112,97,116,104,32,99,97,110,110,111,116,32, - 98,101,32,104,97,110,100,108,101,100,46,10,32,32,32,32, - 32,32,32,32,114,154,0,0,0,41,1,114,201,0,0,0, - 41,2,114,107,0,0,0,114,35,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,218,10,112,97,116, - 104,95,115,116,97,116,115,20,3,0,0,115,2,0,0,0, - 0,11,122,23,83,111,117,114,99,101,76,111,97,100,101,114, - 46,112,97,116,104,95,115,116,97,116,115,99,4,0,0,0, - 0,0,0,0,4,0,0,0,4,0,0,0,67,0,0,0, - 115,12,0,0,0,124,0,160,0,124,2,124,3,161,2,83, - 0,41,1,122,228,79,112,116,105,111,110,97,108,32,109,101, - 116,104,111,100,32,119,104,105,99,104,32,119,114,105,116,101, - 115,32,100,97,116,97,32,40,98,121,116,101,115,41,32,116, - 111,32,97,32,102,105,108,101,32,112,97,116,104,32,40,97, - 32,115,116,114,41,46,10,10,32,32,32,32,32,32,32,32, - 73,109,112,108,101,109,101,110,116,105,110,103,32,116,104,105, - 115,32,109,101,116,104,111,100,32,97,108,108,111,119,115,32, - 102,111,114,32,116,104,101,32,119,114,105,116,105,110,103,32, - 111,102,32,98,121,116,101,99,111,100,101,32,102,105,108,101, - 115,46,10,10,32,32,32,32,32,32,32,32,84,104,101,32, - 115,111,117,114,99,101,32,112,97,116,104,32,105,115,32,110, - 101,101,100,101,100,32,105,110,32,111,114,100,101,114,32,116, - 111,32,99,111,114,114,101,99,116,108,121,32,116,114,97,110, - 115,102,101,114,32,112,101,114,109,105,115,115,105,111,110,115, - 10,32,32,32,32,32,32,32,32,41,1,218,8,115,101,116, - 95,100,97,116,97,41,4,114,107,0,0,0,114,98,0,0, - 0,90,10,99,97,99,104,101,95,112,97,116,104,114,57,0, - 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, - 0,218,15,95,99,97,99,104,101,95,98,121,116,101,99,111, - 100,101,33,3,0,0,115,2,0,0,0,0,8,122,28,83, - 111,117,114,99,101,76,111,97,100,101,114,46,95,99,97,99, - 104,101,95,98,121,116,101,99,111,100,101,99,3,0,0,0, - 0,0,0,0,3,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,83,0,41,2,122,150,79,112,116, - 105,111,110,97,108,32,109,101,116,104,111,100,32,119,104,105, - 99,104,32,119,114,105,116,101,115,32,100,97,116,97,32,40, - 98,121,116,101,115,41,32,116,111,32,97,32,102,105,108,101, - 32,112,97,116,104,32,40,97,32,115,116,114,41,46,10,10, + 115,12,0,0,0,116,0,160,1,124,0,124,1,161,2,83, + 0,41,1,122,26,84,104,105,115,32,109,111,100,117,108,101, + 32,105,115,32,100,101,112,114,101,99,97,116,101,100,46,41, + 2,114,120,0,0,0,218,17,95,108,111,97,100,95,109,111, + 100,117,108,101,95,115,104,105,109,41,2,114,106,0,0,0, + 114,125,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 4,0,0,0,218,11,108,111,97,100,95,109,111,100,117,108, + 101,5,3,0,0,115,2,0,0,0,0,2,122,25,95,76, + 111,97,100,101,114,66,97,115,105,99,115,46,108,111,97,100, + 95,109,111,100,117,108,101,78,41,8,114,111,0,0,0,114, + 110,0,0,0,114,112,0,0,0,114,113,0,0,0,114,165, + 0,0,0,114,191,0,0,0,114,196,0,0,0,114,198,0, + 0,0,114,2,0,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,114,189,0,0,0,237,2,0,0,115, + 10,0,0,0,8,3,4,2,8,8,8,3,8,8,114,189, + 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, + 3,0,0,0,64,0,0,0,115,74,0,0,0,101,0,90, + 1,100,0,90,2,100,1,100,2,132,0,90,3,100,3,100, + 4,132,0,90,4,100,5,100,6,132,0,90,5,100,7,100, + 8,132,0,90,6,100,9,100,10,132,0,90,7,100,11,100, + 12,156,1,100,13,100,14,132,2,90,8,100,15,100,16,132, + 0,90,9,100,17,83,0,41,18,218,12,83,111,117,114,99, + 101,76,111,97,100,101,114,99,2,0,0,0,0,0,0,0, + 2,0,0,0,1,0,0,0,67,0,0,0,115,8,0,0, + 0,116,0,130,1,100,1,83,0,41,2,122,178,79,112,116, + 105,111,110,97,108,32,109,101,116,104,111,100,32,116,104,97, + 116,32,114,101,116,117,114,110,115,32,116,104,101,32,109,111, + 100,105,102,105,99,97,116,105,111,110,32,116,105,109,101,32, + 40,97,110,32,105,110,116,41,32,102,111,114,32,116,104,101, + 10,32,32,32,32,32,32,32,32,115,112,101,99,105,102,105, + 101,100,32,112,97,116,104,44,32,119,104,101,114,101,32,112, + 97,116,104,32,105,115,32,97,32,115,116,114,46,10,10,32, + 32,32,32,32,32,32,32,82,97,105,115,101,115,32,79,83, + 69,114,114,111,114,32,119,104,101,110,32,116,104,101,32,112, + 97,116,104,32,99,97,110,110,111,116,32,98,101,32,104,97, + 110,100,108,101,100,46,10,32,32,32,32,32,32,32,32,78, + 41,1,114,40,0,0,0,41,2,114,106,0,0,0,114,35, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,218,10,112,97,116,104,95,109,116,105,109,101,12,3, + 0,0,115,2,0,0,0,0,6,122,23,83,111,117,114,99, + 101,76,111,97,100,101,114,46,112,97,116,104,95,109,116,105, + 109,101,99,2,0,0,0,0,0,0,0,2,0,0,0,4, + 0,0,0,67,0,0,0,115,14,0,0,0,100,1,124,0, + 160,0,124,1,161,1,105,1,83,0,41,2,97,170,1,0, + 0,79,112,116,105,111,110,97,108,32,109,101,116,104,111,100, + 32,114,101,116,117,114,110,105,110,103,32,97,32,109,101,116, + 97,100,97,116,97,32,100,105,99,116,32,102,111,114,32,116, + 104,101,32,115,112,101,99,105,102,105,101,100,32,112,97,116, + 104,10,32,32,32,32,32,32,32,32,116,111,32,98,121,32, + 116,104,101,32,112,97,116,104,32,40,115,116,114,41,46,10, + 32,32,32,32,32,32,32,32,80,111,115,115,105,98,108,101, + 32,107,101,121,115,58,10,32,32,32,32,32,32,32,32,45, + 32,39,109,116,105,109,101,39,32,40,109,97,110,100,97,116, + 111,114,121,41,32,105,115,32,116,104,101,32,110,117,109,101, + 114,105,99,32,116,105,109,101,115,116,97,109,112,32,111,102, + 32,108,97,115,116,32,115,111,117,114,99,101,10,32,32,32, + 32,32,32,32,32,32,32,99,111,100,101,32,109,111,100,105, + 102,105,99,97,116,105,111,110,59,10,32,32,32,32,32,32, + 32,32,45,32,39,115,105,122,101,39,32,40,111,112,116,105, + 111,110,97,108,41,32,105,115,32,116,104,101,32,115,105,122, + 101,32,105,110,32,98,121,116,101,115,32,111,102,32,116,104, + 101,32,115,111,117,114,99,101,32,99,111,100,101,46,10,10, 32,32,32,32,32,32,32,32,73,109,112,108,101,109,101,110, 116,105,110,103,32,116,104,105,115,32,109,101,116,104,111,100, - 32,97,108,108,111,119,115,32,102,111,114,32,116,104,101,32, - 119,114,105,116,105,110,103,32,111,102,32,98,121,116,101,99, + 32,97,108,108,111,119,115,32,116,104,101,32,108,111,97,100, + 101,114,32,116,111,32,114,101,97,100,32,98,121,116,101,99, 111,100,101,32,102,105,108,101,115,46,10,32,32,32,32,32, - 32,32,32,78,114,2,0,0,0,41,3,114,107,0,0,0, - 114,35,0,0,0,114,57,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,203,0,0,0,43,3, - 0,0,115,2,0,0,0,0,4,122,21,83,111,117,114,99, - 101,76,111,97,100,101,114,46,115,101,116,95,100,97,116,97, - 99,2,0,0,0,0,0,0,0,5,0,0,0,10,0,0, - 0,67,0,0,0,115,82,0,0,0,124,0,160,0,124,1, - 161,1,125,2,122,14,124,0,160,1,124,2,161,1,125,3, - 87,0,110,48,4,0,116,2,107,10,114,72,1,0,125,4, - 1,0,122,18,116,3,100,1,124,1,100,2,141,2,124,4, - 130,2,87,0,53,0,100,3,125,4,126,4,88,0,89,0, - 110,2,88,0,116,4,124,3,131,1,83,0,41,4,122,52, - 67,111,110,99,114,101,116,101,32,105,109,112,108,101,109,101, - 110,116,97,116,105,111,110,32,111,102,32,73,110,115,112,101, - 99,116,76,111,97,100,101,114,46,103,101,116,95,115,111,117, - 114,99,101,46,122,39,115,111,117,114,99,101,32,110,111,116, - 32,97,118,97,105,108,97,98,108,101,32,116,104,114,111,117, - 103,104,32,103,101,116,95,100,97,116,97,40,41,41,1,114, - 105,0,0,0,78,41,5,114,164,0,0,0,218,8,103,101, - 116,95,100,97,116,97,114,40,0,0,0,114,106,0,0,0, - 114,162,0,0,0,41,5,114,107,0,0,0,114,126,0,0, - 0,114,35,0,0,0,114,160,0,0,0,218,3,101,120,99, - 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 10,103,101,116,95,115,111,117,114,99,101,50,3,0,0,115, - 14,0,0,0,0,2,10,1,2,1,14,1,16,1,4,1, - 28,1,122,23,83,111,117,114,99,101,76,111,97,100,101,114, - 46,103,101,116,95,115,111,117,114,99,101,114,95,0,0,0, - 41,1,218,9,95,111,112,116,105,109,105,122,101,99,3,0, - 0,0,1,0,0,0,4,0,0,0,8,0,0,0,67,0, - 0,0,115,22,0,0,0,116,0,106,1,116,2,124,1,124, - 2,100,1,100,2,124,3,100,3,141,6,83,0,41,4,122, - 130,82,101,116,117,114,110,32,116,104,101,32,99,111,100,101, - 32,111,98,106,101,99,116,32,99,111,109,112,105,108,101,100, - 32,102,114,111,109,32,115,111,117,114,99,101,46,10,10,32, - 32,32,32,32,32,32,32,84,104,101,32,39,100,97,116,97, - 39,32,97,114,103,117,109,101,110,116,32,99,97,110,32,98, - 101,32,97,110,121,32,111,98,106,101,99,116,32,116,121,112, - 101,32,116,104,97,116,32,99,111,109,112,105,108,101,40,41, - 32,115,117,112,112,111,114,116,115,46,10,32,32,32,32,32, - 32,32,32,114,195,0,0,0,84,41,2,218,12,100,111,110, - 116,95,105,110,104,101,114,105,116,114,74,0,0,0,41,3, - 114,121,0,0,0,114,194,0,0,0,218,7,99,111,109,112, - 105,108,101,41,4,114,107,0,0,0,114,57,0,0,0,114, - 35,0,0,0,114,208,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,218,14,115,111,117,114,99,101, - 95,116,111,95,99,111,100,101,60,3,0,0,115,4,0,0, - 0,0,5,12,1,122,27,83,111,117,114,99,101,76,111,97, - 100,101,114,46,115,111,117,114,99,101,95,116,111,95,99,111, - 100,101,99,2,0,0,0,0,0,0,0,15,0,0,0,9, - 0,0,0,67,0,0,0,115,46,2,0,0,124,0,160,0, - 124,1,161,1,125,2,100,1,125,3,100,1,125,4,100,1, - 125,5,100,2,125,6,100,3,125,7,122,12,116,1,124,2, - 131,1,125,8,87,0,110,26,4,0,116,2,107,10,114,68, - 1,0,1,0,1,0,100,1,125,8,89,0,144,1,110,48, - 88,0,122,14,124,0,160,3,124,2,161,1,125,9,87,0, - 110,22,4,0,116,4,107,10,114,106,1,0,1,0,1,0, - 89,0,144,1,110,10,88,0,116,5,124,9,100,4,25,0, - 131,1,125,3,122,14,124,0,160,6,124,8,161,1,125,10, - 87,0,110,20,4,0,116,4,107,10,114,154,1,0,1,0, - 1,0,89,0,110,218,88,0,124,1,124,8,100,5,156,2, - 125,11,122,148,116,7,124,10,124,1,124,11,131,3,125,12, - 116,8,124,10,131,1,100,6,100,1,133,2,25,0,125,13, - 124,12,100,7,64,0,100,8,107,3,125,6,124,6,144,1, - 114,36,124,12,100,9,64,0,100,8,107,3,125,7,116,9, - 106,10,100,10,107,3,144,1,114,56,124,7,115,254,116,9, - 106,10,100,11,107,2,144,1,114,56,124,0,160,6,124,2, - 161,1,125,4,116,9,160,11,116,12,124,4,161,2,125,5, - 116,13,124,10,124,5,124,1,124,11,131,4,1,0,110,20, - 116,14,124,10,124,3,124,9,100,12,25,0,124,1,124,11, - 131,5,1,0,87,0,110,26,4,0,116,15,116,16,102,2, - 107,10,144,1,114,84,1,0,1,0,1,0,89,0,110,32, - 88,0,116,17,160,18,100,13,124,8,124,2,161,3,1,0, - 116,19,124,13,124,1,124,8,124,2,100,14,141,4,83,0, - 124,4,100,1,107,8,144,1,114,136,124,0,160,6,124,2, - 161,1,125,4,124,0,160,20,124,4,124,2,161,2,125,14, - 116,17,160,18,100,15,124,2,161,2,1,0,116,21,106,22, - 144,2,115,42,124,8,100,1,107,9,144,2,114,42,124,3, - 100,1,107,9,144,2,114,42,124,6,144,1,114,228,124,5, - 100,1,107,8,144,1,114,214,116,9,160,11,124,4,161,1, - 125,5,116,23,124,14,124,5,124,7,131,3,125,10,110,16, - 116,24,124,14,124,3,116,25,124,4,131,1,131,3,125,10, - 122,30,124,0,160,26,124,2,124,8,124,10,161,3,1,0, - 116,17,160,18,100,16,124,8,161,2,1,0,87,0,110,22, - 4,0,116,2,107,10,144,2,114,40,1,0,1,0,1,0, - 89,0,110,2,88,0,124,14,83,0,41,17,122,190,67,111, - 110,99,114,101,116,101,32,105,109,112,108,101,109,101,110,116, - 97,116,105,111,110,32,111,102,32,73,110,115,112,101,99,116, - 76,111,97,100,101,114,46,103,101,116,95,99,111,100,101,46, - 10,10,32,32,32,32,32,32,32,32,82,101,97,100,105,110, - 103,32,111,102,32,98,121,116,101,99,111,100,101,32,114,101, - 113,117,105,114,101,115,32,112,97,116,104,95,115,116,97,116, - 115,32,116,111,32,98,101,32,105,109,112,108,101,109,101,110, - 116,101,100,46,32,84,111,32,119,114,105,116,101,10,32,32, - 32,32,32,32,32,32,98,121,116,101,99,111,100,101,44,32, - 115,101,116,95,100,97,116,97,32,109,117,115,116,32,97,108, - 115,111,32,98,101,32,105,109,112,108,101,109,101,110,116,101, - 100,46,10,10,32,32,32,32,32,32,32,32,78,70,84,114, - 154,0,0,0,41,2,114,105,0,0,0,114,35,0,0,0, - 114,131,0,0,0,114,29,0,0,0,114,63,0,0,0,114, - 60,0,0,0,90,5,110,101,118,101,114,90,6,97,108,119, - 97,121,115,218,4,115,105,122,101,122,13,123,125,32,109,97, - 116,99,104,101,115,32,123,125,41,3,114,105,0,0,0,114, - 97,0,0,0,114,98,0,0,0,122,19,99,111,100,101,32, - 111,98,106,101,99,116,32,102,114,111,109,32,123,125,122,10, - 119,114,111,116,101,32,123,33,114,125,41,27,114,164,0,0, - 0,114,88,0,0,0,114,72,0,0,0,114,202,0,0,0, - 114,40,0,0,0,114,14,0,0,0,114,205,0,0,0,114, - 138,0,0,0,218,10,109,101,109,111,114,121,118,105,101,119, - 114,148,0,0,0,90,21,99,104,101,99,107,95,104,97,115, - 104,95,98,97,115,101,100,95,112,121,99,115,114,143,0,0, - 0,218,17,95,82,65,87,95,77,65,71,73,67,95,78,85, - 77,66,69,82,114,144,0,0,0,114,142,0,0,0,114,106, - 0,0,0,114,136,0,0,0,114,121,0,0,0,114,135,0, - 0,0,114,150,0,0,0,114,211,0,0,0,114,6,0,0, - 0,218,19,100,111,110,116,95,119,114,105,116,101,95,98,121, - 116,101,99,111,100,101,114,157,0,0,0,114,155,0,0,0, - 114,31,0,0,0,114,204,0,0,0,41,15,114,107,0,0, - 0,114,126,0,0,0,114,98,0,0,0,114,140,0,0,0, - 114,160,0,0,0,114,143,0,0,0,90,10,104,97,115,104, - 95,98,97,115,101,100,90,12,99,104,101,99,107,95,115,111, - 117,114,99,101,114,97,0,0,0,218,2,115,116,114,57,0, - 0,0,114,137,0,0,0,114,73,0,0,0,90,10,98,121, - 116,101,115,95,100,97,116,97,90,11,99,111,100,101,95,111, - 98,106,101,99,116,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,114,193,0,0,0,68,3,0,0,115,134,0, - 0,0,0,7,10,1,4,1,4,1,4,1,4,1,4,1, - 2,1,12,1,14,1,12,2,2,1,14,1,14,1,8,2, - 12,1,2,1,14,1,14,1,6,3,2,1,8,2,2,1, - 12,1,16,1,12,1,6,1,12,1,12,1,4,1,12,1, - 10,1,4,1,2,1,6,2,8,1,8,2,2,1,2,1, - 2,1,6,1,2,1,10,2,20,1,6,2,8,1,6,1, - 6,1,2,1,8,1,10,1,10,1,12,1,12,1,18,1, - 10,1,6,1,10,1,10,1,14,2,6,1,10,1,2,1, - 14,1,16,1,16,1,6,1,122,21,83,111,117,114,99,101, - 76,111,97,100,101,114,46,103,101,116,95,99,111,100,101,78, - 41,10,114,112,0,0,0,114,111,0,0,0,114,113,0,0, - 0,114,201,0,0,0,114,202,0,0,0,114,204,0,0,0, - 114,203,0,0,0,114,207,0,0,0,114,211,0,0,0,114, - 193,0,0,0,114,2,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,114,200,0,0,0,10,3,0, - 0,115,14,0,0,0,8,2,8,8,8,13,8,10,8,7, - 8,10,14,8,114,200,0,0,0,99,0,0,0,0,0,0, - 0,0,0,0,0,0,4,0,0,0,0,0,0,0,115,124, - 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, - 2,100,3,132,0,90,4,100,4,100,5,132,0,90,5,100, - 6,100,7,132,0,90,6,101,7,135,0,102,1,100,8,100, - 9,132,8,131,1,90,8,101,7,100,10,100,11,132,0,131, - 1,90,9,100,12,100,13,132,0,90,10,101,7,100,14,100, - 15,132,0,131,1,90,11,100,16,100,17,132,0,90,12,100, - 18,100,19,132,0,90,13,100,20,100,21,132,0,90,14,100, - 22,100,23,132,0,90,15,135,0,4,0,90,16,83,0,41, - 24,218,10,70,105,108,101,76,111,97,100,101,114,122,103,66, - 97,115,101,32,102,105,108,101,32,108,111,97,100,101,114,32, - 99,108,97,115,115,32,119,104,105,99,104,32,105,109,112,108, - 101,109,101,110,116,115,32,116,104,101,32,108,111,97,100,101, - 114,32,112,114,111,116,111,99,111,108,32,109,101,116,104,111, - 100,115,32,116,104,97,116,10,32,32,32,32,114,101,113,117, - 105,114,101,32,102,105,108,101,32,115,121,115,116,101,109,32, - 117,115,97,103,101,46,99,3,0,0,0,0,0,0,0,3, - 0,0,0,2,0,0,0,67,0,0,0,115,16,0,0,0, - 124,1,124,0,95,0,124,2,124,0,95,1,100,1,83,0, - 41,2,122,75,67,97,99,104,101,32,116,104,101,32,109,111, - 100,117,108,101,32,110,97,109,101,32,97,110,100,32,116,104, - 101,32,112,97,116,104,32,116,111,32,116,104,101,32,102,105, - 108,101,32,102,111,117,110,100,32,98,121,32,116,104,101,10, - 32,32,32,32,32,32,32,32,102,105,110,100,101,114,46,78, - 41,2,114,105,0,0,0,114,35,0,0,0,41,3,114,107, - 0,0,0,114,126,0,0,0,114,35,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,191,0,0, - 0,159,3,0,0,115,4,0,0,0,0,3,6,1,122,19, - 70,105,108,101,76,111,97,100,101,114,46,95,95,105,110,105, - 116,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, - 2,0,0,0,67,0,0,0,115,24,0,0,0,124,0,106, - 0,124,1,106,0,107,2,111,22,124,0,106,1,124,1,106, - 1,107,2,83,0,41,1,78,41,2,218,9,95,95,99,108, - 97,115,115,95,95,114,118,0,0,0,41,2,114,107,0,0, - 0,218,5,111,116,104,101,114,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,218,6,95,95,101,113,95,95,165, - 3,0,0,115,4,0,0,0,0,1,12,1,122,17,70,105, - 108,101,76,111,97,100,101,114,46,95,95,101,113,95,95,99, - 1,0,0,0,0,0,0,0,1,0,0,0,3,0,0,0, - 67,0,0,0,115,20,0,0,0,116,0,124,0,106,1,131, - 1,116,0,124,0,106,2,131,1,65,0,83,0,41,1,78, - 41,3,218,4,104,97,115,104,114,105,0,0,0,114,35,0, - 0,0,41,1,114,107,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,218,8,95,95,104,97,115,104, - 95,95,169,3,0,0,115,2,0,0,0,0,1,122,19,70, - 105,108,101,76,111,97,100,101,114,46,95,95,104,97,115,104, - 95,95,99,2,0,0,0,0,0,0,0,2,0,0,0,3, - 0,0,0,3,0,0,0,115,16,0,0,0,116,0,116,1, - 124,0,131,2,160,2,124,1,161,1,83,0,41,1,122,100, - 76,111,97,100,32,97,32,109,111,100,117,108,101,32,102,114, - 111,109,32,97,32,102,105,108,101,46,10,10,32,32,32,32, - 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, - 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, - 85,115,101,32,101,120,101,99,95,109,111,100,117,108,101,40, - 41,32,105,110,115,116,101,97,100,46,10,10,32,32,32,32, - 32,32,32,32,41,3,218,5,115,117,112,101,114,114,217,0, - 0,0,114,199,0,0,0,41,2,114,107,0,0,0,114,126, - 0,0,0,41,1,114,218,0,0,0,114,2,0,0,0,114, - 4,0,0,0,114,199,0,0,0,172,3,0,0,115,2,0, - 0,0,0,10,122,22,70,105,108,101,76,111,97,100,101,114, - 46,108,111,97,100,95,109,111,100,117,108,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, - 0,115,6,0,0,0,124,0,106,0,83,0,41,1,122,58, - 82,101,116,117,114,110,32,116,104,101,32,112,97,116,104,32, - 116,111,32,116,104,101,32,115,111,117,114,99,101,32,102,105, - 108,101,32,97,115,32,102,111,117,110,100,32,98,121,32,116, - 104,101,32,102,105,110,100,101,114,46,41,1,114,35,0,0, - 0,41,2,114,107,0,0,0,114,126,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,164,0,0, - 0,184,3,0,0,115,2,0,0,0,0,3,122,23,70,105, - 108,101,76,111,97,100,101,114,46,103,101,116,95,102,105,108, - 101,110,97,109,101,99,2,0,0,0,0,0,0,0,3,0, - 0,0,10,0,0,0,67,0,0,0,115,44,0,0,0,116, - 0,160,1,124,1,100,1,161,2,143,22,125,2,124,2,160, - 2,161,0,87,0,2,0,53,0,81,0,82,0,163,0,83, - 0,81,0,82,0,88,0,100,2,83,0,41,3,122,39,82, - 101,116,117,114,110,32,116,104,101,32,100,97,116,97,32,102, - 114,111,109,32,112,97,116,104,32,97,115,32,114,97,119,32, - 98,121,116,101,115,46,218,1,114,78,41,3,114,53,0,0, - 0,114,54,0,0,0,90,4,114,101,97,100,41,3,114,107, - 0,0,0,114,35,0,0,0,114,58,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,205,0,0, - 0,189,3,0,0,115,4,0,0,0,0,2,14,1,122,19, - 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,100, - 97,116,97,99,2,0,0,0,0,0,0,0,2,0,0,0, - 3,0,0,0,67,0,0,0,115,18,0,0,0,124,0,160, - 0,124,1,161,1,114,14,124,0,83,0,100,0,83,0,41, - 1,78,41,1,114,166,0,0,0,41,2,114,107,0,0,0, - 114,196,0,0,0,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,218,19,103,101,116,95,114,101,115,111,117,114, - 99,101,95,114,101,97,100,101,114,196,3,0,0,115,6,0, - 0,0,0,2,10,1,4,1,122,30,70,105,108,101,76,111, - 97,100,101,114,46,103,101,116,95,114,101,115,111,117,114,99, - 101,95,114,101,97,100,101,114,99,2,0,0,0,0,0,0, - 0,3,0,0,0,4,0,0,0,67,0,0,0,115,32,0, - 0,0,116,0,116,1,124,0,106,2,131,1,100,1,25,0, - 124,1,131,2,125,2,116,3,160,4,124,2,100,2,161,2, - 83,0,41,3,78,114,63,0,0,0,114,224,0,0,0,41, - 5,114,28,0,0,0,114,38,0,0,0,114,35,0,0,0, - 114,53,0,0,0,114,54,0,0,0,41,3,114,107,0,0, - 0,218,8,114,101,115,111,117,114,99,101,114,35,0,0,0, - 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 13,111,112,101,110,95,114,101,115,111,117,114,99,101,202,3, - 0,0,115,4,0,0,0,0,1,20,1,122,24,70,105,108, - 101,76,111,97,100,101,114,46,111,112,101,110,95,114,101,115, - 111,117,114,99,101,99,2,0,0,0,0,0,0,0,3,0, - 0,0,3,0,0,0,67,0,0,0,115,38,0,0,0,124, - 0,160,0,124,1,161,1,115,14,116,1,130,1,116,2,116, - 3,124,0,106,4,131,1,100,1,25,0,124,1,131,2,125, - 2,124,2,83,0,41,2,78,114,63,0,0,0,41,5,218, - 11,105,115,95,114,101,115,111,117,114,99,101,218,17,70,105, - 108,101,78,111,116,70,111,117,110,100,69,114,114,111,114,114, - 28,0,0,0,114,38,0,0,0,114,35,0,0,0,41,3, - 114,107,0,0,0,114,226,0,0,0,114,35,0,0,0,114, - 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,13, - 114,101,115,111,117,114,99,101,95,112,97,116,104,206,3,0, - 0,115,8,0,0,0,0,1,10,1,4,1,20,1,122,24, - 70,105,108,101,76,111,97,100,101,114,46,114,101,115,111,117, - 114,99,101,95,112,97,116,104,99,2,0,0,0,0,0,0, - 0,3,0,0,0,3,0,0,0,67,0,0,0,115,40,0, - 0,0,116,0,124,1,107,6,114,12,100,1,83,0,116,1, - 116,2,124,0,106,3,131,1,100,2,25,0,124,1,131,2, - 125,2,116,4,124,2,131,1,83,0,41,3,78,70,114,63, - 0,0,0,41,5,114,25,0,0,0,114,28,0,0,0,114, - 38,0,0,0,114,35,0,0,0,114,44,0,0,0,41,3, - 114,107,0,0,0,114,105,0,0,0,114,35,0,0,0,114, - 2,0,0,0,114,2,0,0,0,114,4,0,0,0,114,228, - 0,0,0,212,3,0,0,115,8,0,0,0,0,1,8,1, - 4,1,20,1,122,22,70,105,108,101,76,111,97,100,101,114, - 46,105,115,95,114,101,115,111,117,114,99,101,99,1,0,0, - 0,0,0,0,0,1,0,0,0,5,0,0,0,67,0,0, - 0,115,24,0,0,0,116,0,116,1,160,2,116,3,124,0, - 106,4,131,1,100,1,25,0,161,1,131,1,83,0,41,2, - 78,114,63,0,0,0,41,5,218,4,105,116,101,114,114,1, - 0,0,0,218,7,108,105,115,116,100,105,114,114,38,0,0, - 0,114,35,0,0,0,41,1,114,107,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,218,8,99,111, - 110,116,101,110,116,115,218,3,0,0,115,2,0,0,0,0, - 1,122,19,70,105,108,101,76,111,97,100,101,114,46,99,111, - 110,116,101,110,116,115,41,17,114,112,0,0,0,114,111,0, - 0,0,114,113,0,0,0,114,114,0,0,0,114,191,0,0, - 0,114,220,0,0,0,114,222,0,0,0,114,123,0,0,0, - 114,199,0,0,0,114,164,0,0,0,114,205,0,0,0,114, - 225,0,0,0,114,227,0,0,0,114,230,0,0,0,114,228, - 0,0,0,114,233,0,0,0,90,13,95,95,99,108,97,115, - 115,99,101,108,108,95,95,114,2,0,0,0,114,2,0,0, - 0,41,1,114,218,0,0,0,114,4,0,0,0,114,217,0, - 0,0,154,3,0,0,115,24,0,0,0,8,3,4,2,8, - 6,8,4,8,3,16,12,12,5,8,7,12,6,8,4,8, - 6,8,6,114,217,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,3,0,0,0,64,0,0,0,115,46,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, - 100,7,156,1,100,8,100,9,132,2,90,6,100,10,83,0, - 41,11,218,16,83,111,117,114,99,101,70,105,108,101,76,111, - 97,100,101,114,122,62,67,111,110,99,114,101,116,101,32,105, - 109,112,108,101,109,101,110,116,97,116,105,111,110,32,111,102, - 32,83,111,117,114,99,101,76,111,97,100,101,114,32,117,115, - 105,110,103,32,116,104,101,32,102,105,108,101,32,115,121,115, - 116,101,109,46,99,2,0,0,0,0,0,0,0,3,0,0, - 0,3,0,0,0,67,0,0,0,115,22,0,0,0,116,0, - 124,1,131,1,125,2,124,2,106,1,124,2,106,2,100,1, - 156,2,83,0,41,2,122,33,82,101,116,117,114,110,32,116, - 104,101,32,109,101,116,97,100,97,116,97,32,102,111,114,32, - 116,104,101,32,112,97,116,104,46,41,2,114,154,0,0,0, - 114,212,0,0,0,41,3,114,39,0,0,0,218,8,115,116, - 95,109,116,105,109,101,90,7,115,116,95,115,105,122,101,41, - 3,114,107,0,0,0,114,35,0,0,0,114,216,0,0,0, - 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,114, - 202,0,0,0,226,3,0,0,115,4,0,0,0,0,2,8, - 1,122,27,83,111,117,114,99,101,70,105,108,101,76,111,97, - 100,101,114,46,112,97,116,104,95,115,116,97,116,115,99,4, - 0,0,0,0,0,0,0,5,0,0,0,5,0,0,0,67, - 0,0,0,115,24,0,0,0,116,0,124,1,131,1,125,4, - 124,0,106,1,124,2,124,3,124,4,100,1,141,3,83,0, - 41,2,78,41,1,218,5,95,109,111,100,101,41,2,114,104, - 0,0,0,114,203,0,0,0,41,5,114,107,0,0,0,114, - 98,0,0,0,114,97,0,0,0,114,57,0,0,0,114,42, + 32,32,32,82,97,105,115,101,115,32,79,83,69,114,114,111, + 114,32,119,104,101,110,32,116,104,101,32,112,97,116,104,32, + 99,97,110,110,111,116,32,98,101,32,104,97,110,100,108,101, + 100,46,10,32,32,32,32,32,32,32,32,114,153,0,0,0, + 41,1,114,200,0,0,0,41,2,114,106,0,0,0,114,35, 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,114,204,0,0,0,231,3,0,0,115,4,0,0,0, - 0,2,8,1,122,32,83,111,117,114,99,101,70,105,108,101, - 76,111,97,100,101,114,46,95,99,97,99,104,101,95,98,121, - 116,101,99,111,100,101,105,182,1,0,0,41,1,114,236,0, - 0,0,99,3,0,0,0,1,0,0,0,9,0,0,0,11, - 0,0,0,67,0,0,0,115,252,0,0,0,116,0,124,1, - 131,1,92,2,125,4,125,5,103,0,125,6,124,4,114,52, - 116,1,124,4,131,1,115,52,116,0,124,4,131,1,92,2, - 125,4,125,7,124,6,160,2,124,7,161,1,1,0,113,16, - 116,3,124,6,131,1,68,0,93,108,125,7,116,4,124,4, - 124,7,131,2,125,4,122,14,116,5,160,6,124,4,161,1, - 1,0,87,0,113,60,4,0,116,7,107,10,114,112,1,0, - 1,0,1,0,89,0,113,60,89,0,113,60,4,0,116,8, - 107,10,114,166,1,0,125,8,1,0,122,26,116,9,160,10, - 100,1,124,4,124,8,161,3,1,0,87,0,89,0,162,6, - 1,0,100,2,83,0,100,2,125,8,126,8,88,0,89,0, - 113,60,88,0,113,60,122,28,116,11,124,1,124,2,124,3, - 131,3,1,0,116,9,160,10,100,3,124,1,161,2,1,0, - 87,0,110,48,4,0,116,8,107,10,114,246,1,0,125,8, - 1,0,122,18,116,9,160,10,100,1,124,1,124,8,161,3, - 1,0,87,0,53,0,100,2,125,8,126,8,88,0,89,0, - 110,2,88,0,100,2,83,0,41,4,122,27,87,114,105,116, - 101,32,98,121,116,101,115,32,100,97,116,97,32,116,111,32, - 97,32,102,105,108,101,46,122,27,99,111,117,108,100,32,110, - 111,116,32,99,114,101,97,116,101,32,123,33,114,125,58,32, - 123,33,114,125,78,122,12,99,114,101,97,116,101,100,32,123, - 33,114,125,41,12,114,38,0,0,0,114,46,0,0,0,114, - 170,0,0,0,114,33,0,0,0,114,28,0,0,0,114,1, - 0,0,0,90,5,109,107,100,105,114,218,15,70,105,108,101, - 69,120,105,115,116,115,69,114,114,111,114,114,40,0,0,0, - 114,121,0,0,0,114,135,0,0,0,114,59,0,0,0,41, - 9,114,107,0,0,0,114,35,0,0,0,114,57,0,0,0, - 114,236,0,0,0,218,6,112,97,114,101,110,116,114,87,0, - 0,0,114,27,0,0,0,114,23,0,0,0,114,206,0,0, + 0,0,218,10,112,97,116,104,95,115,116,97,116,115,20,3, + 0,0,115,2,0,0,0,0,11,122,23,83,111,117,114,99, + 101,76,111,97,100,101,114,46,112,97,116,104,95,115,116,97, + 116,115,99,4,0,0,0,0,0,0,0,4,0,0,0,4, + 0,0,0,67,0,0,0,115,12,0,0,0,124,0,160,0, + 124,2,124,3,161,2,83,0,41,1,122,228,79,112,116,105, + 111,110,97,108,32,109,101,116,104,111,100,32,119,104,105,99, + 104,32,119,114,105,116,101,115,32,100,97,116,97,32,40,98, + 121,116,101,115,41,32,116,111,32,97,32,102,105,108,101,32, + 112,97,116,104,32,40,97,32,115,116,114,41,46,10,10,32, + 32,32,32,32,32,32,32,73,109,112,108,101,109,101,110,116, + 105,110,103,32,116,104,105,115,32,109,101,116,104,111,100,32, + 97,108,108,111,119,115,32,102,111,114,32,116,104,101,32,119, + 114,105,116,105,110,103,32,111,102,32,98,121,116,101,99,111, + 100,101,32,102,105,108,101,115,46,10,10,32,32,32,32,32, + 32,32,32,84,104,101,32,115,111,117,114,99,101,32,112,97, + 116,104,32,105,115,32,110,101,101,100,101,100,32,105,110,32, + 111,114,100,101,114,32,116,111,32,99,111,114,114,101,99,116, + 108,121,32,116,114,97,110,115,102,101,114,32,112,101,114,109, + 105,115,115,105,111,110,115,10,32,32,32,32,32,32,32,32, + 41,1,218,8,115,101,116,95,100,97,116,97,41,4,114,106, + 0,0,0,114,97,0,0,0,90,10,99,97,99,104,101,95, + 112,97,116,104,114,56,0,0,0,114,2,0,0,0,114,2, + 0,0,0,114,4,0,0,0,218,15,95,99,97,99,104,101, + 95,98,121,116,101,99,111,100,101,33,3,0,0,115,2,0, + 0,0,0,8,122,28,83,111,117,114,99,101,76,111,97,100, + 101,114,46,95,99,97,99,104,101,95,98,121,116,101,99,111, + 100,101,99,3,0,0,0,0,0,0,0,3,0,0,0,1, + 0,0,0,67,0,0,0,115,4,0,0,0,100,1,83,0, + 41,2,122,150,79,112,116,105,111,110,97,108,32,109,101,116, + 104,111,100,32,119,104,105,99,104,32,119,114,105,116,101,115, + 32,100,97,116,97,32,40,98,121,116,101,115,41,32,116,111, + 32,97,32,102,105,108,101,32,112,97,116,104,32,40,97,32, + 115,116,114,41,46,10,10,32,32,32,32,32,32,32,32,73, + 109,112,108,101,109,101,110,116,105,110,103,32,116,104,105,115, + 32,109,101,116,104,111,100,32,97,108,108,111,119,115,32,102, + 111,114,32,116,104,101,32,119,114,105,116,105,110,103,32,111, + 102,32,98,121,116,101,99,111,100,101,32,102,105,108,101,115, + 46,10,32,32,32,32,32,32,32,32,78,114,2,0,0,0, + 41,3,114,106,0,0,0,114,35,0,0,0,114,56,0,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 114,203,0,0,0,236,3,0,0,115,42,0,0,0,0,2, - 12,1,4,2,12,1,12,1,12,2,12,1,10,1,2,1, - 14,1,14,2,8,1,16,3,6,1,8,1,28,1,2,1, - 12,1,16,1,16,2,8,1,122,25,83,111,117,114,99,101, - 70,105,108,101,76,111,97,100,101,114,46,115,101,116,95,100, - 97,116,97,78,41,7,114,112,0,0,0,114,111,0,0,0, - 114,113,0,0,0,114,114,0,0,0,114,202,0,0,0,114, - 204,0,0,0,114,203,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,114,234,0, - 0,0,222,3,0,0,115,8,0,0,0,8,2,4,2,8, - 5,8,5,114,234,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,2,0,0,0,64,0,0,0,115,32,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, - 83,0,41,7,218,20,83,111,117,114,99,101,108,101,115,115, - 70,105,108,101,76,111,97,100,101,114,122,45,76,111,97,100, - 101,114,32,119,104,105,99,104,32,104,97,110,100,108,101,115, - 32,115,111,117,114,99,101,108,101,115,115,32,102,105,108,101, - 32,105,109,112,111,114,116,115,46,99,2,0,0,0,0,0, - 0,0,5,0,0,0,5,0,0,0,67,0,0,0,115,68, - 0,0,0,124,0,160,0,124,1,161,1,125,2,124,0,160, - 1,124,2,161,1,125,3,124,1,124,2,100,1,156,2,125, - 4,116,2,124,3,124,1,124,4,131,3,1,0,116,3,116, - 4,124,3,131,1,100,2,100,0,133,2,25,0,124,1,124, - 2,100,3,141,3,83,0,41,4,78,41,2,114,105,0,0, - 0,114,35,0,0,0,114,131,0,0,0,41,2,114,105,0, - 0,0,114,97,0,0,0,41,5,114,164,0,0,0,114,205, - 0,0,0,114,138,0,0,0,114,150,0,0,0,114,213,0, - 0,0,41,5,114,107,0,0,0,114,126,0,0,0,114,35, - 0,0,0,114,57,0,0,0,114,137,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,193,0,0, - 0,15,4,0,0,115,18,0,0,0,0,1,10,1,10,4, - 2,1,8,2,12,1,2,1,14,1,2,1,122,29,83,111, - 117,114,99,101,108,101,115,115,70,105,108,101,76,111,97,100, - 101,114,46,103,101,116,95,99,111,100,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,83,0,41,2,122,39,82,101,116, - 117,114,110,32,78,111,110,101,32,97,115,32,116,104,101,114, - 101,32,105,115,32,110,111,32,115,111,117,114,99,101,32,99, - 111,100,101,46,78,114,2,0,0,0,41,2,114,107,0,0, - 0,114,126,0,0,0,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,114,207,0,0,0,31,4,0,0,115,2, - 0,0,0,0,2,122,31,83,111,117,114,99,101,108,101,115, - 115,70,105,108,101,76,111,97,100,101,114,46,103,101,116,95, - 115,111,117,114,99,101,78,41,6,114,112,0,0,0,114,111, - 0,0,0,114,113,0,0,0,114,114,0,0,0,114,193,0, - 0,0,114,207,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,114,239,0,0,0, - 11,4,0,0,115,6,0,0,0,8,2,4,2,8,16,114, - 239,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, - 0,3,0,0,0,64,0,0,0,115,92,0,0,0,101,0, - 90,1,100,0,90,2,100,1,90,3,100,2,100,3,132,0, - 90,4,100,4,100,5,132,0,90,5,100,6,100,7,132,0, - 90,6,100,8,100,9,132,0,90,7,100,10,100,11,132,0, - 90,8,100,12,100,13,132,0,90,9,100,14,100,15,132,0, - 90,10,100,16,100,17,132,0,90,11,101,12,100,18,100,19, - 132,0,131,1,90,13,100,20,83,0,41,21,218,19,69,120, - 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, - 114,122,93,76,111,97,100,101,114,32,102,111,114,32,101,120, - 116,101,110,115,105,111,110,32,109,111,100,117,108,101,115,46, - 10,10,32,32,32,32,84,104,101,32,99,111,110,115,116,114, - 117,99,116,111,114,32,105,115,32,100,101,115,105,103,110,101, - 100,32,116,111,32,119,111,114,107,32,119,105,116,104,32,70, - 105,108,101,70,105,110,100,101,114,46,10,10,32,32,32,32, - 99,3,0,0,0,0,0,0,0,3,0,0,0,2,0,0, - 0,67,0,0,0,115,16,0,0,0,124,1,124,0,95,0, - 124,2,124,0,95,1,100,0,83,0,41,1,78,41,2,114, - 105,0,0,0,114,35,0,0,0,41,3,114,107,0,0,0, - 114,105,0,0,0,114,35,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,191,0,0,0,48,4, - 0,0,115,4,0,0,0,0,1,6,1,122,28,69,120,116, - 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, - 46,95,95,105,110,105,116,95,95,99,2,0,0,0,0,0, - 0,0,2,0,0,0,2,0,0,0,67,0,0,0,115,24, - 0,0,0,124,0,106,0,124,1,106,0,107,2,111,22,124, - 0,106,1,124,1,106,1,107,2,83,0,41,1,78,41,2, - 114,218,0,0,0,114,118,0,0,0,41,2,114,107,0,0, - 0,114,219,0,0,0,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,114,220,0,0,0,52,4,0,0,115,4, - 0,0,0,0,1,12,1,122,26,69,120,116,101,110,115,105, - 111,110,70,105,108,101,76,111,97,100,101,114,46,95,95,101, - 113,95,95,99,1,0,0,0,0,0,0,0,1,0,0,0, - 3,0,0,0,67,0,0,0,115,20,0,0,0,116,0,124, - 0,106,1,131,1,116,0,124,0,106,2,131,1,65,0,83, - 0,41,1,78,41,3,114,221,0,0,0,114,105,0,0,0, - 114,35,0,0,0,41,1,114,107,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,114,222,0,0,0, - 56,4,0,0,115,2,0,0,0,0,1,122,28,69,120,116, - 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, + 114,202,0,0,0,43,3,0,0,115,2,0,0,0,0,4, + 122,21,83,111,117,114,99,101,76,111,97,100,101,114,46,115, + 101,116,95,100,97,116,97,99,2,0,0,0,0,0,0,0, + 5,0,0,0,10,0,0,0,67,0,0,0,115,82,0,0, + 0,124,0,160,0,124,1,161,1,125,2,122,14,124,0,160, + 1,124,2,161,1,125,3,87,0,110,48,4,0,116,2,107, + 10,114,72,1,0,125,4,1,0,122,18,116,3,100,1,124, + 1,100,2,141,2,124,4,130,2,87,0,53,0,100,3,125, + 4,126,4,88,0,89,0,110,2,88,0,116,4,124,3,131, + 1,83,0,41,4,122,52,67,111,110,99,114,101,116,101,32, + 105,109,112,108,101,109,101,110,116,97,116,105,111,110,32,111, + 102,32,73,110,115,112,101,99,116,76,111,97,100,101,114,46, + 103,101,116,95,115,111,117,114,99,101,46,122,39,115,111,117, + 114,99,101,32,110,111,116,32,97,118,97,105,108,97,98,108, + 101,32,116,104,114,111,117,103,104,32,103,101,116,95,100,97, + 116,97,40,41,41,1,114,104,0,0,0,78,41,5,114,163, + 0,0,0,218,8,103,101,116,95,100,97,116,97,114,40,0, + 0,0,114,105,0,0,0,114,161,0,0,0,41,5,114,106, + 0,0,0,114,125,0,0,0,114,35,0,0,0,114,159,0, + 0,0,218,3,101,120,99,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,10,103,101,116,95,115,111,117,114, + 99,101,50,3,0,0,115,20,0,0,0,0,2,10,1,2, + 1,14,1,16,1,4,1,2,255,4,1,2,255,20,2,122, + 23,83,111,117,114,99,101,76,111,97,100,101,114,46,103,101, + 116,95,115,111,117,114,99,101,114,94,0,0,0,41,1,218, + 9,95,111,112,116,105,109,105,122,101,99,3,0,0,0,1, + 0,0,0,4,0,0,0,8,0,0,0,67,0,0,0,115, + 22,0,0,0,116,0,106,1,116,2,124,1,124,2,100,1, + 100,2,124,3,100,3,141,6,83,0,41,4,122,130,82,101, + 116,117,114,110,32,116,104,101,32,99,111,100,101,32,111,98, + 106,101,99,116,32,99,111,109,112,105,108,101,100,32,102,114, + 111,109,32,115,111,117,114,99,101,46,10,10,32,32,32,32, + 32,32,32,32,84,104,101,32,39,100,97,116,97,39,32,97, + 114,103,117,109,101,110,116,32,99,97,110,32,98,101,32,97, + 110,121,32,111,98,106,101,99,116,32,116,121,112,101,32,116, + 104,97,116,32,99,111,109,112,105,108,101,40,41,32,115,117, + 112,112,111,114,116,115,46,10,32,32,32,32,32,32,32,32, + 114,194,0,0,0,84,41,2,218,12,100,111,110,116,95,105, + 110,104,101,114,105,116,114,73,0,0,0,41,3,114,120,0, + 0,0,114,193,0,0,0,218,7,99,111,109,112,105,108,101, + 41,4,114,106,0,0,0,114,56,0,0,0,114,35,0,0, + 0,114,207,0,0,0,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,218,14,115,111,117,114,99,101,95,116,111, + 95,99,111,100,101,60,3,0,0,115,8,0,0,0,0,5, + 12,1,2,0,2,255,122,27,83,111,117,114,99,101,76,111, + 97,100,101,114,46,115,111,117,114,99,101,95,116,111,95,99, + 111,100,101,99,2,0,0,0,0,0,0,0,15,0,0,0, + 9,0,0,0,67,0,0,0,115,46,2,0,0,124,0,160, + 0,124,1,161,1,125,2,100,1,125,3,100,1,125,4,100, + 1,125,5,100,2,125,6,100,3,125,7,122,12,116,1,124, + 2,131,1,125,8,87,0,110,26,4,0,116,2,107,10,114, + 68,1,0,1,0,1,0,100,1,125,8,89,0,144,1,110, + 48,88,0,122,14,124,0,160,3,124,2,161,1,125,9,87, + 0,110,22,4,0,116,4,107,10,114,106,1,0,1,0,1, + 0,89,0,144,1,110,10,88,0,116,5,124,9,100,4,25, + 0,131,1,125,3,122,14,124,0,160,6,124,8,161,1,125, + 10,87,0,110,20,4,0,116,4,107,10,114,154,1,0,1, + 0,1,0,89,0,110,218,88,0,124,1,124,8,100,5,156, + 2,125,11,122,148,116,7,124,10,124,1,124,11,131,3,125, + 12,116,8,124,10,131,1,100,6,100,1,133,2,25,0,125, + 13,124,12,100,7,64,0,100,8,107,3,125,6,124,6,144, + 1,114,36,124,12,100,9,64,0,100,8,107,3,125,7,116, + 9,106,10,100,10,107,3,144,1,114,34,124,7,115,254,116, + 9,106,10,100,11,107,2,144,1,114,34,124,0,160,6,124, + 2,161,1,125,4,116,9,160,11,116,12,124,4,161,2,125, + 5,116,13,124,10,124,5,124,1,124,11,131,4,1,0,110, + 20,116,14,124,10,124,3,124,9,100,12,25,0,124,1,124, + 11,131,5,1,0,87,0,110,26,4,0,116,15,116,16,102, + 2,107,10,144,1,114,84,1,0,1,0,1,0,89,0,110, + 32,88,0,116,17,160,18,100,13,124,8,124,2,161,3,1, + 0,116,19,124,13,124,1,124,8,124,2,100,14,141,4,83, + 0,124,4,100,1,107,8,144,1,114,136,124,0,160,6,124, + 2,161,1,125,4,124,0,160,20,124,4,124,2,161,2,125, + 14,116,17,160,18,100,15,124,2,161,2,1,0,116,21,106, + 22,144,2,115,42,124,8,100,1,107,9,144,2,114,42,124, + 3,100,1,107,9,144,2,114,42,124,6,144,1,114,228,124, + 5,100,1,107,8,144,1,114,214,116,9,160,11,124,4,161, + 1,125,5,116,23,124,14,124,5,124,7,131,3,125,10,110, + 16,116,24,124,14,124,3,116,25,124,4,131,1,131,3,125, + 10,122,30,124,0,160,26,124,2,124,8,124,10,161,3,1, + 0,116,17,160,18,100,16,124,8,161,2,1,0,87,0,110, + 22,4,0,116,2,107,10,144,2,114,40,1,0,1,0,1, + 0,89,0,110,2,88,0,124,14,83,0,41,17,122,190,67, + 111,110,99,114,101,116,101,32,105,109,112,108,101,109,101,110, + 116,97,116,105,111,110,32,111,102,32,73,110,115,112,101,99, + 116,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, + 46,10,10,32,32,32,32,32,32,32,32,82,101,97,100,105, + 110,103,32,111,102,32,98,121,116,101,99,111,100,101,32,114, + 101,113,117,105,114,101,115,32,112,97,116,104,95,115,116,97, + 116,115,32,116,111,32,98,101,32,105,109,112,108,101,109,101, + 110,116,101,100,46,32,84,111,32,119,114,105,116,101,10,32, + 32,32,32,32,32,32,32,98,121,116,101,99,111,100,101,44, + 32,115,101,116,95,100,97,116,97,32,109,117,115,116,32,97, + 108,115,111,32,98,101,32,105,109,112,108,101,109,101,110,116, + 101,100,46,10,10,32,32,32,32,32,32,32,32,78,70,84, + 114,153,0,0,0,41,2,114,104,0,0,0,114,35,0,0, + 0,114,130,0,0,0,114,29,0,0,0,114,62,0,0,0, + 114,59,0,0,0,90,5,110,101,118,101,114,90,6,97,108, + 119,97,121,115,218,4,115,105,122,101,122,13,123,125,32,109, + 97,116,99,104,101,115,32,123,125,41,3,114,104,0,0,0, + 114,96,0,0,0,114,97,0,0,0,122,19,99,111,100,101, + 32,111,98,106,101,99,116,32,102,114,111,109,32,123,125,122, + 10,119,114,111,116,101,32,123,33,114,125,41,27,114,163,0, + 0,0,114,87,0,0,0,114,71,0,0,0,114,201,0,0, + 0,114,40,0,0,0,114,14,0,0,0,114,204,0,0,0, + 114,137,0,0,0,218,10,109,101,109,111,114,121,118,105,101, + 119,114,147,0,0,0,90,21,99,104,101,99,107,95,104,97, + 115,104,95,98,97,115,101,100,95,112,121,99,115,114,142,0, + 0,0,218,17,95,82,65,87,95,77,65,71,73,67,95,78, + 85,77,66,69,82,114,143,0,0,0,114,141,0,0,0,114, + 105,0,0,0,114,135,0,0,0,114,120,0,0,0,114,134, + 0,0,0,114,149,0,0,0,114,210,0,0,0,114,6,0, + 0,0,218,19,100,111,110,116,95,119,114,105,116,101,95,98, + 121,116,101,99,111,100,101,114,156,0,0,0,114,154,0,0, + 0,114,31,0,0,0,114,203,0,0,0,41,15,114,106,0, + 0,0,114,125,0,0,0,114,97,0,0,0,114,139,0,0, + 0,114,159,0,0,0,114,142,0,0,0,90,10,104,97,115, + 104,95,98,97,115,101,100,90,12,99,104,101,99,107,95,115, + 111,117,114,99,101,114,96,0,0,0,218,2,115,116,114,56, + 0,0,0,114,136,0,0,0,114,72,0,0,0,90,10,98, + 121,116,101,115,95,100,97,116,97,90,11,99,111,100,101,95, + 111,98,106,101,99,116,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,114,192,0,0,0,68,3,0,0,115,154, + 0,0,0,0,7,10,1,4,1,4,1,4,1,4,1,4, + 1,2,1,12,1,14,1,12,2,2,1,14,1,14,1,8, + 2,12,1,2,1,14,1,14,1,6,3,2,1,2,254,6, + 4,2,1,12,1,16,1,12,1,6,1,12,1,12,1,2, + 255,2,2,8,254,4,3,10,1,4,1,2,1,2,254,4, + 4,8,1,2,255,6,3,2,1,2,1,2,1,6,1,2, + 1,2,251,8,7,20,1,6,2,8,1,2,255,4,2,6, + 1,2,1,2,254,6,3,10,1,10,1,12,1,12,1,18, + 1,6,255,4,2,6,1,10,1,10,1,14,2,6,1,6, + 255,4,2,2,1,14,1,16,1,16,1,6,1,122,21,83, + 111,117,114,99,101,76,111,97,100,101,114,46,103,101,116,95, + 99,111,100,101,78,41,10,114,111,0,0,0,114,110,0,0, + 0,114,112,0,0,0,114,200,0,0,0,114,201,0,0,0, + 114,203,0,0,0,114,202,0,0,0,114,206,0,0,0,114, + 210,0,0,0,114,192,0,0,0,114,2,0,0,0,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,199,0, + 0,0,10,3,0,0,115,14,0,0,0,8,2,8,8,8, + 13,8,10,8,7,8,10,14,8,114,199,0,0,0,99,0, + 0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0, + 0,0,0,115,124,0,0,0,101,0,90,1,100,0,90,2, + 100,1,90,3,100,2,100,3,132,0,90,4,100,4,100,5, + 132,0,90,5,100,6,100,7,132,0,90,6,101,7,135,0, + 102,1,100,8,100,9,132,8,131,1,90,8,101,7,100,10, + 100,11,132,0,131,1,90,9,100,12,100,13,132,0,90,10, + 101,7,100,14,100,15,132,0,131,1,90,11,100,16,100,17, + 132,0,90,12,100,18,100,19,132,0,90,13,100,20,100,21, + 132,0,90,14,100,22,100,23,132,0,90,15,135,0,4,0, + 90,16,83,0,41,24,218,10,70,105,108,101,76,111,97,100, + 101,114,122,103,66,97,115,101,32,102,105,108,101,32,108,111, + 97,100,101,114,32,99,108,97,115,115,32,119,104,105,99,104, + 32,105,109,112,108,101,109,101,110,116,115,32,116,104,101,32, + 108,111,97,100,101,114,32,112,114,111,116,111,99,111,108,32, + 109,101,116,104,111,100,115,32,116,104,97,116,10,32,32,32, + 32,114,101,113,117,105,114,101,32,102,105,108,101,32,115,121, + 115,116,101,109,32,117,115,97,103,101,46,99,3,0,0,0, + 0,0,0,0,3,0,0,0,2,0,0,0,67,0,0,0, + 115,16,0,0,0,124,1,124,0,95,0,124,2,124,0,95, + 1,100,1,83,0,41,2,122,75,67,97,99,104,101,32,116, + 104,101,32,109,111,100,117,108,101,32,110,97,109,101,32,97, + 110,100,32,116,104,101,32,112,97,116,104,32,116,111,32,116, + 104,101,32,102,105,108,101,32,102,111,117,110,100,32,98,121, + 32,116,104,101,10,32,32,32,32,32,32,32,32,102,105,110, + 100,101,114,46,78,41,2,114,104,0,0,0,114,35,0,0, + 0,41,3,114,106,0,0,0,114,125,0,0,0,114,35,0, + 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, + 0,114,190,0,0,0,159,3,0,0,115,4,0,0,0,0, + 3,6,1,122,19,70,105,108,101,76,111,97,100,101,114,46, + 95,95,105,110,105,116,95,95,99,2,0,0,0,0,0,0, + 0,2,0,0,0,2,0,0,0,67,0,0,0,115,24,0, + 0,0,124,0,106,0,124,1,106,0,107,2,111,22,124,0, + 106,1,124,1,106,1,107,2,83,0,41,1,78,41,2,218, + 9,95,95,99,108,97,115,115,95,95,114,117,0,0,0,41, + 2,114,106,0,0,0,218,5,111,116,104,101,114,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,218,6,95,95, + 101,113,95,95,165,3,0,0,115,6,0,0,0,0,1,12, + 1,10,255,122,17,70,105,108,101,76,111,97,100,101,114,46, + 95,95,101,113,95,95,99,1,0,0,0,0,0,0,0,1, + 0,0,0,3,0,0,0,67,0,0,0,115,20,0,0,0, + 116,0,124,0,106,1,131,1,116,0,124,0,106,2,131,1, + 65,0,83,0,41,1,78,41,3,218,4,104,97,115,104,114, + 104,0,0,0,114,35,0,0,0,41,1,114,106,0,0,0, + 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, + 8,95,95,104,97,115,104,95,95,169,3,0,0,115,2,0, + 0,0,0,1,122,19,70,105,108,101,76,111,97,100,101,114, 46,95,95,104,97,115,104,95,95,99,2,0,0,0,0,0, - 0,0,3,0,0,0,5,0,0,0,67,0,0,0,115,36, - 0,0,0,116,0,160,1,116,2,106,3,124,1,161,2,125, - 2,116,0,160,4,100,1,124,1,106,5,124,0,106,6,161, - 3,1,0,124,2,83,0,41,2,122,38,67,114,101,97,116, - 101,32,97,110,32,117,110,105,116,105,97,108,105,122,101,100, - 32,101,120,116,101,110,115,105,111,110,32,109,111,100,117,108, - 101,122,38,101,120,116,101,110,115,105,111,110,32,109,111,100, - 117,108,101,32,123,33,114,125,32,108,111,97,100,101,100,32, - 102,114,111,109,32,123,33,114,125,41,7,114,121,0,0,0, - 114,194,0,0,0,114,148,0,0,0,90,14,99,114,101,97, - 116,101,95,100,121,110,97,109,105,99,114,135,0,0,0,114, - 105,0,0,0,114,35,0,0,0,41,3,114,107,0,0,0, - 114,171,0,0,0,114,196,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,192,0,0,0,59,4, - 0,0,115,10,0,0,0,0,2,4,1,10,1,6,1,12, - 1,122,33,69,120,116,101,110,115,105,111,110,70,105,108,101, - 76,111,97,100,101,114,46,99,114,101,97,116,101,95,109,111, - 100,117,108,101,99,2,0,0,0,0,0,0,0,2,0,0, - 0,5,0,0,0,67,0,0,0,115,36,0,0,0,116,0, - 160,1,116,2,106,3,124,1,161,2,1,0,116,0,160,4, - 100,1,124,0,106,5,124,0,106,6,161,3,1,0,100,2, - 83,0,41,3,122,30,73,110,105,116,105,97,108,105,122,101, - 32,97,110,32,101,120,116,101,110,115,105,111,110,32,109,111, - 100,117,108,101,122,40,101,120,116,101,110,115,105,111,110,32, - 109,111,100,117,108,101,32,123,33,114,125,32,101,120,101,99, - 117,116,101,100,32,102,114,111,109,32,123,33,114,125,78,41, - 7,114,121,0,0,0,114,194,0,0,0,114,148,0,0,0, - 90,12,101,120,101,99,95,100,121,110,97,109,105,99,114,135, - 0,0,0,114,105,0,0,0,114,35,0,0,0,41,2,114, - 107,0,0,0,114,196,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,114,197,0,0,0,67,4,0, - 0,115,6,0,0,0,0,2,14,1,6,1,122,31,69,120, - 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, - 114,46,101,120,101,99,95,109,111,100,117,108,101,99,2,0, - 0,0,0,0,0,0,2,0,0,0,4,0,0,0,3,0, - 0,0,115,36,0,0,0,116,0,124,0,106,1,131,1,100, - 1,25,0,137,0,116,2,135,0,102,1,100,2,100,3,132, - 8,116,3,68,0,131,1,131,1,83,0,41,4,122,49,82, - 101,116,117,114,110,32,84,114,117,101,32,105,102,32,116,104, - 101,32,101,120,116,101,110,115,105,111,110,32,109,111,100,117, - 108,101,32,105,115,32,97,32,112,97,99,107,97,103,101,46, - 114,29,0,0,0,99,1,0,0,0,0,0,0,0,2,0, - 0,0,4,0,0,0,51,0,0,0,115,26,0,0,0,124, - 0,93,18,125,1,136,0,100,0,124,1,23,0,107,2,86, - 0,1,0,113,2,100,1,83,0,41,2,114,191,0,0,0, - 78,114,2,0,0,0,41,2,114,22,0,0,0,218,6,115, - 117,102,102,105,120,41,1,218,9,102,105,108,101,95,110,97, - 109,101,114,2,0,0,0,114,4,0,0,0,218,9,60,103, - 101,110,101,120,112,114,62,76,4,0,0,115,2,0,0,0, - 4,1,122,49,69,120,116,101,110,115,105,111,110,70,105,108, - 101,76,111,97,100,101,114,46,105,115,95,112,97,99,107,97, - 103,101,46,60,108,111,99,97,108,115,62,46,60,103,101,110, - 101,120,112,114,62,41,4,114,38,0,0,0,114,35,0,0, - 0,218,3,97,110,121,218,18,69,88,84,69,78,83,73,79, - 78,95,83,85,70,70,73,88,69,83,41,2,114,107,0,0, - 0,114,126,0,0,0,114,2,0,0,0,41,1,114,242,0, - 0,0,114,4,0,0,0,114,166,0,0,0,73,4,0,0, - 115,6,0,0,0,0,2,14,1,12,1,122,30,69,120,116, - 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, - 46,105,115,95,112,97,99,107,97,103,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,83,0,41,2,122,63,82,101,116, - 117,114,110,32,78,111,110,101,32,97,115,32,97,110,32,101, - 120,116,101,110,115,105,111,110,32,109,111,100,117,108,101,32, - 99,97,110,110,111,116,32,99,114,101,97,116,101,32,97,32, - 99,111,100,101,32,111,98,106,101,99,116,46,78,114,2,0, - 0,0,41,2,114,107,0,0,0,114,126,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,114,193,0, - 0,0,79,4,0,0,115,2,0,0,0,0,2,122,28,69, - 120,116,101,110,115,105,111,110,70,105,108,101,76,111,97,100, - 101,114,46,103,101,116,95,99,111,100,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,83,0,41,2,122,53,82,101,116, - 117,114,110,32,78,111,110,101,32,97,115,32,101,120,116,101, - 110,115,105,111,110,32,109,111,100,117,108,101,115,32,104,97, - 118,101,32,110,111,32,115,111,117,114,99,101,32,99,111,100, - 101,46,78,114,2,0,0,0,41,2,114,107,0,0,0,114, - 126,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, - 0,0,0,114,207,0,0,0,83,4,0,0,115,2,0,0, - 0,0,2,122,30,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, - 114,99,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 0,0,2,0,0,0,3,0,0,0,3,0,0,0,115,16, + 0,0,0,116,0,116,1,124,0,131,2,160,2,124,1,161, + 1,83,0,41,1,122,100,76,111,97,100,32,97,32,109,111, + 100,117,108,101,32,102,114,111,109,32,97,32,102,105,108,101, + 46,10,10,32,32,32,32,32,32,32,32,84,104,105,115,32, + 109,101,116,104,111,100,32,105,115,32,100,101,112,114,101,99, + 97,116,101,100,46,32,32,85,115,101,32,101,120,101,99,95, + 109,111,100,117,108,101,40,41,32,105,110,115,116,101,97,100, + 46,10,10,32,32,32,32,32,32,32,32,41,3,218,5,115, + 117,112,101,114,114,216,0,0,0,114,198,0,0,0,41,2, + 114,106,0,0,0,114,125,0,0,0,41,1,114,217,0,0, + 0,114,2,0,0,0,114,4,0,0,0,114,198,0,0,0, + 172,3,0,0,115,2,0,0,0,0,10,122,22,70,105,108, + 101,76,111,97,100,101,114,46,108,111,97,100,95,109,111,100, + 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, 1,0,0,0,67,0,0,0,115,6,0,0,0,124,0,106, 0,83,0,41,1,122,58,82,101,116,117,114,110,32,116,104, 101,32,112,97,116,104,32,116,111,32,116,104,101,32,115,111, 117,114,99,101,32,102,105,108,101,32,97,115,32,102,111,117, 110,100,32,98,121,32,116,104,101,32,102,105,110,100,101,114, - 46,41,1,114,35,0,0,0,41,2,114,107,0,0,0,114, - 126,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, - 0,0,0,114,164,0,0,0,87,4,0,0,115,2,0,0, - 0,0,3,122,32,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,103,101,116,95,102,105,108, - 101,110,97,109,101,78,41,14,114,112,0,0,0,114,111,0, - 0,0,114,113,0,0,0,114,114,0,0,0,114,191,0,0, - 0,114,220,0,0,0,114,222,0,0,0,114,192,0,0,0, - 114,197,0,0,0,114,166,0,0,0,114,193,0,0,0,114, - 207,0,0,0,114,123,0,0,0,114,164,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,114,240,0,0,0,40,4,0,0,115,20,0,0,0, - 8,6,4,2,8,4,8,4,8,3,8,8,8,6,8,6, - 8,4,8,4,114,240,0,0,0,99,0,0,0,0,0,0, - 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,96, + 46,41,1,114,35,0,0,0,41,2,114,106,0,0,0,114, + 125,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, + 0,0,0,114,163,0,0,0,184,3,0,0,115,2,0,0, + 0,0,3,122,23,70,105,108,101,76,111,97,100,101,114,46, + 103,101,116,95,102,105,108,101,110,97,109,101,99,2,0,0, + 0,0,0,0,0,3,0,0,0,10,0,0,0,67,0,0, + 0,115,44,0,0,0,116,0,160,1,124,1,100,1,161,2, + 143,22,125,2,124,2,160,2,161,0,87,0,2,0,53,0, + 81,0,82,0,163,0,83,0,81,0,82,0,88,0,100,2, + 83,0,41,3,122,39,82,101,116,117,114,110,32,116,104,101, + 32,100,97,116,97,32,102,114,111,109,32,112,97,116,104,32, + 97,115,32,114,97,119,32,98,121,116,101,115,46,218,1,114, + 78,41,3,114,52,0,0,0,114,53,0,0,0,90,4,114, + 101,97,100,41,3,114,106,0,0,0,114,35,0,0,0,114, + 57,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, + 0,0,0,114,204,0,0,0,189,3,0,0,115,4,0,0, + 0,0,2,14,1,122,19,70,105,108,101,76,111,97,100,101, + 114,46,103,101,116,95,100,97,116,97,99,2,0,0,0,0, + 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, + 18,0,0,0,124,0,160,0,124,1,161,1,114,14,124,0, + 83,0,100,0,83,0,41,1,78,41,1,114,165,0,0,0, + 41,2,114,106,0,0,0,114,195,0,0,0,114,2,0,0, + 0,114,2,0,0,0,114,4,0,0,0,218,19,103,101,116, + 95,114,101,115,111,117,114,99,101,95,114,101,97,100,101,114, + 196,3,0,0,115,6,0,0,0,0,2,10,1,4,1,122, + 30,70,105,108,101,76,111,97,100,101,114,46,103,101,116,95, + 114,101,115,111,117,114,99,101,95,114,101,97,100,101,114,99, + 2,0,0,0,0,0,0,0,3,0,0,0,4,0,0,0, + 67,0,0,0,115,32,0,0,0,116,0,116,1,124,0,106, + 2,131,1,100,1,25,0,124,1,131,2,125,2,116,3,160, + 4,124,2,100,2,161,2,83,0,41,3,78,114,62,0,0, + 0,114,223,0,0,0,41,5,114,28,0,0,0,114,38,0, + 0,0,114,35,0,0,0,114,52,0,0,0,114,53,0,0, + 0,41,3,114,106,0,0,0,218,8,114,101,115,111,117,114, + 99,101,114,35,0,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,13,111,112,101,110,95,114,101,115, + 111,117,114,99,101,202,3,0,0,115,4,0,0,0,0,1, + 20,1,122,24,70,105,108,101,76,111,97,100,101,114,46,111, + 112,101,110,95,114,101,115,111,117,114,99,101,99,2,0,0, + 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, + 0,115,38,0,0,0,124,0,160,0,124,1,161,1,115,14, + 116,1,130,1,116,2,116,3,124,0,106,4,131,1,100,1, + 25,0,124,1,131,2,125,2,124,2,83,0,41,2,78,114, + 62,0,0,0,41,5,218,11,105,115,95,114,101,115,111,117, + 114,99,101,218,17,70,105,108,101,78,111,116,70,111,117,110, + 100,69,114,114,111,114,114,28,0,0,0,114,38,0,0,0, + 114,35,0,0,0,41,3,114,106,0,0,0,114,225,0,0, + 0,114,35,0,0,0,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,218,13,114,101,115,111,117,114,99,101,95, + 112,97,116,104,206,3,0,0,115,8,0,0,0,0,1,10, + 1,4,1,20,1,122,24,70,105,108,101,76,111,97,100,101, + 114,46,114,101,115,111,117,114,99,101,95,112,97,116,104,99, + 2,0,0,0,0,0,0,0,3,0,0,0,3,0,0,0, + 67,0,0,0,115,40,0,0,0,116,0,124,1,107,6,114, + 12,100,1,83,0,116,1,116,2,124,0,106,3,131,1,100, + 2,25,0,124,1,131,2,125,2,116,4,124,2,131,1,83, + 0,41,3,78,70,114,62,0,0,0,41,5,114,25,0,0, + 0,114,28,0,0,0,114,38,0,0,0,114,35,0,0,0, + 114,44,0,0,0,41,3,114,106,0,0,0,114,104,0,0, + 0,114,35,0,0,0,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,114,227,0,0,0,212,3,0,0,115,8, + 0,0,0,0,1,8,1,4,1,20,1,122,22,70,105,108, + 101,76,111,97,100,101,114,46,105,115,95,114,101,115,111,117, + 114,99,101,99,1,0,0,0,0,0,0,0,1,0,0,0, + 5,0,0,0,67,0,0,0,115,24,0,0,0,116,0,116, + 1,160,2,116,3,124,0,106,4,131,1,100,1,25,0,161, + 1,131,1,83,0,41,2,78,114,62,0,0,0,41,5,218, + 4,105,116,101,114,114,1,0,0,0,218,7,108,105,115,116, + 100,105,114,114,38,0,0,0,114,35,0,0,0,41,1,114, + 106,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, + 0,0,0,218,8,99,111,110,116,101,110,116,115,218,3,0, + 0,115,2,0,0,0,0,1,122,19,70,105,108,101,76,111, + 97,100,101,114,46,99,111,110,116,101,110,116,115,41,17,114, + 111,0,0,0,114,110,0,0,0,114,112,0,0,0,114,113, + 0,0,0,114,190,0,0,0,114,219,0,0,0,114,221,0, + 0,0,114,122,0,0,0,114,198,0,0,0,114,163,0,0, + 0,114,204,0,0,0,114,224,0,0,0,114,226,0,0,0, + 114,229,0,0,0,114,227,0,0,0,114,232,0,0,0,90, + 13,95,95,99,108,97,115,115,99,101,108,108,95,95,114,2, + 0,0,0,114,2,0,0,0,41,1,114,217,0,0,0,114, + 4,0,0,0,114,216,0,0,0,154,3,0,0,115,24,0, + 0,0,8,3,4,2,8,6,8,4,8,3,16,12,12,5, + 8,7,12,6,8,4,8,6,8,6,114,216,0,0,0,99, + 0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0, + 64,0,0,0,115,46,0,0,0,101,0,90,1,100,0,90, + 2,100,1,90,3,100,2,100,3,132,0,90,4,100,4,100, + 5,132,0,90,5,100,6,100,7,156,1,100,8,100,9,132, + 2,90,6,100,10,83,0,41,11,218,16,83,111,117,114,99, + 101,70,105,108,101,76,111,97,100,101,114,122,62,67,111,110, + 99,114,101,116,101,32,105,109,112,108,101,109,101,110,116,97, + 116,105,111,110,32,111,102,32,83,111,117,114,99,101,76,111, + 97,100,101,114,32,117,115,105,110,103,32,116,104,101,32,102, + 105,108,101,32,115,121,115,116,101,109,46,99,2,0,0,0, + 0,0,0,0,3,0,0,0,3,0,0,0,67,0,0,0, + 115,22,0,0,0,116,0,124,1,131,1,125,2,124,2,106, + 1,124,2,106,2,100,1,156,2,83,0,41,2,122,33,82, + 101,116,117,114,110,32,116,104,101,32,109,101,116,97,100,97, + 116,97,32,102,111,114,32,116,104,101,32,112,97,116,104,46, + 41,2,114,153,0,0,0,114,211,0,0,0,41,3,114,39, + 0,0,0,218,8,115,116,95,109,116,105,109,101,90,7,115, + 116,95,115,105,122,101,41,3,114,106,0,0,0,114,35,0, + 0,0,114,215,0,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,114,201,0,0,0,226,3,0,0,115, + 4,0,0,0,0,2,8,1,122,27,83,111,117,114,99,101, + 70,105,108,101,76,111,97,100,101,114,46,112,97,116,104,95, + 115,116,97,116,115,99,4,0,0,0,0,0,0,0,5,0, + 0,0,5,0,0,0,67,0,0,0,115,24,0,0,0,116, + 0,124,1,131,1,125,4,124,0,106,1,124,2,124,3,124, + 4,100,1,141,3,83,0,41,2,78,41,1,218,5,95,109, + 111,100,101,41,2,114,103,0,0,0,114,202,0,0,0,41, + 5,114,106,0,0,0,114,97,0,0,0,114,96,0,0,0, + 114,56,0,0,0,114,42,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,203,0,0,0,231,3, + 0,0,115,4,0,0,0,0,2,8,1,122,32,83,111,117, + 114,99,101,70,105,108,101,76,111,97,100,101,114,46,95,99, + 97,99,104,101,95,98,121,116,101,99,111,100,101,105,182,1, + 0,0,41,1,114,235,0,0,0,99,3,0,0,0,1,0, + 0,0,9,0,0,0,11,0,0,0,67,0,0,0,115,0, + 1,0,0,116,0,124,1,131,1,92,2,125,4,125,5,103, + 0,125,6,124,4,114,52,116,1,124,4,131,1,115,52,116, + 0,124,4,131,1,92,2,125,4,125,7,124,6,160,2,124, + 7,161,1,1,0,113,16,116,3,124,6,131,1,68,0,93, + 112,125,7,116,4,124,4,124,7,131,2,125,4,122,14,116, + 5,160,6,124,4,161,1,1,0,87,0,110,82,4,0,116, + 7,107,10,114,112,1,0,1,0,1,0,89,0,113,60,89, + 0,110,60,4,0,116,8,107,10,114,170,1,0,125,8,1, + 0,122,30,116,9,160,10,100,1,124,4,124,8,161,3,1, + 0,87,0,89,0,162,10,1,0,100,2,83,0,87,0,53, + 0,100,2,125,8,126,8,88,0,89,0,110,2,88,0,113, + 60,122,28,116,11,124,1,124,2,124,3,131,3,1,0,116, + 9,160,10,100,3,124,1,161,2,1,0,87,0,110,48,4, + 0,116,8,107,10,114,250,1,0,125,8,1,0,122,18,116, + 9,160,10,100,1,124,1,124,8,161,3,1,0,87,0,53, + 0,100,2,125,8,126,8,88,0,89,0,110,2,88,0,100, + 2,83,0,41,4,122,27,87,114,105,116,101,32,98,121,116, + 101,115,32,100,97,116,97,32,116,111,32,97,32,102,105,108, + 101,46,122,27,99,111,117,108,100,32,110,111,116,32,99,114, + 101,97,116,101,32,123,33,114,125,58,32,123,33,114,125,78, + 122,12,99,114,101,97,116,101,100,32,123,33,114,125,41,12, + 114,38,0,0,0,114,46,0,0,0,114,169,0,0,0,114, + 33,0,0,0,114,28,0,0,0,114,1,0,0,0,90,5, + 109,107,100,105,114,218,15,70,105,108,101,69,120,105,115,116, + 115,69,114,114,111,114,114,40,0,0,0,114,120,0,0,0, + 114,134,0,0,0,114,58,0,0,0,41,9,114,106,0,0, + 0,114,35,0,0,0,114,56,0,0,0,114,235,0,0,0, + 218,6,112,97,114,101,110,116,114,86,0,0,0,114,27,0, + 0,0,114,23,0,0,0,114,205,0,0,0,114,2,0,0, + 0,114,2,0,0,0,114,4,0,0,0,114,202,0,0,0, + 236,3,0,0,115,48,0,0,0,0,2,12,1,4,2,12, + 1,12,1,12,2,12,1,10,1,2,1,14,1,14,2,8, + 1,16,3,6,1,2,0,2,255,4,2,32,1,2,1,12, + 1,16,1,16,2,8,1,2,255,122,25,83,111,117,114,99, + 101,70,105,108,101,76,111,97,100,101,114,46,115,101,116,95, + 100,97,116,97,78,41,7,114,111,0,0,0,114,110,0,0, + 0,114,112,0,0,0,114,113,0,0,0,114,201,0,0,0, + 114,203,0,0,0,114,202,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,114,233, + 0,0,0,222,3,0,0,115,8,0,0,0,8,2,4,2, + 8,5,8,5,114,233,0,0,0,99,0,0,0,0,0,0, + 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,32, + 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, + 2,100,3,132,0,90,4,100,4,100,5,132,0,90,5,100, + 6,83,0,41,7,218,20,83,111,117,114,99,101,108,101,115, + 115,70,105,108,101,76,111,97,100,101,114,122,45,76,111,97, + 100,101,114,32,119,104,105,99,104,32,104,97,110,100,108,101, + 115,32,115,111,117,114,99,101,108,101,115,115,32,102,105,108, + 101,32,105,109,112,111,114,116,115,46,99,2,0,0,0,0, + 0,0,0,5,0,0,0,5,0,0,0,67,0,0,0,115, + 68,0,0,0,124,0,160,0,124,1,161,1,125,2,124,0, + 160,1,124,2,161,1,125,3,124,1,124,2,100,1,156,2, + 125,4,116,2,124,3,124,1,124,4,131,3,1,0,116,3, + 116,4,124,3,131,1,100,2,100,0,133,2,25,0,124,1, + 124,2,100,3,141,3,83,0,41,4,78,41,2,114,104,0, + 0,0,114,35,0,0,0,114,130,0,0,0,41,2,114,104, + 0,0,0,114,96,0,0,0,41,5,114,163,0,0,0,114, + 204,0,0,0,114,137,0,0,0,114,149,0,0,0,114,212, + 0,0,0,41,5,114,106,0,0,0,114,125,0,0,0,114, + 35,0,0,0,114,56,0,0,0,114,136,0,0,0,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,192,0, + 0,0,15,4,0,0,115,22,0,0,0,0,1,10,1,10, + 4,2,1,2,254,6,4,12,1,2,1,14,1,2,1,2, + 253,122,29,83,111,117,114,99,101,108,101,115,115,70,105,108, + 101,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, + 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, + 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, + 122,39,82,101,116,117,114,110,32,78,111,110,101,32,97,115, + 32,116,104,101,114,101,32,105,115,32,110,111,32,115,111,117, + 114,99,101,32,99,111,100,101,46,78,114,2,0,0,0,41, + 2,114,106,0,0,0,114,125,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,114,206,0,0,0,31, + 4,0,0,115,2,0,0,0,0,2,122,31,83,111,117,114, + 99,101,108,101,115,115,70,105,108,101,76,111,97,100,101,114, + 46,103,101,116,95,115,111,117,114,99,101,78,41,6,114,111, + 0,0,0,114,110,0,0,0,114,112,0,0,0,114,113,0, + 0,0,114,192,0,0,0,114,206,0,0,0,114,2,0,0, + 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 114,238,0,0,0,11,4,0,0,115,6,0,0,0,8,2, + 4,2,8,16,114,238,0,0,0,99,0,0,0,0,0,0, + 0,0,0,0,0,0,3,0,0,0,64,0,0,0,115,92, 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, 2,100,3,132,0,90,4,100,4,100,5,132,0,90,5,100, 6,100,7,132,0,90,6,100,8,100,9,132,0,90,7,100, 10,100,11,132,0,90,8,100,12,100,13,132,0,90,9,100, - 14,100,15,132,0,90,10,100,16,100,17,132,0,90,11,100, - 18,100,19,132,0,90,12,100,20,100,21,132,0,90,13,100, - 22,83,0,41,23,218,14,95,78,97,109,101,115,112,97,99, - 101,80,97,116,104,97,38,1,0,0,82,101,112,114,101,115, - 101,110,116,115,32,97,32,110,97,109,101,115,112,97,99,101, - 32,112,97,99,107,97,103,101,39,115,32,112,97,116,104,46, - 32,32,73,116,32,117,115,101,115,32,116,104,101,32,109,111, - 100,117,108,101,32,110,97,109,101,10,32,32,32,32,116,111, - 32,102,105,110,100,32,105,116,115,32,112,97,114,101,110,116, - 32,109,111,100,117,108,101,44,32,97,110,100,32,102,114,111, - 109,32,116,104,101,114,101,32,105,116,32,108,111,111,107,115, - 32,117,112,32,116,104,101,32,112,97,114,101,110,116,39,115, - 10,32,32,32,32,95,95,112,97,116,104,95,95,46,32,32, - 87,104,101,110,32,116,104,105,115,32,99,104,97,110,103,101, - 115,44,32,116,104,101,32,109,111,100,117,108,101,39,115,32, - 111,119,110,32,112,97,116,104,32,105,115,32,114,101,99,111, - 109,112,117,116,101,100,44,10,32,32,32,32,117,115,105,110, - 103,32,112,97,116,104,95,102,105,110,100,101,114,46,32,32, - 70,111,114,32,116,111,112,45,108,101,118,101,108,32,109,111, - 100,117,108,101,115,44,32,116,104,101,32,112,97,114,101,110, - 116,32,109,111,100,117,108,101,39,115,32,112,97,116,104,10, - 32,32,32,32,105,115,32,115,121,115,46,112,97,116,104,46, - 99,4,0,0,0,0,0,0,0,4,0,0,0,3,0,0, - 0,67,0,0,0,115,36,0,0,0,124,1,124,0,95,0, - 124,2,124,0,95,1,116,2,124,0,160,3,161,0,131,1, - 124,0,95,4,124,3,124,0,95,5,100,0,83,0,41,1, - 78,41,6,218,5,95,110,97,109,101,218,5,95,112,97,116, - 104,114,101,0,0,0,218,16,95,103,101,116,95,112,97,114, - 101,110,116,95,112,97,116,104,218,17,95,108,97,115,116,95, - 112,97,114,101,110,116,95,112,97,116,104,218,12,95,112,97, - 116,104,95,102,105,110,100,101,114,41,4,114,107,0,0,0, - 114,105,0,0,0,114,35,0,0,0,218,11,112,97,116,104, - 95,102,105,110,100,101,114,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,114,191,0,0,0,100,4,0,0,115, - 8,0,0,0,0,1,6,1,6,1,14,1,122,23,95,78, - 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,105, - 110,105,116,95,95,99,1,0,0,0,0,0,0,0,4,0, - 0,0,3,0,0,0,67,0,0,0,115,38,0,0,0,124, - 0,106,0,160,1,100,1,161,1,92,3,125,1,125,2,125, - 3,124,2,100,2,107,2,114,30,100,3,83,0,124,1,100, - 4,102,2,83,0,41,5,122,62,82,101,116,117,114,110,115, - 32,97,32,116,117,112,108,101,32,111,102,32,40,112,97,114, - 101,110,116,45,109,111,100,117,108,101,45,110,97,109,101,44, - 32,112,97,114,101,110,116,45,112,97,116,104,45,97,116,116, - 114,45,110,97,109,101,41,114,62,0,0,0,114,30,0,0, - 0,41,2,114,6,0,0,0,114,35,0,0,0,90,8,95, - 95,112,97,116,104,95,95,41,2,114,247,0,0,0,114,32, - 0,0,0,41,4,114,107,0,0,0,114,238,0,0,0,218, - 3,100,111,116,90,2,109,101,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,218,23,95,102,105,110,100,95,112, + 14,100,15,132,0,90,10,100,16,100,17,132,0,90,11,101, + 12,100,18,100,19,132,0,131,1,90,13,100,20,83,0,41, + 21,218,19,69,120,116,101,110,115,105,111,110,70,105,108,101, + 76,111,97,100,101,114,122,93,76,111,97,100,101,114,32,102, + 111,114,32,101,120,116,101,110,115,105,111,110,32,109,111,100, + 117,108,101,115,46,10,10,32,32,32,32,84,104,101,32,99, + 111,110,115,116,114,117,99,116,111,114,32,105,115,32,100,101, + 115,105,103,110,101,100,32,116,111,32,119,111,114,107,32,119, + 105,116,104,32,70,105,108,101,70,105,110,100,101,114,46,10, + 10,32,32,32,32,99,3,0,0,0,0,0,0,0,3,0, + 0,0,2,0,0,0,67,0,0,0,115,16,0,0,0,124, + 1,124,0,95,0,124,2,124,0,95,1,100,0,83,0,41, + 1,78,41,2,114,104,0,0,0,114,35,0,0,0,41,3, + 114,106,0,0,0,114,104,0,0,0,114,35,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,114,190, + 0,0,0,48,4,0,0,115,4,0,0,0,0,1,6,1, + 122,28,69,120,116,101,110,115,105,111,110,70,105,108,101,76, + 111,97,100,101,114,46,95,95,105,110,105,116,95,95,99,2, + 0,0,0,0,0,0,0,2,0,0,0,2,0,0,0,67, + 0,0,0,115,24,0,0,0,124,0,106,0,124,1,106,0, + 107,2,111,22,124,0,106,1,124,1,106,1,107,2,83,0, + 41,1,78,41,2,114,217,0,0,0,114,117,0,0,0,41, + 2,114,106,0,0,0,114,218,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,114,219,0,0,0,52, + 4,0,0,115,6,0,0,0,0,1,12,1,10,255,122,26, + 69,120,116,101,110,115,105,111,110,70,105,108,101,76,111,97, + 100,101,114,46,95,95,101,113,95,95,99,1,0,0,0,0, + 0,0,0,1,0,0,0,3,0,0,0,67,0,0,0,115, + 20,0,0,0,116,0,124,0,106,1,131,1,116,0,124,0, + 106,2,131,1,65,0,83,0,41,1,78,41,3,114,220,0, + 0,0,114,104,0,0,0,114,35,0,0,0,41,1,114,106, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,114,221,0,0,0,56,4,0,0,115,2,0,0,0, + 0,1,122,28,69,120,116,101,110,115,105,111,110,70,105,108, + 101,76,111,97,100,101,114,46,95,95,104,97,115,104,95,95, + 99,2,0,0,0,0,0,0,0,3,0,0,0,5,0,0, + 0,67,0,0,0,115,36,0,0,0,116,0,160,1,116,2, + 106,3,124,1,161,2,125,2,116,0,160,4,100,1,124,1, + 106,5,124,0,106,6,161,3,1,0,124,2,83,0,41,2, + 122,38,67,114,101,97,116,101,32,97,110,32,117,110,105,116, + 105,97,108,105,122,101,100,32,101,120,116,101,110,115,105,111, + 110,32,109,111,100,117,108,101,122,38,101,120,116,101,110,115, + 105,111,110,32,109,111,100,117,108,101,32,123,33,114,125,32, + 108,111,97,100,101,100,32,102,114,111,109,32,123,33,114,125, + 41,7,114,120,0,0,0,114,193,0,0,0,114,147,0,0, + 0,90,14,99,114,101,97,116,101,95,100,121,110,97,109,105, + 99,114,134,0,0,0,114,104,0,0,0,114,35,0,0,0, + 41,3,114,106,0,0,0,114,170,0,0,0,114,195,0,0, + 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 114,191,0,0,0,59,4,0,0,115,18,0,0,0,0,2, + 4,1,4,0,2,255,4,2,6,1,4,0,4,255,4,2, + 122,33,69,120,116,101,110,115,105,111,110,70,105,108,101,76, + 111,97,100,101,114,46,99,114,101,97,116,101,95,109,111,100, + 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 5,0,0,0,67,0,0,0,115,36,0,0,0,116,0,160, + 1,116,2,106,3,124,1,161,2,1,0,116,0,160,4,100, + 1,124,0,106,5,124,0,106,6,161,3,1,0,100,2,83, + 0,41,3,122,30,73,110,105,116,105,97,108,105,122,101,32, + 97,110,32,101,120,116,101,110,115,105,111,110,32,109,111,100, + 117,108,101,122,40,101,120,116,101,110,115,105,111,110,32,109, + 111,100,117,108,101,32,123,33,114,125,32,101,120,101,99,117, + 116,101,100,32,102,114,111,109,32,123,33,114,125,78,41,7, + 114,120,0,0,0,114,193,0,0,0,114,147,0,0,0,90, + 12,101,120,101,99,95,100,121,110,97,109,105,99,114,134,0, + 0,0,114,104,0,0,0,114,35,0,0,0,41,2,114,106, + 0,0,0,114,195,0,0,0,114,2,0,0,0,114,2,0, + 0,0,114,4,0,0,0,114,196,0,0,0,67,4,0,0, + 115,10,0,0,0,0,2,14,1,6,1,4,0,4,255,122, + 31,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, + 97,100,101,114,46,101,120,101,99,95,109,111,100,117,108,101, + 99,2,0,0,0,0,0,0,0,2,0,0,0,4,0,0, + 0,3,0,0,0,115,36,0,0,0,116,0,124,0,106,1, + 131,1,100,1,25,0,137,0,116,2,135,0,102,1,100,2, + 100,3,132,8,116,3,68,0,131,1,131,1,83,0,41,4, + 122,49,82,101,116,117,114,110,32,84,114,117,101,32,105,102, + 32,116,104,101,32,101,120,116,101,110,115,105,111,110,32,109, + 111,100,117,108,101,32,105,115,32,97,32,112,97,99,107,97, + 103,101,46,114,29,0,0,0,99,1,0,0,0,0,0,0, + 0,2,0,0,0,4,0,0,0,51,0,0,0,115,26,0, + 0,0,124,0,93,18,125,1,136,0,100,0,124,1,23,0, + 107,2,86,0,1,0,113,2,100,1,83,0,41,2,114,190, + 0,0,0,78,114,2,0,0,0,41,2,114,22,0,0,0, + 218,6,115,117,102,102,105,120,41,1,218,9,102,105,108,101, + 95,110,97,109,101,114,2,0,0,0,114,4,0,0,0,218, + 9,60,103,101,110,101,120,112,114,62,76,4,0,0,115,4, + 0,0,0,4,1,2,255,122,49,69,120,116,101,110,115,105, + 111,110,70,105,108,101,76,111,97,100,101,114,46,105,115,95, + 112,97,99,107,97,103,101,46,60,108,111,99,97,108,115,62, + 46,60,103,101,110,101,120,112,114,62,41,4,114,38,0,0, + 0,114,35,0,0,0,218,3,97,110,121,218,18,69,88,84, + 69,78,83,73,79,78,95,83,85,70,70,73,88,69,83,41, + 2,114,106,0,0,0,114,125,0,0,0,114,2,0,0,0, + 41,1,114,241,0,0,0,114,4,0,0,0,114,165,0,0, + 0,73,4,0,0,115,8,0,0,0,0,2,14,1,12,1, + 2,255,122,30,69,120,116,101,110,115,105,111,110,70,105,108, + 101,76,111,97,100,101,114,46,105,115,95,112,97,99,107,97, + 103,101,99,2,0,0,0,0,0,0,0,2,0,0,0,1, + 0,0,0,67,0,0,0,115,4,0,0,0,100,1,83,0, + 41,2,122,63,82,101,116,117,114,110,32,78,111,110,101,32, + 97,115,32,97,110,32,101,120,116,101,110,115,105,111,110,32, + 109,111,100,117,108,101,32,99,97,110,110,111,116,32,99,114, + 101,97,116,101,32,97,32,99,111,100,101,32,111,98,106,101, + 99,116,46,78,114,2,0,0,0,41,2,114,106,0,0,0, + 114,125,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 4,0,0,0,114,192,0,0,0,79,4,0,0,115,2,0, + 0,0,0,2,122,28,69,120,116,101,110,115,105,111,110,70, + 105,108,101,76,111,97,100,101,114,46,103,101,116,95,99,111, + 100,101,99,2,0,0,0,0,0,0,0,2,0,0,0,1, + 0,0,0,67,0,0,0,115,4,0,0,0,100,1,83,0, + 41,2,122,53,82,101,116,117,114,110,32,78,111,110,101,32, + 97,115,32,101,120,116,101,110,115,105,111,110,32,109,111,100, + 117,108,101,115,32,104,97,118,101,32,110,111,32,115,111,117, + 114,99,101,32,99,111,100,101,46,78,114,2,0,0,0,41, + 2,114,106,0,0,0,114,125,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,114,206,0,0,0,83, + 4,0,0,115,2,0,0,0,0,2,122,30,69,120,116,101, + 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, + 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, + 0,0,0,2,0,0,0,1,0,0,0,67,0,0,0,115, + 6,0,0,0,124,0,106,0,83,0,41,1,122,58,82,101, + 116,117,114,110,32,116,104,101,32,112,97,116,104,32,116,111, + 32,116,104,101,32,115,111,117,114,99,101,32,102,105,108,101, + 32,97,115,32,102,111,117,110,100,32,98,121,32,116,104,101, + 32,102,105,110,100,101,114,46,41,1,114,35,0,0,0,41, + 2,114,106,0,0,0,114,125,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,114,163,0,0,0,87, + 4,0,0,115,2,0,0,0,0,3,122,32,69,120,116,101, + 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, + 103,101,116,95,102,105,108,101,110,97,109,101,78,41,14,114, + 111,0,0,0,114,110,0,0,0,114,112,0,0,0,114,113, + 0,0,0,114,190,0,0,0,114,219,0,0,0,114,221,0, + 0,0,114,191,0,0,0,114,196,0,0,0,114,165,0,0, + 0,114,192,0,0,0,114,206,0,0,0,114,122,0,0,0, + 114,163,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,239,0,0,0,40,4, + 0,0,115,20,0,0,0,8,6,4,2,8,4,8,4,8, + 3,8,8,8,6,8,6,8,4,8,4,114,239,0,0,0, + 99,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0, + 0,64,0,0,0,115,96,0,0,0,101,0,90,1,100,0, + 90,2,100,1,90,3,100,2,100,3,132,0,90,4,100,4, + 100,5,132,0,90,5,100,6,100,7,132,0,90,6,100,8, + 100,9,132,0,90,7,100,10,100,11,132,0,90,8,100,12, + 100,13,132,0,90,9,100,14,100,15,132,0,90,10,100,16, + 100,17,132,0,90,11,100,18,100,19,132,0,90,12,100,20, + 100,21,132,0,90,13,100,22,83,0,41,23,218,14,95,78, + 97,109,101,115,112,97,99,101,80,97,116,104,97,38,1,0, + 0,82,101,112,114,101,115,101,110,116,115,32,97,32,110,97, + 109,101,115,112,97,99,101,32,112,97,99,107,97,103,101,39, + 115,32,112,97,116,104,46,32,32,73,116,32,117,115,101,115, + 32,116,104,101,32,109,111,100,117,108,101,32,110,97,109,101, + 10,32,32,32,32,116,111,32,102,105,110,100,32,105,116,115, + 32,112,97,114,101,110,116,32,109,111,100,117,108,101,44,32, + 97,110,100,32,102,114,111,109,32,116,104,101,114,101,32,105, + 116,32,108,111,111,107,115,32,117,112,32,116,104,101,32,112, + 97,114,101,110,116,39,115,10,32,32,32,32,95,95,112,97, + 116,104,95,95,46,32,32,87,104,101,110,32,116,104,105,115, + 32,99,104,97,110,103,101,115,44,32,116,104,101,32,109,111, + 100,117,108,101,39,115,32,111,119,110,32,112,97,116,104,32, + 105,115,32,114,101,99,111,109,112,117,116,101,100,44,10,32, + 32,32,32,117,115,105,110,103,32,112,97,116,104,95,102,105, + 110,100,101,114,46,32,32,70,111,114,32,116,111,112,45,108, + 101,118,101,108,32,109,111,100,117,108,101,115,44,32,116,104, + 101,32,112,97,114,101,110,116,32,109,111,100,117,108,101,39, + 115,32,112,97,116,104,10,32,32,32,32,105,115,32,115,121, + 115,46,112,97,116,104,46,99,4,0,0,0,0,0,0,0, + 4,0,0,0,3,0,0,0,67,0,0,0,115,36,0,0, + 0,124,1,124,0,95,0,124,2,124,0,95,1,116,2,124, + 0,160,3,161,0,131,1,124,0,95,4,124,3,124,0,95, + 5,100,0,83,0,41,1,78,41,6,218,5,95,110,97,109, + 101,218,5,95,112,97,116,104,114,100,0,0,0,218,16,95, + 103,101,116,95,112,97,114,101,110,116,95,112,97,116,104,218, + 17,95,108,97,115,116,95,112,97,114,101,110,116,95,112,97, + 116,104,218,12,95,112,97,116,104,95,102,105,110,100,101,114, + 41,4,114,106,0,0,0,114,104,0,0,0,114,35,0,0, + 0,218,11,112,97,116,104,95,102,105,110,100,101,114,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,190,0, + 0,0,100,4,0,0,115,8,0,0,0,0,1,6,1,6, + 1,14,1,122,23,95,78,97,109,101,115,112,97,99,101,80, + 97,116,104,46,95,95,105,110,105,116,95,95,99,1,0,0, + 0,0,0,0,0,4,0,0,0,3,0,0,0,67,0,0, + 0,115,38,0,0,0,124,0,106,0,160,1,100,1,161,1, + 92,3,125,1,125,2,125,3,124,2,100,2,107,2,114,30, + 100,3,83,0,124,1,100,4,102,2,83,0,41,5,122,62, + 82,101,116,117,114,110,115,32,97,32,116,117,112,108,101,32, + 111,102,32,40,112,97,114,101,110,116,45,109,111,100,117,108, + 101,45,110,97,109,101,44,32,112,97,114,101,110,116,45,112, + 97,116,104,45,97,116,116,114,45,110,97,109,101,41,114,61, + 0,0,0,114,30,0,0,0,41,2,114,6,0,0,0,114, + 35,0,0,0,90,8,95,95,112,97,116,104,95,95,41,2, + 114,246,0,0,0,114,32,0,0,0,41,4,114,106,0,0, + 0,114,237,0,0,0,218,3,100,111,116,90,2,109,101,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,23, + 95,102,105,110,100,95,112,97,114,101,110,116,95,112,97,116, + 104,95,110,97,109,101,115,106,4,0,0,115,8,0,0,0, + 0,2,18,1,8,2,4,3,122,38,95,78,97,109,101,115, + 112,97,99,101,80,97,116,104,46,95,102,105,110,100,95,112, 97,114,101,110,116,95,112,97,116,104,95,110,97,109,101,115, - 106,4,0,0,115,8,0,0,0,0,2,18,1,8,2,4, - 3,122,38,95,78,97,109,101,115,112,97,99,101,80,97,116, - 104,46,95,102,105,110,100,95,112,97,114,101,110,116,95,112, - 97,116,104,95,110,97,109,101,115,99,1,0,0,0,0,0, - 0,0,3,0,0,0,3,0,0,0,67,0,0,0,115,28, - 0,0,0,124,0,160,0,161,0,92,2,125,1,125,2,116, - 1,116,2,106,3,124,1,25,0,124,2,131,2,83,0,41, - 1,78,41,4,114,254,0,0,0,114,117,0,0,0,114,6, - 0,0,0,218,7,109,111,100,117,108,101,115,41,3,114,107, - 0,0,0,90,18,112,97,114,101,110,116,95,109,111,100,117, - 108,101,95,110,97,109,101,90,14,112,97,116,104,95,97,116, - 116,114,95,110,97,109,101,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,114,249,0,0,0,116,4,0,0,115, - 4,0,0,0,0,1,12,1,122,31,95,78,97,109,101,115, - 112,97,99,101,80,97,116,104,46,95,103,101,116,95,112,97, - 114,101,110,116,95,112,97,116,104,99,1,0,0,0,0,0, - 0,0,3,0,0,0,4,0,0,0,67,0,0,0,115,80, - 0,0,0,116,0,124,0,160,1,161,0,131,1,125,1,124, - 1,124,0,106,2,107,3,114,74,124,0,160,3,124,0,106, - 4,124,1,161,2,125,2,124,2,100,0,107,9,114,68,124, - 2,106,5,100,0,107,8,114,68,124,2,106,6,114,68,124, - 2,106,6,124,0,95,7,124,1,124,0,95,2,124,0,106, - 7,83,0,41,1,78,41,8,114,101,0,0,0,114,249,0, - 0,0,114,250,0,0,0,114,251,0,0,0,114,247,0,0, - 0,114,127,0,0,0,114,163,0,0,0,114,248,0,0,0, - 41,3,114,107,0,0,0,90,11,112,97,114,101,110,116,95, - 112,97,116,104,114,171,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,218,12,95,114,101,99,97,108, - 99,117,108,97,116,101,120,4,0,0,115,16,0,0,0,0, - 2,12,1,10,1,14,3,18,1,6,1,8,1,6,1,122, - 27,95,78,97,109,101,115,112,97,99,101,80,97,116,104,46, - 95,114,101,99,97,108,99,117,108,97,116,101,99,1,0,0, - 0,0,0,0,0,1,0,0,0,3,0,0,0,67,0,0, - 0,115,12,0,0,0,116,0,124,0,160,1,161,0,131,1, - 83,0,41,1,78,41,2,114,231,0,0,0,114,0,1,0, - 0,41,1,114,107,0,0,0,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,218,8,95,95,105,116,101,114,95, - 95,133,4,0,0,115,2,0,0,0,0,1,122,23,95,78, - 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,105, - 116,101,114,95,95,99,3,0,0,0,0,0,0,0,3,0, - 0,0,3,0,0,0,67,0,0,0,115,14,0,0,0,124, - 2,124,0,106,0,124,1,60,0,100,0,83,0,41,1,78, - 41,1,114,248,0,0,0,41,3,114,107,0,0,0,218,5, - 105,110,100,101,120,114,35,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,218,11,95,95,115,101,116, - 105,116,101,109,95,95,136,4,0,0,115,2,0,0,0,0, - 1,122,26,95,78,97,109,101,115,112,97,99,101,80,97,116, - 104,46,95,95,115,101,116,105,116,101,109,95,95,99,1,0, - 0,0,0,0,0,0,1,0,0,0,3,0,0,0,67,0, - 0,0,115,12,0,0,0,116,0,124,0,160,1,161,0,131, - 1,83,0,41,1,78,41,2,114,31,0,0,0,114,0,1, - 0,0,41,1,114,107,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,218,7,95,95,108,101,110,95, - 95,139,4,0,0,115,2,0,0,0,0,1,122,22,95,78, - 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,108, - 101,110,95,95,99,1,0,0,0,0,0,0,0,1,0,0, - 0,3,0,0,0,67,0,0,0,115,12,0,0,0,100,1, - 160,0,124,0,106,1,161,1,83,0,41,2,78,122,20,95, - 78,97,109,101,115,112,97,99,101,80,97,116,104,40,123,33, - 114,125,41,41,2,114,51,0,0,0,114,248,0,0,0,41, - 1,114,107,0,0,0,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,218,8,95,95,114,101,112,114,95,95,142, - 4,0,0,115,2,0,0,0,0,1,122,23,95,78,97,109, - 101,115,112,97,99,101,80,97,116,104,46,95,95,114,101,112, - 114,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, - 3,0,0,0,67,0,0,0,115,12,0,0,0,124,1,124, - 0,160,0,161,0,107,6,83,0,41,1,78,41,1,114,0, - 1,0,0,41,2,114,107,0,0,0,218,4,105,116,101,109, + 99,1,0,0,0,0,0,0,0,3,0,0,0,3,0,0, + 0,67,0,0,0,115,28,0,0,0,124,0,160,0,161,0, + 92,2,125,1,125,2,116,1,116,2,106,3,124,1,25,0, + 124,2,131,2,83,0,41,1,78,41,4,114,253,0,0,0, + 114,116,0,0,0,114,6,0,0,0,218,7,109,111,100,117, + 108,101,115,41,3,114,106,0,0,0,90,18,112,97,114,101, + 110,116,95,109,111,100,117,108,101,95,110,97,109,101,90,14, + 112,97,116,104,95,97,116,116,114,95,110,97,109,101,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,248,0, + 0,0,116,4,0,0,115,4,0,0,0,0,1,12,1,122, + 31,95,78,97,109,101,115,112,97,99,101,80,97,116,104,46, + 95,103,101,116,95,112,97,114,101,110,116,95,112,97,116,104, + 99,1,0,0,0,0,0,0,0,3,0,0,0,4,0,0, + 0,67,0,0,0,115,80,0,0,0,116,0,124,0,160,1, + 161,0,131,1,125,1,124,1,124,0,106,2,107,3,114,74, + 124,0,160,3,124,0,106,4,124,1,161,2,125,2,124,2, + 100,0,107,9,114,68,124,2,106,5,100,0,107,8,114,68, + 124,2,106,6,114,68,124,2,106,6,124,0,95,7,124,1, + 124,0,95,2,124,0,106,7,83,0,41,1,78,41,8,114, + 100,0,0,0,114,248,0,0,0,114,249,0,0,0,114,250, + 0,0,0,114,246,0,0,0,114,126,0,0,0,114,162,0, + 0,0,114,247,0,0,0,41,3,114,106,0,0,0,90,11, + 112,97,114,101,110,116,95,112,97,116,104,114,170,0,0,0, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 12,95,95,99,111,110,116,97,105,110,115,95,95,145,4,0, - 0,115,2,0,0,0,0,1,122,27,95,78,97,109,101,115, - 112,97,99,101,80,97,116,104,46,95,95,99,111,110,116,97, - 105,110,115,95,95,99,2,0,0,0,0,0,0,0,2,0, - 0,0,3,0,0,0,67,0,0,0,115,16,0,0,0,124, - 0,106,0,160,1,124,1,161,1,1,0,100,0,83,0,41, - 1,78,41,2,114,248,0,0,0,114,170,0,0,0,41,2, - 114,107,0,0,0,114,6,1,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,170,0,0,0,148,4, - 0,0,115,2,0,0,0,0,1,122,21,95,78,97,109,101, - 115,112,97,99,101,80,97,116,104,46,97,112,112,101,110,100, - 78,41,14,114,112,0,0,0,114,111,0,0,0,114,113,0, - 0,0,114,114,0,0,0,114,191,0,0,0,114,254,0,0, - 0,114,249,0,0,0,114,0,1,0,0,114,1,1,0,0, - 114,3,1,0,0,114,4,1,0,0,114,5,1,0,0,114, - 7,1,0,0,114,170,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,114,246,0, - 0,0,93,4,0,0,115,22,0,0,0,8,5,4,2,8, - 6,8,10,8,4,8,13,8,3,8,3,8,3,8,3,8, - 3,114,246,0,0,0,99,0,0,0,0,0,0,0,0,0, - 0,0,0,3,0,0,0,64,0,0,0,115,80,0,0,0, - 101,0,90,1,100,0,90,2,100,1,100,2,132,0,90,3, - 101,4,100,3,100,4,132,0,131,1,90,5,100,5,100,6, - 132,0,90,6,100,7,100,8,132,0,90,7,100,9,100,10, - 132,0,90,8,100,11,100,12,132,0,90,9,100,13,100,14, - 132,0,90,10,100,15,100,16,132,0,90,11,100,17,83,0, - 41,18,218,16,95,78,97,109,101,115,112,97,99,101,76,111, - 97,100,101,114,99,4,0,0,0,0,0,0,0,4,0,0, - 0,4,0,0,0,67,0,0,0,115,18,0,0,0,116,0, - 124,1,124,2,124,3,131,3,124,0,95,1,100,0,83,0, - 41,1,78,41,2,114,246,0,0,0,114,248,0,0,0,41, - 4,114,107,0,0,0,114,105,0,0,0,114,35,0,0,0, - 114,252,0,0,0,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,114,191,0,0,0,154,4,0,0,115,2,0, - 0,0,0,1,122,25,95,78,97,109,101,115,112,97,99,101, - 76,111,97,100,101,114,46,95,95,105,110,105,116,95,95,99, - 2,0,0,0,0,0,0,0,2,0,0,0,3,0,0,0, - 67,0,0,0,115,12,0,0,0,100,1,160,0,124,1,106, - 1,161,1,83,0,41,2,122,115,82,101,116,117,114,110,32, - 114,101,112,114,32,102,111,114,32,116,104,101,32,109,111,100, - 117,108,101,46,10,10,32,32,32,32,32,32,32,32,84,104, - 101,32,109,101,116,104,111,100,32,105,115,32,100,101,112,114, - 101,99,97,116,101,100,46,32,32,84,104,101,32,105,109,112, - 111,114,116,32,109,97,99,104,105,110,101,114,121,32,100,111, - 101,115,32,116,104,101,32,106,111,98,32,105,116,115,101,108, - 102,46,10,10,32,32,32,32,32,32,32,32,122,25,60,109, - 111,100,117,108,101,32,123,33,114,125,32,40,110,97,109,101, - 115,112,97,99,101,41,62,41,2,114,51,0,0,0,114,112, - 0,0,0,41,2,114,177,0,0,0,114,196,0,0,0,114, - 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,11, - 109,111,100,117,108,101,95,114,101,112,114,157,4,0,0,115, - 2,0,0,0,0,7,122,28,95,78,97,109,101,115,112,97, - 99,101,76,111,97,100,101,114,46,109,111,100,117,108,101,95, - 114,101,112,114,99,2,0,0,0,0,0,0,0,2,0,0, - 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, - 83,0,41,2,78,84,114,2,0,0,0,41,2,114,107,0, - 0,0,114,126,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,114,166,0,0,0,166,4,0,0,115, - 2,0,0,0,0,1,122,27,95,78,97,109,101,115,112,97, - 99,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, - 97,103,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 12,95,114,101,99,97,108,99,117,108,97,116,101,120,4,0, + 0,115,16,0,0,0,0,2,12,1,10,1,14,3,18,1, + 6,1,8,1,6,1,122,27,95,78,97,109,101,115,112,97, + 99,101,80,97,116,104,46,95,114,101,99,97,108,99,117,108, + 97,116,101,99,1,0,0,0,0,0,0,0,1,0,0,0, + 3,0,0,0,67,0,0,0,115,12,0,0,0,116,0,124, + 0,160,1,161,0,131,1,83,0,41,1,78,41,2,114,230, + 0,0,0,114,255,0,0,0,41,1,114,106,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,8, + 95,95,105,116,101,114,95,95,133,4,0,0,115,2,0,0, + 0,0,1,122,23,95,78,97,109,101,115,112,97,99,101,80, + 97,116,104,46,95,95,105,116,101,114,95,95,99,3,0,0, + 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, + 0,115,14,0,0,0,124,2,124,0,106,0,124,1,60,0, + 100,0,83,0,41,1,78,41,1,114,247,0,0,0,41,3, + 114,106,0,0,0,218,5,105,110,100,101,120,114,35,0,0, + 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 218,11,95,95,115,101,116,105,116,101,109,95,95,136,4,0, + 0,115,2,0,0,0,0,1,122,26,95,78,97,109,101,115, + 112,97,99,101,80,97,116,104,46,95,95,115,101,116,105,116, + 101,109,95,95,99,1,0,0,0,0,0,0,0,1,0,0, + 0,3,0,0,0,67,0,0,0,115,12,0,0,0,116,0, + 124,0,160,1,161,0,131,1,83,0,41,1,78,41,2,114, + 31,0,0,0,114,255,0,0,0,41,1,114,106,0,0,0, + 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, + 7,95,95,108,101,110,95,95,139,4,0,0,115,2,0,0, + 0,0,1,122,22,95,78,97,109,101,115,112,97,99,101,80, + 97,116,104,46,95,95,108,101,110,95,95,99,1,0,0,0, + 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, + 115,12,0,0,0,100,1,160,0,124,0,106,1,161,1,83, + 0,41,2,78,122,20,95,78,97,109,101,115,112,97,99,101, + 80,97,116,104,40,123,33,114,125,41,41,2,114,50,0,0, + 0,114,247,0,0,0,41,1,114,106,0,0,0,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,218,8,95,95, + 114,101,112,114,95,95,142,4,0,0,115,2,0,0,0,0, + 1,122,23,95,78,97,109,101,115,112,97,99,101,80,97,116, + 104,46,95,95,114,101,112,114,95,95,99,2,0,0,0,0, + 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, + 12,0,0,0,124,1,124,0,160,0,161,0,107,6,83,0, + 41,1,78,41,1,114,255,0,0,0,41,2,114,106,0,0, + 0,218,4,105,116,101,109,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,12,95,95,99,111,110,116,97,105, + 110,115,95,95,145,4,0,0,115,2,0,0,0,0,1,122, + 27,95,78,97,109,101,115,112,97,99,101,80,97,116,104,46, + 95,95,99,111,110,116,97,105,110,115,95,95,99,2,0,0, + 0,0,0,0,0,2,0,0,0,3,0,0,0,67,0,0, + 0,115,16,0,0,0,124,0,106,0,160,1,124,1,161,1, + 1,0,100,0,83,0,41,1,78,41,2,114,247,0,0,0, + 114,169,0,0,0,41,2,114,106,0,0,0,114,5,1,0, + 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 114,169,0,0,0,148,4,0,0,115,2,0,0,0,0,1, + 122,21,95,78,97,109,101,115,112,97,99,101,80,97,116,104, + 46,97,112,112,101,110,100,78,41,14,114,111,0,0,0,114, + 110,0,0,0,114,112,0,0,0,114,113,0,0,0,114,190, + 0,0,0,114,253,0,0,0,114,248,0,0,0,114,255,0, + 0,0,114,0,1,0,0,114,2,1,0,0,114,3,1,0, + 0,114,4,1,0,0,114,6,1,0,0,114,169,0,0,0, + 114,2,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 4,0,0,0,114,245,0,0,0,93,4,0,0,115,22,0, + 0,0,8,5,4,2,8,6,8,10,8,4,8,13,8,3, + 8,3,8,3,8,3,8,3,114,245,0,0,0,99,0,0, + 0,0,0,0,0,0,0,0,0,0,3,0,0,0,64,0, + 0,0,115,80,0,0,0,101,0,90,1,100,0,90,2,100, + 1,100,2,132,0,90,3,101,4,100,3,100,4,132,0,131, + 1,90,5,100,5,100,6,132,0,90,6,100,7,100,8,132, + 0,90,7,100,9,100,10,132,0,90,8,100,11,100,12,132, + 0,90,9,100,13,100,14,132,0,90,10,100,15,100,16,132, + 0,90,11,100,17,83,0,41,18,218,16,95,78,97,109,101, + 115,112,97,99,101,76,111,97,100,101,114,99,4,0,0,0, + 0,0,0,0,4,0,0,0,4,0,0,0,67,0,0,0, + 115,18,0,0,0,116,0,124,1,124,2,124,3,131,3,124, + 0,95,1,100,0,83,0,41,1,78,41,2,114,245,0,0, + 0,114,247,0,0,0,41,4,114,106,0,0,0,114,104,0, + 0,0,114,35,0,0,0,114,251,0,0,0,114,2,0,0, + 0,114,2,0,0,0,114,4,0,0,0,114,190,0,0,0, + 154,4,0,0,115,2,0,0,0,0,1,122,25,95,78,97, + 109,101,115,112,97,99,101,76,111,97,100,101,114,46,95,95, + 105,110,105,116,95,95,99,2,0,0,0,0,0,0,0,2, + 0,0,0,3,0,0,0,67,0,0,0,115,12,0,0,0, + 100,1,160,0,124,1,106,1,161,1,83,0,41,2,122,115, + 82,101,116,117,114,110,32,114,101,112,114,32,102,111,114,32, + 116,104,101,32,109,111,100,117,108,101,46,10,10,32,32,32, + 32,32,32,32,32,84,104,101,32,109,101,116,104,111,100,32, + 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, + 84,104,101,32,105,109,112,111,114,116,32,109,97,99,104,105, + 110,101,114,121,32,100,111,101,115,32,116,104,101,32,106,111, + 98,32,105,116,115,101,108,102,46,10,10,32,32,32,32,32, + 32,32,32,122,25,60,109,111,100,117,108,101,32,123,33,114, + 125,32,40,110,97,109,101,115,112,97,99,101,41,62,41,2, + 114,50,0,0,0,114,111,0,0,0,41,2,114,176,0,0, + 0,114,195,0,0,0,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,218,11,109,111,100,117,108,101,95,114,101, + 112,114,157,4,0,0,115,2,0,0,0,0,7,122,28,95, + 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, + 109,111,100,117,108,101,95,114,101,112,114,99,2,0,0,0, + 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, + 115,4,0,0,0,100,1,83,0,41,2,78,84,114,2,0, + 0,0,41,2,114,106,0,0,0,114,125,0,0,0,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,165,0, + 0,0,166,4,0,0,115,2,0,0,0,0,1,122,27,95, + 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, + 105,115,95,112,97,99,107,97,103,101,99,2,0,0,0,0, + 0,0,0,2,0,0,0,1,0,0,0,67,0,0,0,115, + 4,0,0,0,100,1,83,0,41,2,78,114,30,0,0,0, + 114,2,0,0,0,41,2,114,106,0,0,0,114,125,0,0, + 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 114,206,0,0,0,169,4,0,0,115,2,0,0,0,0,1, + 122,27,95,78,97,109,101,115,112,97,99,101,76,111,97,100, + 101,114,46,103,101,116,95,115,111,117,114,99,101,99,2,0, + 0,0,0,0,0,0,2,0,0,0,6,0,0,0,67,0, + 0,0,115,16,0,0,0,116,0,100,1,100,2,100,3,100, + 4,100,5,141,4,83,0,41,6,78,114,30,0,0,0,122, + 8,60,115,116,114,105,110,103,62,114,194,0,0,0,84,41, + 1,114,208,0,0,0,41,1,114,209,0,0,0,41,2,114, + 106,0,0,0,114,125,0,0,0,114,2,0,0,0,114,2, + 0,0,0,114,4,0,0,0,114,192,0,0,0,172,4,0, + 0,115,2,0,0,0,0,1,122,25,95,78,97,109,101,115, + 112,97,99,101,76,111,97,100,101,114,46,103,101,116,95,99, + 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, - 0,41,2,78,114,30,0,0,0,114,2,0,0,0,41,2, - 114,107,0,0,0,114,126,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,207,0,0,0,169,4, - 0,0,115,2,0,0,0,0,1,122,27,95,78,97,109,101, - 115,112,97,99,101,76,111,97,100,101,114,46,103,101,116,95, - 115,111,117,114,99,101,99,2,0,0,0,0,0,0,0,2, - 0,0,0,6,0,0,0,67,0,0,0,115,16,0,0,0, - 116,0,100,1,100,2,100,3,100,4,100,5,141,4,83,0, - 41,6,78,114,30,0,0,0,122,8,60,115,116,114,105,110, - 103,62,114,195,0,0,0,84,41,1,114,209,0,0,0,41, - 1,114,210,0,0,0,41,2,114,107,0,0,0,114,126,0, + 0,41,2,122,42,85,115,101,32,100,101,102,97,117,108,116, + 32,115,101,109,97,110,116,105,99,115,32,102,111,114,32,109, + 111,100,117,108,101,32,99,114,101,97,116,105,111,110,46,78, + 114,2,0,0,0,41,2,114,106,0,0,0,114,170,0,0, + 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 114,191,0,0,0,175,4,0,0,115,2,0,0,0,0,1, + 122,30,95,78,97,109,101,115,112,97,99,101,76,111,97,100, + 101,114,46,99,114,101,97,116,101,95,109,111,100,117,108,101, + 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, + 0,67,0,0,0,115,4,0,0,0,100,0,83,0,41,1, + 78,114,2,0,0,0,41,2,114,106,0,0,0,114,195,0, 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, - 0,114,193,0,0,0,172,4,0,0,115,2,0,0,0,0, - 1,122,25,95,78,97,109,101,115,112,97,99,101,76,111,97, - 100,101,114,46,103,101,116,95,99,111,100,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, - 0,115,4,0,0,0,100,1,83,0,41,2,122,42,85,115, - 101,32,100,101,102,97,117,108,116,32,115,101,109,97,110,116, - 105,99,115,32,102,111,114,32,109,111,100,117,108,101,32,99, - 114,101,97,116,105,111,110,46,78,114,2,0,0,0,41,2, - 114,107,0,0,0,114,171,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,192,0,0,0,175,4, - 0,0,115,2,0,0,0,0,1,122,30,95,78,97,109,101, - 115,112,97,99,101,76,111,97,100,101,114,46,99,114,101,97, - 116,101,95,109,111,100,117,108,101,99,2,0,0,0,0,0, - 0,0,2,0,0,0,1,0,0,0,67,0,0,0,115,4, - 0,0,0,100,0,83,0,41,1,78,114,2,0,0,0,41, - 2,114,107,0,0,0,114,196,0,0,0,114,2,0,0,0, - 114,2,0,0,0,114,4,0,0,0,114,197,0,0,0,178, - 4,0,0,115,2,0,0,0,0,1,122,28,95,78,97,109, - 101,115,112,97,99,101,76,111,97,100,101,114,46,101,120,101, - 99,95,109,111,100,117,108,101,99,2,0,0,0,0,0,0, - 0,2,0,0,0,4,0,0,0,67,0,0,0,115,26,0, - 0,0,116,0,160,1,100,1,124,0,106,2,161,2,1,0, - 116,0,160,3,124,0,124,1,161,2,83,0,41,2,122,98, - 76,111,97,100,32,97,32,110,97,109,101,115,112,97,99,101, - 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, + 0,114,196,0,0,0,178,4,0,0,115,2,0,0,0,0, + 1,122,28,95,78,97,109,101,115,112,97,99,101,76,111,97, + 100,101,114,46,101,120,101,99,95,109,111,100,117,108,101,99, + 2,0,0,0,0,0,0,0,2,0,0,0,4,0,0,0, + 67,0,0,0,115,26,0,0,0,116,0,160,1,100,1,124, + 0,106,2,161,2,1,0,116,0,160,3,124,0,124,1,161, + 2,83,0,41,2,122,98,76,111,97,100,32,97,32,110,97, + 109,101,115,112,97,99,101,32,109,111,100,117,108,101,46,10, + 10,32,32,32,32,32,32,32,32,84,104,105,115,32,109,101, + 116,104,111,100,32,105,115,32,100,101,112,114,101,99,97,116, + 101,100,46,32,32,85,115,101,32,101,120,101,99,95,109,111, + 100,117,108,101,40,41,32,105,110,115,116,101,97,100,46,10, + 10,32,32,32,32,32,32,32,32,122,38,110,97,109,101,115, + 112,97,99,101,32,109,111,100,117,108,101,32,108,111,97,100, + 101,100,32,119,105,116,104,32,112,97,116,104,32,123,33,114, + 125,41,4,114,120,0,0,0,114,134,0,0,0,114,247,0, + 0,0,114,197,0,0,0,41,2,114,106,0,0,0,114,125, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,114,198,0,0,0,181,4,0,0,115,8,0,0,0, + 0,7,6,1,4,255,4,2,122,28,95,78,97,109,101,115, + 112,97,99,101,76,111,97,100,101,114,46,108,111,97,100,95, + 109,111,100,117,108,101,78,41,12,114,111,0,0,0,114,110, + 0,0,0,114,112,0,0,0,114,190,0,0,0,114,188,0, + 0,0,114,8,1,0,0,114,165,0,0,0,114,206,0,0, + 0,114,192,0,0,0,114,191,0,0,0,114,196,0,0,0, + 114,198,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,7,1,0,0,153,4, + 0,0,115,16,0,0,0,8,1,8,3,12,9,8,3,8, + 3,8,3,8,3,8,3,114,7,1,0,0,99,0,0,0, + 0,0,0,0,0,0,0,0,0,4,0,0,0,64,0,0, + 0,115,114,0,0,0,101,0,90,1,100,0,90,2,100,1, + 90,3,101,4,100,2,100,3,132,0,131,1,90,5,101,4, + 100,4,100,5,132,0,131,1,90,6,101,4,100,6,100,7, + 132,0,131,1,90,7,101,4,100,8,100,9,132,0,131,1, + 90,8,101,4,100,10,102,1,100,11,100,12,132,1,131,1, + 90,9,101,4,100,10,100,10,102,2,100,13,100,14,132,1, + 131,1,90,10,101,4,100,10,102,1,100,15,100,16,132,1, + 131,1,90,11,100,10,83,0,41,17,218,10,80,97,116,104, + 70,105,110,100,101,114,122,62,77,101,116,97,32,112,97,116, + 104,32,102,105,110,100,101,114,32,102,111,114,32,115,121,115, + 46,112,97,116,104,32,97,110,100,32,112,97,99,107,97,103, + 101,32,95,95,112,97,116,104,95,95,32,97,116,116,114,105, + 98,117,116,101,115,46,99,1,0,0,0,0,0,0,0,3, + 0,0,0,4,0,0,0,67,0,0,0,115,64,0,0,0, + 116,0,116,1,106,2,160,3,161,0,131,1,68,0,93,44, + 92,2,125,1,125,2,124,2,100,1,107,8,114,40,116,1, + 106,2,124,1,61,0,113,14,116,4,124,2,100,2,131,2, + 114,14,124,2,160,5,161,0,1,0,113,14,100,1,83,0, + 41,3,122,125,67,97,108,108,32,116,104,101,32,105,110,118, + 97,108,105,100,97,116,101,95,99,97,99,104,101,115,40,41, + 32,109,101,116,104,111,100,32,111,110,32,97,108,108,32,112, + 97,116,104,32,101,110,116,114,121,32,102,105,110,100,101,114, + 115,10,32,32,32,32,32,32,32,32,115,116,111,114,101,100, + 32,105,110,32,115,121,115,46,112,97,116,104,95,105,109,112, + 111,114,116,101,114,95,99,97,99,104,101,115,32,40,119,104, + 101,114,101,32,105,109,112,108,101,109,101,110,116,101,100,41, + 46,78,218,17,105,110,118,97,108,105,100,97,116,101,95,99, + 97,99,104,101,115,41,6,218,4,108,105,115,116,114,6,0, + 0,0,218,19,112,97,116,104,95,105,109,112,111,114,116,101, + 114,95,99,97,99,104,101,218,5,105,116,101,109,115,114,114, + 0,0,0,114,10,1,0,0,41,3,114,176,0,0,0,114, + 104,0,0,0,218,6,102,105,110,100,101,114,114,2,0,0, + 0,114,2,0,0,0,114,4,0,0,0,114,10,1,0,0, + 199,4,0,0,115,10,0,0,0,0,4,22,1,8,1,10, + 1,10,1,122,28,80,97,116,104,70,105,110,100,101,114,46, + 105,110,118,97,108,105,100,97,116,101,95,99,97,99,104,101, + 115,99,2,0,0,0,0,0,0,0,3,0,0,0,9,0, + 0,0,67,0,0,0,115,84,0,0,0,116,0,106,1,100, + 1,107,9,114,28,116,0,106,1,115,28,116,2,160,3,100, + 2,116,4,161,2,1,0,116,0,106,1,68,0,93,44,125, + 2,122,14,124,2,124,1,131,1,87,0,2,0,1,0,83, + 0,4,0,116,5,107,10,114,76,1,0,1,0,1,0,89, + 0,113,34,89,0,113,34,88,0,113,34,100,1,83,0,41, + 3,122,46,83,101,97,114,99,104,32,115,121,115,46,112,97, + 116,104,95,104,111,111,107,115,32,102,111,114,32,97,32,102, + 105,110,100,101,114,32,102,111,114,32,39,112,97,116,104,39, + 46,78,122,23,115,121,115,46,112,97,116,104,95,104,111,111, + 107,115,32,105,115,32,101,109,112,116,121,41,6,114,6,0, + 0,0,218,10,112,97,116,104,95,104,111,111,107,115,114,64, + 0,0,0,114,65,0,0,0,114,124,0,0,0,114,105,0, + 0,0,41,3,114,176,0,0,0,114,35,0,0,0,90,4, + 104,111,111,107,114,2,0,0,0,114,2,0,0,0,114,4, + 0,0,0,218,11,95,112,97,116,104,95,104,111,111,107,115, + 209,4,0,0,115,16,0,0,0,0,3,16,1,12,1,10, + 1,2,1,14,1,14,1,12,2,122,22,80,97,116,104,70, + 105,110,100,101,114,46,95,112,97,116,104,95,104,111,111,107, + 115,99,2,0,0,0,0,0,0,0,3,0,0,0,8,0, + 0,0,67,0,0,0,115,104,0,0,0,124,1,100,1,107, + 2,114,44,122,12,116,0,160,1,161,0,125,1,87,0,110, + 22,4,0,116,2,107,10,114,42,1,0,1,0,1,0,89, + 0,100,2,83,0,88,0,122,14,116,3,106,4,124,1,25, + 0,125,2,87,0,110,40,4,0,116,5,107,10,114,98,1, + 0,1,0,1,0,124,0,160,6,124,1,161,1,125,2,124, + 2,116,3,106,4,124,1,60,0,89,0,110,2,88,0,124, + 2,83,0,41,3,122,210,71,101,116,32,116,104,101,32,102, + 105,110,100,101,114,32,102,111,114,32,116,104,101,32,112,97, + 116,104,32,101,110,116,114,121,32,102,114,111,109,32,115,121, + 115,46,112,97,116,104,95,105,109,112,111,114,116,101,114,95, + 99,97,99,104,101,46,10,10,32,32,32,32,32,32,32,32, + 73,102,32,116,104,101,32,112,97,116,104,32,101,110,116,114, + 121,32,105,115,32,110,111,116,32,105,110,32,116,104,101,32, + 99,97,99,104,101,44,32,102,105,110,100,32,116,104,101,32, + 97,112,112,114,111,112,114,105,97,116,101,32,102,105,110,100, + 101,114,10,32,32,32,32,32,32,32,32,97,110,100,32,99, + 97,99,104,101,32,105,116,46,32,73,102,32,110,111,32,102, + 105,110,100,101,114,32,105,115,32,97,118,97,105,108,97,98, + 108,101,44,32,115,116,111,114,101,32,78,111,110,101,46,10, + 10,32,32,32,32,32,32,32,32,114,30,0,0,0,78,41, + 7,114,1,0,0,0,114,45,0,0,0,114,228,0,0,0, + 114,6,0,0,0,114,12,1,0,0,218,8,75,101,121,69, + 114,114,111,114,114,16,1,0,0,41,3,114,176,0,0,0, + 114,35,0,0,0,114,14,1,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,218,20,95,112,97,116,104, + 95,105,109,112,111,114,116,101,114,95,99,97,99,104,101,222, + 4,0,0,115,22,0,0,0,0,8,8,1,2,1,12,1, + 14,3,8,1,2,1,14,1,14,1,10,1,16,1,122,31, + 80,97,116,104,70,105,110,100,101,114,46,95,112,97,116,104, + 95,105,109,112,111,114,116,101,114,95,99,97,99,104,101,99, + 3,0,0,0,0,0,0,0,6,0,0,0,4,0,0,0, + 67,0,0,0,115,82,0,0,0,116,0,124,2,100,1,131, + 2,114,26,124,2,160,1,124,1,161,1,92,2,125,3,125, + 4,110,14,124,2,160,2,124,1,161,1,125,3,103,0,125, + 4,124,3,100,0,107,9,114,60,116,3,160,4,124,1,124, + 3,161,2,83,0,116,3,160,5,124,1,100,0,161,2,125, + 5,124,4,124,5,95,6,124,5,83,0,41,2,78,114,123, + 0,0,0,41,7,114,114,0,0,0,114,123,0,0,0,114, + 187,0,0,0,114,120,0,0,0,114,184,0,0,0,114,166, + 0,0,0,114,162,0,0,0,41,6,114,176,0,0,0,114, + 125,0,0,0,114,14,1,0,0,114,126,0,0,0,114,127, + 0,0,0,114,170,0,0,0,114,2,0,0,0,114,2,0, + 0,0,114,4,0,0,0,218,16,95,108,101,103,97,99,121, + 95,103,101,116,95,115,112,101,99,244,4,0,0,115,18,0, + 0,0,0,4,10,1,16,2,10,1,4,1,8,1,12,1, + 12,1,6,1,122,27,80,97,116,104,70,105,110,100,101,114, + 46,95,108,101,103,97,99,121,95,103,101,116,95,115,112,101, + 99,78,99,4,0,0,0,0,0,0,0,9,0,0,0,5, + 0,0,0,67,0,0,0,115,166,0,0,0,103,0,125,4, + 124,2,68,0,93,134,125,5,116,0,124,5,116,1,116,2, + 102,2,131,2,115,28,113,8,124,0,160,3,124,5,161,1, + 125,6,124,6,100,1,107,9,114,8,116,4,124,6,100,2, + 131,2,114,70,124,6,160,5,124,1,124,3,161,2,125,7, + 110,12,124,0,160,6,124,1,124,6,161,2,125,7,124,7, + 100,1,107,8,114,92,113,8,124,7,106,7,100,1,107,9, + 114,110,124,7,2,0,1,0,83,0,124,7,106,8,125,8, + 124,8,100,1,107,8,114,132,116,9,100,3,131,1,130,1, + 124,4,160,10,124,8,161,1,1,0,113,8,116,11,160,12, + 124,1,100,1,161,2,125,7,124,4,124,7,95,8,124,7, + 83,0,41,4,122,63,70,105,110,100,32,116,104,101,32,108, + 111,97,100,101,114,32,111,114,32,110,97,109,101,115,112,97, + 99,101,95,112,97,116,104,32,102,111,114,32,116,104,105,115, + 32,109,111,100,117,108,101,47,112,97,99,107,97,103,101,32, + 110,97,109,101,46,78,114,186,0,0,0,122,19,115,112,101, + 99,32,109,105,115,115,105,110,103,32,108,111,97,100,101,114, + 41,13,114,145,0,0,0,114,74,0,0,0,218,5,98,121, + 116,101,115,114,18,1,0,0,114,114,0,0,0,114,186,0, + 0,0,114,19,1,0,0,114,126,0,0,0,114,162,0,0, + 0,114,105,0,0,0,114,151,0,0,0,114,120,0,0,0, + 114,166,0,0,0,41,9,114,176,0,0,0,114,125,0,0, + 0,114,35,0,0,0,114,185,0,0,0,218,14,110,97,109, + 101,115,112,97,99,101,95,112,97,116,104,90,5,101,110,116, + 114,121,114,14,1,0,0,114,170,0,0,0,114,127,0,0, + 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 218,9,95,103,101,116,95,115,112,101,99,3,5,0,0,115, + 40,0,0,0,0,5,4,1,8,1,14,1,2,1,10,1, + 8,1,10,1,14,2,12,1,8,1,2,1,10,1,8,1, + 6,1,8,1,8,5,12,2,12,1,6,1,122,20,80,97, + 116,104,70,105,110,100,101,114,46,95,103,101,116,95,115,112, + 101,99,99,4,0,0,0,0,0,0,0,6,0,0,0,5, + 0,0,0,67,0,0,0,115,100,0,0,0,124,2,100,1, + 107,8,114,14,116,0,106,1,125,2,124,0,160,2,124,1, + 124,2,124,3,161,3,125,4,124,4,100,1,107,8,114,40, + 100,1,83,0,124,4,106,3,100,1,107,8,114,92,124,4, + 106,4,125,5,124,5,114,86,100,1,124,4,95,5,116,6, + 124,1,124,5,124,0,106,2,131,3,124,4,95,4,124,4, + 83,0,100,1,83,0,110,4,124,4,83,0,100,1,83,0, + 41,2,122,141,84,114,121,32,116,111,32,102,105,110,100,32, + 97,32,115,112,101,99,32,102,111,114,32,39,102,117,108,108, + 110,97,109,101,39,32,111,110,32,115,121,115,46,112,97,116, + 104,32,111,114,32,39,112,97,116,104,39,46,10,10,32,32, + 32,32,32,32,32,32,84,104,101,32,115,101,97,114,99,104, + 32,105,115,32,98,97,115,101,100,32,111,110,32,115,121,115, + 46,112,97,116,104,95,104,111,111,107,115,32,97,110,100,32, + 115,121,115,46,112,97,116,104,95,105,109,112,111,114,116,101, + 114,95,99,97,99,104,101,46,10,32,32,32,32,32,32,32, + 32,78,41,7,114,6,0,0,0,114,35,0,0,0,114,22, + 1,0,0,114,126,0,0,0,114,162,0,0,0,114,164,0, + 0,0,114,245,0,0,0,41,6,114,176,0,0,0,114,125, + 0,0,0,114,35,0,0,0,114,185,0,0,0,114,170,0, + 0,0,114,21,1,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,114,186,0,0,0,35,5,0,0,115, + 26,0,0,0,0,6,8,1,6,1,14,1,8,1,4,1, + 10,1,6,1,4,3,6,1,16,1,4,2,6,2,122,20, + 80,97,116,104,70,105,110,100,101,114,46,102,105,110,100,95, + 115,112,101,99,99,3,0,0,0,0,0,0,0,4,0,0, + 0,4,0,0,0,67,0,0,0,115,30,0,0,0,124,0, + 160,0,124,1,124,2,161,2,125,3,124,3,100,1,107,8, + 114,24,100,1,83,0,124,3,106,1,83,0,41,2,122,170, + 102,105,110,100,32,116,104,101,32,109,111,100,117,108,101,32, + 111,110,32,115,121,115,46,112,97,116,104,32,111,114,32,39, + 112,97,116,104,39,32,98,97,115,101,100,32,111,110,32,115, + 121,115,46,112,97,116,104,95,104,111,111,107,115,32,97,110, + 100,10,32,32,32,32,32,32,32,32,115,121,115,46,112,97, + 116,104,95,105,109,112,111,114,116,101,114,95,99,97,99,104, + 101,46,10,10,32,32,32,32,32,32,32,32,84,104,105,115, + 32,109,101,116,104,111,100,32,105,115,32,100,101,112,114,101, + 99,97,116,101,100,46,32,32,85,115,101,32,102,105,110,100, + 95,115,112,101,99,40,41,32,105,110,115,116,101,97,100,46, + 10,10,32,32,32,32,32,32,32,32,78,41,2,114,186,0, + 0,0,114,126,0,0,0,41,4,114,176,0,0,0,114,125, + 0,0,0,114,35,0,0,0,114,170,0,0,0,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,114,187,0,0, + 0,59,5,0,0,115,8,0,0,0,0,8,12,1,8,1, + 4,1,122,22,80,97,116,104,70,105,110,100,101,114,46,102, + 105,110,100,95,109,111,100,117,108,101,41,12,114,111,0,0, + 0,114,110,0,0,0,114,112,0,0,0,114,113,0,0,0, + 114,188,0,0,0,114,10,1,0,0,114,16,1,0,0,114, + 18,1,0,0,114,19,1,0,0,114,22,1,0,0,114,186, + 0,0,0,114,187,0,0,0,114,2,0,0,0,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,114,9,1,0, + 0,195,4,0,0,115,30,0,0,0,8,2,4,2,12,10, + 12,13,12,22,12,15,2,1,2,255,12,32,2,1,2,0, + 2,255,12,24,2,1,2,255,114,9,1,0,0,99,0,0, + 0,0,0,0,0,0,0,0,0,0,3,0,0,0,64,0, + 0,0,115,90,0,0,0,101,0,90,1,100,0,90,2,100, + 1,90,3,100,2,100,3,132,0,90,4,100,4,100,5,132, + 0,90,5,101,6,90,7,100,6,100,7,132,0,90,8,100, + 8,100,9,132,0,90,9,100,19,100,11,100,12,132,1,90, + 10,100,13,100,14,132,0,90,11,101,12,100,15,100,16,132, + 0,131,1,90,13,100,17,100,18,132,0,90,14,100,10,83, + 0,41,20,218,10,70,105,108,101,70,105,110,100,101,114,122, + 172,70,105,108,101,45,98,97,115,101,100,32,102,105,110,100, + 101,114,46,10,10,32,32,32,32,73,110,116,101,114,97,99, + 116,105,111,110,115,32,119,105,116,104,32,116,104,101,32,102, + 105,108,101,32,115,121,115,116,101,109,32,97,114,101,32,99, + 97,99,104,101,100,32,102,111,114,32,112,101,114,102,111,114, + 109,97,110,99,101,44,32,98,101,105,110,103,10,32,32,32, + 32,114,101,102,114,101,115,104,101,100,32,119,104,101,110,32, + 116,104,101,32,100,105,114,101,99,116,111,114,121,32,116,104, + 101,32,102,105,110,100,101,114,32,105,115,32,104,97,110,100, + 108,105,110,103,32,104,97,115,32,98,101,101,110,32,109,111, + 100,105,102,105,101,100,46,10,10,32,32,32,32,99,2,0, + 0,0,0,0,0,0,5,0,0,0,6,0,0,0,7,0, + 0,0,115,84,0,0,0,103,0,125,3,124,2,68,0,93, + 32,92,2,137,0,125,4,124,3,160,0,135,0,102,1,100, + 1,100,2,132,8,124,4,68,0,131,1,161,1,1,0,113, + 8,124,3,124,0,95,1,124,1,112,54,100,3,124,0,95, + 2,100,4,124,0,95,3,116,4,131,0,124,0,95,5,116, + 4,131,0,124,0,95,6,100,5,83,0,41,6,122,154,73, + 110,105,116,105,97,108,105,122,101,32,119,105,116,104,32,116, + 104,101,32,112,97,116,104,32,116,111,32,115,101,97,114,99, + 104,32,111,110,32,97,110,100,32,97,32,118,97,114,105,97, + 98,108,101,32,110,117,109,98,101,114,32,111,102,10,32,32, + 32,32,32,32,32,32,50,45,116,117,112,108,101,115,32,99, + 111,110,116,97,105,110,105,110,103,32,116,104,101,32,108,111, + 97,100,101,114,32,97,110,100,32,116,104,101,32,102,105,108, + 101,32,115,117,102,102,105,120,101,115,32,116,104,101,32,108, + 111,97,100,101,114,10,32,32,32,32,32,32,32,32,114,101, + 99,111,103,110,105,122,101,115,46,99,1,0,0,0,0,0, + 0,0,2,0,0,0,3,0,0,0,51,0,0,0,115,22, + 0,0,0,124,0,93,14,125,1,124,1,136,0,102,2,86, + 0,1,0,113,2,100,0,83,0,41,1,78,114,2,0,0, + 0,41,2,114,22,0,0,0,114,240,0,0,0,41,1,114, + 126,0,0,0,114,2,0,0,0,114,4,0,0,0,114,242, + 0,0,0,88,5,0,0,115,4,0,0,0,4,0,2,0, + 122,38,70,105,108,101,70,105,110,100,101,114,46,95,95,105, + 110,105,116,95,95,46,60,108,111,99,97,108,115,62,46,60, + 103,101,110,101,120,112,114,62,114,61,0,0,0,114,94,0, + 0,0,78,41,7,114,151,0,0,0,218,8,95,108,111,97, + 100,101,114,115,114,35,0,0,0,218,11,95,112,97,116,104, + 95,109,116,105,109,101,218,3,115,101,116,218,11,95,112,97, + 116,104,95,99,97,99,104,101,218,19,95,114,101,108,97,120, + 101,100,95,112,97,116,104,95,99,97,99,104,101,41,5,114, + 106,0,0,0,114,35,0,0,0,218,14,108,111,97,100,101, + 114,95,100,101,116,97,105,108,115,90,7,108,111,97,100,101, + 114,115,114,172,0,0,0,114,2,0,0,0,41,1,114,126, + 0,0,0,114,4,0,0,0,114,190,0,0,0,82,5,0, + 0,115,16,0,0,0,0,4,4,1,12,1,26,1,6,2, + 10,1,6,1,8,1,122,19,70,105,108,101,70,105,110,100, + 101,114,46,95,95,105,110,105,116,95,95,99,1,0,0,0, + 0,0,0,0,1,0,0,0,2,0,0,0,67,0,0,0, + 115,10,0,0,0,100,1,124,0,95,0,100,2,83,0,41, + 3,122,31,73,110,118,97,108,105,100,97,116,101,32,116,104, + 101,32,100,105,114,101,99,116,111,114,121,32,109,116,105,109, + 101,46,114,94,0,0,0,78,41,1,114,25,1,0,0,41, + 1,114,106,0,0,0,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,114,10,1,0,0,96,5,0,0,115,2, + 0,0,0,0,2,122,28,70,105,108,101,70,105,110,100,101, + 114,46,105,110,118,97,108,105,100,97,116,101,95,99,97,99, + 104,101,115,99,2,0,0,0,0,0,0,0,3,0,0,0, + 3,0,0,0,67,0,0,0,115,42,0,0,0,124,0,160, + 0,124,1,161,1,125,2,124,2,100,1,107,8,114,26,100, + 1,103,0,102,2,83,0,124,2,106,1,124,2,106,2,112, + 38,103,0,102,2,83,0,41,2,122,197,84,114,121,32,116, + 111,32,102,105,110,100,32,97,32,108,111,97,100,101,114,32, + 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, + 100,32,109,111,100,117,108,101,44,32,111,114,32,116,104,101, + 32,110,97,109,101,115,112,97,99,101,10,32,32,32,32,32, + 32,32,32,112,97,99,107,97,103,101,32,112,111,114,116,105, + 111,110,115,46,32,82,101,116,117,114,110,115,32,40,108,111, + 97,100,101,114,44,32,108,105,115,116,45,111,102,45,112,111, + 114,116,105,111,110,115,41,46,10,10,32,32,32,32,32,32, 32,32,84,104,105,115,32,109,101,116,104,111,100,32,105,115, 32,100,101,112,114,101,99,97,116,101,100,46,32,32,85,115, - 101,32,101,120,101,99,95,109,111,100,117,108,101,40,41,32, - 105,110,115,116,101,97,100,46,10,10,32,32,32,32,32,32, - 32,32,122,38,110,97,109,101,115,112,97,99,101,32,109,111, - 100,117,108,101,32,108,111,97,100,101,100,32,119,105,116,104, - 32,112,97,116,104,32,123,33,114,125,41,4,114,121,0,0, - 0,114,135,0,0,0,114,248,0,0,0,114,198,0,0,0, - 41,2,114,107,0,0,0,114,126,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,114,199,0,0,0, - 181,4,0,0,115,6,0,0,0,0,7,6,1,8,1,122, - 28,95,78,97,109,101,115,112,97,99,101,76,111,97,100,101, - 114,46,108,111,97,100,95,109,111,100,117,108,101,78,41,12, - 114,112,0,0,0,114,111,0,0,0,114,113,0,0,0,114, - 191,0,0,0,114,189,0,0,0,114,9,1,0,0,114,166, - 0,0,0,114,207,0,0,0,114,193,0,0,0,114,192,0, - 0,0,114,197,0,0,0,114,199,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 114,8,1,0,0,153,4,0,0,115,16,0,0,0,8,1, - 8,3,12,9,8,3,8,3,8,3,8,3,8,3,114,8, - 1,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, - 4,0,0,0,64,0,0,0,115,106,0,0,0,101,0,90, - 1,100,0,90,2,100,1,90,3,101,4,100,2,100,3,132, - 0,131,1,90,5,101,4,100,4,100,5,132,0,131,1,90, - 6,101,4,100,6,100,7,132,0,131,1,90,7,101,4,100, - 8,100,9,132,0,131,1,90,8,101,4,100,17,100,11,100, - 12,132,1,131,1,90,9,101,4,100,18,100,13,100,14,132, - 1,131,1,90,10,101,4,100,19,100,15,100,16,132,1,131, - 1,90,11,100,10,83,0,41,20,218,10,80,97,116,104,70, - 105,110,100,101,114,122,62,77,101,116,97,32,112,97,116,104, - 32,102,105,110,100,101,114,32,102,111,114,32,115,121,115,46, - 112,97,116,104,32,97,110,100,32,112,97,99,107,97,103,101, - 32,95,95,112,97,116,104,95,95,32,97,116,116,114,105,98, - 117,116,101,115,46,99,1,0,0,0,0,0,0,0,3,0, - 0,0,4,0,0,0,67,0,0,0,115,64,0,0,0,116, - 0,116,1,106,2,160,3,161,0,131,1,68,0,93,44,92, - 2,125,1,125,2,124,2,100,1,107,8,114,40,116,1,106, - 2,124,1,61,0,113,14,116,4,124,2,100,2,131,2,114, - 14,124,2,160,5,161,0,1,0,113,14,100,1,83,0,41, - 3,122,125,67,97,108,108,32,116,104,101,32,105,110,118,97, - 108,105,100,97,116,101,95,99,97,99,104,101,115,40,41,32, - 109,101,116,104,111,100,32,111,110,32,97,108,108,32,112,97, - 116,104,32,101,110,116,114,121,32,102,105,110,100,101,114,115, - 10,32,32,32,32,32,32,32,32,115,116,111,114,101,100,32, - 105,110,32,115,121,115,46,112,97,116,104,95,105,109,112,111, - 114,116,101,114,95,99,97,99,104,101,115,32,40,119,104,101, - 114,101,32,105,109,112,108,101,109,101,110,116,101,100,41,46, - 78,218,17,105,110,118,97,108,105,100,97,116,101,95,99,97, - 99,104,101,115,41,6,218,4,108,105,115,116,114,6,0,0, - 0,218,19,112,97,116,104,95,105,109,112,111,114,116,101,114, - 95,99,97,99,104,101,218,5,105,116,101,109,115,114,115,0, - 0,0,114,11,1,0,0,41,3,114,177,0,0,0,114,105, - 0,0,0,218,6,102,105,110,100,101,114,114,2,0,0,0, - 114,2,0,0,0,114,4,0,0,0,114,11,1,0,0,199, - 4,0,0,115,10,0,0,0,0,4,22,1,8,1,10,1, - 10,1,122,28,80,97,116,104,70,105,110,100,101,114,46,105, - 110,118,97,108,105,100,97,116,101,95,99,97,99,104,101,115, - 99,2,0,0,0,0,0,0,0,3,0,0,0,9,0,0, - 0,67,0,0,0,115,84,0,0,0,116,0,106,1,100,1, - 107,9,114,28,116,0,106,1,115,28,116,2,160,3,100,2, - 116,4,161,2,1,0,116,0,106,1,68,0,93,44,125,2, - 122,14,124,2,124,1,131,1,87,0,2,0,1,0,83,0, - 4,0,116,5,107,10,114,76,1,0,1,0,1,0,89,0, - 113,34,89,0,113,34,88,0,113,34,100,1,83,0,41,3, - 122,46,83,101,97,114,99,104,32,115,121,115,46,112,97,116, - 104,95,104,111,111,107,115,32,102,111,114,32,97,32,102,105, - 110,100,101,114,32,102,111,114,32,39,112,97,116,104,39,46, - 78,122,23,115,121,115,46,112,97,116,104,95,104,111,111,107, - 115,32,105,115,32,101,109,112,116,121,41,6,114,6,0,0, - 0,218,10,112,97,116,104,95,104,111,111,107,115,114,65,0, - 0,0,114,66,0,0,0,114,125,0,0,0,114,106,0,0, - 0,41,3,114,177,0,0,0,114,35,0,0,0,90,4,104, - 111,111,107,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,218,11,95,112,97,116,104,95,104,111,111,107,115,209, - 4,0,0,115,16,0,0,0,0,3,16,1,12,1,10,1, - 2,1,14,1,14,1,12,2,122,22,80,97,116,104,70,105, - 110,100,101,114,46,95,112,97,116,104,95,104,111,111,107,115, - 99,2,0,0,0,0,0,0,0,3,0,0,0,8,0,0, - 0,67,0,0,0,115,104,0,0,0,124,1,100,1,107,2, - 114,44,122,12,116,0,160,1,161,0,125,1,87,0,110,22, - 4,0,116,2,107,10,114,42,1,0,1,0,1,0,89,0, - 100,2,83,0,88,0,122,14,116,3,106,4,124,1,25,0, - 125,2,87,0,110,40,4,0,116,5,107,10,114,98,1,0, - 1,0,1,0,124,0,160,6,124,1,161,1,125,2,124,2, - 116,3,106,4,124,1,60,0,89,0,110,2,88,0,124,2, - 83,0,41,3,122,210,71,101,116,32,116,104,101,32,102,105, - 110,100,101,114,32,102,111,114,32,116,104,101,32,112,97,116, - 104,32,101,110,116,114,121,32,102,114,111,109,32,115,121,115, - 46,112,97,116,104,95,105,109,112,111,114,116,101,114,95,99, - 97,99,104,101,46,10,10,32,32,32,32,32,32,32,32,73, - 102,32,116,104,101,32,112,97,116,104,32,101,110,116,114,121, - 32,105,115,32,110,111,116,32,105,110,32,116,104,101,32,99, - 97,99,104,101,44,32,102,105,110,100,32,116,104,101,32,97, - 112,112,114,111,112,114,105,97,116,101,32,102,105,110,100,101, - 114,10,32,32,32,32,32,32,32,32,97,110,100,32,99,97, - 99,104,101,32,105,116,46,32,73,102,32,110,111,32,102,105, - 110,100,101,114,32,105,115,32,97,118,97,105,108,97,98,108, - 101,44,32,115,116,111,114,101,32,78,111,110,101,46,10,10, - 32,32,32,32,32,32,32,32,114,30,0,0,0,78,41,7, - 114,1,0,0,0,114,45,0,0,0,114,229,0,0,0,114, - 6,0,0,0,114,13,1,0,0,218,8,75,101,121,69,114, - 114,111,114,114,17,1,0,0,41,3,114,177,0,0,0,114, - 35,0,0,0,114,15,1,0,0,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,218,20,95,112,97,116,104,95, - 105,109,112,111,114,116,101,114,95,99,97,99,104,101,222,4, - 0,0,115,22,0,0,0,0,8,8,1,2,1,12,1,14, - 3,8,1,2,1,14,1,14,1,10,1,16,1,122,31,80, - 97,116,104,70,105,110,100,101,114,46,95,112,97,116,104,95, - 105,109,112,111,114,116,101,114,95,99,97,99,104,101,99,3, - 0,0,0,0,0,0,0,6,0,0,0,4,0,0,0,67, - 0,0,0,115,82,0,0,0,116,0,124,2,100,1,131,2, - 114,26,124,2,160,1,124,1,161,1,92,2,125,3,125,4, - 110,14,124,2,160,2,124,1,161,1,125,3,103,0,125,4, - 124,3,100,0,107,9,114,60,116,3,160,4,124,1,124,3, - 161,2,83,0,116,3,160,5,124,1,100,0,161,2,125,5, - 124,4,124,5,95,6,124,5,83,0,41,2,78,114,124,0, - 0,0,41,7,114,115,0,0,0,114,124,0,0,0,114,188, - 0,0,0,114,121,0,0,0,114,185,0,0,0,114,167,0, - 0,0,114,163,0,0,0,41,6,114,177,0,0,0,114,126, - 0,0,0,114,15,1,0,0,114,127,0,0,0,114,128,0, - 0,0,114,171,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,218,16,95,108,101,103,97,99,121,95, - 103,101,116,95,115,112,101,99,244,4,0,0,115,18,0,0, - 0,0,4,10,1,16,2,10,1,4,1,8,1,12,1,12, - 1,6,1,122,27,80,97,116,104,70,105,110,100,101,114,46, - 95,108,101,103,97,99,121,95,103,101,116,95,115,112,101,99, - 78,99,4,0,0,0,0,0,0,0,9,0,0,0,5,0, - 0,0,67,0,0,0,115,166,0,0,0,103,0,125,4,124, - 2,68,0,93,134,125,5,116,0,124,5,116,1,116,2,102, - 2,131,2,115,28,113,8,124,0,160,3,124,5,161,1,125, - 6,124,6,100,1,107,9,114,8,116,4,124,6,100,2,131, - 2,114,70,124,6,160,5,124,1,124,3,161,2,125,7,110, - 12,124,0,160,6,124,1,124,6,161,2,125,7,124,7,100, - 1,107,8,114,92,113,8,124,7,106,7,100,1,107,9,114, - 110,124,7,2,0,1,0,83,0,124,7,106,8,125,8,124, - 8,100,1,107,8,114,132,116,9,100,3,131,1,130,1,124, - 4,160,10,124,8,161,1,1,0,113,8,116,11,160,12,124, - 1,100,1,161,2,125,7,124,4,124,7,95,8,124,7,83, - 0,41,4,122,63,70,105,110,100,32,116,104,101,32,108,111, - 97,100,101,114,32,111,114,32,110,97,109,101,115,112,97,99, - 101,95,112,97,116,104,32,102,111,114,32,116,104,105,115,32, - 109,111,100,117,108,101,47,112,97,99,107,97,103,101,32,110, - 97,109,101,46,78,114,187,0,0,0,122,19,115,112,101,99, - 32,109,105,115,115,105,110,103,32,108,111,97,100,101,114,41, - 13,114,146,0,0,0,114,75,0,0,0,218,5,98,121,116, - 101,115,114,19,1,0,0,114,115,0,0,0,114,187,0,0, - 0,114,20,1,0,0,114,127,0,0,0,114,163,0,0,0, - 114,106,0,0,0,114,152,0,0,0,114,121,0,0,0,114, - 167,0,0,0,41,9,114,177,0,0,0,114,126,0,0,0, - 114,35,0,0,0,114,186,0,0,0,218,14,110,97,109,101, - 115,112,97,99,101,95,112,97,116,104,90,5,101,110,116,114, - 121,114,15,1,0,0,114,171,0,0,0,114,128,0,0,0, - 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 9,95,103,101,116,95,115,112,101,99,3,5,0,0,115,40, - 0,0,0,0,5,4,1,8,1,14,1,2,1,10,1,8, - 1,10,1,14,2,12,1,8,1,2,1,10,1,8,1,6, - 1,8,1,8,5,12,2,12,1,6,1,122,20,80,97,116, - 104,70,105,110,100,101,114,46,95,103,101,116,95,115,112,101, - 99,99,4,0,0,0,0,0,0,0,6,0,0,0,5,0, - 0,0,67,0,0,0,115,100,0,0,0,124,2,100,1,107, - 8,114,14,116,0,106,1,125,2,124,0,160,2,124,1,124, - 2,124,3,161,3,125,4,124,4,100,1,107,8,114,40,100, - 1,83,0,124,4,106,3,100,1,107,8,114,92,124,4,106, - 4,125,5,124,5,114,86,100,1,124,4,95,5,116,6,124, - 1,124,5,124,0,106,2,131,3,124,4,95,4,124,4,83, - 0,100,1,83,0,110,4,124,4,83,0,100,1,83,0,41, - 2,122,141,84,114,121,32,116,111,32,102,105,110,100,32,97, - 32,115,112,101,99,32,102,111,114,32,39,102,117,108,108,110, - 97,109,101,39,32,111,110,32,115,121,115,46,112,97,116,104, - 32,111,114,32,39,112,97,116,104,39,46,10,10,32,32,32, - 32,32,32,32,32,84,104,101,32,115,101,97,114,99,104,32, - 105,115,32,98,97,115,101,100,32,111,110,32,115,121,115,46, - 112,97,116,104,95,104,111,111,107,115,32,97,110,100,32,115, - 121,115,46,112,97,116,104,95,105,109,112,111,114,116,101,114, - 95,99,97,99,104,101,46,10,32,32,32,32,32,32,32,32, - 78,41,7,114,6,0,0,0,114,35,0,0,0,114,23,1, - 0,0,114,127,0,0,0,114,163,0,0,0,114,165,0,0, - 0,114,246,0,0,0,41,6,114,177,0,0,0,114,126,0, - 0,0,114,35,0,0,0,114,186,0,0,0,114,171,0,0, - 0,114,22,1,0,0,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,114,187,0,0,0,35,5,0,0,115,26, - 0,0,0,0,6,8,1,6,1,14,1,8,1,4,1,10, - 1,6,1,4,3,6,1,16,1,4,2,6,2,122,20,80, - 97,116,104,70,105,110,100,101,114,46,102,105,110,100,95,115, - 112,101,99,99,3,0,0,0,0,0,0,0,4,0,0,0, - 4,0,0,0,67,0,0,0,115,30,0,0,0,124,0,160, - 0,124,1,124,2,161,2,125,3,124,3,100,1,107,8,114, - 24,100,1,83,0,124,3,106,1,83,0,41,2,122,170,102, - 105,110,100,32,116,104,101,32,109,111,100,117,108,101,32,111, - 110,32,115,121,115,46,112,97,116,104,32,111,114,32,39,112, - 97,116,104,39,32,98,97,115,101,100,32,111,110,32,115,121, - 115,46,112,97,116,104,95,104,111,111,107,115,32,97,110,100, - 10,32,32,32,32,32,32,32,32,115,121,115,46,112,97,116, - 104,95,105,109,112,111,114,116,101,114,95,99,97,99,104,101, - 46,10,10,32,32,32,32,32,32,32,32,84,104,105,115,32, - 109,101,116,104,111,100,32,105,115,32,100,101,112,114,101,99, - 97,116,101,100,46,32,32,85,115,101,32,102,105,110,100,95, - 115,112,101,99,40,41,32,105,110,115,116,101,97,100,46,10, - 10,32,32,32,32,32,32,32,32,78,41,2,114,187,0,0, - 0,114,127,0,0,0,41,4,114,177,0,0,0,114,126,0, - 0,0,114,35,0,0,0,114,171,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,114,188,0,0,0, - 59,5,0,0,115,8,0,0,0,0,8,12,1,8,1,4, - 1,122,22,80,97,116,104,70,105,110,100,101,114,46,102,105, - 110,100,95,109,111,100,117,108,101,41,1,78,41,2,78,78, - 41,1,78,41,12,114,112,0,0,0,114,111,0,0,0,114, - 113,0,0,0,114,114,0,0,0,114,189,0,0,0,114,11, - 1,0,0,114,17,1,0,0,114,19,1,0,0,114,20,1, - 0,0,114,23,1,0,0,114,187,0,0,0,114,188,0,0, - 0,114,2,0,0,0,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,114,10,1,0,0,195,4,0,0,115,22, - 0,0,0,8,2,4,2,12,10,12,13,12,22,12,15,2, - 1,12,31,2,1,12,23,2,1,114,10,1,0,0,99,0, - 0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,64, - 0,0,0,115,90,0,0,0,101,0,90,1,100,0,90,2, - 100,1,90,3,100,2,100,3,132,0,90,4,100,4,100,5, - 132,0,90,5,101,6,90,7,100,6,100,7,132,0,90,8, - 100,8,100,9,132,0,90,9,100,19,100,11,100,12,132,1, - 90,10,100,13,100,14,132,0,90,11,101,12,100,15,100,16, - 132,0,131,1,90,13,100,17,100,18,132,0,90,14,100,10, - 83,0,41,20,218,10,70,105,108,101,70,105,110,100,101,114, - 122,172,70,105,108,101,45,98,97,115,101,100,32,102,105,110, - 100,101,114,46,10,10,32,32,32,32,73,110,116,101,114,97, - 99,116,105,111,110,115,32,119,105,116,104,32,116,104,101,32, - 102,105,108,101,32,115,121,115,116,101,109,32,97,114,101,32, - 99,97,99,104,101,100,32,102,111,114,32,112,101,114,102,111, - 114,109,97,110,99,101,44,32,98,101,105,110,103,10,32,32, - 32,32,114,101,102,114,101,115,104,101,100,32,119,104,101,110, - 32,116,104,101,32,100,105,114,101,99,116,111,114,121,32,116, - 104,101,32,102,105,110,100,101,114,32,105,115,32,104,97,110, - 100,108,105,110,103,32,104,97,115,32,98,101,101,110,32,109, - 111,100,105,102,105,101,100,46,10,10,32,32,32,32,99,2, - 0,0,0,0,0,0,0,5,0,0,0,6,0,0,0,7, - 0,0,0,115,84,0,0,0,103,0,125,3,124,2,68,0, - 93,32,92,2,137,0,125,4,124,3,160,0,135,0,102,1, - 100,1,100,2,132,8,124,4,68,0,131,1,161,1,1,0, - 113,8,124,3,124,0,95,1,124,1,112,54,100,3,124,0, - 95,2,100,4,124,0,95,3,116,4,131,0,124,0,95,5, - 116,4,131,0,124,0,95,6,100,5,83,0,41,6,122,154, - 73,110,105,116,105,97,108,105,122,101,32,119,105,116,104,32, - 116,104,101,32,112,97,116,104,32,116,111,32,115,101,97,114, - 99,104,32,111,110,32,97,110,100,32,97,32,118,97,114,105, - 97,98,108,101,32,110,117,109,98,101,114,32,111,102,10,32, - 32,32,32,32,32,32,32,50,45,116,117,112,108,101,115,32, - 99,111,110,116,97,105,110,105,110,103,32,116,104,101,32,108, - 111,97,100,101,114,32,97,110,100,32,116,104,101,32,102,105, - 108,101,32,115,117,102,102,105,120,101,115,32,116,104,101,32, - 108,111,97,100,101,114,10,32,32,32,32,32,32,32,32,114, - 101,99,111,103,110,105,122,101,115,46,99,1,0,0,0,0, - 0,0,0,2,0,0,0,3,0,0,0,51,0,0,0,115, - 22,0,0,0,124,0,93,14,125,1,124,1,136,0,102,2, - 86,0,1,0,113,2,100,0,83,0,41,1,78,114,2,0, - 0,0,41,2,114,22,0,0,0,114,241,0,0,0,41,1, - 114,127,0,0,0,114,2,0,0,0,114,4,0,0,0,114, - 243,0,0,0,88,5,0,0,115,2,0,0,0,4,0,122, - 38,70,105,108,101,70,105,110,100,101,114,46,95,95,105,110, - 105,116,95,95,46,60,108,111,99,97,108,115,62,46,60,103, - 101,110,101,120,112,114,62,114,62,0,0,0,114,95,0,0, - 0,78,41,7,114,152,0,0,0,218,8,95,108,111,97,100, - 101,114,115,114,35,0,0,0,218,11,95,112,97,116,104,95, - 109,116,105,109,101,218,3,115,101,116,218,11,95,112,97,116, - 104,95,99,97,99,104,101,218,19,95,114,101,108,97,120,101, - 100,95,112,97,116,104,95,99,97,99,104,101,41,5,114,107, - 0,0,0,114,35,0,0,0,218,14,108,111,97,100,101,114, - 95,100,101,116,97,105,108,115,90,7,108,111,97,100,101,114, - 115,114,173,0,0,0,114,2,0,0,0,41,1,114,127,0, - 0,0,114,4,0,0,0,114,191,0,0,0,82,5,0,0, - 115,16,0,0,0,0,4,4,1,12,1,26,1,6,2,10, - 1,6,1,8,1,122,19,70,105,108,101,70,105,110,100,101, - 114,46,95,95,105,110,105,116,95,95,99,1,0,0,0,0, - 0,0,0,1,0,0,0,2,0,0,0,67,0,0,0,115, - 10,0,0,0,100,1,124,0,95,0,100,2,83,0,41,3, - 122,31,73,110,118,97,108,105,100,97,116,101,32,116,104,101, - 32,100,105,114,101,99,116,111,114,121,32,109,116,105,109,101, - 46,114,95,0,0,0,78,41,1,114,26,1,0,0,41,1, - 114,107,0,0,0,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,114,11,1,0,0,96,5,0,0,115,2,0, - 0,0,0,2,122,28,70,105,108,101,70,105,110,100,101,114, - 46,105,110,118,97,108,105,100,97,116,101,95,99,97,99,104, - 101,115,99,2,0,0,0,0,0,0,0,3,0,0,0,3, - 0,0,0,67,0,0,0,115,42,0,0,0,124,0,160,0, - 124,1,161,1,125,2,124,2,100,1,107,8,114,26,100,1, - 103,0,102,2,83,0,124,2,106,1,124,2,106,2,112,38, - 103,0,102,2,83,0,41,2,122,197,84,114,121,32,116,111, - 32,102,105,110,100,32,97,32,108,111,97,100,101,114,32,102, - 111,114,32,116,104,101,32,115,112,101,99,105,102,105,101,100, - 32,109,111,100,117,108,101,44,32,111,114,32,116,104,101,32, - 110,97,109,101,115,112,97,99,101,10,32,32,32,32,32,32, - 32,32,112,97,99,107,97,103,101,32,112,111,114,116,105,111, - 110,115,46,32,82,101,116,117,114,110,115,32,40,108,111,97, - 100,101,114,44,32,108,105,115,116,45,111,102,45,112,111,114, - 116,105,111,110,115,41,46,10,10,32,32,32,32,32,32,32, - 32,84,104,105,115,32,109,101,116,104,111,100,32,105,115,32, - 100,101,112,114,101,99,97,116,101,100,46,32,32,85,115,101, - 32,102,105,110,100,95,115,112,101,99,40,41,32,105,110,115, - 116,101,97,100,46,10,10,32,32,32,32,32,32,32,32,78, - 41,3,114,187,0,0,0,114,127,0,0,0,114,163,0,0, - 0,41,3,114,107,0,0,0,114,126,0,0,0,114,171,0, - 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, - 0,114,124,0,0,0,102,5,0,0,115,8,0,0,0,0, - 7,10,1,8,1,8,1,122,22,70,105,108,101,70,105,110, - 100,101,114,46,102,105,110,100,95,108,111,97,100,101,114,99, - 6,0,0,0,0,0,0,0,7,0,0,0,6,0,0,0, - 67,0,0,0,115,26,0,0,0,124,1,124,2,124,3,131, - 2,125,6,116,0,124,2,124,3,124,6,124,4,100,1,141, - 4,83,0,41,2,78,41,2,114,127,0,0,0,114,163,0, - 0,0,41,1,114,174,0,0,0,41,7,114,107,0,0,0, - 114,172,0,0,0,114,126,0,0,0,114,35,0,0,0,90, - 4,115,109,115,108,114,186,0,0,0,114,127,0,0,0,114, - 2,0,0,0,114,2,0,0,0,114,4,0,0,0,114,23, - 1,0,0,114,5,0,0,115,6,0,0,0,0,1,10,1, - 8,1,122,20,70,105,108,101,70,105,110,100,101,114,46,95, - 103,101,116,95,115,112,101,99,78,99,3,0,0,0,0,0, - 0,0,14,0,0,0,8,0,0,0,67,0,0,0,115,98, - 1,0,0,100,1,125,3,124,1,160,0,100,2,161,1,100, - 3,25,0,125,4,122,24,116,1,124,0,106,2,112,34,116, - 3,160,4,161,0,131,1,106,5,125,5,87,0,110,24,4, - 0,116,6,107,10,114,66,1,0,1,0,1,0,100,4,125, - 5,89,0,110,2,88,0,124,5,124,0,106,7,107,3,114, - 92,124,0,160,8,161,0,1,0,124,5,124,0,95,7,116, - 9,131,0,114,114,124,0,106,10,125,6,124,4,160,11,161, - 0,125,7,110,10,124,0,106,12,125,6,124,4,125,7,124, - 7,124,6,107,6,114,218,116,13,124,0,106,2,124,4,131, - 2,125,8,124,0,106,14,68,0,93,58,92,2,125,9,125, - 10,100,5,124,9,23,0,125,11,116,13,124,8,124,11,131, - 2,125,12,116,15,124,12,131,1,114,150,124,0,160,16,124, - 10,124,1,124,12,124,8,103,1,124,2,161,5,2,0,1, - 0,83,0,113,150,116,17,124,8,131,1,125,3,124,0,106, - 14,68,0,93,82,92,2,125,9,125,10,116,13,124,0,106, - 2,124,4,124,9,23,0,131,2,125,12,116,18,106,19,100, - 6,124,12,100,3,100,7,141,3,1,0,124,7,124,9,23, - 0,124,6,107,6,114,224,116,15,124,12,131,1,114,224,124, - 0,160,16,124,10,124,1,124,12,100,8,124,2,161,5,2, - 0,1,0,83,0,113,224,124,3,144,1,114,94,116,18,160, - 19,100,9,124,8,161,2,1,0,116,18,160,20,124,1,100, - 8,161,2,125,13,124,8,103,1,124,13,95,21,124,13,83, - 0,100,8,83,0,41,10,122,111,84,114,121,32,116,111,32, - 102,105,110,100,32,97,32,115,112,101,99,32,102,111,114,32, - 116,104,101,32,115,112,101,99,105,102,105,101,100,32,109,111, - 100,117,108,101,46,10,10,32,32,32,32,32,32,32,32,82, - 101,116,117,114,110,115,32,116,104,101,32,109,97,116,99,104, - 105,110,103,32,115,112,101,99,44,32,111,114,32,78,111,110, - 101,32,105,102,32,110,111,116,32,102,111,117,110,100,46,10, - 32,32,32,32,32,32,32,32,70,114,62,0,0,0,114,60, - 0,0,0,114,95,0,0,0,114,191,0,0,0,122,9,116, - 114,121,105,110,103,32,123,125,41,1,90,9,118,101,114,98, - 111,115,105,116,121,78,122,25,112,111,115,115,105,98,108,101, - 32,110,97,109,101,115,112,97,99,101,32,102,111,114,32,123, - 125,41,22,114,32,0,0,0,114,39,0,0,0,114,35,0, - 0,0,114,1,0,0,0,114,45,0,0,0,114,235,0,0, - 0,114,40,0,0,0,114,26,1,0,0,218,11,95,102,105, - 108,108,95,99,97,99,104,101,114,5,0,0,0,114,29,1, - 0,0,114,96,0,0,0,114,28,1,0,0,114,28,0,0, - 0,114,25,1,0,0,114,44,0,0,0,114,23,1,0,0, - 114,46,0,0,0,114,121,0,0,0,114,135,0,0,0,114, - 167,0,0,0,114,163,0,0,0,41,14,114,107,0,0,0, - 114,126,0,0,0,114,186,0,0,0,90,12,105,115,95,110, - 97,109,101,115,112,97,99,101,90,11,116,97,105,108,95,109, - 111,100,117,108,101,114,154,0,0,0,90,5,99,97,99,104, - 101,90,12,99,97,99,104,101,95,109,111,100,117,108,101,90, - 9,98,97,115,101,95,112,97,116,104,114,241,0,0,0,114, - 172,0,0,0,90,13,105,110,105,116,95,102,105,108,101,110, - 97,109,101,90,9,102,117,108,108,95,112,97,116,104,114,171, + 101,32,102,105,110,100,95,115,112,101,99,40,41,32,105,110, + 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, + 78,41,3,114,186,0,0,0,114,126,0,0,0,114,162,0, + 0,0,41,3,114,106,0,0,0,114,125,0,0,0,114,170, 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,114,187,0,0,0,119,5,0,0,115,70,0,0,0, - 0,5,4,1,14,1,2,1,24,1,14,1,10,1,10,1, - 8,1,6,2,6,1,6,1,10,2,6,1,4,2,8,1, - 12,1,14,1,8,1,10,1,8,1,26,4,8,2,14,1, - 16,1,16,1,12,1,8,1,10,1,14,1,6,1,12,1, - 12,1,8,1,4,1,122,20,70,105,108,101,70,105,110,100, - 101,114,46,102,105,110,100,95,115,112,101,99,99,1,0,0, - 0,0,0,0,0,9,0,0,0,10,0,0,0,67,0,0, - 0,115,190,0,0,0,124,0,106,0,125,1,122,22,116,1, - 160,2,124,1,112,22,116,1,160,3,161,0,161,1,125,2, - 87,0,110,30,4,0,116,4,116,5,116,6,102,3,107,10, - 114,58,1,0,1,0,1,0,103,0,125,2,89,0,110,2, - 88,0,116,7,106,8,160,9,100,1,161,1,115,84,116,10, - 124,2,131,1,124,0,95,11,110,74,116,10,131,0,125,3, - 124,2,68,0,93,56,125,4,124,4,160,12,100,2,161,1, - 92,3,125,5,125,6,125,7,124,6,114,136,100,3,160,13, - 124,5,124,7,160,14,161,0,161,2,125,8,110,4,124,5, - 125,8,124,3,160,15,124,8,161,1,1,0,113,94,124,3, - 124,0,95,11,116,7,106,8,160,9,116,16,161,1,114,186, - 100,4,100,5,132,0,124,2,68,0,131,1,124,0,95,17, - 100,6,83,0,41,7,122,68,70,105,108,108,32,116,104,101, - 32,99,97,99,104,101,32,111,102,32,112,111,116,101,110,116, - 105,97,108,32,109,111,100,117,108,101,115,32,97,110,100,32, - 112,97,99,107,97,103,101,115,32,102,111,114,32,116,104,105, - 115,32,100,105,114,101,99,116,111,114,121,46,114,0,0,0, - 0,114,62,0,0,0,122,5,123,125,46,123,125,99,1,0, - 0,0,0,0,0,0,2,0,0,0,4,0,0,0,83,0, - 0,0,115,20,0,0,0,104,0,124,0,93,12,125,1,124, - 1,160,0,161,0,146,2,113,4,83,0,114,2,0,0,0, - 41,1,114,96,0,0,0,41,2,114,22,0,0,0,90,2, - 102,110,114,2,0,0,0,114,2,0,0,0,114,4,0,0, - 0,218,9,60,115,101,116,99,111,109,112,62,196,5,0,0, - 115,2,0,0,0,6,0,122,41,70,105,108,101,70,105,110, - 100,101,114,46,95,102,105,108,108,95,99,97,99,104,101,46, - 60,108,111,99,97,108,115,62,46,60,115,101,116,99,111,109, - 112,62,78,41,18,114,35,0,0,0,114,1,0,0,0,114, - 232,0,0,0,114,45,0,0,0,114,229,0,0,0,218,15, - 80,101,114,109,105,115,115,105,111,110,69,114,114,111,114,218, - 18,78,111,116,65,68,105,114,101,99,116,111,114,121,69,114, - 114,111,114,114,6,0,0,0,114,7,0,0,0,114,8,0, - 0,0,114,27,1,0,0,114,28,1,0,0,114,91,0,0, - 0,114,51,0,0,0,114,96,0,0,0,218,3,97,100,100, - 114,9,0,0,0,114,29,1,0,0,41,9,114,107,0,0, - 0,114,35,0,0,0,114,233,0,0,0,90,21,108,111,119, - 101,114,95,115,117,102,102,105,120,95,99,111,110,116,101,110, - 116,115,114,6,1,0,0,114,105,0,0,0,114,253,0,0, - 0,114,241,0,0,0,90,8,110,101,119,95,110,97,109,101, + 0,0,114,123,0,0,0,102,5,0,0,115,8,0,0,0, + 0,7,10,1,8,1,8,1,122,22,70,105,108,101,70,105, + 110,100,101,114,46,102,105,110,100,95,108,111,97,100,101,114, + 99,6,0,0,0,0,0,0,0,7,0,0,0,6,0,0, + 0,67,0,0,0,115,26,0,0,0,124,1,124,2,124,3, + 131,2,125,6,116,0,124,2,124,3,124,6,124,4,100,1, + 141,4,83,0,41,2,78,41,2,114,126,0,0,0,114,162, + 0,0,0,41,1,114,173,0,0,0,41,7,114,106,0,0, + 0,114,171,0,0,0,114,125,0,0,0,114,35,0,0,0, + 90,4,115,109,115,108,114,185,0,0,0,114,126,0,0,0, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,114, - 31,1,0,0,167,5,0,0,115,34,0,0,0,0,2,6, - 1,2,1,22,1,20,3,10,3,12,1,12,7,6,1,8, - 1,16,1,4,1,18,2,4,1,12,1,6,1,12,1,122, - 22,70,105,108,101,70,105,110,100,101,114,46,95,102,105,108, - 108,95,99,97,99,104,101,99,1,0,0,0,0,0,0,0, - 3,0,0,0,3,0,0,0,7,0,0,0,115,18,0,0, - 0,135,0,135,1,102,2,100,1,100,2,132,8,125,2,124, - 2,83,0,41,3,97,20,1,0,0,65,32,99,108,97,115, - 115,32,109,101,116,104,111,100,32,119,104,105,99,104,32,114, - 101,116,117,114,110,115,32,97,32,99,108,111,115,117,114,101, - 32,116,111,32,117,115,101,32,111,110,32,115,121,115,46,112, - 97,116,104,95,104,111,111,107,10,32,32,32,32,32,32,32, - 32,119,104,105,99,104,32,119,105,108,108,32,114,101,116,117, - 114,110,32,97,110,32,105,110,115,116,97,110,99,101,32,117, - 115,105,110,103,32,116,104,101,32,115,112,101,99,105,102,105, - 101,100,32,108,111,97,100,101,114,115,32,97,110,100,32,116, - 104,101,32,112,97,116,104,10,32,32,32,32,32,32,32,32, - 99,97,108,108,101,100,32,111,110,32,116,104,101,32,99,108, - 111,115,117,114,101,46,10,10,32,32,32,32,32,32,32,32, - 73,102,32,116,104,101,32,112,97,116,104,32,99,97,108,108, - 101,100,32,111,110,32,116,104,101,32,99,108,111,115,117,114, - 101,32,105,115,32,110,111,116,32,97,32,100,105,114,101,99, - 116,111,114,121,44,32,73,109,112,111,114,116,69,114,114,111, - 114,32,105,115,10,32,32,32,32,32,32,32,32,114,97,105, - 115,101,100,46,10,10,32,32,32,32,32,32,32,32,99,1, - 0,0,0,0,0,0,0,1,0,0,0,4,0,0,0,19, - 0,0,0,115,34,0,0,0,116,0,124,0,131,1,115,20, - 116,1,100,1,124,0,100,2,141,2,130,1,136,0,124,0, - 102,1,136,1,158,2,142,0,83,0,41,3,122,45,80,97, - 116,104,32,104,111,111,107,32,102,111,114,32,105,109,112,111, - 114,116,108,105,98,46,109,97,99,104,105,110,101,114,121,46, - 70,105,108,101,70,105,110,100,101,114,46,122,30,111,110,108, - 121,32,100,105,114,101,99,116,111,114,105,101,115,32,97,114, - 101,32,115,117,112,112,111,114,116,101,100,41,1,114,35,0, - 0,0,41,2,114,46,0,0,0,114,106,0,0,0,41,1, - 114,35,0,0,0,41,2,114,177,0,0,0,114,30,1,0, - 0,114,2,0,0,0,114,4,0,0,0,218,24,112,97,116, - 104,95,104,111,111,107,95,102,111,114,95,70,105,108,101,70, - 105,110,100,101,114,208,5,0,0,115,6,0,0,0,0,2, - 8,1,12,1,122,54,70,105,108,101,70,105,110,100,101,114, - 46,112,97,116,104,95,104,111,111,107,46,60,108,111,99,97, - 108,115,62,46,112,97,116,104,95,104,111,111,107,95,102,111, - 114,95,70,105,108,101,70,105,110,100,101,114,114,2,0,0, - 0,41,3,114,177,0,0,0,114,30,1,0,0,114,36,1, - 0,0,114,2,0,0,0,41,2,114,177,0,0,0,114,30, - 1,0,0,114,4,0,0,0,218,9,112,97,116,104,95,104, - 111,111,107,198,5,0,0,115,4,0,0,0,0,10,14,6, - 122,20,70,105,108,101,70,105,110,100,101,114,46,112,97,116, - 104,95,104,111,111,107,99,1,0,0,0,0,0,0,0,1, - 0,0,0,3,0,0,0,67,0,0,0,115,12,0,0,0, - 100,1,160,0,124,0,106,1,161,1,83,0,41,2,78,122, - 16,70,105,108,101,70,105,110,100,101,114,40,123,33,114,125, - 41,41,2,114,51,0,0,0,114,35,0,0,0,41,1,114, - 107,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, - 0,0,0,114,5,1,0,0,216,5,0,0,115,2,0,0, - 0,0,1,122,19,70,105,108,101,70,105,110,100,101,114,46, - 95,95,114,101,112,114,95,95,41,1,78,41,15,114,112,0, - 0,0,114,111,0,0,0,114,113,0,0,0,114,114,0,0, - 0,114,191,0,0,0,114,11,1,0,0,114,130,0,0,0, - 114,188,0,0,0,114,124,0,0,0,114,23,1,0,0,114, - 187,0,0,0,114,31,1,0,0,114,189,0,0,0,114,37, - 1,0,0,114,5,1,0,0,114,2,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,24,1,0, - 0,73,5,0,0,115,20,0,0,0,8,7,4,2,8,14, - 8,4,4,2,8,12,8,5,10,48,8,31,12,18,114,24, - 1,0,0,99,4,0,0,0,0,0,0,0,6,0,0,0, - 8,0,0,0,67,0,0,0,115,146,0,0,0,124,0,160, - 0,100,1,161,1,125,4,124,0,160,0,100,2,161,1,125, - 5,124,4,115,66,124,5,114,36,124,5,106,1,125,4,110, - 30,124,2,124,3,107,2,114,56,116,2,124,1,124,2,131, - 2,125,4,110,10,116,3,124,1,124,2,131,2,125,4,124, - 5,115,84,116,4,124,1,124,2,124,4,100,3,141,3,125, - 5,122,36,124,5,124,0,100,2,60,0,124,4,124,0,100, - 1,60,0,124,2,124,0,100,4,60,0,124,3,124,0,100, - 5,60,0,87,0,110,20,4,0,116,5,107,10,114,140,1, - 0,1,0,1,0,89,0,110,2,88,0,100,0,83,0,41, - 6,78,218,10,95,95,108,111,97,100,101,114,95,95,218,8, - 95,95,115,112,101,99,95,95,41,1,114,127,0,0,0,90, - 8,95,95,102,105,108,101,95,95,90,10,95,95,99,97,99, - 104,101,100,95,95,41,6,218,3,103,101,116,114,127,0,0, - 0,114,239,0,0,0,114,234,0,0,0,114,174,0,0,0, - 218,9,69,120,99,101,112,116,105,111,110,41,6,90,2,110, - 115,114,105,0,0,0,90,8,112,97,116,104,110,97,109,101, - 90,9,99,112,97,116,104,110,97,109,101,114,127,0,0,0, - 114,171,0,0,0,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,218,14,95,102,105,120,95,117,112,95,109,111, - 100,117,108,101,222,5,0,0,115,34,0,0,0,0,2,10, - 1,10,1,4,1,4,1,8,1,8,1,12,2,10,1,4, - 1,14,1,2,1,8,1,8,1,8,1,12,1,14,2,114, - 42,1,0,0,99,0,0,0,0,0,0,0,0,3,0,0, - 0,3,0,0,0,67,0,0,0,115,38,0,0,0,116,0, - 116,1,160,2,161,0,102,2,125,0,116,3,116,4,102,2, - 125,1,116,5,116,6,102,2,125,2,124,0,124,1,124,2, - 103,3,83,0,41,1,122,95,82,101,116,117,114,110,115,32, - 97,32,108,105,115,116,32,111,102,32,102,105,108,101,45,98, - 97,115,101,100,32,109,111,100,117,108,101,32,108,111,97,100, - 101,114,115,46,10,10,32,32,32,32,69,97,99,104,32,105, - 116,101,109,32,105,115,32,97,32,116,117,112,108,101,32,40, - 108,111,97,100,101,114,44,32,115,117,102,102,105,120,101,115, - 41,46,10,32,32,32,32,41,7,114,240,0,0,0,114,148, - 0,0,0,218,18,101,120,116,101,110,115,105,111,110,95,115, - 117,102,102,105,120,101,115,114,234,0,0,0,114,92,0,0, - 0,114,239,0,0,0,114,79,0,0,0,41,3,90,10,101, - 120,116,101,110,115,105,111,110,115,90,6,115,111,117,114,99, - 101,90,8,98,121,116,101,99,111,100,101,114,2,0,0,0, - 114,2,0,0,0,114,4,0,0,0,114,168,0,0,0,245, - 5,0,0,115,8,0,0,0,0,5,12,1,8,1,8,1, - 114,168,0,0,0,99,1,0,0,0,0,0,0,0,12,0, - 0,0,9,0,0,0,67,0,0,0,115,178,1,0,0,124, - 0,97,0,116,0,106,1,97,1,116,0,106,2,97,2,116, - 1,106,3,116,4,25,0,125,1,100,1,68,0,93,48,125, - 2,124,2,116,1,106,3,107,7,114,56,116,0,160,5,124, - 2,161,1,125,3,110,10,116,1,106,3,124,2,25,0,125, - 3,116,6,124,1,124,2,124,3,131,3,1,0,113,30,100, - 2,100,3,103,1,102,2,100,4,100,5,100,3,103,2,102, - 2,102,2,125,4,124,4,68,0,93,110,92,2,125,5,125, - 6,116,7,100,6,100,7,132,0,124,6,68,0,131,1,131, - 1,115,136,116,8,130,1,124,6,100,8,25,0,125,7,124, - 5,116,1,106,3,107,6,114,170,116,1,106,3,124,5,25, - 0,125,8,1,0,113,226,113,106,122,20,116,0,160,5,124, - 5,161,1,125,8,87,0,1,0,113,226,87,0,113,106,4, - 0,116,9,107,10,114,214,1,0,1,0,1,0,89,0,113, - 106,89,0,113,106,88,0,113,106,116,9,100,9,131,1,130, - 1,116,6,124,1,100,10,124,8,131,3,1,0,116,6,124, - 1,100,11,124,7,131,3,1,0,116,6,124,1,100,12,100, - 13,160,10,124,6,161,1,131,3,1,0,116,6,124,1,100, - 14,100,15,100,16,132,0,124,6,68,0,131,1,131,3,1, - 0,116,0,160,5,100,17,161,1,125,9,116,6,124,1,100, - 17,124,9,131,3,1,0,116,0,160,5,100,18,161,1,125, - 10,116,6,124,1,100,18,124,10,131,3,1,0,124,5,100, - 4,107,2,144,1,114,110,116,0,160,5,100,19,161,1,125, - 11,116,6,124,1,100,20,124,11,131,3,1,0,116,6,124, - 1,100,21,116,11,131,0,131,3,1,0,116,12,160,13,116, - 2,160,14,161,0,161,1,1,0,124,5,100,4,107,2,144, - 1,114,174,116,15,160,16,100,22,161,1,1,0,100,23,116, - 12,107,6,144,1,114,174,100,24,116,17,95,18,100,25,83, - 0,41,26,122,205,83,101,116,117,112,32,116,104,101,32,112, - 97,116,104,45,98,97,115,101,100,32,105,109,112,111,114,116, - 101,114,115,32,102,111,114,32,105,109,112,111,114,116,108,105, - 98,32,98,121,32,105,109,112,111,114,116,105,110,103,32,110, - 101,101,100,101,100,10,32,32,32,32,98,117,105,108,116,45, - 105,110,32,109,111,100,117,108,101,115,32,97,110,100,32,105, - 110,106,101,99,116,105,110,103,32,116,104,101,109,32,105,110, - 116,111,32,116,104,101,32,103,108,111,98,97,108,32,110,97, - 109,101,115,112,97,99,101,46,10,10,32,32,32,32,79,116, - 104,101,114,32,99,111,109,112,111,110,101,110,116,115,32,97, - 114,101,32,101,120,116,114,97,99,116,101,100,32,102,114,111, - 109,32,116,104,101,32,99,111,114,101,32,98,111,111,116,115, - 116,114,97,112,32,109,111,100,117,108,101,46,10,10,32,32, - 32,32,41,4,114,53,0,0,0,114,65,0,0,0,218,8, - 98,117,105,108,116,105,110,115,114,145,0,0,0,90,5,112, - 111,115,105,120,250,1,47,90,2,110,116,250,1,92,99,1, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,115, - 0,0,0,115,26,0,0,0,124,0,93,18,125,1,116,0, - 124,1,131,1,100,0,107,2,86,0,1,0,113,2,100,1, - 83,0,41,2,114,29,0,0,0,78,41,1,114,31,0,0, - 0,41,2,114,22,0,0,0,114,85,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,243,0,0, - 0,25,6,0,0,115,2,0,0,0,4,0,122,25,95,115, - 101,116,117,112,46,60,108,111,99,97,108,115,62,46,60,103, - 101,110,101,120,112,114,62,114,63,0,0,0,122,30,105,109, - 112,111,114,116,108,105,98,32,114,101,113,117,105,114,101,115, - 32,112,111,115,105,120,32,111,114,32,110,116,114,1,0,0, - 0,114,25,0,0,0,114,21,0,0,0,114,30,0,0,0, - 114,48,0,0,0,99,1,0,0,0,0,0,0,0,2,0, - 0,0,4,0,0,0,83,0,0,0,115,22,0,0,0,104, - 0,124,0,93,14,125,1,100,0,124,1,155,0,157,2,146, - 2,113,4,83,0,41,1,114,64,0,0,0,114,2,0,0, - 0,41,2,114,22,0,0,0,218,1,115,114,2,0,0,0, - 114,2,0,0,0,114,4,0,0,0,114,32,1,0,0,41, - 6,0,0,115,2,0,0,0,6,0,122,25,95,115,101,116, - 117,112,46,60,108,111,99,97,108,115,62,46,60,115,101,116, - 99,111,109,112,62,90,7,95,116,104,114,101,97,100,90,8, - 95,119,101,97,107,114,101,102,90,6,119,105,110,114,101,103, - 114,176,0,0,0,114,5,0,0,0,122,4,46,112,121,119, - 122,6,95,100,46,112,121,100,84,78,41,19,114,121,0,0, - 0,114,6,0,0,0,114,148,0,0,0,114,255,0,0,0, - 114,112,0,0,0,90,18,95,98,117,105,108,116,105,110,95, - 102,114,111,109,95,110,97,109,101,114,116,0,0,0,218,3, - 97,108,108,114,156,0,0,0,114,106,0,0,0,114,26,0, - 0,0,114,11,0,0,0,114,245,0,0,0,114,152,0,0, - 0,114,43,1,0,0,114,92,0,0,0,114,170,0,0,0, - 114,175,0,0,0,114,179,0,0,0,41,12,218,17,95,98, - 111,111,116,115,116,114,97,112,95,109,111,100,117,108,101,90, - 11,115,101,108,102,95,109,111,100,117,108,101,90,12,98,117, - 105,108,116,105,110,95,110,97,109,101,90,14,98,117,105,108, - 116,105,110,95,109,111,100,117,108,101,90,10,111,115,95,100, - 101,116,97,105,108,115,90,10,98,117,105,108,116,105,110,95, - 111,115,114,21,0,0,0,114,25,0,0,0,90,9,111,115, - 95,109,111,100,117,108,101,90,13,116,104,114,101,97,100,95, - 109,111,100,117,108,101,90,14,119,101,97,107,114,101,102,95, - 109,111,100,117,108,101,90,13,119,105,110,114,101,103,95,109, - 111,100,117,108,101,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,218,6,95,115,101,116,117,112,0,6,0,0, - 115,78,0,0,0,0,8,4,1,6,1,6,3,10,1,8, - 1,10,1,12,2,10,1,14,3,22,1,12,2,22,1,8, - 1,10,1,10,1,6,2,2,1,10,1,10,1,14,1,12, - 2,8,1,12,1,12,1,18,1,22,3,10,1,12,3,10, - 1,12,3,10,1,10,1,12,3,14,1,14,1,10,1,10, - 1,10,1,114,50,1,0,0,99,1,0,0,0,0,0,0, - 0,2,0,0,0,4,0,0,0,67,0,0,0,115,50,0, - 0,0,116,0,124,0,131,1,1,0,116,1,131,0,125,1, - 116,2,106,3,160,4,116,5,106,6,124,1,142,0,103,1, - 161,1,1,0,116,2,106,7,160,8,116,9,161,1,1,0, - 100,1,83,0,41,2,122,41,73,110,115,116,97,108,108,32, - 116,104,101,32,112,97,116,104,45,98,97,115,101,100,32,105, - 109,112,111,114,116,32,99,111,109,112,111,110,101,110,116,115, - 46,78,41,10,114,50,1,0,0,114,168,0,0,0,114,6, - 0,0,0,114,16,1,0,0,114,152,0,0,0,114,24,1, - 0,0,114,37,1,0,0,218,9,109,101,116,97,95,112,97, - 116,104,114,170,0,0,0,114,10,1,0,0,41,2,114,49, - 1,0,0,90,17,115,117,112,112,111,114,116,101,100,95,108, - 111,97,100,101,114,115,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,218,8,95,105,110,115,116,97,108,108,65, - 6,0,0,115,8,0,0,0,0,2,8,1,6,1,20,1, - 114,52,1,0,0,41,1,114,50,0,0,0,41,1,78,41, - 3,78,78,78,41,2,114,63,0,0,0,114,63,0,0,0, - 41,1,84,41,1,78,41,1,78,41,62,114,114,0,0,0, - 114,10,0,0,0,90,37,95,67,65,83,69,95,73,78,83, - 69,78,83,73,84,73,86,69,95,80,76,65,84,70,79,82, - 77,83,95,66,89,84,69,83,95,75,69,89,114,9,0,0, - 0,114,11,0,0,0,114,17,0,0,0,114,19,0,0,0, - 114,28,0,0,0,114,38,0,0,0,114,39,0,0,0,114, - 43,0,0,0,114,44,0,0,0,114,46,0,0,0,114,49, - 0,0,0,114,59,0,0,0,218,4,116,121,112,101,218,8, - 95,95,99,111,100,101,95,95,114,147,0,0,0,114,15,0, - 0,0,114,134,0,0,0,114,14,0,0,0,114,18,0,0, - 0,114,214,0,0,0,114,82,0,0,0,114,78,0,0,0, - 114,92,0,0,0,114,79,0,0,0,90,23,68,69,66,85, - 71,95,66,89,84,69,67,79,68,69,95,83,85,70,70,73, - 88,69,83,90,27,79,80,84,73,77,73,90,69,68,95,66, + 22,1,0,0,114,5,0,0,115,8,0,0,0,0,1,10, + 1,8,1,2,255,122,20,70,105,108,101,70,105,110,100,101, + 114,46,95,103,101,116,95,115,112,101,99,78,99,3,0,0, + 0,0,0,0,0,14,0,0,0,8,0,0,0,67,0,0, + 0,115,102,1,0,0,100,1,125,3,124,1,160,0,100,2, + 161,1,100,3,25,0,125,4,122,24,116,1,124,0,106,2, + 112,34,116,3,160,4,161,0,131,1,106,5,125,5,87,0, + 110,24,4,0,116,6,107,10,114,66,1,0,1,0,1,0, + 100,4,125,5,89,0,110,2,88,0,124,5,124,0,106,7, + 107,3,114,92,124,0,160,8,161,0,1,0,124,5,124,0, + 95,7,116,9,131,0,114,114,124,0,106,10,125,6,124,4, + 160,11,161,0,125,7,110,10,124,0,106,12,125,6,124,4, + 125,7,124,7,124,6,107,6,114,218,116,13,124,0,106,2, + 124,4,131,2,125,8,124,0,106,14,68,0,93,58,92,2, + 125,9,125,10,100,5,124,9,23,0,125,11,116,13,124,8, + 124,11,131,2,125,12,116,15,124,12,131,1,114,208,124,0, + 160,16,124,10,124,1,124,12,124,8,103,1,124,2,161,5, + 2,0,1,0,83,0,113,150,116,17,124,8,131,1,125,3, + 124,0,106,14,68,0,93,86,92,2,125,9,125,10,116,13, + 124,0,106,2,124,4,124,9,23,0,131,2,125,12,116,18, + 106,19,100,6,124,12,100,3,100,7,141,3,1,0,124,7, + 124,9,23,0,124,6,107,6,144,1,114,54,116,15,124,12, + 131,1,144,1,114,54,124,0,160,16,124,10,124,1,124,12, + 100,8,124,2,161,5,2,0,1,0,83,0,113,224,124,3, + 144,1,114,98,116,18,160,19,100,9,124,8,161,2,1,0, + 116,18,160,20,124,1,100,8,161,2,125,13,124,8,103,1, + 124,13,95,21,124,13,83,0,100,8,83,0,41,10,122,111, + 84,114,121,32,116,111,32,102,105,110,100,32,97,32,115,112, + 101,99,32,102,111,114,32,116,104,101,32,115,112,101,99,105, + 102,105,101,100,32,109,111,100,117,108,101,46,10,10,32,32, + 32,32,32,32,32,32,82,101,116,117,114,110,115,32,116,104, + 101,32,109,97,116,99,104,105,110,103,32,115,112,101,99,44, + 32,111,114,32,78,111,110,101,32,105,102,32,110,111,116,32, + 102,111,117,110,100,46,10,32,32,32,32,32,32,32,32,70, + 114,61,0,0,0,114,59,0,0,0,114,94,0,0,0,114, + 190,0,0,0,122,9,116,114,121,105,110,103,32,123,125,41, + 1,90,9,118,101,114,98,111,115,105,116,121,78,122,25,112, + 111,115,115,105,98,108,101,32,110,97,109,101,115,112,97,99, + 101,32,102,111,114,32,123,125,41,22,114,32,0,0,0,114, + 39,0,0,0,114,35,0,0,0,114,1,0,0,0,114,45, + 0,0,0,114,234,0,0,0,114,40,0,0,0,114,25,1, + 0,0,218,11,95,102,105,108,108,95,99,97,99,104,101,114, + 5,0,0,0,114,28,1,0,0,114,95,0,0,0,114,27, + 1,0,0,114,28,0,0,0,114,24,1,0,0,114,44,0, + 0,0,114,22,1,0,0,114,46,0,0,0,114,120,0,0, + 0,114,134,0,0,0,114,166,0,0,0,114,162,0,0,0, + 41,14,114,106,0,0,0,114,125,0,0,0,114,185,0,0, + 0,90,12,105,115,95,110,97,109,101,115,112,97,99,101,90, + 11,116,97,105,108,95,109,111,100,117,108,101,114,153,0,0, + 0,90,5,99,97,99,104,101,90,12,99,97,99,104,101,95, + 109,111,100,117,108,101,90,9,98,97,115,101,95,112,97,116, + 104,114,240,0,0,0,114,171,0,0,0,90,13,105,110,105, + 116,95,102,105,108,101,110,97,109,101,90,9,102,117,108,108, + 95,112,97,116,104,114,170,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,186,0,0,0,119,5, + 0,0,115,74,0,0,0,0,5,4,1,14,1,2,1,24, + 1,14,1,10,1,10,1,8,1,6,2,6,1,6,1,10, + 2,6,1,4,2,8,1,12,1,14,1,8,1,10,1,8, + 1,26,4,8,2,14,1,16,1,16,1,14,1,10,1,10, + 1,2,0,2,255,10,2,6,1,12,1,12,1,8,1,4, + 1,122,20,70,105,108,101,70,105,110,100,101,114,46,102,105, + 110,100,95,115,112,101,99,99,1,0,0,0,0,0,0,0, + 9,0,0,0,10,0,0,0,67,0,0,0,115,190,0,0, + 0,124,0,106,0,125,1,122,22,116,1,160,2,124,1,112, + 22,116,1,160,3,161,0,161,1,125,2,87,0,110,30,4, + 0,116,4,116,5,116,6,102,3,107,10,114,58,1,0,1, + 0,1,0,103,0,125,2,89,0,110,2,88,0,116,7,106, + 8,160,9,100,1,161,1,115,84,116,10,124,2,131,1,124, + 0,95,11,110,74,116,10,131,0,125,3,124,2,68,0,93, + 56,125,4,124,4,160,12,100,2,161,1,92,3,125,5,125, + 6,125,7,124,6,114,136,100,3,160,13,124,5,124,7,160, + 14,161,0,161,2,125,8,110,4,124,5,125,8,124,3,160, + 15,124,8,161,1,1,0,113,94,124,3,124,0,95,11,116, + 7,106,8,160,9,116,16,161,1,114,186,100,4,100,5,132, + 0,124,2,68,0,131,1,124,0,95,17,100,6,83,0,41, + 7,122,68,70,105,108,108,32,116,104,101,32,99,97,99,104, + 101,32,111,102,32,112,111,116,101,110,116,105,97,108,32,109, + 111,100,117,108,101,115,32,97,110,100,32,112,97,99,107,97, + 103,101,115,32,102,111,114,32,116,104,105,115,32,100,105,114, + 101,99,116,111,114,121,46,114,0,0,0,0,114,61,0,0, + 0,122,5,123,125,46,123,125,99,1,0,0,0,0,0,0, + 0,2,0,0,0,4,0,0,0,83,0,0,0,115,20,0, + 0,0,104,0,124,0,93,12,125,1,124,1,160,0,161,0, + 146,2,113,4,83,0,114,2,0,0,0,41,1,114,95,0, + 0,0,41,2,114,22,0,0,0,90,2,102,110,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,218,9,60,115, + 101,116,99,111,109,112,62,196,5,0,0,115,4,0,0,0, + 6,0,2,0,122,41,70,105,108,101,70,105,110,100,101,114, + 46,95,102,105,108,108,95,99,97,99,104,101,46,60,108,111, + 99,97,108,115,62,46,60,115,101,116,99,111,109,112,62,78, + 41,18,114,35,0,0,0,114,1,0,0,0,114,231,0,0, + 0,114,45,0,0,0,114,228,0,0,0,218,15,80,101,114, + 109,105,115,115,105,111,110,69,114,114,111,114,218,18,78,111, + 116,65,68,105,114,101,99,116,111,114,121,69,114,114,111,114, + 114,6,0,0,0,114,7,0,0,0,114,8,0,0,0,114, + 26,1,0,0,114,27,1,0,0,114,90,0,0,0,114,50, + 0,0,0,114,95,0,0,0,218,3,97,100,100,114,9,0, + 0,0,114,28,1,0,0,41,9,114,106,0,0,0,114,35, + 0,0,0,114,232,0,0,0,90,21,108,111,119,101,114,95, + 115,117,102,102,105,120,95,99,111,110,116,101,110,116,115,114, + 5,1,0,0,114,104,0,0,0,114,252,0,0,0,114,240, + 0,0,0,90,8,110,101,119,95,110,97,109,101,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,114,30,1,0, + 0,167,5,0,0,115,34,0,0,0,0,2,6,1,2,1, + 22,1,20,3,10,3,12,1,12,7,6,1,8,1,16,1, + 4,1,18,2,4,1,12,1,6,1,12,1,122,22,70,105, + 108,101,70,105,110,100,101,114,46,95,102,105,108,108,95,99, + 97,99,104,101,99,1,0,0,0,0,0,0,0,3,0,0, + 0,3,0,0,0,7,0,0,0,115,18,0,0,0,135,0, + 135,1,102,2,100,1,100,2,132,8,125,2,124,2,83,0, + 41,3,97,20,1,0,0,65,32,99,108,97,115,115,32,109, + 101,116,104,111,100,32,119,104,105,99,104,32,114,101,116,117, + 114,110,115,32,97,32,99,108,111,115,117,114,101,32,116,111, + 32,117,115,101,32,111,110,32,115,121,115,46,112,97,116,104, + 95,104,111,111,107,10,32,32,32,32,32,32,32,32,119,104, + 105,99,104,32,119,105,108,108,32,114,101,116,117,114,110,32, + 97,110,32,105,110,115,116,97,110,99,101,32,117,115,105,110, + 103,32,116,104,101,32,115,112,101,99,105,102,105,101,100,32, + 108,111,97,100,101,114,115,32,97,110,100,32,116,104,101,32, + 112,97,116,104,10,32,32,32,32,32,32,32,32,99,97,108, + 108,101,100,32,111,110,32,116,104,101,32,99,108,111,115,117, + 114,101,46,10,10,32,32,32,32,32,32,32,32,73,102,32, + 116,104,101,32,112,97,116,104,32,99,97,108,108,101,100,32, + 111,110,32,116,104,101,32,99,108,111,115,117,114,101,32,105, + 115,32,110,111,116,32,97,32,100,105,114,101,99,116,111,114, + 121,44,32,73,109,112,111,114,116,69,114,114,111,114,32,105, + 115,10,32,32,32,32,32,32,32,32,114,97,105,115,101,100, + 46,10,10,32,32,32,32,32,32,32,32,99,1,0,0,0, + 0,0,0,0,1,0,0,0,4,0,0,0,19,0,0,0, + 115,34,0,0,0,116,0,124,0,131,1,115,20,116,1,100, + 1,124,0,100,2,141,2,130,1,136,0,124,0,102,1,136, + 1,158,2,142,0,83,0,41,3,122,45,80,97,116,104,32, + 104,111,111,107,32,102,111,114,32,105,109,112,111,114,116,108, + 105,98,46,109,97,99,104,105,110,101,114,121,46,70,105,108, + 101,70,105,110,100,101,114,46,122,30,111,110,108,121,32,100, + 105,114,101,99,116,111,114,105,101,115,32,97,114,101,32,115, + 117,112,112,111,114,116,101,100,41,1,114,35,0,0,0,41, + 2,114,46,0,0,0,114,105,0,0,0,41,1,114,35,0, + 0,0,41,2,114,176,0,0,0,114,29,1,0,0,114,2, + 0,0,0,114,4,0,0,0,218,24,112,97,116,104,95,104, + 111,111,107,95,102,111,114,95,70,105,108,101,70,105,110,100, + 101,114,208,5,0,0,115,6,0,0,0,0,2,8,1,12, + 1,122,54,70,105,108,101,70,105,110,100,101,114,46,112,97, + 116,104,95,104,111,111,107,46,60,108,111,99,97,108,115,62, + 46,112,97,116,104,95,104,111,111,107,95,102,111,114,95,70, + 105,108,101,70,105,110,100,101,114,114,2,0,0,0,41,3, + 114,176,0,0,0,114,29,1,0,0,114,35,1,0,0,114, + 2,0,0,0,41,2,114,176,0,0,0,114,29,1,0,0, + 114,4,0,0,0,218,9,112,97,116,104,95,104,111,111,107, + 198,5,0,0,115,4,0,0,0,0,10,14,6,122,20,70, + 105,108,101,70,105,110,100,101,114,46,112,97,116,104,95,104, + 111,111,107,99,1,0,0,0,0,0,0,0,1,0,0,0, + 3,0,0,0,67,0,0,0,115,12,0,0,0,100,1,160, + 0,124,0,106,1,161,1,83,0,41,2,78,122,16,70,105, + 108,101,70,105,110,100,101,114,40,123,33,114,125,41,41,2, + 114,50,0,0,0,114,35,0,0,0,41,1,114,106,0,0, + 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 114,4,1,0,0,216,5,0,0,115,2,0,0,0,0,1, + 122,19,70,105,108,101,70,105,110,100,101,114,46,95,95,114, + 101,112,114,95,95,41,1,78,41,15,114,111,0,0,0,114, + 110,0,0,0,114,112,0,0,0,114,113,0,0,0,114,190, + 0,0,0,114,10,1,0,0,114,129,0,0,0,114,187,0, + 0,0,114,123,0,0,0,114,22,1,0,0,114,186,0,0, + 0,114,30,1,0,0,114,188,0,0,0,114,36,1,0,0, + 114,4,1,0,0,114,2,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,23,1,0,0,73,5, + 0,0,115,20,0,0,0,8,7,4,2,8,14,8,4,4, + 2,8,12,8,5,10,48,8,31,12,18,114,23,1,0,0, + 99,4,0,0,0,0,0,0,0,6,0,0,0,8,0,0, + 0,67,0,0,0,115,146,0,0,0,124,0,160,0,100,1, + 161,1,125,4,124,0,160,0,100,2,161,1,125,5,124,4, + 115,66,124,5,114,36,124,5,106,1,125,4,110,30,124,2, + 124,3,107,2,114,56,116,2,124,1,124,2,131,2,125,4, + 110,10,116,3,124,1,124,2,131,2,125,4,124,5,115,84, + 116,4,124,1,124,2,124,4,100,3,141,3,125,5,122,36, + 124,5,124,0,100,2,60,0,124,4,124,0,100,1,60,0, + 124,2,124,0,100,4,60,0,124,3,124,0,100,5,60,0, + 87,0,110,20,4,0,116,5,107,10,114,140,1,0,1,0, + 1,0,89,0,110,2,88,0,100,0,83,0,41,6,78,218, + 10,95,95,108,111,97,100,101,114,95,95,218,8,95,95,115, + 112,101,99,95,95,41,1,114,126,0,0,0,90,8,95,95, + 102,105,108,101,95,95,90,10,95,95,99,97,99,104,101,100, + 95,95,41,6,218,3,103,101,116,114,126,0,0,0,114,238, + 0,0,0,114,233,0,0,0,114,173,0,0,0,218,9,69, + 120,99,101,112,116,105,111,110,41,6,90,2,110,115,114,104, + 0,0,0,90,8,112,97,116,104,110,97,109,101,90,9,99, + 112,97,116,104,110,97,109,101,114,126,0,0,0,114,170,0, + 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, + 0,218,14,95,102,105,120,95,117,112,95,109,111,100,117,108, + 101,222,5,0,0,115,34,0,0,0,0,2,10,1,10,1, + 4,1,4,1,8,1,8,1,12,2,10,1,4,1,14,1, + 2,1,8,1,8,1,8,1,12,1,14,2,114,41,1,0, + 0,99,0,0,0,0,0,0,0,0,3,0,0,0,3,0, + 0,0,67,0,0,0,115,38,0,0,0,116,0,116,1,160, + 2,161,0,102,2,125,0,116,3,116,4,102,2,125,1,116, + 5,116,6,102,2,125,2,124,0,124,1,124,2,103,3,83, + 0,41,1,122,95,82,101,116,117,114,110,115,32,97,32,108, + 105,115,116,32,111,102,32,102,105,108,101,45,98,97,115,101, + 100,32,109,111,100,117,108,101,32,108,111,97,100,101,114,115, + 46,10,10,32,32,32,32,69,97,99,104,32,105,116,101,109, + 32,105,115,32,97,32,116,117,112,108,101,32,40,108,111,97, + 100,101,114,44,32,115,117,102,102,105,120,101,115,41,46,10, + 32,32,32,32,41,7,114,239,0,0,0,114,147,0,0,0, + 218,18,101,120,116,101,110,115,105,111,110,95,115,117,102,102, + 105,120,101,115,114,233,0,0,0,114,91,0,0,0,114,238, + 0,0,0,114,78,0,0,0,41,3,90,10,101,120,116,101, + 110,115,105,111,110,115,90,6,115,111,117,114,99,101,90,8, + 98,121,116,101,99,111,100,101,114,2,0,0,0,114,2,0, + 0,0,114,4,0,0,0,114,167,0,0,0,245,5,0,0, + 115,8,0,0,0,0,5,12,1,8,1,8,1,114,167,0, + 0,0,99,1,0,0,0,0,0,0,0,12,0,0,0,9, + 0,0,0,67,0,0,0,115,178,1,0,0,124,0,97,0, + 116,0,106,1,97,1,116,0,106,2,97,2,116,1,106,3, + 116,4,25,0,125,1,100,1,68,0,93,48,125,2,124,2, + 116,1,106,3,107,7,114,56,116,0,160,5,124,2,161,1, + 125,3,110,10,116,1,106,3,124,2,25,0,125,3,116,6, + 124,1,124,2,124,3,131,3,1,0,113,30,100,2,100,3, + 103,1,102,2,100,4,100,5,100,3,103,2,102,2,102,2, + 125,4,124,4,68,0,93,110,92,2,125,5,125,6,116,7, + 100,6,100,7,132,0,124,6,68,0,131,1,131,1,115,136, + 116,8,130,1,124,6,100,8,25,0,125,7,124,5,116,1, + 106,3,107,6,114,170,116,1,106,3,124,5,25,0,125,8, + 1,0,113,226,113,106,122,20,116,0,160,5,124,5,161,1, + 125,8,87,0,1,0,113,226,87,0,113,106,4,0,116,9, + 107,10,114,214,1,0,1,0,1,0,89,0,113,106,89,0, + 113,106,88,0,113,106,116,9,100,9,131,1,130,1,116,6, + 124,1,100,10,124,8,131,3,1,0,116,6,124,1,100,11, + 124,7,131,3,1,0,116,6,124,1,100,12,100,13,160,10, + 124,6,161,1,131,3,1,0,116,6,124,1,100,14,100,15, + 100,16,132,0,124,6,68,0,131,1,131,3,1,0,116,0, + 160,5,100,17,161,1,125,9,116,6,124,1,100,17,124,9, + 131,3,1,0,116,0,160,5,100,18,161,1,125,10,116,6, + 124,1,100,18,124,10,131,3,1,0,124,5,100,4,107,2, + 144,1,114,110,116,0,160,5,100,19,161,1,125,11,116,6, + 124,1,100,20,124,11,131,3,1,0,116,6,124,1,100,21, + 116,11,131,0,131,3,1,0,116,12,160,13,116,2,160,14, + 161,0,161,1,1,0,124,5,100,4,107,2,144,1,114,174, + 116,15,160,16,100,22,161,1,1,0,100,23,116,12,107,6, + 144,1,114,174,100,24,116,17,95,18,100,25,83,0,41,26, + 122,205,83,101,116,117,112,32,116,104,101,32,112,97,116,104, + 45,98,97,115,101,100,32,105,109,112,111,114,116,101,114,115, + 32,102,111,114,32,105,109,112,111,114,116,108,105,98,32,98, + 121,32,105,109,112,111,114,116,105,110,103,32,110,101,101,100, + 101,100,10,32,32,32,32,98,117,105,108,116,45,105,110,32, + 109,111,100,117,108,101,115,32,97,110,100,32,105,110,106,101, + 99,116,105,110,103,32,116,104,101,109,32,105,110,116,111,32, + 116,104,101,32,103,108,111,98,97,108,32,110,97,109,101,115, + 112,97,99,101,46,10,10,32,32,32,32,79,116,104,101,114, + 32,99,111,109,112,111,110,101,110,116,115,32,97,114,101,32, + 101,120,116,114,97,99,116,101,100,32,102,114,111,109,32,116, + 104,101,32,99,111,114,101,32,98,111,111,116,115,116,114,97, + 112,32,109,111,100,117,108,101,46,10,10,32,32,32,32,41, + 4,114,52,0,0,0,114,64,0,0,0,218,8,98,117,105, + 108,116,105,110,115,114,144,0,0,0,90,5,112,111,115,105, + 120,250,1,47,90,2,110,116,250,1,92,99,1,0,0,0, + 0,0,0,0,2,0,0,0,3,0,0,0,115,0,0,0, + 115,26,0,0,0,124,0,93,18,125,1,116,0,124,1,131, + 1,100,0,107,2,86,0,1,0,113,2,100,1,83,0,41, + 2,114,29,0,0,0,78,41,1,114,31,0,0,0,41,2, + 114,22,0,0,0,114,84,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,242,0,0,0,25,6, + 0,0,115,4,0,0,0,4,0,2,0,122,25,95,115,101, + 116,117,112,46,60,108,111,99,97,108,115,62,46,60,103,101, + 110,101,120,112,114,62,114,62,0,0,0,122,30,105,109,112, + 111,114,116,108,105,98,32,114,101,113,117,105,114,101,115,32, + 112,111,115,105,120,32,111,114,32,110,116,114,1,0,0,0, + 114,25,0,0,0,114,21,0,0,0,114,30,0,0,0,114, + 48,0,0,0,99,1,0,0,0,0,0,0,0,2,0,0, + 0,4,0,0,0,83,0,0,0,115,22,0,0,0,104,0, + 124,0,93,14,125,1,100,0,124,1,155,0,157,2,146,2, + 113,4,83,0,41,1,114,63,0,0,0,114,2,0,0,0, + 41,2,114,22,0,0,0,218,1,115,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,31,1,0,0,41,6, + 0,0,115,4,0,0,0,6,0,2,0,122,25,95,115,101, + 116,117,112,46,60,108,111,99,97,108,115,62,46,60,115,101, + 116,99,111,109,112,62,90,7,95,116,104,114,101,97,100,90, + 8,95,119,101,97,107,114,101,102,90,6,119,105,110,114,101, + 103,114,175,0,0,0,114,5,0,0,0,122,4,46,112,121, + 119,122,6,95,100,46,112,121,100,84,78,41,19,114,120,0, + 0,0,114,6,0,0,0,114,147,0,0,0,114,254,0,0, + 0,114,111,0,0,0,90,18,95,98,117,105,108,116,105,110, + 95,102,114,111,109,95,110,97,109,101,114,115,0,0,0,218, + 3,97,108,108,114,155,0,0,0,114,105,0,0,0,114,26, + 0,0,0,114,11,0,0,0,114,244,0,0,0,114,151,0, + 0,0,114,42,1,0,0,114,91,0,0,0,114,169,0,0, + 0,114,174,0,0,0,114,178,0,0,0,41,12,218,17,95, + 98,111,111,116,115,116,114,97,112,95,109,111,100,117,108,101, + 90,11,115,101,108,102,95,109,111,100,117,108,101,90,12,98, + 117,105,108,116,105,110,95,110,97,109,101,90,14,98,117,105, + 108,116,105,110,95,109,111,100,117,108,101,90,10,111,115,95, + 100,101,116,97,105,108,115,90,10,98,117,105,108,116,105,110, + 95,111,115,114,21,0,0,0,114,25,0,0,0,90,9,111, + 115,95,109,111,100,117,108,101,90,13,116,104,114,101,97,100, + 95,109,111,100,117,108,101,90,14,119,101,97,107,114,101,102, + 95,109,111,100,117,108,101,90,13,119,105,110,114,101,103,95, + 109,111,100,117,108,101,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,218,6,95,115,101,116,117,112,0,6,0, + 0,115,78,0,0,0,0,8,4,1,6,1,6,3,10,1, + 8,1,10,1,12,2,10,1,14,3,22,1,12,2,22,1, + 8,1,10,1,10,1,6,2,2,1,10,1,10,1,14,1, + 12,2,8,1,12,1,12,1,18,1,22,3,10,1,12,3, + 10,1,12,3,10,1,10,1,12,3,14,1,14,1,10,1, + 10,1,10,1,114,49,1,0,0,99,1,0,0,0,0,0, + 0,0,2,0,0,0,4,0,0,0,67,0,0,0,115,50, + 0,0,0,116,0,124,0,131,1,1,0,116,1,131,0,125, + 1,116,2,106,3,160,4,116,5,106,6,124,1,142,0,103, + 1,161,1,1,0,116,2,106,7,160,8,116,9,161,1,1, + 0,100,1,83,0,41,2,122,41,73,110,115,116,97,108,108, + 32,116,104,101,32,112,97,116,104,45,98,97,115,101,100,32, + 105,109,112,111,114,116,32,99,111,109,112,111,110,101,110,116, + 115,46,78,41,10,114,49,1,0,0,114,167,0,0,0,114, + 6,0,0,0,114,15,1,0,0,114,151,0,0,0,114,23, + 1,0,0,114,36,1,0,0,218,9,109,101,116,97,95,112, + 97,116,104,114,169,0,0,0,114,9,1,0,0,41,2,114, + 48,1,0,0,90,17,115,117,112,112,111,114,116,101,100,95, + 108,111,97,100,101,114,115,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,8,95,105,110,115,116,97,108,108, + 65,6,0,0,115,8,0,0,0,0,2,8,1,6,1,20, + 1,114,51,1,0,0,41,62,114,113,0,0,0,114,10,0, + 0,0,90,37,95,67,65,83,69,95,73,78,83,69,78,83, + 73,84,73,86,69,95,80,76,65,84,70,79,82,77,83,95, + 66,89,84,69,83,95,75,69,89,114,9,0,0,0,114,11, + 0,0,0,114,17,0,0,0,114,19,0,0,0,114,28,0, + 0,0,114,38,0,0,0,114,39,0,0,0,114,43,0,0, + 0,114,44,0,0,0,114,46,0,0,0,114,49,0,0,0, + 114,58,0,0,0,218,4,116,121,112,101,218,8,95,95,99, + 111,100,101,95,95,114,146,0,0,0,114,15,0,0,0,114, + 133,0,0,0,114,14,0,0,0,114,18,0,0,0,114,213, + 0,0,0,114,81,0,0,0,114,77,0,0,0,114,91,0, + 0,0,114,78,0,0,0,90,23,68,69,66,85,71,95,66, 89,84,69,67,79,68,69,95,83,85,70,70,73,88,69,83, - 114,88,0,0,0,114,93,0,0,0,114,99,0,0,0,114, - 102,0,0,0,114,104,0,0,0,114,123,0,0,0,114,130, - 0,0,0,114,138,0,0,0,114,142,0,0,0,114,144,0, - 0,0,114,150,0,0,0,114,155,0,0,0,114,157,0,0, - 0,114,162,0,0,0,218,6,111,98,106,101,99,116,114,169, - 0,0,0,114,174,0,0,0,114,175,0,0,0,114,190,0, - 0,0,114,200,0,0,0,114,217,0,0,0,114,234,0,0, - 0,114,239,0,0,0,114,245,0,0,0,114,240,0,0,0, - 114,246,0,0,0,114,8,1,0,0,114,10,1,0,0,114, - 24,1,0,0,114,42,1,0,0,114,168,0,0,0,114,50, - 1,0,0,114,52,1,0,0,114,2,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,218,8,60,109, - 111,100,117,108,101,62,8,0,0,0,115,120,0,0,0,4, - 15,4,1,4,1,2,1,6,3,8,17,8,5,8,5,8, - 6,8,12,8,10,8,9,8,5,8,7,8,9,10,22,10, - 127,0,7,16,1,12,2,4,1,4,2,6,2,6,2,8, - 2,16,71,8,40,8,19,8,12,8,12,8,28,8,17,8, - 33,8,28,8,24,10,13,10,10,10,11,8,14,6,3,4, - 1,14,67,14,64,14,29,16,127,0,17,14,68,18,45,18, - 26,4,3,18,53,14,60,14,42,14,127,0,7,14,127,0, - 22,10,23,8,11,8,65, + 90,27,79,80,84,73,77,73,90,69,68,95,66,89,84,69, + 67,79,68,69,95,83,85,70,70,73,88,69,83,114,87,0, + 0,0,114,92,0,0,0,114,98,0,0,0,114,101,0,0, + 0,114,103,0,0,0,114,122,0,0,0,114,129,0,0,0, + 114,137,0,0,0,114,141,0,0,0,114,143,0,0,0,114, + 149,0,0,0,114,154,0,0,0,114,156,0,0,0,114,161, + 0,0,0,218,6,111,98,106,101,99,116,114,168,0,0,0, + 114,173,0,0,0,114,174,0,0,0,114,189,0,0,0,114, + 199,0,0,0,114,216,0,0,0,114,233,0,0,0,114,238, + 0,0,0,114,244,0,0,0,114,239,0,0,0,114,245,0, + 0,0,114,7,1,0,0,114,9,1,0,0,114,23,1,0, + 0,114,41,1,0,0,114,167,0,0,0,114,49,1,0,0, + 114,51,1,0,0,114,2,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,218,8,60,109,111,100,117, + 108,101,62,8,0,0,0,115,124,0,0,0,4,15,4,1, + 4,1,2,1,2,255,4,4,8,17,8,5,8,5,8,6, + 8,12,8,10,8,9,8,5,8,7,8,9,12,22,10,127, + 0,7,16,1,12,2,4,1,4,2,6,2,6,2,8,2, + 18,71,8,40,8,19,8,12,8,12,8,28,8,17,8,33, + 8,28,8,24,16,13,14,10,12,11,8,14,6,3,6,1, + 2,255,12,68,14,64,14,29,16,127,0,17,14,68,18,45, + 18,26,4,3,18,53,14,60,14,42,14,127,0,7,14,127, + 0,22,12,23,8,11,8,65, }; From webhook-mailer at python.org Mon Sep 17 08:18:25 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 12:18:25 -0000 Subject: [Python-checkins] bpo-34710: fix SSL module build (GH-9347) Message-ID: https://github.com/python/cpython/commit/e2c0aea670d603b187733606a4601d77f355bc47 commit: e2c0aea670d603b187733606a4601d77f355bc47 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T05:18:23-07:00 summary: bpo-34710: fix SSL module build (GH-9347) Include ``openssl/dh.h`` header file to fix implicit function declaration of ``DH_free()``. Signed-off-by: Alexandru Ardelean (cherry picked from commit b3a271fc0ce3e13e427be8914decfc205a220ca8) Co-authored-by: Alexandru Ardelean files: A Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst b/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst new file mode 100644 index 000000000000..b06289d091e5 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst @@ -0,0 +1 @@ +Fixed SSL module build with OpenSSL & pedantic CFLAGS. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index bf379f01a1d2..6af48ee0ff20 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -63,6 +63,7 @@ static PySocketModule_APIObject PySocketModule; #include "openssl/err.h" #include "openssl/rand.h" #include "openssl/bio.h" +#include "openssl/dh.h" #ifndef HAVE_X509_VERIFY_PARAM_SET1_HOST # ifdef LIBRESSL_VERSION_NUMBER From webhook-mailer at python.org Mon Sep 17 08:19:11 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 12:19:11 -0000 Subject: [Python-checkins] bpo-34710: fix SSL module build (GH-9347) Message-ID: https://github.com/python/cpython/commit/fff869e2a0ee3d2d65973d82e57f05479828f5a3 commit: fff869e2a0ee3d2d65973d82e57f05479828f5a3 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T05:19:08-07:00 summary: bpo-34710: fix SSL module build (GH-9347) Include ``openssl/dh.h`` header file to fix implicit function declaration of ``DH_free()``. Signed-off-by: Alexandru Ardelean (cherry picked from commit b3a271fc0ce3e13e427be8914decfc205a220ca8) Co-authored-by: Alexandru Ardelean files: A Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst b/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst new file mode 100644 index 000000000000..b06289d091e5 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst @@ -0,0 +1 @@ +Fixed SSL module build with OpenSSL & pedantic CFLAGS. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index e0c7dfbdaacb..99f24a5b37f2 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -78,6 +78,7 @@ static PySocketModule_APIObject PySocketModule; #include "openssl/err.h" #include "openssl/rand.h" #include "openssl/bio.h" +#include "openssl/dh.h" /* SSL error object */ static PyObject *PySSLErrorObject; From webhook-mailer at python.org Mon Sep 17 08:36:45 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Mon, 17 Sep 2018 12:36:45 -0000 Subject: [Python-checkins] bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683) Message-ID: https://github.com/python/cpython/commit/9bdb7be482aef8f60daa1d36606568a132dcb616 commit: 9bdb7be482aef8f60daa1d36606568a132dcb616 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-17T15:36:40+03:00 summary: bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683) files: A Misc/NEWS.d/next/Library/2018-08-06-11-01-18.bpo-34341.E0b9p2.rst M Lib/test/test_zipfile.py M Lib/zipfile.py diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index 4c6f57c07590..7b8922f4e054 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -770,6 +770,20 @@ def test_absolute_arcnames(self): with zipfile.ZipFile(TESTFN2, "r", zipfile.ZIP_STORED) as zipfp: self.assertEqual(zipfp.namelist(), ["absolute"]) + def test_append(self): + # Test that appending to the Zip64 archive doesn't change + # extra fields of existing entries. + with zipfile.ZipFile(TESTFN2, "w", allowZip64=True) as zipfp: + zipfp.writestr("strfile", self.data) + with zipfile.ZipFile(TESTFN2, "r", allowZip64=True) as zipfp: + zinfo = zipfp.getinfo("strfile") + extra = zinfo.extra + with zipfile.ZipFile(TESTFN2, "a", allowZip64=True) as zipfp: + zipfp.writestr("strfile2", self.data) + with zipfile.ZipFile(TESTFN2, "r", allowZip64=True) as zipfp: + zinfo = zipfp.getinfo("strfile") + self.assertEqual(zinfo.extra, extra) + @requires_zlib class DeflateTestZip64InSmallFiles(AbstractTestZip64InSmallFiles, unittest.TestCase): diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 7f237783773d..89df90b25209 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -159,6 +159,27 @@ class LargeZipFile(Exception): _CD64_DIRECTORY_SIZE = 8 _CD64_OFFSET_START_CENTDIR = 9 +_EXTRA_FIELD_STRUCT = struct.Struct(' https://github.com/python/cpython/commit/12a69db9080d43b2d1a07527aa1e83c8b59e5a6d commit: 12a69db9080d43b2d1a07527aa1e83c8b59e5a6d branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-17T15:38:27+03:00 summary: Convert os.readlink() to Argument Clinic. (GH-8778) Also convert os.get_blocking() and os.set_blocking(). files: M Modules/clinic/posixmodule.c.h M Modules/posixmodule.c diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h index c3849a93c0c2..2c46d4bf172e 100644 --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -3186,6 +3186,50 @@ os_wait(PyObject *module, PyObject *Py_UNUSED(ignored)) #endif /* defined(HAVE_WAIT) */ +#if (defined(HAVE_READLINK) || defined(MS_WINDOWS)) + +PyDoc_STRVAR(os_readlink__doc__, +"readlink($module, /, path, *, dir_fd=None)\n" +"--\n" +"\n" +"Return a string representing the path to which the symbolic link points.\n" +"\n" +"If dir_fd is not None, it should be a file descriptor open to a directory,\n" +"and path should be relative; path will then be relative to that directory.\n" +"\n" +"dir_fd may not be implemented on your platform. If it is unavailable,\n" +"using it will raise a NotImplementedError."); + +#define OS_READLINK_METHODDEF \ + {"readlink", (PyCFunction)os_readlink, METH_FASTCALL|METH_KEYWORDS, os_readlink__doc__}, + +static PyObject * +os_readlink_impl(PyObject *module, path_t *path, int dir_fd); + +static PyObject * +os_readlink(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"path", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&|$O&:readlink", _keywords, 0}; + path_t path = PATH_T_INITIALIZE("readlink", "path", 0, 0); + int dir_fd = DEFAULT_DIR_FD; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + path_converter, &path, READLINKAT_DIR_FD_CONVERTER, &dir_fd)) { + goto exit; + } + return_value = os_readlink_impl(module, &path, dir_fd); + +exit: + /* Cleanup for path */ + path_cleanup(&path); + + return return_value; +} + +#endif /* (defined(HAVE_READLINK) || defined(MS_WINDOWS)) */ + #if defined(HAVE_SYMLINK) PyDoc_STRVAR(os_symlink__doc__, @@ -5873,6 +5917,80 @@ os_set_handle_inheritable(PyObject *module, PyObject *const *args, Py_ssize_t na #endif /* defined(MS_WINDOWS) */ +#if !defined(MS_WINDOWS) + +PyDoc_STRVAR(os_get_blocking__doc__, +"get_blocking($module, fd, /)\n" +"--\n" +"\n" +"Get the blocking mode of the file descriptor.\n" +"\n" +"Return False if the O_NONBLOCK flag is set, True if the flag is cleared."); + +#define OS_GET_BLOCKING_METHODDEF \ + {"get_blocking", (PyCFunction)os_get_blocking, METH_O, os_get_blocking__doc__}, + +static int +os_get_blocking_impl(PyObject *module, int fd); + +static PyObject * +os_get_blocking(PyObject *module, PyObject *arg) +{ + PyObject *return_value = NULL; + int fd; + int _return_value; + + if (!PyArg_Parse(arg, "i:get_blocking", &fd)) { + goto exit; + } + _return_value = os_get_blocking_impl(module, fd); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} + +#endif /* !defined(MS_WINDOWS) */ + +#if !defined(MS_WINDOWS) + +PyDoc_STRVAR(os_set_blocking__doc__, +"set_blocking($module, fd, blocking, /)\n" +"--\n" +"\n" +"Set the blocking mode of the specified file descriptor.\n" +"\n" +"Set the O_NONBLOCK flag if blocking is False,\n" +"clear the O_NONBLOCK flag otherwise."); + +#define OS_SET_BLOCKING_METHODDEF \ + {"set_blocking", (PyCFunction)os_set_blocking, METH_FASTCALL, os_set_blocking__doc__}, + +static PyObject * +os_set_blocking_impl(PyObject *module, int fd, int blocking); + +static PyObject * +os_set_blocking(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + int fd; + int blocking; + + if (!_PyArg_ParseStack(args, nargs, "ii:set_blocking", + &fd, &blocking)) { + goto exit; + } + return_value = os_set_blocking_impl(module, fd, blocking); + +exit: + return return_value; +} + +#endif /* !defined(MS_WINDOWS) */ + PyDoc_STRVAR(os_DirEntry_is_symlink__doc__, "is_symlink($self, /)\n" "--\n" @@ -6425,6 +6543,10 @@ os_getrandom(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject #define OS_WAIT_METHODDEF #endif /* !defined(OS_WAIT_METHODDEF) */ +#ifndef OS_READLINK_METHODDEF + #define OS_READLINK_METHODDEF +#endif /* !defined(OS_READLINK_METHODDEF) */ + #ifndef OS_SYMLINK_METHODDEF #define OS_SYMLINK_METHODDEF #endif /* !defined(OS_SYMLINK_METHODDEF) */ @@ -6645,7 +6767,15 @@ os_getrandom(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ +#ifndef OS_GET_BLOCKING_METHODDEF + #define OS_GET_BLOCKING_METHODDEF +#endif /* !defined(OS_GET_BLOCKING_METHODDEF) */ + +#ifndef OS_SET_BLOCKING_METHODDEF + #define OS_SET_BLOCKING_METHODDEF +#endif /* !defined(OS_SET_BLOCKING_METHODDEF) */ + #ifndef OS_GETRANDOM_METHODDEF #define OS_GETRANDOM_METHODDEF #endif /* !defined(OS_GETRANDOM_METHODDEF) */ -/*[clinic end generated code: output=758ee0434fb03d90 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=0f23518dd4482e66 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 0ac0042f124d..53d2ce22439e 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -7524,67 +7524,61 @@ os_wait_impl(PyObject *module) #if defined(HAVE_READLINK) || defined(MS_WINDOWS) -PyDoc_STRVAR(readlink__doc__, -"readlink(path, *, dir_fd=None) -> path\n\n\ -Return a string representing the path to which the symbolic link points.\n\ -\n\ -If dir_fd is not None, it should be a file descriptor open to a directory,\n\ - and path should be relative; path will then be relative to that directory.\n\ -dir_fd may not be implemented on your platform.\n\ - If it is unavailable, using it will raise a NotImplementedError."); +/*[clinic input] +os.readlink + + path: path_t + * + dir_fd: dir_fd(requires='readlinkat') = None + +Return a string representing the path to which the symbolic link points. + +If dir_fd is not None, it should be a file descriptor open to a directory, +and path should be relative; path will then be relative to that directory. + +dir_fd may not be implemented on your platform. If it is unavailable, +using it will raise a NotImplementedError. +[clinic start generated code]*/ static PyObject * -posix_readlink(PyObject *self, PyObject *args, PyObject *kwargs) +os_readlink_impl(PyObject *module, path_t *path, int dir_fd) +/*[clinic end generated code: output=d21b732a2e814030 input=113c87e0db1ecaf2]*/ { - path_t path; - int dir_fd = DEFAULT_DIR_FD; #if defined(HAVE_READLINK) char buffer[MAXPATHLEN+1]; ssize_t length; -#elif defined(MS_WINDOWS) - DWORD n_bytes_returned; - DWORD io_result; - HANDLE reparse_point_handle; - - char target_buffer[_Py_MAXIMUM_REPARSE_DATA_BUFFER_SIZE]; - _Py_REPARSE_DATA_BUFFER *rdb = (_Py_REPARSE_DATA_BUFFER *)target_buffer; - const wchar_t *print_name; -#endif - PyObject *return_value = NULL; - static char *keywords[] = {"path", "dir_fd", NULL}; - - memset(&path, 0, sizeof(path)); - path.function_name = "readlink"; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|$O&:readlink", keywords, - path_converter, &path, - READLINKAT_DIR_FD_CONVERTER, &dir_fd)) - return NULL; -#if defined(HAVE_READLINK) Py_BEGIN_ALLOW_THREADS #ifdef HAVE_READLINKAT if (dir_fd != DEFAULT_DIR_FD) - length = readlinkat(dir_fd, path.narrow, buffer, MAXPATHLEN); + length = readlinkat(dir_fd, path->narrow, buffer, MAXPATHLEN); else #endif - length = readlink(path.narrow, buffer, MAXPATHLEN); + length = readlink(path->narrow, buffer, MAXPATHLEN); Py_END_ALLOW_THREADS if (length < 0) { - return_value = path_error(&path); - goto exit; + return path_error(path); } buffer[length] = '\0'; - if (PyUnicode_Check(path.object)) - return_value = PyUnicode_DecodeFSDefaultAndSize(buffer, length); + if (PyUnicode_Check(path->object)) + return PyUnicode_DecodeFSDefaultAndSize(buffer, length); else - return_value = PyBytes_FromStringAndSize(buffer, length); + return PyBytes_FromStringAndSize(buffer, length); #elif defined(MS_WINDOWS) + DWORD n_bytes_returned; + DWORD io_result; + HANDLE reparse_point_handle; + char target_buffer[_Py_MAXIMUM_REPARSE_DATA_BUFFER_SIZE]; + _Py_REPARSE_DATA_BUFFER *rdb = (_Py_REPARSE_DATA_BUFFER *)target_buffer; + const wchar_t *print_name; + PyObject *result; + /* First get a handle to the reparse point */ Py_BEGIN_ALLOW_THREADS reparse_point_handle = CreateFileW( - path.wide, + path->wide, 0, 0, 0, @@ -7594,8 +7588,7 @@ posix_readlink(PyObject *self, PyObject *args, PyObject *kwargs) Py_END_ALLOW_THREADS if (reparse_point_handle == INVALID_HANDLE_VALUE) { - return_value = path_error(&path); - goto exit; + return path_error(path); } Py_BEGIN_ALLOW_THREADS @@ -7612,28 +7605,25 @@ posix_readlink(PyObject *self, PyObject *args, PyObject *kwargs) Py_END_ALLOW_THREADS if (io_result == 0) { - return_value = path_error(&path); - goto exit; + return path_error(path); } if (rdb->ReparseTag != IO_REPARSE_TAG_SYMLINK) { PyErr_SetString(PyExc_ValueError, "not a symbolic link"); - goto exit; + return NULL; } print_name = (wchar_t *)((char*)rdb->SymbolicLinkReparseBuffer.PathBuffer + rdb->SymbolicLinkReparseBuffer.PrintNameOffset); - return_value = PyUnicode_FromWideChar(print_name, - rdb->SymbolicLinkReparseBuffer.PrintNameLength / sizeof(wchar_t)); - if (path.narrow) { - Py_SETREF(return_value, PyUnicode_EncodeFSDefault(return_value)); + result = PyUnicode_FromWideChar(print_name, + rdb->SymbolicLinkReparseBuffer.PrintNameLength / sizeof(wchar_t)); + if (path->narrow) { + Py_SETREF(result, PyUnicode_EncodeFSDefault(result)); } + return result; #endif -exit: - path_cleanup(&path); - return return_value; } #endif /* defined(HAVE_READLINK) || defined(MS_WINDOWS) */ @@ -11918,43 +11908,45 @@ os_set_handle_inheritable_impl(PyObject *module, intptr_t handle, #endif /* MS_WINDOWS */ #ifndef MS_WINDOWS -PyDoc_STRVAR(get_blocking__doc__, - "get_blocking(fd) -> bool\n" \ - "\n" \ - "Get the blocking mode of the file descriptor:\n" \ - "False if the O_NONBLOCK flag is set, True if the flag is cleared."); +/*[clinic input] +os.get_blocking -> bool + fd: int + / -static PyObject* -posix_get_blocking(PyObject *self, PyObject *args) +Get the blocking mode of the file descriptor. + +Return False if the O_NONBLOCK flag is set, True if the flag is cleared. +[clinic start generated code]*/ + +static int +os_get_blocking_impl(PyObject *module, int fd) +/*[clinic end generated code: output=336a12ad76a61482 input=f4afb59d51560179]*/ { - int fd; int blocking; - if (!PyArg_ParseTuple(args, "i:get_blocking", &fd)) - return NULL; - _Py_BEGIN_SUPPRESS_IPH blocking = _Py_get_blocking(fd); _Py_END_SUPPRESS_IPH - if (blocking < 0) - return NULL; - return PyBool_FromLong(blocking); + return blocking; } -PyDoc_STRVAR(set_blocking__doc__, - "set_blocking(fd, blocking)\n" \ - "\n" \ - "Set the blocking mode of the specified file descriptor.\n" \ - "Set the O_NONBLOCK flag if blocking is False,\n" \ - "clear the O_NONBLOCK flag otherwise."); +/*[clinic input] +os.set_blocking + fd: int + blocking: bool(accept={int}) + / -static PyObject* -posix_set_blocking(PyObject *self, PyObject *args) -{ - int fd, blocking, result; +Set the blocking mode of the specified file descriptor. - if (!PyArg_ParseTuple(args, "ii:set_blocking", &fd, &blocking)) - return NULL; +Set the O_NONBLOCK flag if blocking is False, +clear the O_NONBLOCK flag otherwise. +[clinic start generated code]*/ + +static PyObject * +os_set_blocking_impl(PyObject *module, int fd, int blocking) +/*[clinic end generated code: output=384eb43aa0762a9d input=bf5c8efdc5860ff3]*/ +{ + int result; _Py_BEGIN_SUPPRESS_IPH result = _Py_set_blocking(fd, blocking); @@ -13048,11 +13040,7 @@ static PyMethodDef posix_methods[] = { OS_GETPRIORITY_METHODDEF OS_SETPRIORITY_METHODDEF OS_POSIX_SPAWN_METHODDEF -#if defined(HAVE_READLINK) || defined(MS_WINDOWS) - {"readlink", (PyCFunction)posix_readlink, - METH_VARARGS | METH_KEYWORDS, - readlink__doc__}, -#endif /* defined(HAVE_READLINK) || defined(MS_WINDOWS) */ + OS_READLINK_METHODDEF OS_RENAME_METHODDEF OS_REPLACE_METHODDEF OS_RMDIR_METHODDEF @@ -13206,8 +13194,8 @@ static PyMethodDef posix_methods[] = { OS_GET_HANDLE_INHERITABLE_METHODDEF OS_SET_HANDLE_INHERITABLE_METHODDEF #ifndef MS_WINDOWS - {"get_blocking", posix_get_blocking, METH_VARARGS, get_blocking__doc__}, - {"set_blocking", posix_set_blocking, METH_VARARGS, set_blocking__doc__}, + OS_GET_BLOCKING_METHODDEF + OS_SET_BLOCKING_METHODDEF #endif OS_SCANDIR_METHODDEF OS_FSPATH_METHODDEF From webhook-mailer at python.org Mon Sep 17 09:08:53 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 13:08:53 -0000 Subject: [Python-checkins] bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683) Message-ID: https://github.com/python/cpython/commit/efdf316d23c2bc81f499c46faaba5e1b49509afa commit: efdf316d23c2bc81f499c46faaba5e1b49509afa branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T06:08:45-07:00 summary: bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683) (cherry picked from commit 9bdb7be482aef8f60daa1d36606568a132dcb616) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2018-08-06-11-01-18.bpo-34341.E0b9p2.rst M Lib/test/test_zipfile.py M Lib/zipfile.py diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index ac9a4ff6fef0..b48366a53263 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -750,6 +750,20 @@ def test_absolute_arcnames(self): with zipfile.ZipFile(TESTFN2, "r", zipfile.ZIP_STORED) as zipfp: self.assertEqual(zipfp.namelist(), ["absolute"]) + def test_append(self): + # Test that appending to the Zip64 archive doesn't change + # extra fields of existing entries. + with zipfile.ZipFile(TESTFN2, "w", allowZip64=True) as zipfp: + zipfp.writestr("strfile", self.data) + with zipfile.ZipFile(TESTFN2, "r", allowZip64=True) as zipfp: + zinfo = zipfp.getinfo("strfile") + extra = zinfo.extra + with zipfile.ZipFile(TESTFN2, "a", allowZip64=True) as zipfp: + zipfp.writestr("strfile2", self.data) + with zipfile.ZipFile(TESTFN2, "r", allowZip64=True) as zipfp: + zinfo = zipfp.getinfo("strfile") + self.assertEqual(zinfo.extra, extra) + @requires_zlib class DeflateTestZip64InSmallFiles(AbstractTestZip64InSmallFiles, unittest.TestCase): diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 2757ce91cf48..9f88512d9838 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -159,6 +159,27 @@ class LargeZipFile(Exception): _CD64_DIRECTORY_SIZE = 8 _CD64_OFFSET_START_CENTDIR = 9 +_EXTRA_FIELD_STRUCT = struct.Struct(' https://github.com/python/cpython/commit/f8e34eee74871b3343f6c3545bce41242ba378e8 commit: f8e34eee74871b3343f6c3545bce41242ba378e8 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T06:09:32-07:00 summary: bpo-33216: Improve the documentation for CALL_FUNCTION_* (GH-8338) (GH-8784) (cherry picked from commit 5e99b56d6b249995a4fa2bc09c0bb03841f49572) Co-authored-by: Serhiy Storchaka files: M Doc/library/dis.rst diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 535b36efbf2e..4f84e5b637bf 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -1059,18 +1059,20 @@ All of the following opcodes use their arguments. .. opcode:: RAISE_VARARGS (argc) - Raises an exception. *argc* indicates the number of parameters to the raise + Raises an exception. *argc* indicates the number of arguments to the raise statement, ranging from 0 to 3. The handler will find the traceback as TOS2, the parameter as TOS1, and the exception as TOS. .. opcode:: CALL_FUNCTION (argc) - Calls a function. *argc* indicates the number of positional arguments. - The positional arguments are on the stack, with the right-most argument - on top. Below the arguments, the function object to call is on the stack. - Pops all function arguments, and the function itself off the stack, and - pushes the return value. + Calls a callable object with positional arguments. + *argc* indicates the number of positional arguments. + The top of the stack contains positional arguments, with the right-most + argument on top. Below the arguments is a callable object to call. + ``CALL_FUNCTION`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. versionchanged:: 3.6 This opcode is used only for calls with positional arguments. @@ -1078,31 +1080,36 @@ All of the following opcodes use their arguments. .. opcode:: CALL_FUNCTION_KW (argc) - Calls a function. *argc* indicates the number of arguments (positional - and keyword). The top element on the stack contains a tuple of keyword - argument names. Below the tuple, keyword arguments are on the stack, in - the order corresponding to the tuple. Below the keyword arguments, the - positional arguments are on the stack, with the right-most parameter on - top. Below the arguments, the function object to call is on the stack. - Pops all function arguments, and the function itself off the stack, and - pushes the return value. + Calls a callable object with positional (if any) and keyword arguments. + *argc* indicates the total number of positional and keyword arguments. + The top element on the stack contains a tuple of keyword argument names. + Below that are keyword arguments in the order corresponding to the tuple. + Below that are positional arguments, with the right-most parameter on + top. Below the arguments is a callable object to call. + ``CALL_FUNCTION_KW`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. versionchanged:: 3.6 Keyword arguments are packed in a tuple instead of a dictionary, - *argc* indicates the total number of arguments + *argc* indicates the total number of arguments. .. opcode:: CALL_FUNCTION_EX (flags) - Calls a function. The lowest bit of *flags* indicates whether the - var-keyword argument is placed at the top of the stack. Below the - var-keyword argument, the var-positional argument is on the stack. - Below the arguments, the function object to call is placed. - Pops all function arguments, and the function itself off the stack, and - pushes the return value. Note that this opcode pops at most three items - from the stack. Var-positional and var-keyword arguments are packed - by :opcode:`BUILD_TUPLE_UNPACK_WITH_CALL` and - :opcode:`BUILD_MAP_UNPACK_WITH_CALL`. + Calls a callable object with variable set of positional and keyword + arguments. If the lowest bit of *flags* is set, the top of the stack + contains a mapping object containing additional keyword arguments. + Below that is an iterable object containing positional arguments and + a callable object to call. :opcode:`BUILD_MAP_UNPACK_WITH_CALL` and + :opcode:`BUILD_TUPLE_UNPACK_WITH_CALL` can be used for merging multiple + mapping objects and iterables containing arguments. + Before the callable is called, the mapping object and iterable object + are each "unpacked" and their contents passed in as keyword and + positional arguments respectively. + ``CALL_FUNCTION_EX`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. versionadded:: 3.6 @@ -1134,7 +1141,8 @@ All of the following opcodes use their arguments. Pushes a new function object on the stack. From bottom to top, the consumed stack must consist of values if the argument carries a specified flag value - * ``0x01`` a tuple of default argument objects in positional order + * ``0x01`` a tuple of default values for positional-only and + positional-or-keyword parameters in positional order * ``0x02`` a dictionary of keyword-only parameters' default values * ``0x04`` an annotation dictionary * ``0x08`` a tuple containing cells for free variables, making a closure @@ -1217,7 +1225,7 @@ instructions: .. data:: hasconst - Sequence of bytecodes that have a constant parameter. + Sequence of bytecodes that access a constant. .. data:: hasfree From webhook-mailer at python.org Mon Sep 17 09:09:45 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 13:09:45 -0000 Subject: [Python-checkins] bpo-33216: Improve the documentation for CALL_FUNCTION_* (GH-8338) (GH-8784) Message-ID: https://github.com/python/cpython/commit/1e7193bd027a5a4aa1a387e3f201a3e465c25f01 commit: 1e7193bd027a5a4aa1a387e3f201a3e465c25f01 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T06:09:39-07:00 summary: bpo-33216: Improve the documentation for CALL_FUNCTION_* (GH-8338) (GH-8784) (cherry picked from commit 5e99b56d6b249995a4fa2bc09c0bb03841f49572) Co-authored-by: Serhiy Storchaka files: M Doc/library/dis.rst diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 3f615dbd15ce..d5757c2c6628 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -1043,18 +1043,20 @@ All of the following opcodes use their arguments. .. opcode:: RAISE_VARARGS (argc) - Raises an exception. *argc* indicates the number of parameters to the raise + Raises an exception. *argc* indicates the number of arguments to the raise statement, ranging from 0 to 3. The handler will find the traceback as TOS2, the parameter as TOS1, and the exception as TOS. .. opcode:: CALL_FUNCTION (argc) - Calls a function. *argc* indicates the number of positional arguments. - The positional arguments are on the stack, with the right-most argument - on top. Below the arguments, the function object to call is on the stack. - Pops all function arguments, and the function itself off the stack, and - pushes the return value. + Calls a callable object with positional arguments. + *argc* indicates the number of positional arguments. + The top of the stack contains positional arguments, with the right-most + argument on top. Below the arguments is a callable object to call. + ``CALL_FUNCTION`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. versionchanged:: 3.6 This opcode is used only for calls with positional arguments. @@ -1062,31 +1064,36 @@ All of the following opcodes use their arguments. .. opcode:: CALL_FUNCTION_KW (argc) - Calls a function. *argc* indicates the number of arguments (positional - and keyword). The top element on the stack contains a tuple of keyword - argument names. Below the tuple, keyword arguments are on the stack, in - the order corresponding to the tuple. Below the keyword arguments, the - positional arguments are on the stack, with the right-most parameter on - top. Below the arguments, the function object to call is on the stack. - Pops all function arguments, and the function itself off the stack, and - pushes the return value. + Calls a callable object with positional (if any) and keyword arguments. + *argc* indicates the total number of positional and keyword arguments. + The top element on the stack contains a tuple of keyword argument names. + Below that are keyword arguments in the order corresponding to the tuple. + Below that are positional arguments, with the right-most parameter on + top. Below the arguments is a callable object to call. + ``CALL_FUNCTION_KW`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. versionchanged:: 3.6 Keyword arguments are packed in a tuple instead of a dictionary, - *argc* indicates the total number of arguments + *argc* indicates the total number of arguments. .. opcode:: CALL_FUNCTION_EX (flags) - Calls a function. The lowest bit of *flags* indicates whether the - var-keyword argument is placed at the top of the stack. Below the - var-keyword argument, the var-positional argument is on the stack. - Below the arguments, the function object to call is placed. - Pops all function arguments, and the function itself off the stack, and - pushes the return value. Note that this opcode pops at most three items - from the stack. Var-positional and var-keyword arguments are packed - by :opcode:`BUILD_TUPLE_UNPACK_WITH_CALL` and - :opcode:`BUILD_MAP_UNPACK_WITH_CALL`. + Calls a callable object with variable set of positional and keyword + arguments. If the lowest bit of *flags* is set, the top of the stack + contains a mapping object containing additional keyword arguments. + Below that is an iterable object containing positional arguments and + a callable object to call. :opcode:`BUILD_MAP_UNPACK_WITH_CALL` and + :opcode:`BUILD_TUPLE_UNPACK_WITH_CALL` can be used for merging multiple + mapping objects and iterables containing arguments. + Before the callable is called, the mapping object and iterable object + are each "unpacked" and their contents passed in as keyword and + positional arguments respectively. + ``CALL_FUNCTION_EX`` pops all arguments and the callable object off the stack, + calls the callable object with those arguments, and pushes the return value + returned by the callable object. .. versionadded:: 3.6 @@ -1096,7 +1103,8 @@ All of the following opcodes use their arguments. Pushes a new function object on the stack. From bottom to top, the consumed stack must consist of values if the argument carries a specified flag value - * ``0x01`` a tuple of default argument objects in positional order + * ``0x01`` a tuple of default values for positional-only and + positional-or-keyword parameters in positional order * ``0x02`` a dictionary of keyword-only parameters' default values * ``0x04`` an annotation dictionary * ``0x08`` a tuple containing cells for free variables, making a closure @@ -1179,7 +1187,7 @@ instructions: .. data:: hasconst - Sequence of bytecodes that have a constant parameter. + Sequence of bytecodes that access a constant. .. data:: hasfree From webhook-mailer at python.org Mon Sep 17 09:11:45 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 13:11:45 -0000 Subject: [Python-checkins] bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683) Message-ID: https://github.com/python/cpython/commit/83a0985165abf340294c10d732840e9d46ba218c commit: 83a0985165abf340294c10d732840e9d46ba218c branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T06:11:42-07:00 summary: bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683) (cherry picked from commit 9bdb7be482aef8f60daa1d36606568a132dcb616) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2018-08-06-11-01-18.bpo-34341.E0b9p2.rst M Lib/test/test_zipfile.py M Lib/zipfile.py diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index d09ad96fc8ba..e62b82e1d31d 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -718,6 +718,20 @@ def test_absolute_arcnames(self): with zipfile.ZipFile(TESTFN2, "r", zipfile.ZIP_STORED) as zipfp: self.assertEqual(zipfp.namelist(), ["absolute"]) + def test_append(self): + # Test that appending to the Zip64 archive doesn't change + # extra fields of existing entries. + with zipfile.ZipFile(TESTFN2, "w", allowZip64=True) as zipfp: + zipfp.writestr("strfile", self.data) + with zipfile.ZipFile(TESTFN2, "r", allowZip64=True) as zipfp: + zinfo = zipfp.getinfo("strfile") + extra = zinfo.extra + with zipfile.ZipFile(TESTFN2, "a", allowZip64=True) as zipfp: + zipfp.writestr("strfile2", self.data) + with zipfile.ZipFile(TESTFN2, "r", allowZip64=True) as zipfp: + zinfo = zipfp.getinfo("strfile") + self.assertEqual(zinfo.extra, extra) + @requires_zlib class DeflateTestZip64InSmallFiles(AbstractTestZip64InSmallFiles, unittest.TestCase): diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 9164f8ab086a..bc757a37b5c4 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -164,6 +164,27 @@ class LargeZipFile(Exception): _CD64_DIRECTORY_SIZE = 8 _CD64_OFFSET_START_CENTDIR = 9 +_EXTRA_FIELD_STRUCT = struct.Struct(' https://github.com/python/cpython/commit/5f883fcb9b50b8335953b3849b7f224d5cd59d56 commit: 5f883fcb9b50b8335953b3849b7f224d5cd59d56 branch: 2.7 author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-17T06:42:30-07:00 summary: [2.7] bpo-34710: fix SSL module build (GH-9347) (GH-9353) Include ``openssl/dh.h`` header file to fix implicit function declaration of ``DH_free()``. Signed-off-by: Alexandru Ardelean . (cherry picked from commit b3a271fc0ce3e13e427be8914decfc205a220ca8) Co-authored-by: Alexandru Ardelean https://bugs.python.org/issue34710 files: A Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst b/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst new file mode 100644 index 000000000000..b06289d091e5 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-09-17-13-56-12.bpo-34710.ARqIAK.rst @@ -0,0 +1 @@ +Fixed SSL module build with OpenSSL & pedantic CFLAGS. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index d0ce913d3d89..a96c41926036 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -74,6 +74,7 @@ #include "openssl/ssl.h" #include "openssl/err.h" #include "openssl/rand.h" +#include "openssl/dh.h" /* SSL error object */ static PyObject *PySSLErrorObject; From webhook-mailer at python.org Mon Sep 17 14:34:58 2018 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 17 Sep 2018 18:34:58 -0000 Subject: [Python-checkins] bpo-32533: Fixed thread-safety of error handling in _ssl. (GH-7158) Message-ID: https://github.com/python/cpython/commit/c6fd1c1c3a65217958b68df3a4991e4f306e9b7d commit: c6fd1c1c3a65217958b68df3a4991e4f306e9b7d branch: master author: Steve Dower committer: GitHub date: 2018-09-17T11:34:47-07:00 summary: bpo-32533: Fixed thread-safety of error handling in _ssl. (GH-7158) files: A Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst b/Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst new file mode 100644 index 000000000000..a3642258edaf --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst @@ -0,0 +1 @@ +Fixed thread-safety of error handling in _ssl. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 3c9388480941..4750b930c642 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -423,6 +423,14 @@ typedef struct { int protocol; } PySSLContext; +typedef struct { + int ssl; /* last seen error from SSL */ + int c; /* last seen error from libc */ +#ifdef MS_WINDOWS + int ws; /* last seen error from winsock */ +#endif +} _PySSLError; + typedef struct { PyObject_HEAD PyObject *Socket; /* weakref to socket on which we're layered */ @@ -432,11 +440,7 @@ typedef struct { enum py_ssl_server_or_client socket_type; PyObject *owner; /* Python level "owner" passed to servername callback */ PyObject *server_hostname; - int ssl_errno; /* last seen error from SSL */ - int c_errno; /* last seen error from libc */ -#ifdef MS_WINDOWS - int ws_errno; /* last seen error from winsock */ -#endif + _PySSLError err; /* last seen error from various sources */ } PySSLSocket; typedef struct { @@ -456,20 +460,19 @@ static PyTypeObject PySSLSocket_Type; static PyTypeObject PySSLMemoryBIO_Type; static PyTypeObject PySSLSession_Type; +static inline _PySSLError _PySSL_errno(int failed, const SSL *ssl, int retcode) +{ + _PySSLError err = { 0 }; + if (failed) { #ifdef MS_WINDOWS -#define _PySSL_UPDATE_ERRNO_IF(cond, sock, retcode) if (cond) { \ - (sock)->ws_errno = WSAGetLastError(); \ - _PySSL_FIX_ERRNO; \ - (sock)->c_errno = errno; \ - (sock)->ssl_errno = SSL_get_error((sock->ssl), (retcode)); \ - } else { sock->ws_errno = 0; sock->c_errno = 0; sock->ssl_errno = 0; } -#else -#define _PySSL_UPDATE_ERRNO_IF(cond, sock, retcode) if (cond) { \ - (sock)->c_errno = errno; \ - (sock)->ssl_errno = SSL_get_error((sock->ssl), (retcode)); \ - } else { (sock)->c_errno = 0; (sock)->ssl_errno = 0; } + err.ws = WSAGetLastError(); + _PySSL_FIX_ERRNO; #endif -#define _PySSL_UPDATE_ERRNO(sock, retcode) _PySSL_UPDATE_ERRNO_IF(1, (sock), (retcode)) + err.c = errno; + err.ssl = SSL_get_error(ssl, retcode); + } + return err; +} /*[clinic input] module _ssl @@ -703,7 +706,7 @@ PySSL_SetError(PySSLSocket *sslsock, int ret, const char *filename, int lineno) { PyObject *type = PySSLErrorObject; char *errstr = NULL; - int err; + _PySSLError err; enum py_ssl_error p = PY_SSL_ERROR_NONE; unsigned long e = 0; @@ -711,9 +714,9 @@ PySSL_SetError(PySSLSocket *sslsock, int ret, const char *filename, int lineno) e = ERR_peek_last_error(); if (sslsock->ssl != NULL) { - err = sslsock->ssl_errno; + err = sslsock->err; - switch (err) { + switch (err.ssl) { case SSL_ERROR_ZERO_RETURN: errstr = "TLS/SSL connection has been closed (EOF)"; type = PySSLZeroReturnErrorObject; @@ -749,11 +752,12 @@ PySSL_SetError(PySSLSocket *sslsock, int ret, const char *filename, int lineno) /* underlying BIO reported an I/O error */ ERR_clear_error(); #ifdef MS_WINDOWS - if (sslsock->ws_errno) - return PyErr_SetFromWindowsErr(sslsock->ws_errno); + if (err.ws) { + return PyErr_SetFromWindowsErr(err.ws); + } #endif - if (sslsock->c_errno) { - errno = sslsock->c_errno; + if (err.c) { + errno = err.c; return PyErr_SetFromErrno(PyExc_OSError); } Py_INCREF(s); @@ -883,6 +887,7 @@ newPySSLSocket(PySSLContext *sslctx, PySocketSockObject *sock, { PySSLSocket *self; SSL_CTX *ctx = sslctx->ctx; + _PySSLError err = { 0 }; self = PyObject_New(PySSLSocket, &PySSLSocket_Type); if (self == NULL) @@ -895,11 +900,7 @@ newPySSLSocket(PySSLContext *sslctx, PySocketSockObject *sock, self->shutdown_seen_zero = 0; self->owner = NULL; self->server_hostname = NULL; - self->ssl_errno = 0; - self->c_errno = 0; -#ifdef MS_WINDOWS - self->ws_errno = 0; -#endif + self->err = err; /* Make sure the SSL error state is initialized */ ERR_clear_error(); @@ -976,7 +977,7 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) /*[clinic end generated code: output=6c0898a8936548f6 input=d2d737de3df018c8]*/ { int ret; - int err; + _PySSLError err; int sockstate, nonblocking; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -1006,9 +1007,9 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) do { PySSL_BEGIN_ALLOW_THREADS ret = SSL_do_handshake(self->ssl); - _PySSL_UPDATE_ERRNO_IF(ret < 1, self, ret); + err = _PySSL_errno(ret < 1, self->ssl, ret); PySSL_END_ALLOW_THREADS - err = self->ssl_errno; + self->err = err; if (PyErr_CheckSignals()) goto error; @@ -1016,9 +1017,9 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) if (has_timeout) timeout = deadline - _PyTime_GetMonotonicClock(); - if (err == SSL_ERROR_WANT_READ) { + if (err.ssl == SSL_ERROR_WANT_READ) { sockstate = PySSL_select(sock, 0, timeout); - } else if (err == SSL_ERROR_WANT_WRITE) { + } else if (err.ssl == SSL_ERROR_WANT_WRITE) { sockstate = PySSL_select(sock, 1, timeout); } else { sockstate = SOCKET_OPERATION_OK; @@ -1039,7 +1040,8 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) } else if (sockstate == SOCKET_IS_NONBLOCKING) { break; } - } while (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE); + } while (err.ssl == SSL_ERROR_WANT_READ || + err.ssl == SSL_ERROR_WANT_WRITE); Py_XDECREF(sock); if (ret < 1) return PySSL_SetError(self, ret, __FILE__, __LINE__); @@ -2228,7 +2230,7 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) { int len; int sockstate; - int err; + _PySSLError err; int nonblocking; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -2279,9 +2281,9 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) do { PySSL_BEGIN_ALLOW_THREADS len = SSL_write(self->ssl, b->buf, (int)b->len); - _PySSL_UPDATE_ERRNO_IF(len <= 0, self, len); + err = _PySSL_errno(len <= 0, self->ssl, len); PySSL_END_ALLOW_THREADS - err = self->ssl_errno; + self->err = err; if (PyErr_CheckSignals()) goto error; @@ -2289,9 +2291,9 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) if (has_timeout) timeout = deadline - _PyTime_GetMonotonicClock(); - if (err == SSL_ERROR_WANT_READ) { + if (err.ssl == SSL_ERROR_WANT_READ) { sockstate = PySSL_select(sock, 0, timeout); - } else if (err == SSL_ERROR_WANT_WRITE) { + } else if (err.ssl == SSL_ERROR_WANT_WRITE) { sockstate = PySSL_select(sock, 1, timeout); } else { sockstate = SOCKET_OPERATION_OK; @@ -2308,7 +2310,8 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) } else if (sockstate == SOCKET_IS_NONBLOCKING) { break; } - } while (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE); + } while (err.ssl == SSL_ERROR_WANT_READ || + err.ssl == SSL_ERROR_WANT_WRITE); Py_XDECREF(sock); if (len > 0) @@ -2332,11 +2335,14 @@ _ssl__SSLSocket_pending_impl(PySSLSocket *self) /*[clinic end generated code: output=983d9fecdc308a83 input=2b77487d6dfd597f]*/ { int count = 0; + _PySSLError err; PySSL_BEGIN_ALLOW_THREADS count = SSL_pending(self->ssl); - _PySSL_UPDATE_ERRNO_IF(count < 0, self, count); + err = _PySSL_errno(count < 0, self->ssl, count); PySSL_END_ALLOW_THREADS + self->err = err; + if (count < 0) return PySSL_SetError(self, count, __FILE__, __LINE__); else @@ -2363,7 +2369,7 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, char *mem; int count; int sockstate; - int err; + _PySSLError err; int nonblocking; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -2424,8 +2430,9 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, do { PySSL_BEGIN_ALLOW_THREADS count = SSL_read(self->ssl, mem, len); - _PySSL_UPDATE_ERRNO_IF(count <= 0, self, count); + err = _PySSL_errno(count <= 0, self->ssl, count); PySSL_END_ALLOW_THREADS + self->err = err; if (PyErr_CheckSignals()) goto error; @@ -2433,12 +2440,11 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, if (has_timeout) timeout = deadline - _PyTime_GetMonotonicClock(); - err = self->ssl_errno; - if (err == SSL_ERROR_WANT_READ) { + if (err.ssl == SSL_ERROR_WANT_READ) { sockstate = PySSL_select(sock, 0, timeout); - } else if (err == SSL_ERROR_WANT_WRITE) { + } else if (err.ssl == SSL_ERROR_WANT_WRITE) { sockstate = PySSL_select(sock, 1, timeout); - } else if (err == SSL_ERROR_ZERO_RETURN && + } else if (err.ssl == SSL_ERROR_ZERO_RETURN && SSL_get_shutdown(self->ssl) == SSL_RECEIVED_SHUTDOWN) { count = 0; @@ -2454,7 +2460,8 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, } else if (sockstate == SOCKET_IS_NONBLOCKING) { break; } - } while (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE); + } while (err.ssl == SSL_ERROR_WANT_READ || + err.ssl == SSL_ERROR_WANT_WRITE); if (count <= 0) { PySSL_SetError(self, count, __FILE__, __LINE__); @@ -2488,7 +2495,8 @@ static PyObject * _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) /*[clinic end generated code: output=ca1aa7ed9d25ca42 input=11d39e69b0a2bf4a]*/ { - int err, sockstate, nonblocking; + _PySSLError err; + int sockstate, nonblocking, ret; int zeros = 0; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -2526,14 +2534,15 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) */ if (self->shutdown_seen_zero) SSL_set_read_ahead(self->ssl, 0); - err = SSL_shutdown(self->ssl); - _PySSL_UPDATE_ERRNO_IF(err < 0, self, err); + ret = SSL_shutdown(self->ssl); + err = _PySSL_errno(ret < 0, self->ssl, ret); PySSL_END_ALLOW_THREADS + self->err = err; /* If err == 1, a secure shutdown with SSL_shutdown() is complete */ - if (err > 0) + if (ret > 0) break; - if (err == 0) { + if (ret == 0) { /* Don't loop endlessly; instead preserve legacy behaviour of trying SSL_shutdown() only twice. This looks necessary for OpenSSL < 0.9.8m */ @@ -2548,16 +2557,15 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) timeout = deadline - _PyTime_GetMonotonicClock(); /* Possibly retry shutdown until timeout or failure */ - _PySSL_UPDATE_ERRNO(self, err); - if (self->ssl_errno == SSL_ERROR_WANT_READ) + if (err.ssl == SSL_ERROR_WANT_READ) sockstate = PySSL_select(sock, 0, timeout); - else if (self->ssl_errno == SSL_ERROR_WANT_WRITE) + else if (err.ssl == SSL_ERROR_WANT_WRITE) sockstate = PySSL_select(sock, 1, timeout); else break; if (sockstate == SOCKET_HAS_TIMED_OUT) { - if (self->ssl_errno == SSL_ERROR_WANT_READ) + if (err.ssl == SSL_ERROR_WANT_READ) PyErr_SetString(PySocketModule.timeout_error, "The read operation timed out"); else @@ -2575,9 +2583,9 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) break; } - if (err < 0) { + if (err.ssl < 0) { Py_XDECREF(sock); - return PySSL_SetError(self, err, __FILE__, __LINE__); + return PySSL_SetError(self, err.ssl, __FILE__, __LINE__); } if (sock) /* It's already INCREF'ed */ From webhook-mailer at python.org Mon Sep 17 15:12:22 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 19:12:22 -0000 Subject: [Python-checkins] bpo-32533: Fixed thread-safety of error handling in _ssl. (GH-7158) Message-ID: https://github.com/python/cpython/commit/1229664f30dd5fd4da32174a19258f8312464d45 commit: 1229664f30dd5fd4da32174a19258f8312464d45 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T12:12:13-07:00 summary: bpo-32533: Fixed thread-safety of error handling in _ssl. (GH-7158) (cherry picked from commit c6fd1c1c3a65217958b68df3a4991e4f306e9b7d) Co-authored-by: Steve Dower files: A Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst b/Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst new file mode 100644 index 000000000000..a3642258edaf --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst @@ -0,0 +1 @@ +Fixed thread-safety of error handling in _ssl. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 6af48ee0ff20..7ff616eacdce 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -423,6 +423,14 @@ typedef struct { int protocol; } PySSLContext; +typedef struct { + int ssl; /* last seen error from SSL */ + int c; /* last seen error from libc */ +#ifdef MS_WINDOWS + int ws; /* last seen error from winsock */ +#endif +} _PySSLError; + typedef struct { PyObject_HEAD PyObject *Socket; /* weakref to socket on which we're layered */ @@ -432,11 +440,7 @@ typedef struct { enum py_ssl_server_or_client socket_type; PyObject *owner; /* Python level "owner" passed to servername callback */ PyObject *server_hostname; - int ssl_errno; /* last seen error from SSL */ - int c_errno; /* last seen error from libc */ -#ifdef MS_WINDOWS - int ws_errno; /* last seen error from winsock */ -#endif + _PySSLError err; /* last seen error from various sources */ } PySSLSocket; typedef struct { @@ -456,20 +460,19 @@ static PyTypeObject PySSLSocket_Type; static PyTypeObject PySSLMemoryBIO_Type; static PyTypeObject PySSLSession_Type; +static inline _PySSLError _PySSL_errno(int failed, const SSL *ssl, int retcode) +{ + _PySSLError err = { 0 }; + if (failed) { #ifdef MS_WINDOWS -#define _PySSL_UPDATE_ERRNO_IF(cond, sock, retcode) if (cond) { \ - (sock)->ws_errno = WSAGetLastError(); \ - _PySSL_FIX_ERRNO; \ - (sock)->c_errno = errno; \ - (sock)->ssl_errno = SSL_get_error((sock->ssl), (retcode)); \ - } else { sock->ws_errno = 0; sock->c_errno = 0; sock->ssl_errno = 0; } -#else -#define _PySSL_UPDATE_ERRNO_IF(cond, sock, retcode) if (cond) { \ - (sock)->c_errno = errno; \ - (sock)->ssl_errno = SSL_get_error((sock->ssl), (retcode)); \ - } else { (sock)->c_errno = 0; (sock)->ssl_errno = 0; } + err.ws = WSAGetLastError(); + _PySSL_FIX_ERRNO; #endif -#define _PySSL_UPDATE_ERRNO(sock, retcode) _PySSL_UPDATE_ERRNO_IF(1, (sock), (retcode)) + err.c = errno; + err.ssl = SSL_get_error(ssl, retcode); + } + return err; +} /*[clinic input] module _ssl @@ -703,7 +706,7 @@ PySSL_SetError(PySSLSocket *sslsock, int ret, const char *filename, int lineno) { PyObject *type = PySSLErrorObject; char *errstr = NULL; - int err; + _PySSLError err; enum py_ssl_error p = PY_SSL_ERROR_NONE; unsigned long e = 0; @@ -711,9 +714,9 @@ PySSL_SetError(PySSLSocket *sslsock, int ret, const char *filename, int lineno) e = ERR_peek_last_error(); if (sslsock->ssl != NULL) { - err = sslsock->ssl_errno; + err = sslsock->err; - switch (err) { + switch (err.ssl) { case SSL_ERROR_ZERO_RETURN: errstr = "TLS/SSL connection has been closed (EOF)"; type = PySSLZeroReturnErrorObject; @@ -749,11 +752,12 @@ PySSL_SetError(PySSLSocket *sslsock, int ret, const char *filename, int lineno) /* underlying BIO reported an I/O error */ ERR_clear_error(); #ifdef MS_WINDOWS - if (sslsock->ws_errno) - return PyErr_SetFromWindowsErr(sslsock->ws_errno); + if (err.ws) { + return PyErr_SetFromWindowsErr(err.ws); + } #endif - if (sslsock->c_errno) { - errno = sslsock->c_errno; + if (err.c) { + errno = err.c; return PyErr_SetFromErrno(PyExc_OSError); } Py_INCREF(s); @@ -883,6 +887,7 @@ newPySSLSocket(PySSLContext *sslctx, PySocketSockObject *sock, { PySSLSocket *self; SSL_CTX *ctx = sslctx->ctx; + _PySSLError err = { 0 }; self = PyObject_New(PySSLSocket, &PySSLSocket_Type); if (self == NULL) @@ -895,11 +900,7 @@ newPySSLSocket(PySSLContext *sslctx, PySocketSockObject *sock, self->shutdown_seen_zero = 0; self->owner = NULL; self->server_hostname = NULL; - self->ssl_errno = 0; - self->c_errno = 0; -#ifdef MS_WINDOWS - self->ws_errno = 0; -#endif + self->err = err; /* Make sure the SSL error state is initialized */ (void) ERR_get_state(); @@ -977,7 +978,7 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) /*[clinic end generated code: output=6c0898a8936548f6 input=d2d737de3df018c8]*/ { int ret; - int err; + _PySSLError err; int sockstate, nonblocking; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -1007,9 +1008,9 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) do { PySSL_BEGIN_ALLOW_THREADS ret = SSL_do_handshake(self->ssl); - _PySSL_UPDATE_ERRNO_IF(ret < 1, self, ret); + err = _PySSL_errno(ret < 1, self->ssl, ret); PySSL_END_ALLOW_THREADS - err = self->ssl_errno; + self->err = err; if (PyErr_CheckSignals()) goto error; @@ -1017,9 +1018,9 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) if (has_timeout) timeout = deadline - _PyTime_GetMonotonicClock(); - if (err == SSL_ERROR_WANT_READ) { + if (err.ssl == SSL_ERROR_WANT_READ) { sockstate = PySSL_select(sock, 0, timeout); - } else if (err == SSL_ERROR_WANT_WRITE) { + } else if (err.ssl == SSL_ERROR_WANT_WRITE) { sockstate = PySSL_select(sock, 1, timeout); } else { sockstate = SOCKET_OPERATION_OK; @@ -1040,7 +1041,8 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) } else if (sockstate == SOCKET_IS_NONBLOCKING) { break; } - } while (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE); + } while (err.ssl == SSL_ERROR_WANT_READ || + err.ssl == SSL_ERROR_WANT_WRITE); Py_XDECREF(sock); if (ret < 1) return PySSL_SetError(self, ret, __FILE__, __LINE__); @@ -2229,7 +2231,7 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) { int len; int sockstate; - int err; + _PySSLError err; int nonblocking; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -2280,9 +2282,9 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) do { PySSL_BEGIN_ALLOW_THREADS len = SSL_write(self->ssl, b->buf, (int)b->len); - _PySSL_UPDATE_ERRNO_IF(len <= 0, self, len); + err = _PySSL_errno(len <= 0, self->ssl, len); PySSL_END_ALLOW_THREADS - err = self->ssl_errno; + self->err = err; if (PyErr_CheckSignals()) goto error; @@ -2290,9 +2292,9 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) if (has_timeout) timeout = deadline - _PyTime_GetMonotonicClock(); - if (err == SSL_ERROR_WANT_READ) { + if (err.ssl == SSL_ERROR_WANT_READ) { sockstate = PySSL_select(sock, 0, timeout); - } else if (err == SSL_ERROR_WANT_WRITE) { + } else if (err.ssl == SSL_ERROR_WANT_WRITE) { sockstate = PySSL_select(sock, 1, timeout); } else { sockstate = SOCKET_OPERATION_OK; @@ -2309,7 +2311,8 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) } else if (sockstate == SOCKET_IS_NONBLOCKING) { break; } - } while (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE); + } while (err.ssl == SSL_ERROR_WANT_READ || + err.ssl == SSL_ERROR_WANT_WRITE); Py_XDECREF(sock); if (len > 0) @@ -2333,11 +2336,14 @@ _ssl__SSLSocket_pending_impl(PySSLSocket *self) /*[clinic end generated code: output=983d9fecdc308a83 input=2b77487d6dfd597f]*/ { int count = 0; + _PySSLError err; PySSL_BEGIN_ALLOW_THREADS count = SSL_pending(self->ssl); - _PySSL_UPDATE_ERRNO_IF(count < 0, self, count); + err = _PySSL_errno(count < 0, self->ssl, count); PySSL_END_ALLOW_THREADS + self->err = err; + if (count < 0) return PySSL_SetError(self, count, __FILE__, __LINE__); else @@ -2364,7 +2370,7 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, char *mem; int count; int sockstate; - int err; + _PySSLError err; int nonblocking; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -2425,8 +2431,9 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, do { PySSL_BEGIN_ALLOW_THREADS count = SSL_read(self->ssl, mem, len); - _PySSL_UPDATE_ERRNO_IF(count <= 0, self, count); + err = _PySSL_errno(count <= 0, self->ssl, count); PySSL_END_ALLOW_THREADS + self->err = err; if (PyErr_CheckSignals()) goto error; @@ -2434,12 +2441,11 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, if (has_timeout) timeout = deadline - _PyTime_GetMonotonicClock(); - err = self->ssl_errno; - if (err == SSL_ERROR_WANT_READ) { + if (err.ssl == SSL_ERROR_WANT_READ) { sockstate = PySSL_select(sock, 0, timeout); - } else if (err == SSL_ERROR_WANT_WRITE) { + } else if (err.ssl == SSL_ERROR_WANT_WRITE) { sockstate = PySSL_select(sock, 1, timeout); - } else if (err == SSL_ERROR_ZERO_RETURN && + } else if (err.ssl == SSL_ERROR_ZERO_RETURN && SSL_get_shutdown(self->ssl) == SSL_RECEIVED_SHUTDOWN) { count = 0; @@ -2455,7 +2461,8 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, } else if (sockstate == SOCKET_IS_NONBLOCKING) { break; } - } while (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE); + } while (err.ssl == SSL_ERROR_WANT_READ || + err.ssl == SSL_ERROR_WANT_WRITE); if (count <= 0) { PySSL_SetError(self, count, __FILE__, __LINE__); @@ -2489,7 +2496,8 @@ static PyObject * _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) /*[clinic end generated code: output=ca1aa7ed9d25ca42 input=11d39e69b0a2bf4a]*/ { - int err, sockstate, nonblocking; + _PySSLError err; + int sockstate, nonblocking, ret; int zeros = 0; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -2527,14 +2535,15 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) */ if (self->shutdown_seen_zero) SSL_set_read_ahead(self->ssl, 0); - err = SSL_shutdown(self->ssl); - _PySSL_UPDATE_ERRNO_IF(err < 0, self, err); + ret = SSL_shutdown(self->ssl); + err = _PySSL_errno(ret < 0, self->ssl, ret); PySSL_END_ALLOW_THREADS + self->err = err; /* If err == 1, a secure shutdown with SSL_shutdown() is complete */ - if (err > 0) + if (ret > 0) break; - if (err == 0) { + if (ret == 0) { /* Don't loop endlessly; instead preserve legacy behaviour of trying SSL_shutdown() only twice. This looks necessary for OpenSSL < 0.9.8m */ @@ -2549,16 +2558,15 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) timeout = deadline - _PyTime_GetMonotonicClock(); /* Possibly retry shutdown until timeout or failure */ - _PySSL_UPDATE_ERRNO(self, err); - if (self->ssl_errno == SSL_ERROR_WANT_READ) + if (err.ssl == SSL_ERROR_WANT_READ) sockstate = PySSL_select(sock, 0, timeout); - else if (self->ssl_errno == SSL_ERROR_WANT_WRITE) + else if (err.ssl == SSL_ERROR_WANT_WRITE) sockstate = PySSL_select(sock, 1, timeout); else break; if (sockstate == SOCKET_HAS_TIMED_OUT) { - if (self->ssl_errno == SSL_ERROR_WANT_READ) + if (err.ssl == SSL_ERROR_WANT_READ) PyErr_SetString(PySocketModule.timeout_error, "The read operation timed out"); else @@ -2576,9 +2584,9 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) break; } - if (err < 0) { + if (err.ssl < 0) { Py_XDECREF(sock); - return PySSL_SetError(self, err, __FILE__, __LINE__); + return PySSL_SetError(self, err.ssl, __FILE__, __LINE__); } if (sock) /* It's already INCREF'ed */ From webhook-mailer at python.org Mon Sep 17 15:35:28 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Mon, 17 Sep 2018 19:35:28 -0000 Subject: [Python-checkins] bpo-33649: Add low-level APIs index. (GH-9364) Message-ID: https://github.com/python/cpython/commit/394374e30c85f6eacddbbfc7471aab62b54ce021 commit: 394374e30c85f6eacddbbfc7471aab62b54ce021 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-17T15:35:24-04:00 summary: bpo-33649: Add low-level APIs index. (GH-9364) files: A Doc/library/asyncio-llapi-index.rst M Doc/library/asyncio-api-index.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-future.rst M Doc/library/asyncio-protocol.rst M Doc/library/asyncio-stream.rst M Doc/library/asyncio-subprocess.rst M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst M Doc/tools/templates/layout.html diff --git a/Doc/library/asyncio-api-index.rst b/Doc/library/asyncio-api-index.rst index dc691000471c..d5b5659abc65 100644 --- a/Doc/library/asyncio-api-index.rst +++ b/Doc/library/asyncio-api-index.rst @@ -1,9 +1,9 @@ .. currentmodule:: asyncio -===================== -High-level APIs Index -===================== +==================== +High-level API Index +==================== This page lists all high-level async/await enabled asyncio APIs. @@ -16,6 +16,7 @@ await on multiple things with timeouts. .. list-table:: :widths: 50 50 + :class: full-width-table * - :func:`run` - Create event loop, run a coroutine, close the loop. @@ -36,7 +37,7 @@ await on multiple things with timeouts. - Shield from cancellation. * - ``await`` :func:`wait` - - Monitor for completeness. + - Monitor for completion. * - :func:`current_task` - Return the current Task. @@ -47,6 +48,12 @@ await on multiple things with timeouts. * - :class:`Task` - Task object. + * - :func:`run_coroutine_threadsafe` + - Schedule a coroutine from another OS thread. + + * - ``for in`` :func:`as_completed` + - Monitor for completion with a ``for`` loop. + .. rubric:: Examples @@ -72,6 +79,7 @@ implement connection pools, and pub/sub patterns. .. list-table:: :widths: 50 50 + :class: full-width-table * - :class:`Queue` - A FIFO queue. @@ -98,6 +106,7 @@ Utilities to spawn subprocesses and run shell commands. .. list-table:: :widths: 50 50 + :class: full-width-table * - ``await`` :func:`create_subprocess_exec` - Create a subprocess. @@ -121,6 +130,7 @@ High-level APIs to work with network IO. .. list-table:: :widths: 50 50 + :class: full-width-table * - ``await`` :func:`open_connection` - Establish a TCP connection. @@ -156,6 +166,7 @@ Threading-like synchronization primitives that can be used in Tasks. .. list-table:: :widths: 50 50 + :class: full-width-table * - :class:`Lock` - A mutex lock. @@ -186,6 +197,7 @@ Exceptions .. list-table:: :widths: 50 50 + :class: full-width-table * - :exc:`asyncio.TimeoutError` diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 3e1571f72453..909d3ea33632 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -980,7 +980,7 @@ Availability: UNIX. Executing code in thread or process pools ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. method:: loop.run_in_executor(executor, func, \*args) +.. coroutinemethod:: loop.run_in_executor(executor, func, \*args) Arrange for a *func* to be called in the specified executor. @@ -1418,7 +1418,7 @@ need to be written this way; consider using high-level functions like :func:`asyncio.run`. -.. _asyncio-hello-world-callback: +.. _asyncio_example_lowlevel_helloworld: Hello World with call_soon() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1451,7 +1451,7 @@ event loop:: example created with a coroutine and the :func:`run` function. -.. _asyncio-date-callback: +.. _asyncio_example_call_later: Display the current date with call_later() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1488,7 +1488,7 @@ during 5 seconds, and then stops the event loop:: created with a coroutine and the :func:`run` function. -.. _asyncio-watch-read-event: +.. _asyncio_example_watch_fd: Watch a file descriptor for read events ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1531,15 +1531,17 @@ Wait until a file descriptor received some data using the .. seealso:: - * A similar :ref:`example ` + * A similar :ref:`example ` using transports, protocols, and the :meth:`loop.create_connection` method. - * Another similar :ref:`example ` + * Another similar :ref:`example ` using the high-level :func:`asyncio.open_connection` function and streams. +.. _asyncio_example_unix_signals: + Set signal handlers for SIGINT and SIGTERM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index 19bf8a6e058b..d6c5335c0e18 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -191,6 +191,8 @@ Future Object .. versionadded:: 3.7 +.. _asyncio_example_future: + This example creates a Future object, creates and schedules an asynchronous Task to set result for the Future, and waits until the Future has a result:: diff --git a/Doc/library/asyncio-llapi-index.rst b/Doc/library/asyncio-llapi-index.rst new file mode 100644 index 000000000000..7fb1e60f50de --- /dev/null +++ b/Doc/library/asyncio-llapi-index.rst @@ -0,0 +1,510 @@ +.. currentmodule:: asyncio + + +=================== +Low-level API Index +=================== + +This page lists all low-level asyncio APIs. + + +Obtaining the Event Loop +======================== + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :func:`asyncio.get_running_loop` + - The **preferred** function to get the running event loop. + + * - :func:`asyncio.get_event_loop` + - Get an event loop instance (current or via the policy). + + * - :func:`asyncio.set_event_loop` + - Set the event loop as current via the current policy. + + * - :func:`asyncio.new_event_loop` + - Create a new event loop. + + +.. rubric:: Examples + +* :ref:`Using asyncio.get_running_loop() `. + + +Event Loop Methods +================== + +See also the main documentation section about the +:ref:`event loop methods `. + +.. rubric:: Lifecycle +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.run_until_complete` + - Run a Future/Task/awaitable until complete. + + * - :meth:`loop.run_forever` + - Run the event loop forever. + + * - :meth:`loop.stop` + - Stop the event loop. + + * - :meth:`loop.stop` + - Close the event loop. + + * - :meth:`loop.is_running()` + - Return ``True`` if the event loop is running. + + * - :meth:`loop.is_closed()` + - Return ``True`` if the event loop is closed. + + * - ``await`` :meth:`loop.shutdown_asyncgens` + - Close asynchronous generators. + + +.. rubric:: Debugging +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.set_debug` + - Enable or disable the debug mode. + + * - :meth:`loop.get_debug` + - Get the current debug mode. + + +.. rubric:: Scheduling Callbacks +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.call_soon` + - Invoke a callback soon. + + * - :meth:`loop.call_soon_threadsafe` + - A thread-safe variant of :meth:`loop.call_soon`. + + * - :meth:`loop.call_later` + - Invoke a callback *after* the given time. + + * - :meth:`loop.call_at` + - Invoke a callback *at* the given time. + + +.. rubric:: Thread/Process Pool +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :meth:`loop.run_in_executor` + - Run a CPU-bound or other blocking function in + a :mod:`concurrent.futures` executor. + + * - :meth:`loop.set_default_executor` + - Set the default executor for :meth:`loop.run_in_executor`. + + +.. rubric:: Tasks and Futures +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.create_future` + - Create a :class:`Future` object. + + * - :meth:`loop.create_task` + - Schedule coroutine as a :class:`Task`. + + * - :meth:`loop.set_task_factory` + - Set a factory used by :meth:`loop.create_task` to + create :class:`Tasks `. + + * - :meth:`loop.get_task_factory` + - Get the factory :meth:`loop.create_task` uses + to create :class:`Tasks `. + + +.. rubric:: DNS +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :meth:`loop.getaddrinfo` + - Asynchronous version of :meth:`socket.getaddrinfo`. + + * - ``await`` :meth:`loop.getnameinfo` + - Asynchronous version of :meth:`socket.getnameinfo`. + + +.. rubric:: Networking and IPC +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :meth:`loop.create_connection` + - Open a TCP connection. + + * - ``await`` :meth:`loop.create_server` + - Create a TCP server. + + * - ``await`` :meth:`loop.create_unix_connection` + - Open a Unix socket connection. + + * - ``await`` :meth:`loop.create_unix_server` + - Create a Unix socket server. + + * - ``await`` :meth:`loop.connect_accepted_socket` + - Wrap a :class:`~socket.socket` into a ``(transport, protocol)`` + pair. + + * - ``await`` :meth:`loop.create_datagram_endpoint` + - Open a datagram (UDP) connection. + + * - ``await`` :meth:`loop.sendfile` + - Send a file over a transport. + + * - ``await`` :meth:`loop.start_tls` + - Upgrade an existing connection to TLS. + + * - ``await`` :meth:`loop.connect_read_pipe` + - Wrap a read end of a pipe into a ``(transport, protocol)`` pair. + + * - ``await`` :meth:`loop.connect_write_pipe` + - Wrap a write end of a pipe into a ``(transport, protocol)`` pair. + + +.. rubric:: Sockets +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :meth:`loop.sock_recv` + - Receive data from the :class:`~socket.socket`. + + * - ``await`` :meth:`loop.sock_recv_into` + - Receive data from the :class:`~socket.socket` into a buffer. + + * - ``await`` :meth:`loop.sock_sendall` + - Send data to the :class:`~socket.socket`. + + * - ``await`` :meth:`loop.sock_connect` + - Connect the :class:`~socket.socket`. + + * - ``await`` :meth:`loop.sock_accept` + - Accept a :class:`~socket.socket` connection. + + * - ``await`` :meth:`loop.sock_sendfile` + - Send a file over the :class:`~socket.socket`. + + * - :meth:`loop.add_reader` + - Start watching a file descriptor for read availability. + + * - :meth:`loop.remove_reader` + - Stop watching a file descriptor for read availability. + + * - :meth:`loop.add_writer` + - Start watching a file descriptor for write availability. + + * - :meth:`loop.remove_writer` + - Stop watching a file descriptor for write availability. + + +.. rubric:: Unix Signals +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.add_signal_handler` + - Add a handler for a :mod:`signal`. + + * - :meth:`loop.remove_signal_handler` + - Remove a handler for a :mod:`signal`. + + +.. rubric:: Subprocesses +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.subprocess_exec` + - Spawn a subprocess. + + * - :meth:`loop.subprocess_shell` + - Spawn a subprocess from a shell command. + + +.. rubric:: Error Handling +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.call_exception_handler` + - Call the exception handler. + + * - :meth:`loop.set_exception_handler` + - Set a new exception handler. + + * - :meth:`loop.get_exception_handler` + - Get the current exception handler. + + * - :meth:`loop.default_exception_handler` + - The default exception handler implementation. + + +.. rubric:: Examples + +* :ref:`Using asyncio.get_event_loop() and loop.run_forever() + `. + +* :ref:`Using loop.call_later() `. + +* Using ``loop.create_connection()`` to implement + :ref:`an echo-client `. + +* Using ``loop.create_connection()`` to + :ref:`connect a socket `. + +* :ref:`Using add_reader() to watch an FD for read events + `. + +* :ref:`Using loop.add_signal_handler() `. + +* :ref:`Using loop.subprocess_exec() `. + + +Transports +========== + +All transports implement the following methods: + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.close() ` + - Close the transport. + + * - :meth:`transport.is_closing() ` + - Return ``True`` if the transport is closing or is closed. + + * - :meth:`transport.get_extra_info() ` + - Request for information about the transport. + + * - :meth:`transport.set_protocol() ` + - Set a new protocol. + + * - :meth:`transport.get_protocol() ` + - Return the current protocol. + + +Transports that can receive data (TCP and Unix connections, +pipes, etc). Returned from methods like +:meth:`loop.create_connection`, :meth:`loop.create_unix_connection`, +:meth:`loop.connect_read_pipe`, etc: + +.. rubric:: Read Transports +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.is_reading() ` + - Return ``True`` if the transport is receiving. + + * - :meth:`transport.pause_reading() ` + - Pause receiving. + + * - :meth:`transport.resume_reading() ` + - Resume receiving. + + +Transports that can Send data (TCP and Unix connections, +pipes, etc). Returned from methods like +:meth:`loop.create_connection`, :meth:`loop.create_unix_connection`, +:meth:`loop.connect_write_pipe`, etc: + +.. rubric:: Write Transports +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.write() ` + - Write data to the transport. + + * - :meth:`transport.writelines() ` + - Write buffers to the transport. + + * - :meth:`transport.can_write_eof() ` + - Return :const:`True` if the transport supports sending EOF. + + * - :meth:`transport.write_eof() ` + - Close and send EOF after flushing buffered data. + + * - :meth:`transport.abort() ` + - Close the transport immediately. + + * - :meth:`transport.get_write_buffer_size() + ` + - Return high and low water marks for write flow control. + + * - :meth:`transport.set_write_buffer_limits() + ` + - Set new high and low water marks for write flow control. + + +Transports returned by :meth:`loop.create_datagram_endpoint`: + +.. rubric:: Datagram Transports +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.sendto() ` + - Send data to the remote peer. + + * - :meth:`transport.abort() ` + - Close the transport immediately. + + +Low-level transport abstraction over subprocesses. +Returned by :meth:`loop.subprocess_exec` and +:meth:`loop.subprocess_shell`: + +.. rubric:: Subprocess Transports +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.get_pid() ` + - Return the subprocess process id. + + * - :meth:`transport.get_pipe_transport() + ` + - Return the transport for the requested communication pipe + (*stdin*, *stdout*, or *stderr*). + + * - :meth:`transport.get_returncode() ` + - Return the subprocess return code. + + * - :meth:`transport.kill() ` + - Kill the subprocess. + + * - :meth:`transport.send_signal() ` + - Send a signal to the subprocess. + + * - :meth:`transport.terminate() ` + - Stop the subprocess. + + * - :meth:`transport.close() ` + - Kill the subprocess and close all pipes. + + +Protocols +========= + +Protocol classes can implement the following **callback methods**: + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`connection_made() ` + - Called when a connection is made. + + * - ``callback`` :meth:`connection_lost() ` + - Called when the connection is lost or closed. + + * - ``callback`` :meth:`pause_writing() ` + - Called when the transport's buffer goes over the high water mark. + + * - ``callback`` :meth:`resume_writing() ` + - Called when the transport's buffer drains below the low water mark. + + +.. rubric:: Streaming Protocols (TCP, Unix Sockets, Pipes) +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`data_received() ` + - Called when some data is received. + + * - ``callback`` :meth:`eof_received() ` + - Called when an EOF is received. + + +.. rubric:: Buffered Streaming Protocols +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`get_buffer() ` + - Called to allocate a new receive buffer. + + * - ``callback`` :meth:`buffer_updated() ` + - Called when the buffer was updated with the received data. + + * - ``callback`` :meth:`eof_received() ` + - Called when an EOF is received. + + +.. rubric:: Datagram Protocols +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`datagram_received() + ` + - Called when a datagram is received. + + * - ``callback`` :meth:`error_received() ` + - Called when a previous send or receive operation raises an + :class:`OSError`. + + +.. rubric:: Subprocess Protocols +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`pipe_data_received() + ` + - Called when the child process writes data into its + *stdout* or *stderr* pipe. + + * - ``callback`` :meth:`pipe_connection_lost() + ` + - Called when one of the pipes communicating with + the child process is closed. + + * - ``callback`` :meth:`process_exited() + ` + - Called when the child process has exited. + + +Event Loop Policies +=================== + +Policies is a low-level mechanism to alter the behavior of +functions like :func:`asyncio.get_event_loop`. See also +the main :ref:`policies section ` for more +details. + + +.. rubric:: Accessing Policies +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`asyncio.get_event_loop_policy` + - Return the current process-wide policy. + + * - :meth:`asyncio.set_event_loop_policy` + - Set a new process-wide policy. + + * - :class:`AbstractEventLoopPolicy` + - Base class for policy objects. diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst index e4aed647c338..cb16acd58be2 100644 --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -707,7 +707,7 @@ factories passed to the :meth:`loop.subprocess_exec` and Examples ======== -.. _asyncio-tcp-echo-server-protocol: +.. _asyncio_example_tcp_echo_server_protocol: TCP Echo Server --------------- @@ -756,7 +756,7 @@ received data, and close the connection:: The :ref:`TCP echo server using streams ` example uses the high-level :func:`asyncio.start_server` function. -.. _asyncio-tcp-echo-client-protocol: +.. _asyncio_example_tcp_echo_client_protocol: TCP Echo Client --------------- @@ -914,7 +914,7 @@ method, sends data and closes the transport when it receives the answer:: asyncio.run(main()) -.. _asyncio-register-socket: +.. _asyncio_example_create_connection: Connecting Existing Sockets --------------------------- @@ -973,14 +973,14 @@ Wait until a socket receives data using the .. seealso:: The :ref:`watch a file descriptor for read events - ` example uses the low-level + ` example uses the low-level :meth:`loop.add_reader` method to register an FD. The :ref:`register an open socket to wait for data using streams - ` example uses high-level streams + ` example uses high-level streams created by the :func:`open_connection` function in a coroutine. -.. _asyncio-subprocess-proto-example: +.. _asyncio_example_subprocess_proto: loop.subprocess_exec() and SubprocessProtocol --------------------------------------------- @@ -1037,3 +1037,6 @@ The subprocess is created by th :meth:`loop.subprocess_exec` method:: date = asyncio.run(get_date()) print(f"Current date: {date}") + +See also the :ref:`same example ` +written using high-level APIs. diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 29163a217dce..0489201b4d02 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -348,7 +348,7 @@ TCP echo client using the :func:`asyncio.open_connection` function:: .. seealso:: - The :ref:`TCP echo client protocol ` + The :ref:`TCP echo client protocol ` example uses the low-level :meth:`loop.create_connection` method. @@ -390,7 +390,7 @@ TCP echo server using the :func:`asyncio.start_server` function:: .. seealso:: - The :ref:`TCP echo server protocol ` + The :ref:`TCP echo server protocol ` example uses the :meth:`loop.create_server` method. @@ -444,7 +444,7 @@ or with HTTPS:: python example.py https://example.com/path/page.html -.. _asyncio-register-socket-streams: +.. _asyncio_example_create_connection-streams: Register an open socket to wait for data using streams ------------------------------------------------------ @@ -484,9 +484,9 @@ Coroutine waiting until a socket receives data using the .. seealso:: The :ref:`register an open socket to wait for data using a protocol - ` example uses a low-level protocol and + ` example uses a low-level protocol and the :meth:`loop.create_connection` method. The :ref:`watch a file descriptor for read events - ` example uses the low-level + ` example uses the low-level :meth:`loop.add_reader` method to watch a file descriptor. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index ef8a1cbb4b1c..57a7a378b8fd 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -318,6 +318,8 @@ An example using the :class:`~asyncio.subprocess.Process` class to control a subprocess and the :class:`StreamReader` class to read from the *stdout*. +.. _asyncio_example_create_subprocess_exec: + The subprocess is created by the :func:`create_subprocess_exec` function:: @@ -349,5 +351,5 @@ function:: print(f"Current date: {date}") -See also the :ref:`same example ` +See also the :ref:`same example ` written using low-level APIs. diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index d3cfd5ff31ce..3faaf0cdb9b8 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -130,7 +130,7 @@ Running an asyncio Program programs, and should ideally only be called once. .. versionadded:: 3.7 - **Important:** this has been been added to asyncio in Python 3.7 + **Important:** this has been added to asyncio in Python 3.7 on a :term:`provisional basis `. @@ -188,7 +188,7 @@ Sleeping Running Tasks Concurrently ========================== -.. coroutinefunction:: gather(\*fs, loop=None, return_exceptions=False) +.. function:: gather(\*fs, loop=None, return_exceptions=False) Return a Future aggregating results from the given coroutine objects, Tasks, or Futures. diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 9f45f7458c37..7895826c65a0 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -43,12 +43,13 @@ as well as **low-level** APIs for *library and framework developers* to: with async/await syntax. -Reference ---------- +.. We use the "rubric" directive here to avoid creating + the "Reference" subsection in the TOC. -.. rubric:: High-level APIs +.. rubric:: Reference .. toctree:: + :caption: High-level APIs :maxdepth: 1 asyncio-task.rst @@ -58,9 +59,8 @@ Reference asyncio-queue.rst asyncio-exceptions.rst -.. rubric:: Low-level APIs - .. toctree:: + :caption: Low-level APIs :maxdepth: 1 asyncio-eventloop.rst @@ -69,10 +69,10 @@ Reference asyncio-policy.rst asyncio-platforms.rst -.. rubric:: Guides and Tutorials - .. toctree:: + :caption: Guides and Tutorials :maxdepth: 1 asyncio-api-index.rst + asyncio-llapi-index.rst asyncio-dev.rst diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index 37811725d86b..c39922456140 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -21,5 +21,14 @@ {% if pagename == 'whatsnew/changelog' and not embedded %} {% endif %} {% endif %} + + {# custom CSS; used in asyncio docs! #} + {{ super() }} {% endblock %} From webhook-mailer at python.org Mon Sep 17 16:56:20 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 17 Sep 2018 20:56:20 -0000 Subject: [Python-checkins] bpo-34715: Revert "Simplify PyInit_timezone. (GH-9323)" (GH-9366) Message-ID: https://github.com/python/cpython/commit/1fb399ba4e977e697d194769070316247237f68e commit: 1fb399ba4e977e697d194769070316247237f68e branch: master author: Victor Stinner committer: GitHub date: 2018-09-17T13:56:17-07:00 summary: bpo-34715: Revert "Simplify PyInit_timezone. (GH-9323)" (GH-9366) This reverts commit afde1c1a05cc8a1e8adf6403c451f6708509a605. files: M Modules/timemodule.c diff --git a/Modules/timemodule.c b/Modules/timemodule.c index b9a71b48414a..1a4cff23d65e 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1522,6 +1522,29 @@ PyDoc_STRVAR(get_clock_info_doc, \n\ Get information of the specified clock."); +#if !defined(HAVE_TZNAME) || defined(__GLIBC__) || defined(__CYGWIN__) +static void +get_zone(char *zone, int n, struct tm *p) +{ +#ifdef HAVE_STRUCT_TM_TM_ZONE + strncpy(zone, p->tm_zone ? p->tm_zone : " ", n); +#else + tzset(); + strftime(zone, n, "%Z", p); +#endif +} + +static int +get_gmtoff(time_t t, struct tm *p) +{ +#ifdef HAVE_STRUCT_TM_TM_ZONE + return p->tm_gmtoff; +#else + return timegm(p) - t; +#endif +} +#endif /* !defined(HAVE_TZNAME) || defined(__GLIBC__) || defined(__CYGWIN__) */ + static void PyInit_timezone(PyObject *m) { /* This code moved from PyInit_time wholesale to allow calling it from @@ -1540,35 +1563,65 @@ PyInit_timezone(PyObject *m) { And I'm lazy and hate C so nyer. */ +#if defined(HAVE_TZNAME) && !defined(__GLIBC__) && !defined(__CYGWIN__) PyObject *otz0, *otz1; tzset(); PyModule_AddIntConstant(m, "timezone", timezone); #ifdef HAVE_ALTZONE PyModule_AddIntConstant(m, "altzone", altzone); -#elif defined(HAVE_STRUCT_TM_TM_ZONE) +#else + PyModule_AddIntConstant(m, "altzone", timezone-3600); +#endif + PyModule_AddIntConstant(m, "daylight", daylight); + otz0 = PyUnicode_DecodeLocale(tzname[0], "surrogateescape"); + otz1 = PyUnicode_DecodeLocale(tzname[1], "surrogateescape"); + PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1)); +#else /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/ { - static const time_t YEAR = (365 * 24 + 6) * 3600; +#define YEAR ((time_t)((365 * 24 + 6) * 3600)) time_t t; struct tm p; long janzone, julyzone; + char janname[10], julyname[10]; t = (time((time_t *)0) / YEAR) * YEAR; _PyTime_localtime(t, &p); - janzone = -p.tm_gmtoff; + get_zone(janname, 9, &p); + janzone = -get_gmtoff(t, &p); + janname[9] = '\0'; t += YEAR/2; _PyTime_localtime(t, &p); - julyzone = -p.tm_gmtoff; - - // DST is reversed in the southern hemisphere. - PyModule_AddIntConstant(m, "altzone", - (janzone < julyzone) ? janzone : julyzone); + get_zone(julyname, 9, &p); + julyzone = -get_gmtoff(t, &p); + julyname[9] = '\0'; + + if( janzone < julyzone ) { + /* DST is reversed in the southern hemisphere */ + PyModule_AddIntConstant(m, "timezone", julyzone); + PyModule_AddIntConstant(m, "altzone", janzone); + PyModule_AddIntConstant(m, "daylight", + janzone != julyzone); + PyModule_AddObject(m, "tzname", + Py_BuildValue("(zz)", + julyname, janname)); + } else { + PyModule_AddIntConstant(m, "timezone", janzone); + PyModule_AddIntConstant(m, "altzone", julyzone); + PyModule_AddIntConstant(m, "daylight", + janzone != julyzone); + PyModule_AddObject(m, "tzname", + Py_BuildValue("(zz)", + janname, julyname)); + } } -#else - PyModule_AddIntConstant(m, "altzone", timezone-3600); -#endif - PyModule_AddIntConstant(m, "daylight", daylight); - otz0 = PyUnicode_DecodeLocale(tzname[0], "surrogateescape"); - otz1 = PyUnicode_DecodeLocale(tzname[1], "surrogateescape"); - PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1)); +#ifdef __CYGWIN__ + tzset(); + PyModule_AddIntConstant(m, "timezone", _timezone); + PyModule_AddIntConstant(m, "altzone", _timezone-3600); + PyModule_AddIntConstant(m, "daylight", _daylight); + PyModule_AddObject(m, "tzname", + Py_BuildValue("(zz)", _tzname[0], _tzname[1])); +#endif /* __CYGWIN__ */ +#endif /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/ } From webhook-mailer at python.org Mon Sep 17 17:01:25 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 17 Sep 2018 21:01:25 -0000 Subject: [Python-checkins] bpo-34587, test_socket: remove RDSTest.testCongestion() (GH-9277) Message-ID: https://github.com/python/cpython/commit/7484bdfd1e2e33fdd2c44dd4ffa044aacd495337 commit: 7484bdfd1e2e33fdd2c44dd4ffa044aacd495337 branch: master author: Victor Stinner committer: GitHub date: 2018-09-17T14:01:20-07:00 summary: bpo-34587, test_socket: remove RDSTest.testCongestion() (GH-9277) The test tries to fill the receiver's socket buffer and expects an error. But the RDS protocol doesn't require that. Moreover, the Linux implementation of RDS expects that the producer of the messages reduces its rate, it's not the role of the receiver to trigger an error. The test fails on Fedora 28 by design, so remove it. files: A Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index 4f3c4774e40f..f4d58ebf7157 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -2054,33 +2054,6 @@ def _testSelect(self): self.data = b'select' self.cli.sendto(self.data, 0, (HOST, self.port)) - def testCongestion(self): - # wait until the sender is done - self.evt.wait() - - def _testCongestion(self): - # test the behavior in case of congestion - self.data = b'fill' - self.cli.setblocking(False) - try: - # try to lower the receiver's socket buffer size - self.cli.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 16384) - except OSError: - pass - with self.assertRaises(OSError) as cm: - try: - # fill the receiver's socket buffer - while True: - self.cli.sendto(self.data, 0, (HOST, self.port)) - finally: - # signal the receiver we're done - self.evt.set() - # sendto() should have failed with ENOBUFS - self.assertEqual(cm.exception.errno, errno.ENOBUFS) - # and we should have received a congestion notification through poll - r, w, x = select.select([self.serv], [], [], 3.0) - self.assertIn(self.serv, r) - @unittest.skipIf(fcntl is None, "need fcntl") @unittest.skipUnless(HAVE_SOCKET_VSOCK, diff --git a/Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst b/Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst new file mode 100644 index 000000000000..8d45418aeab4 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst @@ -0,0 +1,5 @@ +test_socket: Remove RDSTest.testCongestion(). The test tries to fill the +receiver's socket buffer and expects an error. But the RDS protocol doesn't +require that. Moreover, the Linux implementation of RDS expects that the +producer of the messages reduces its rate, it's not the role of the receiver to +trigger an error. The test fails on Fedora 28 by design, so just remove it. From webhook-mailer at python.org Mon Sep 17 17:28:03 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 21:28:03 -0000 Subject: [Python-checkins] bpo-34587, test_socket: remove RDSTest.testCongestion() (GH-9277) Message-ID: https://github.com/python/cpython/commit/b7f58d7f80f80f0e20cad84773f158a379a19280 commit: b7f58d7f80f80f0e20cad84773f158a379a19280 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T14:27:59-07:00 summary: bpo-34587, test_socket: remove RDSTest.testCongestion() (GH-9277) The test tries to fill the receiver's socket buffer and expects an error. But the RDS protocol doesn't require that. Moreover, the Linux implementation of RDS expects that the producer of the messages reduces its rate, it's not the role of the receiver to trigger an error. The test fails on Fedora 28 by design, so remove it. (cherry picked from commit 7484bdfd1e2e33fdd2c44dd4ffa044aacd495337) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index 4f3c4774e40f..f4d58ebf7157 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -2054,33 +2054,6 @@ def _testSelect(self): self.data = b'select' self.cli.sendto(self.data, 0, (HOST, self.port)) - def testCongestion(self): - # wait until the sender is done - self.evt.wait() - - def _testCongestion(self): - # test the behavior in case of congestion - self.data = b'fill' - self.cli.setblocking(False) - try: - # try to lower the receiver's socket buffer size - self.cli.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 16384) - except OSError: - pass - with self.assertRaises(OSError) as cm: - try: - # fill the receiver's socket buffer - while True: - self.cli.sendto(self.data, 0, (HOST, self.port)) - finally: - # signal the receiver we're done - self.evt.set() - # sendto() should have failed with ENOBUFS - self.assertEqual(cm.exception.errno, errno.ENOBUFS) - # and we should have received a congestion notification through poll - r, w, x = select.select([self.serv], [], [], 3.0) - self.assertIn(self.serv, r) - @unittest.skipIf(fcntl is None, "need fcntl") @unittest.skipUnless(HAVE_SOCKET_VSOCK, diff --git a/Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst b/Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst new file mode 100644 index 000000000000..8d45418aeab4 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst @@ -0,0 +1,5 @@ +test_socket: Remove RDSTest.testCongestion(). The test tries to fill the +receiver's socket buffer and expects an error. But the RDS protocol doesn't +require that. Moreover, the Linux implementation of RDS expects that the +producer of the messages reduces its rate, it's not the role of the receiver to +trigger an error. The test fails on Fedora 28 by design, so just remove it. From webhook-mailer at python.org Mon Sep 17 17:40:25 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 21:40:25 -0000 Subject: [Python-checkins] bpo-34587, test_socket: remove RDSTest.testCongestion() (GH-9277) Message-ID: https://github.com/python/cpython/commit/68a8f041051e8387583c66b91c7a3bbda6cf7e63 commit: 68a8f041051e8387583c66b91c7a3bbda6cf7e63 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T14:40:22-07:00 summary: bpo-34587, test_socket: remove RDSTest.testCongestion() (GH-9277) The test tries to fill the receiver's socket buffer and expects an error. But the RDS protocol doesn't require that. Moreover, the Linux implementation of RDS expects that the producer of the messages reduces its rate, it's not the role of the receiver to trigger an error. The test fails on Fedora 28 by design, so remove it. (cherry picked from commit 7484bdfd1e2e33fdd2c44dd4ffa044aacd495337) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index 77357868d9e1..cd249ff9b444 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -1773,33 +1773,6 @@ def _testSelect(self): self.data = b'select' self.cli.sendto(self.data, 0, (HOST, self.port)) - def testCongestion(self): - # wait until the sender is done - self.evt.wait() - - def _testCongestion(self): - # test the behavior in case of congestion - self.data = b'fill' - self.cli.setblocking(False) - try: - # try to lower the receiver's socket buffer size - self.cli.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 16384) - except OSError: - pass - with self.assertRaises(OSError) as cm: - try: - # fill the receiver's socket buffer - while True: - self.cli.sendto(self.data, 0, (HOST, self.port)) - finally: - # signal the receiver we're done - self.evt.set() - # sendto() should have failed with ENOBUFS - self.assertEqual(cm.exception.errno, errno.ENOBUFS) - # and we should have received a congestion notification through poll - r, w, x = select.select([self.serv], [], [], 3.0) - self.assertIn(self.serv, r) - @unittest.skipUnless(thread, 'Threading required for this test.') class BasicTCPTest(SocketConnectedTest): diff --git a/Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst b/Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst new file mode 100644 index 000000000000..8d45418aeab4 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-13-20-58-07.bpo-34587.rCcxp3.rst @@ -0,0 +1,5 @@ +test_socket: Remove RDSTest.testCongestion(). The test tries to fill the +receiver's socket buffer and expects an error. But the RDS protocol doesn't +require that. Moreover, the Linux implementation of RDS expects that the +producer of the messages reduces its rate, it's not the role of the receiver to +trigger an error. The test fails on Fedora 28 by design, so just remove it. From webhook-mailer at python.org Mon Sep 17 17:41:46 2018 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 17 Sep 2018 21:41:46 -0000 Subject: [Python-checkins] bpo-32533: Fixed thread-safety of error handling in _ssl. (GH-7158) Message-ID: https://github.com/python/cpython/commit/1a107eea8e91e50c5c9025e945c78eb1aa9b874d commit: 1a107eea8e91e50c5c9025e945c78eb1aa9b874d branch: 3.6 author: Steve Dower committer: GitHub date: 2018-09-17T14:41:43-07:00 summary: bpo-32533: Fixed thread-safety of error handling in _ssl. (GH-7158) files: A Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst b/Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst new file mode 100644 index 000000000000..a3642258edaf --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-05-28-08-55-30.bpo-32533.IzwkBI.rst @@ -0,0 +1 @@ +Fixed thread-safety of error handling in _ssl. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 99f24a5b37f2..327f6ae7018b 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -325,6 +325,14 @@ typedef struct { int check_hostname; } PySSLContext; +typedef struct { + int ssl; /* last seen error from SSL */ + int c; /* last seen error from libc */ +#ifdef MS_WINDOWS + int ws; /* last seen error from winsock */ +#endif +} _PySSLError; + typedef struct { PyObject_HEAD PyObject *Socket; /* weakref to socket on which we're layered */ @@ -334,11 +342,7 @@ typedef struct { enum py_ssl_server_or_client socket_type; PyObject *owner; /* Python level "owner" passed to servername callback */ PyObject *server_hostname; - int ssl_errno; /* last seen error from SSL */ - int c_errno; /* last seen error from libc */ -#ifdef MS_WINDOWS - int ws_errno; /* last seen error from winsock */ -#endif + _PySSLError err; /* last seen error from various sources */ } PySSLSocket; typedef struct { @@ -358,19 +362,18 @@ static PyTypeObject PySSLSocket_Type; static PyTypeObject PySSLMemoryBIO_Type; static PyTypeObject PySSLSession_Type; +static inline _PySSLError _PySSL_errno(int failed, const SSL *ssl, int retcode) +{ + _PySSLError err = { 0 }; + if (failed) { #ifdef MS_WINDOWS -#define _PySSL_UPDATE_ERRNO_IF(cond, sock, retcode) if (cond) { \ - (sock)->ws_errno = WSAGetLastError(); \ - (sock)->c_errno = errno; \ - (sock)->ssl_errno = SSL_get_error((sock->ssl), (retcode)); \ - } else { sock->ws_errno = 0; sock->c_errno = 0; sock->ssl_errno = 0; } -#else -#define _PySSL_UPDATE_ERRNO_IF(cond, sock, retcode) if (cond) { \ - (sock)->c_errno = errno; \ - (sock)->ssl_errno = SSL_get_error((sock->ssl), (retcode)); \ - } else { (sock)->c_errno = 0; (sock)->ssl_errno = 0; } + err.ws = WSAGetLastError(); #endif -#define _PySSL_UPDATE_ERRNO(sock, retcode) _PySSL_UPDATE_ERRNO_IF(1, (sock), (retcode)) + err.c = errno; + err.ssl = SSL_get_error(ssl, retcode); + } + return err; +} /*[clinic input] module _ssl @@ -537,7 +540,7 @@ PySSL_SetError(PySSLSocket *obj, int ret, const char *filename, int lineno) { PyObject *type = PySSLErrorObject; char *errstr = NULL; - int err; + _PySSLError err; enum py_ssl_error p = PY_SSL_ERROR_NONE; unsigned long e = 0; @@ -545,9 +548,9 @@ PySSL_SetError(PySSLSocket *obj, int ret, const char *filename, int lineno) e = ERR_peek_last_error(); if (obj->ssl != NULL) { - err = obj->ssl_errno; + err = obj->err; - switch (err) { + switch (err.ssl) { case SSL_ERROR_ZERO_RETURN: errstr = "TLS/SSL connection has been closed (EOF)"; type = PySSLZeroReturnErrorObject; @@ -583,11 +586,12 @@ PySSL_SetError(PySSLSocket *obj, int ret, const char *filename, int lineno) /* underlying BIO reported an I/O error */ ERR_clear_error(); #ifdef MS_WINDOWS - if (obj->ws_errno) - return PyErr_SetFromWindowsErr(obj->ws_errno); + if (err.ws) { + return PyErr_SetFromWindowsErr(err.ws); + } #endif - if (obj->c_errno) { - errno = obj->c_errno; + if (err.c) { + errno = err.c; return PyErr_SetFromErrno(PyExc_OSError); } Py_INCREF(s); @@ -647,6 +651,7 @@ newPySSLSocket(PySSLContext *sslctx, PySocketSockObject *sock, PySSLSocket *self; SSL_CTX *ctx = sslctx->ctx; long mode; + _PySSLError err = { 0 }; self = PyObject_New(PySSLSocket, &PySSLSocket_Type); if (self == NULL) @@ -668,11 +673,7 @@ newPySSLSocket(PySSLContext *sslctx, PySocketSockObject *sock, } self->server_hostname = hostname; } - self->ssl_errno = 0; - self->c_errno = 0; -#ifdef MS_WINDOWS - self->ws_errno = 0; -#endif + self->err = err; /* Make sure the SSL error state is initialized */ (void) ERR_get_state(); @@ -773,7 +774,7 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) /*[clinic end generated code: output=6c0898a8936548f6 input=d2d737de3df018c8]*/ { int ret; - int err; + _PySSLError err; int sockstate, nonblocking; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -803,9 +804,9 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) do { PySSL_BEGIN_ALLOW_THREADS ret = SSL_do_handshake(self->ssl); - _PySSL_UPDATE_ERRNO_IF(ret < 1, self, ret); + err = _PySSL_errno(ret < 1, self->ssl, ret); PySSL_END_ALLOW_THREADS - err = self->ssl_errno; + self->err = err; if (PyErr_CheckSignals()) goto error; @@ -813,9 +814,9 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) if (has_timeout) timeout = deadline - _PyTime_GetMonotonicClock(); - if (err == SSL_ERROR_WANT_READ) { + if (err.ssl == SSL_ERROR_WANT_READ) { sockstate = PySSL_select(sock, 0, timeout); - } else if (err == SSL_ERROR_WANT_WRITE) { + } else if (err.ssl == SSL_ERROR_WANT_WRITE) { sockstate = PySSL_select(sock, 1, timeout); } else { sockstate = SOCKET_OPERATION_OK; @@ -836,7 +837,8 @@ _ssl__SSLSocket_do_handshake_impl(PySSLSocket *self) } else if (sockstate == SOCKET_IS_NONBLOCKING) { break; } - } while (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE); + } while (err.ssl == SSL_ERROR_WANT_READ || + err.ssl == SSL_ERROR_WANT_WRITE); Py_XDECREF(sock); if (ret < 1) return PySSL_SetError(self, ret, __FILE__, __LINE__); @@ -2050,7 +2052,7 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) { int len; int sockstate; - int err; + _PySSLError err; int nonblocking; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -2101,9 +2103,9 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) do { PySSL_BEGIN_ALLOW_THREADS len = SSL_write(self->ssl, b->buf, (int)b->len); - _PySSL_UPDATE_ERRNO_IF(len <= 0, self, len); + err = _PySSL_errno(len <= 0, self->ssl, len); PySSL_END_ALLOW_THREADS - err = self->ssl_errno; + self->err = err; if (PyErr_CheckSignals()) goto error; @@ -2111,9 +2113,9 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) if (has_timeout) timeout = deadline - _PyTime_GetMonotonicClock(); - if (err == SSL_ERROR_WANT_READ) { + if (err.ssl == SSL_ERROR_WANT_READ) { sockstate = PySSL_select(sock, 0, timeout); - } else if (err == SSL_ERROR_WANT_WRITE) { + } else if (err.ssl == SSL_ERROR_WANT_WRITE) { sockstate = PySSL_select(sock, 1, timeout); } else { sockstate = SOCKET_OPERATION_OK; @@ -2130,7 +2132,8 @@ _ssl__SSLSocket_write_impl(PySSLSocket *self, Py_buffer *b) } else if (sockstate == SOCKET_IS_NONBLOCKING) { break; } - } while (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE); + } while (err.ssl == SSL_ERROR_WANT_READ || + err.ssl == SSL_ERROR_WANT_WRITE); Py_XDECREF(sock); if (len > 0) @@ -2154,11 +2157,14 @@ _ssl__SSLSocket_pending_impl(PySSLSocket *self) /*[clinic end generated code: output=983d9fecdc308a83 input=2b77487d6dfd597f]*/ { int count = 0; + _PySSLError err; PySSL_BEGIN_ALLOW_THREADS count = SSL_pending(self->ssl); - _PySSL_UPDATE_ERRNO_IF(count < 0, self, count); + err = _PySSL_errno(count < 0, self->ssl, count); PySSL_END_ALLOW_THREADS + self->err = err; + if (count < 0) return PySSL_SetError(self, count, __FILE__, __LINE__); else @@ -2185,7 +2191,7 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, char *mem; int count; int sockstate; - int err; + _PySSLError err; int nonblocking; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -2246,8 +2252,9 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, do { PySSL_BEGIN_ALLOW_THREADS count = SSL_read(self->ssl, mem, len); - _PySSL_UPDATE_ERRNO_IF(count <= 0, self, count); + err = _PySSL_errno(count <= 0, self->ssl, count); PySSL_END_ALLOW_THREADS + self->err = err; if (PyErr_CheckSignals()) goto error; @@ -2255,12 +2262,11 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, if (has_timeout) timeout = deadline - _PyTime_GetMonotonicClock(); - err = self->ssl_errno; - if (err == SSL_ERROR_WANT_READ) { + if (err.ssl == SSL_ERROR_WANT_READ) { sockstate = PySSL_select(sock, 0, timeout); - } else if (err == SSL_ERROR_WANT_WRITE) { + } else if (err.ssl == SSL_ERROR_WANT_WRITE) { sockstate = PySSL_select(sock, 1, timeout); - } else if (err == SSL_ERROR_ZERO_RETURN && + } else if (err.ssl == SSL_ERROR_ZERO_RETURN && SSL_get_shutdown(self->ssl) == SSL_RECEIVED_SHUTDOWN) { count = 0; @@ -2276,7 +2282,8 @@ _ssl__SSLSocket_read_impl(PySSLSocket *self, int len, int group_right_1, } else if (sockstate == SOCKET_IS_NONBLOCKING) { break; } - } while (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE); + } while (err.ssl == SSL_ERROR_WANT_READ || + err.ssl == SSL_ERROR_WANT_WRITE); if (count <= 0) { PySSL_SetError(self, count, __FILE__, __LINE__); @@ -2312,7 +2319,8 @@ static PyObject * _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) /*[clinic end generated code: output=ca1aa7ed9d25ca42 input=ede2cc1a2ddf0ee4]*/ { - int err, sockstate, nonblocking; + _PySSLError err; + int sockstate, nonblocking, ret; int zeros = 0; PySocketSockObject *sock = GET_SOCKET(self); _PyTime_t timeout, deadline = 0; @@ -2350,14 +2358,15 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) */ if (self->shutdown_seen_zero) SSL_set_read_ahead(self->ssl, 0); - err = SSL_shutdown(self->ssl); - _PySSL_UPDATE_ERRNO_IF(err < 0, self, err); + ret = SSL_shutdown(self->ssl); + err = _PySSL_errno(ret < 0, self->ssl, ret); PySSL_END_ALLOW_THREADS + self->err = err; /* If err == 1, a secure shutdown with SSL_shutdown() is complete */ - if (err > 0) + if (ret > 0) break; - if (err == 0) { + if (ret == 0) { /* Don't loop endlessly; instead preserve legacy behaviour of trying SSL_shutdown() only twice. This looks necessary for OpenSSL < 0.9.8m */ @@ -2372,16 +2381,15 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) timeout = deadline - _PyTime_GetMonotonicClock(); /* Possibly retry shutdown until timeout or failure */ - _PySSL_UPDATE_ERRNO(self, err); - if (self->ssl_errno == SSL_ERROR_WANT_READ) + if (err.ssl == SSL_ERROR_WANT_READ) sockstate = PySSL_select(sock, 0, timeout); - else if (self->ssl_errno == SSL_ERROR_WANT_WRITE) + else if (err.ssl == SSL_ERROR_WANT_WRITE) sockstate = PySSL_select(sock, 1, timeout); else break; if (sockstate == SOCKET_HAS_TIMED_OUT) { - if (self->ssl_errno == SSL_ERROR_WANT_READ) + if (err.ssl == SSL_ERROR_WANT_READ) PyErr_SetString(PySocketModule.timeout_error, "The read operation timed out"); else @@ -2399,9 +2407,9 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) break; } - if (err < 0) { + if (err.ssl < 0) { Py_XDECREF(sock); - return PySSL_SetError(self, err, __FILE__, __LINE__); + return PySSL_SetError(self, err.ssl, __FILE__, __LINE__); } if (sock) /* It's already INCREF'ed */ From webhook-mailer at python.org Mon Sep 17 17:41:56 2018 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 17 Sep 2018 21:41:56 -0000 Subject: [Python-checkins] bpo-34267: Update find_python.bat to use 3.7 if available (GH-8552) Message-ID: https://github.com/python/cpython/commit/6750922f8d3428d84a016c34d6fcd99659e8610c commit: 6750922f8d3428d84a016c34d6fcd99659e8610c branch: master author: Steve Dower committer: GitHub date: 2018-09-17T14:41:53-07:00 summary: bpo-34267: Update find_python.bat to use 3.7 if available (GH-8552) files: M PCbuild/find_python.bat diff --git a/PCbuild/find_python.bat b/PCbuild/find_python.bat index 17ce7ac609c1..d0778ddd7347 100644 --- a/PCbuild/find_python.bat +++ b/PCbuild/find_python.bat @@ -34,7 +34,7 @@ @if NOT "%HOST_PYTHON%"=="" @%HOST_PYTHON% -Ec "import sys; assert sys.version_info[:2] >= (3, 6)" >nul 2>nul && (set PYTHON="%HOST_PYTHON%") && (set _Py_Python_Source=found as HOST_PYTHON) && goto :found @rem If py.exe finds a recent enough version, use that one - at py -3.6 -EV >nul 2>&1 && (set PYTHON=py -3.6) && (set _Py_Python_Source=found with py.exe) && goto :found + at for %%p in (3.7 3.6) do @py -%%p -EV >nul 2>&1 && (set PYTHON=py -%%p) && (set _Py_Python_Source=found %%p with py.exe) && goto :found @if NOT exist "%_Py_EXTERNALS_DIR%" mkdir "%_Py_EXTERNALS_DIR%" @set _Py_NUGET=%NUGET% @@ -55,7 +55,7 @@ ) @echo Installing Python via nuget... @"%_Py_NUGET%" install pythonx86 -ExcludeVersion -OutputDirectory "%_Py_EXTERNALS_DIR%" - at rem Quote it here; it's not quoted later because "py -3.6" wouldn't work + at rem Quote it here; it's not quoted later because "py -x.y" wouldn't work @if not errorlevel 1 (set PYTHON="%_Py_EXTERNALS_DIR%\pythonx86\tools\python.exe") & (set _Py_Python_Source=found on nuget.org) & goto :found From webhook-mailer at python.org Mon Sep 17 18:12:24 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Mon, 17 Sep 2018 22:12:24 -0000 Subject: [Python-checkins] bpo-34717: Stop numbering stdlib titles/sections in the docs (GH-9370) Message-ID: https://github.com/python/cpython/commit/c62ab2862db2382808bb2228760eebdda3f608bd commit: c62ab2862db2382808bb2228760eebdda3f608bd branch: master author: Yury Selivanov committer: GitHub date: 2018-09-17T18:12:21-04:00 summary: bpo-34717: Stop numbering stdlib titles/sections in the docs (GH-9370) files: M Doc/library/index.rst diff --git a/Doc/library/index.rst b/Doc/library/index.rst index b8fbf44ae49f..bebf7429b0e6 100644 --- a/Doc/library/index.rst +++ b/Doc/library/index.rst @@ -32,10 +32,11 @@ several thousand components (from individual programs and modules to packages and entire application development frameworks), available from the `Python Package Index `_. - +.. We don't use :numbered: option for the TOC below as it enforces + numbered sections for the entire stdlib docs. If desired, + :numbered: can be enabled on a per-module basis. .. toctree:: :maxdepth: 2 - :numbered: intro.rst functions.rst From webhook-mailer at python.org Mon Sep 17 18:13:21 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 17 Sep 2018 22:13:21 -0000 Subject: [Python-checkins] bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371) Message-ID: https://github.com/python/cpython/commit/188ebfa475a6f6aa2d0ea14ca8e1fbe7865b6d27 commit: 188ebfa475a6f6aa2d0ea14ca8e1fbe7865b6d27 branch: master author: Victor Stinner committer: GitHub date: 2018-09-17T15:13:17-07:00 summary: bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371) _PyCoreConfig: * Rename coerce_c_locale to _coerce_c_locale * Rename coerce_c_locale_warn to _coerce_c_locale_warn These fields are now private (name prefixed by "_"). files: M Include/coreconfig.h M Lib/test/test_embed.py M Modules/_testcapimodule.c M Modules/main.c M Programs/_testembed.c M Python/coreconfig.c M Python/pylifecycle.c diff --git a/Include/coreconfig.h b/Include/coreconfig.h index ef043ab02df6..8944ec23df09 100644 --- a/Include/coreconfig.h +++ b/Include/coreconfig.h @@ -63,8 +63,6 @@ typedef struct { int show_alloc_count; /* -X showalloccount */ int dump_refs; /* PYTHONDUMPREFS */ int malloc_stats; /* PYTHONMALLOCSTATS */ - int coerce_c_locale; /* PYTHONCOERCECLOCALE, -1 means unknown */ - int coerce_c_locale_warn; /* PYTHONCOERCECLOCALE=warn */ /* Python filesystem encoding and error handler: sys.getfilesystemencoding() and sys.getfilesystemencodeerrors(). @@ -297,6 +295,22 @@ typedef struct { If set to -1 (default), inherit Py_FrozenFlag value. */ int _frozen; + /* C locale coercion (PEP 538). + + The option is enabled by the PYTHONCOERCECLOCALE environment + variable. The option is also enabled if the LC_CTYPE locale is "C" + and a target locale (ex: "C.UTF-8") is supported by the platform. + + See also the _coerce_c_locale_warn option. */ + int _coerce_c_locale; + + /* C locale coercion warning (PEP 538). + + Enabled by the PYTHONCOERCECLOCALE=warn environment variable. + + See also the _coerce_c_locale option. */ + int _coerce_c_locale_warn; + } _PyCoreConfig; #ifdef MS_WINDOWS @@ -314,7 +328,7 @@ typedef struct { .use_hash_seed = -1, \ .faulthandler = -1, \ .tracemalloc = -1, \ - .coerce_c_locale = -1, \ + ._coerce_c_locale = -1, \ .utf8_mode = -1, \ .argc = -1, \ .nmodule_search_path = -1, \ diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 80233a54b0b0..c00000e68131 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -277,8 +277,6 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): 'filesystem_errors': None, 'utf8_mode': 0, - 'coerce_c_locale': 0, - 'coerce_c_locale_warn': 0, 'pycache_prefix': NULL_STR, 'program_name': './_testembed', @@ -306,6 +304,8 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): '_install_importlib': 1, '_check_hash_pycs_mode': 'default', '_frozen': 0, + '_coerce_c_locale': 0, + '_coerce_c_locale_warn': 0, } def get_stdio_encoding(self, env): diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index add642f223af..c21a6e305836 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -4701,10 +4701,6 @@ get_coreconfig(PyObject *self, PyObject *Py_UNUSED(args)) PyLong_FromLong(config->dump_refs)); SET_ITEM("malloc_stats", PyLong_FromLong(config->malloc_stats)); - SET_ITEM("coerce_c_locale", - PyLong_FromLong(config->coerce_c_locale)); - SET_ITEM("coerce_c_locale_warn", - PyLong_FromLong(config->coerce_c_locale_warn)); SET_ITEM("filesystem_encoding", FROM_STRING(config->filesystem_encoding)); SET_ITEM("filesystem_errors", @@ -4783,6 +4779,10 @@ get_coreconfig(PyObject *self, PyObject *Py_UNUSED(args)) FROM_STRING(config->_check_hash_pycs_mode)); SET_ITEM("_frozen", PyLong_FromLong(config->_frozen)); + SET_ITEM("_coerce_c_locale", + PyLong_FromLong(config->_coerce_c_locale)); + SET_ITEM("_coerce_c_locale_warn", + PyLong_FromLong(config->_coerce_c_locale_warn)); return dict; diff --git a/Modules/main.c b/Modules/main.c index 455178af8bab..6bc2917e2ccc 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -1342,9 +1342,9 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, * See the documentation of the PYTHONCOERCECLOCALE setting for more * details. */ - if (config->coerce_c_locale && !locale_coerced) { + if (config->_coerce_c_locale && !locale_coerced) { locale_coerced = 1; - _Py_CoerceLegacyLocale(config->coerce_c_locale_warn); + _Py_CoerceLegacyLocale(config->_coerce_c_locale_warn); encoding_changed = 1; } @@ -1367,7 +1367,7 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, /* Reset the configuration before reading again the configuration, just keep UTF-8 Mode value. */ int new_utf8_mode = config->utf8_mode; - int new_coerce_c_locale = config->coerce_c_locale; + int new_coerce_c_locale = config->_coerce_c_locale; if (_PyCoreConfig_Copy(config, &save_config) < 0) { pymain->err = _Py_INIT_NO_MEMORY(); goto done; @@ -1375,7 +1375,7 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, pymain_clear_cmdline(pymain, cmdline); memset(cmdline, 0, sizeof(*cmdline)); config->utf8_mode = new_utf8_mode; - config->coerce_c_locale = new_coerce_c_locale; + config->_coerce_c_locale = new_coerce_c_locale; /* The encoding changed: read again the configuration with the new encoding */ diff --git a/Programs/_testembed.c b/Programs/_testembed.c index 99772eacbdc4..12880325b254 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -330,8 +330,6 @@ dump_config(void) printf("filesystem_encoding = %s\n", config->filesystem_encoding); printf("filesystem_errors = %s\n", config->filesystem_errors); - printf("coerce_c_locale = %i\n", config->coerce_c_locale); - printf("coerce_c_locale_warn = %i\n", config->coerce_c_locale_warn); printf("utf8_mode = %i\n", config->utf8_mode); printf("pycache_prefix = %ls\n", config->pycache_prefix); @@ -385,6 +383,8 @@ dump_config(void) printf("_install_importlib = %i\n", config->_install_importlib); printf("_check_hash_pycs_mode = %s\n", config->_check_hash_pycs_mode); printf("_frozen = %i\n", config->_frozen); + printf("_coerce_c_locale = %i\n", config->_coerce_c_locale); + printf("_coerce_c_locale_warn = %i\n", config->_coerce_c_locale_warn); #undef ASSERT_EQUAL #undef ASSERT_STR_EQUAL @@ -482,7 +482,7 @@ static int test_init_from_config(void) putenv("PYTHONMALLOCSTATS=0"); config.malloc_stats = 1; - /* FIXME: test coerce_c_locale and coerce_c_locale_warn */ + /* FIXME: test _coerce_c_locale and _coerce_c_locale_warn */ putenv("PYTHONUTF8=0"); Py_UTF8Mode = 0; @@ -606,8 +606,8 @@ static int test_init_isolated(void) /* Test _PyCoreConfig.isolated=1 */ _PyCoreConfig config = _PyCoreConfig_INIT; - /* Set coerce_c_locale and utf8_mode to not depend on the locale */ - config.coerce_c_locale = 0; + /* Set _coerce_c_locale and utf8_mode to not depend on the locale */ + config._coerce_c_locale = 0; config.utf8_mode = 0; /* Use path starting with "./" avoids a search along the PATH */ config.program_name = L"./_testembed"; diff --git a/Python/coreconfig.c b/Python/coreconfig.c index fae32e533aa9..1e03ce31f3d5 100644 --- a/Python/coreconfig.c +++ b/Python/coreconfig.c @@ -303,8 +303,8 @@ _PyCoreConfig_Copy(_PyCoreConfig *config, const _PyCoreConfig *config2) COPY_ATTR(dump_refs); COPY_ATTR(malloc_stats); - COPY_ATTR(coerce_c_locale); - COPY_ATTR(coerce_c_locale_warn); + COPY_ATTR(_coerce_c_locale); + COPY_ATTR(_coerce_c_locale_warn); COPY_ATTR(utf8_mode); COPY_WSTR_ATTR(pycache_prefix); @@ -811,16 +811,16 @@ config_read_env_vars(_PyCoreConfig *config) const char *env = _PyCoreConfig_GetEnv(config, "PYTHONCOERCECLOCALE"); if (env) { if (strcmp(env, "0") == 0) { - if (config->coerce_c_locale < 0) { - config->coerce_c_locale = 0; + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 0; } } else if (strcmp(env, "warn") == 0) { - config->coerce_c_locale_warn = 1; + config->_coerce_c_locale_warn = 1; } else { - if (config->coerce_c_locale < 0) { - config->coerce_c_locale = 1; + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 1; } } } @@ -967,10 +967,10 @@ config_read_complex_options(_PyCoreConfig *config) static void config_init_locale(_PyCoreConfig *config) { - if (config->coerce_c_locale < 0) { + if (config->_coerce_c_locale < 0) { /* The C locale enables the C locale coercion (PEP 538) */ if (_Py_LegacyLocaleDetected()) { - config->coerce_c_locale = 1; + config->_coerce_c_locale = 1; } } @@ -1291,7 +1291,7 @@ _PyCoreConfig_Read(_PyCoreConfig *config) } } - if (config->utf8_mode < 0 || config->coerce_c_locale < 0) { + if (config->utf8_mode < 0 || config->_coerce_c_locale < 0) { config_init_locale(config); } @@ -1321,8 +1321,8 @@ _PyCoreConfig_Read(_PyCoreConfig *config) if (config->tracemalloc < 0) { config->tracemalloc = 0; } - if (config->coerce_c_locale < 0) { - config->coerce_c_locale = 0; + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 0; } if (config->utf8_mode < 0) { config->utf8_mode = 0; @@ -1343,7 +1343,7 @@ _PyCoreConfig_Read(_PyCoreConfig *config) return err; } - assert(config->coerce_c_locale >= 0); + assert(config->_coerce_c_locale >= 0); assert(config->use_environment >= 0); assert(config->filesystem_encoding != NULL); assert(config->filesystem_errors != NULL); diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 379e860b5ba4..b735228ea56d 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -301,7 +301,7 @@ static const char *_C_LOCALE_WARNING = static void _emit_stderr_warning_for_legacy_locale(const _PyCoreConfig *core_config) { - if (core_config->coerce_c_locale_warn && _Py_LegacyLocaleDetected()) { + if (core_config->_coerce_c_locale_warn && _Py_LegacyLocaleDetected()) { PySys_FormatStderr("%s", _C_LOCALE_WARNING); } } From webhook-mailer at python.org Mon Sep 17 18:39:28 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 22:39:28 -0000 Subject: [Python-checkins] bpo-34267: Update find_python.bat to use 3.7 if available (GH-8552) Message-ID: https://github.com/python/cpython/commit/a8a8cc71c7fc7585e15652ff7ab354665f787d5a commit: a8a8cc71c7fc7585e15652ff7ab354665f787d5a branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T15:39:18-07:00 summary: bpo-34267: Update find_python.bat to use 3.7 if available (GH-8552) (cherry picked from commit 6750922f8d3428d84a016c34d6fcd99659e8610c) Co-authored-by: Steve Dower files: M PCbuild/find_python.bat diff --git a/PCbuild/find_python.bat b/PCbuild/find_python.bat index 17ce7ac609c1..d0778ddd7347 100644 --- a/PCbuild/find_python.bat +++ b/PCbuild/find_python.bat @@ -34,7 +34,7 @@ @if NOT "%HOST_PYTHON%"=="" @%HOST_PYTHON% -Ec "import sys; assert sys.version_info[:2] >= (3, 6)" >nul 2>nul && (set PYTHON="%HOST_PYTHON%") && (set _Py_Python_Source=found as HOST_PYTHON) && goto :found @rem If py.exe finds a recent enough version, use that one - at py -3.6 -EV >nul 2>&1 && (set PYTHON=py -3.6) && (set _Py_Python_Source=found with py.exe) && goto :found + at for %%p in (3.7 3.6) do @py -%%p -EV >nul 2>&1 && (set PYTHON=py -%%p) && (set _Py_Python_Source=found %%p with py.exe) && goto :found @if NOT exist "%_Py_EXTERNALS_DIR%" mkdir "%_Py_EXTERNALS_DIR%" @set _Py_NUGET=%NUGET% @@ -55,7 +55,7 @@ ) @echo Installing Python via nuget... @"%_Py_NUGET%" install pythonx86 -ExcludeVersion -OutputDirectory "%_Py_EXTERNALS_DIR%" - at rem Quote it here; it's not quoted later because "py -3.6" wouldn't work + at rem Quote it here; it's not quoted later because "py -x.y" wouldn't work @if not errorlevel 1 (set PYTHON="%_Py_EXTERNALS_DIR%\pythonx86\tools\python.exe") & (set _Py_Python_Source=found on nuget.org) & goto :found From webhook-mailer at python.org Mon Sep 17 18:42:04 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Mon, 17 Sep 2018 22:42:04 -0000 Subject: [Python-checkins] bpo-33649: Add a hello world example to asyncio.rst (GH-9374) Message-ID: https://github.com/python/cpython/commit/3085534c398e6b181e7a9ac0cb9c80f3c670f2b9 commit: 3085534c398e6b181e7a9ac0cb9c80f3c670f2b9 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-17T18:41:59-04:00 summary: bpo-33649: Add a hello world example to asyncio.rst (GH-9374) files: M Doc/library/asyncio.rst diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 7895826c65a0..73b0e63a68db 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -6,6 +6,19 @@ -------------- +.. sidebar:: Hello World! + + .. code-block:: python + + import asyncio + + async def main(): + print('Hello ...') + await asyncio.sleep(1) + print('... World!') + + asyncio.run(main()) + asyncio is a library to write **concurrent** code using **async/await** syntax. From webhook-mailer at python.org Mon Sep 17 18:42:13 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 17 Sep 2018 22:42:13 -0000 Subject: [Python-checkins] bpo-34717: Stop numbering stdlib titles/sections in the docs (GH-9370) Message-ID: https://github.com/python/cpython/commit/c63d81b3feaa008a6be4c7c83c324174e8d95c24 commit: c63d81b3feaa008a6be4c7c83c324174e8d95c24 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T15:42:09-07:00 summary: bpo-34717: Stop numbering stdlib titles/sections in the docs (GH-9370) (cherry picked from commit c62ab2862db2382808bb2228760eebdda3f608bd) Co-authored-by: Yury Selivanov files: M Doc/library/index.rst diff --git a/Doc/library/index.rst b/Doc/library/index.rst index b8fbf44ae49f..bebf7429b0e6 100644 --- a/Doc/library/index.rst +++ b/Doc/library/index.rst @@ -32,10 +32,11 @@ several thousand components (from individual programs and modules to packages and entire application development frameworks), available from the `Python Package Index `_. - +.. We don't use :numbered: option for the TOC below as it enforces + numbered sections for the entire stdlib docs. If desired, + :numbered: can be enabled on a per-module basis. .. toctree:: :maxdepth: 2 - :numbered: intro.rst functions.rst From webhook-mailer at python.org Mon Sep 17 19:16:51 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Mon, 17 Sep 2018 23:16:51 -0000 Subject: [Python-checkins] bpo-33649: A copy-editing pass on asyncio documentation (GH-9376) Message-ID: https://github.com/python/cpython/commit/1fa2ec49bec50bea1847b558b883c5c904334734 commit: 1fa2ec49bec50bea1847b558b883c5c904334734 branch: master author: Elvis Pranskevichus committer: Yury Selivanov date: 2018-09-17T19:16:44-04:00 summary: bpo-33649: A copy-editing pass on asyncio documentation (GH-9376) files: M Doc/library/asyncio-dev.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-exceptions.rst M Doc/library/asyncio-platforms.rst M Doc/library/asyncio-policy.rst M Doc/library/asyncio-protocol.rst M Doc/library/asyncio-queue.rst M Doc/library/asyncio-stream.rst M Doc/library/asyncio-subprocess.rst M Doc/library/asyncio-sync.rst M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index e8153186760b..5f926fceb22d 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -6,7 +6,7 @@ Developing with asyncio ======================= -Asynchronous programming is different from classical "sequential" +Asynchronous programming is different from classic "sequential" programming. This page lists common mistakes and traps and explains how @@ -21,19 +21,17 @@ Debug Mode By default asyncio runs in production mode. In order to ease the development asyncio has a *debug mode*. -To enable debugging for an application: +There are several ways to enable asyncio debug mode: -* Enable the debug mode globally by setting the environment variable - :envvar:`PYTHONASYNCIODEBUG` to ``1``. +* Setting the :envvar:`PYTHONASYNCIODEBUG` environment variable to ``1``. -* Alternatively, the debug mode can be enabled by using the ``-X dev`` - command line option for Python (see the :option:`-X` option). +* Using the :option:`-X` ``dev`` Python command line option. -* Yet another way to enable the debug mode is by calling - :meth:`loop.set_debug` or by passing ``debug=True`` to - :func:`asyncio.run`. +* Passing ``debug=True`` to :func:`asyncio.run`. -In addition to enabling debug mode, consider also: +* Calling :meth:`loop.set_debug`. + +In addition to enabling the debug mode, consider also: * setting the log level of the :ref:`asyncio logger ` to :py:data:`logging.DEBUG`, for example the following snippet of code @@ -43,25 +41,25 @@ In addition to enabling debug mode, consider also: * configuring the :mod:`warnings` module to display :exc:`ResourceWarning` warnings. One way of doing that is by - using the ``-Wdefault`` command line option. + using the :option:`-W` ``default`` command line option. -In asyncio debug mode the following checks are performed: +When the debug mode is enabled: -* Log :ref:`coroutines that were not awaited - `; this mitigates the "forgotten - await" pitfall. +* asyncio checks for :ref:`coroutines that were not awaited + ` and logs them; this mitigates + the "forgotten await" pitfall. * Many non-treadsafe asyncio APIs (such as :meth:`loop.call_soon` and :meth:`loop.call_at` methods) raise an exception if they are called from a wrong thread. -* Log the execution time of the IO selector if it takes too long to - perform an IO operation. +* The execution time of the I/O selector is logged if it takes too long to + perform an I/O operation. -* Log callbacks taking longer than 100 ms to be executed. The - :attr:`loop.slow_callback_duration` attribute is the minimum - duration in seconds of "slow" callbacks. +* Callbacks taking longer than 100ms are logged. The + :attr:`loop.slow_callback_duration` attribute can be used to set the + minimum execution duration in seconds that is considered "slow". .. _asyncio-multithreading: @@ -134,7 +132,7 @@ Logging asyncio uses the :mod:`logging` module and all logging is performed via the ``"asyncio"`` logger. -The default log level is :py:data:`logging.INFO`, which can easily be +The default log level is :py:data:`logging.INFO`, which can be easily adjusted:: logging.getLogger("asyncio").setLevel(logging.WARNING) @@ -142,12 +140,13 @@ adjusted:: .. _asyncio-coroutine-not-scheduled: -Detect never awaited coroutines +Detect never-awaited coroutines =============================== -When a coroutine is called (e.g. ``coro()`` instead of ``await coro()``) -the call is not wrapped with :meth:`asyncio.create_task`, the execution -of the coroutine object will never be scheduled. For example:: +When a coroutine function is called, but not awaited +(e.g. ``coro()`` instead of ``await coro()``) +or the coroutine is not scheduled with :meth:`asyncio.create_task`, asyncio +will emit a :exc:`RuntimeWarning`:: import asyncio @@ -184,8 +183,8 @@ The usual fix is to either await the coroutine or call the await test() -Detect never consumed exceptions -================================ +Detect never-retrieved exceptions +================================= If a :meth:`Future.set_exception` is called but the Future object is never awaited on, the exception would never be propagated to the diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 909d3ea33632..0320d798c1e5 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -8,20 +8,17 @@ Event Loop .. rubric:: Preface -The event loop is a central component of every asyncio application. +The event loop is the core of every asyncio application. Event loops run asynchronous tasks and callbacks, perform network IO operations, and run subprocesses. -Application developers will typically use high-level asyncio functions -to interact with the event loop. In general, high-level asyncio applications -should not need to work directly with event loops and, instead, should use -the :func:`asyncio.run` function to initialize, manage the event loop, and -run asynchronous code. +Application developers should typically use the high-level asyncio functions, +such as :func:`asyncio.run`, and should rarely need to reference the loop +object or call its methods. This section is intended mostly for authors +of lower-level code, libraries, and frameworks, who need finer control over +the event loop behavior. -Alternatively, developers of low-level code, such as libraries and -framework, may need access to the event loop. - -.. rubric:: Accessing Event Loop +.. rubric:: Obtaining the Event Loop The following low-level functions can be used to get, set, or create an event loop: @@ -68,16 +65,16 @@ and :func:`new_event_loop` functions can be altered by This documentation page contains the following sections: * The `Event Loop Methods`_ section is the reference documentation of - event loop APIs; + the event loop APIs; * The `Callback Handles`_ section documents the :class:`Handle` and - :class:`TimerHandle`, instances which are returned from functions, such - as :meth:`loop.call_soon` and :meth:`loop.call_later`; + :class:`TimerHandle` instances which are returned from scheduling + methods such as :meth:`loop.call_soon` and :meth:`loop.call_later`; -* The `Server Objects`_ sections document types returned from +* The `Server Objects`_ section documents types returned from event loop methods like :meth:`loop.create_server`; -* The `Event Loops Implementations`_ section documents the +* The `Event Loop Implementations`_ section documents the :class:`SelectorEventLoop` and :class:`ProactorEventLoop` classes; * The `Examples`_ section showcases how to work with some event @@ -101,11 +98,11 @@ Running and stopping the loop .. method:: loop.run_until_complete(future) - Run until the *future* (an instance of :class:`Future`) is + Run until the *future* (an instance of :class:`Future`) has completed. If the argument is a :ref:`coroutine object ` it - is implicitly wrapped into an :class:`asyncio.Task`. + is implicitly scheduled to run as a :class:`asyncio.Task`. Return the Future's result or raise its exception. @@ -120,7 +117,7 @@ Running and stopping the loop If :meth:`stop` is called while :meth:`run_forever` is running, the loop will run the current batch of callbacks and then exit. - Note that callbacks scheduled by callbacks will not run in this + Note that new callbacks scheduled by callbacks will not run in this case; instead, they will run the next time :meth:`run_forever` or :meth:`run_until_complete` is called. @@ -167,8 +164,6 @@ Running and stopping the loop .. versionadded:: 3.6 -.. _asyncio-pass-keywords: - Scheduling callbacks ^^^^^^^^^^^^^^^^^^^^ @@ -201,18 +196,19 @@ Scheduling callbacks The *context* keyword-only parameter was added. See :pep:`567` for more details. +.. _asyncio-pass-keywords: + .. note:: Most :mod:`asyncio` scheduling functions don't allow passing - keyword arguments. To do that, use :func:`functools.partial`, - e.g.:: + keyword arguments. To do that, use :func:`functools.partial`:: # will schedule "print("Hello", flush=True)" loop.call_soon( functools.partial(print, "Hello", flush=True)) Using partial objects is usually more convenient than using lambdas, - as asyncio can better render partial objects in debug and error + as asyncio can render partial objects better in debug and error messages. @@ -235,8 +231,8 @@ clocks to track time. be used to cancel the callback. *callback* will be called exactly once. If two callbacks are - scheduled for exactly the same time, it is undefined which one will - be called first. + scheduled for exactly the same time, the order in which they + are called is undefined. The optional positional *args* will be passed to the callback when it is called. If you want the callback to be called with keyword @@ -250,6 +246,11 @@ clocks to track time. The *context* keyword-only parameter was added. See :pep:`567` for more details. + .. versionchanged:: 3.8 + In Python 3.7 and earlier with the default event loop implementation, + the *delay* could not exceed one day. + This has been fixed in Python 3.8. + .. method:: loop.call_at(when, callback, *args, context=None) Schedule *callback* to be called at the given absolute timestamp @@ -265,6 +266,11 @@ clocks to track time. The *context* keyword-only parameter was added. See :pep:`567` for more details. + .. versionchanged:: 3.8 + In Python 3.7 and earlier with the default event loop implementation, + the difference between *when* and the current time could not exceed + one day. This has been fixed in Python 3.8. + .. method:: loop.time() Return the current time, as a :class:`float` value, according to @@ -314,11 +320,10 @@ Creating Futures and Tasks :meth:`loop.create_task`. If *factory* is ``None`` the default task factory will be set. - - If *factory* is a *callable*, it should have a signature matching - ``(loop, coro)``, where *loop* will be a reference to the active - event loop, *coro* will be a coroutine object. The callable - must return an :class:`asyncio.Future` compatible object. + Otherwise, *factory* must be a *callable* with the signature matching + ``(loop, coro)``, where *loop* is a reference to the active + event loop, and *coro* is a coroutine object. The callable + must return a :class:`asyncio.Future`-compatible object. .. method:: loop.get_task_factory() @@ -365,28 +370,23 @@ Opening network connections The created transport is an implementation-dependent bidirectional stream. - .. note:: - *protocol_factory* can be any kind of callable, not necessarily - a class. For example, if you want to use a pre-created - protocol instance, you can pass ``lambda: my_protocol``. - Other arguments: - * *ssl*: if given and not false, an SSL/TLS transport is created + * *ssl*: if given and not false, a SSL/TLS transport is created (by default a plain TCP transport is created). If *ssl* is a :class:`ssl.SSLContext` object, this context is used to create - the transport; if *ssl* is :const:`True`, a context with some - unspecified default settings is used. + the transport; if *ssl* is :const:`True`, a default context returned + from :func:`ssl.create_default_context` is used. .. seealso:: :ref:`SSL/TLS security considerations ` - * *server_hostname*, is only for use together with *ssl*, - and sets or overrides the hostname that the target server's certificate - will be matched against. By default the value of the *host* argument + * *server_hostname* sets or overrides the hostname that the target + server's certificate will be matched against. Should only be passed + if *ssl* is not ``None``. By default the value of the *host* argument is used. If *host* is empty, there is no default and you must pass a value for *server_hostname*. If *server_hostname* is an empty string, hostname matching is disabled (which is a serious security - risk, allowing for man-in-the-middle-attacks). + risk, allowing for potential man-in-the-middle attacks). * *family*, *proto*, *flags* are the optional address family, protocol and flags to be passed through to getaddrinfo() for *host* resolution. @@ -402,8 +402,8 @@ Opening network connections to bind the socket to locally. The *local_host* and *local_port* are looked up using ``getaddrinfo()``, similarly to *host* and *port*. - * *ssl_handshake_timeout* is (for an SSL connection) the time in seconds - to wait for the SSL handshake to complete before aborting the connection. + * *ssl_handshake_timeout* is (for a TLS connection) the time in seconds + to wait for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). .. versionadded:: 3.7 @@ -417,7 +417,7 @@ Opening network connections .. versionchanged:: 3.5 - Added support for SSL/TLS for :class:`ProactorEventLoop`. + Added support for SSL/TLS in :class:`ProactorEventLoop`. .. seealso:: @@ -462,12 +462,12 @@ Opening network connections * *reuse_address* tells the kernel to reuse a local socket in ``TIME_WAIT`` state, without waiting for its natural timeout to expire. If not specified will automatically be set to ``True`` on - UNIX. + Unix. * *reuse_port* tells the kernel to allow this endpoint to be bound to the same port as other existing endpoints are bound to, so long as they all set this flag when being created. This option is not supported on Windows - and some UNIX's. If the :py:data:`~socket.SO_REUSEPORT` constant is not + and some Unixes. If the :py:data:`~socket.SO_REUSEPORT` constant is not defined then this capability is unsupported. * *allow_broadcast* tells the kernel to allow this endpoint to send @@ -478,7 +478,7 @@ Opening network connections transport. If specified, *local_addr* and *remote_addr* should be omitted (must be :const:`None`). - On Windows with :class:`ProactorEventLoop`, this method is not supported. + On Windows, with :class:`ProactorEventLoop`, this method is not supported. See :ref:`UDP echo client protocol ` and :ref:`UDP echo server protocol ` examples. @@ -491,22 +491,22 @@ Opening network connections path=None, \*, ssl=None, sock=None, \ server_hostname=None, ssl_handshake_timeout=None) - Create UNIX connection. + Create a Unix connection. The socket family will be :py:data:`~socket.AF_UNIX`; socket type will be :py:data:`~socket.SOCK_STREAM`. A tuple of ``(transport, protocol)`` is returned on success. - *path* is the name of a UNIX domain socket and is required, - unless a *sock* parameter is specified. Abstract UNIX sockets, + *path* is the name of a Unix domain socket and is required, + unless a *sock* parameter is specified. Abstract Unix sockets, :class:`str`, :class:`bytes`, and :class:`~pathlib.Path` paths are supported. See the documentation of the :meth:`loop.create_connection` method for information about arguments to this method. - Availability: UNIX. + Availability: Unix. .. versionadded:: 3.7 @@ -529,7 +529,7 @@ Creating network servers ssl_handshake_timeout=None, start_serving=True) Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) listening - on the *host* and *port* address. + on *port* of the *host* address. Returns a :class:`Server` object. @@ -538,13 +538,18 @@ Creating network servers * *protocol_factory* must be a callable returning a :ref:`protocol ` implementation. - * The *host* parameter can be set to several types which determine behavior: - - If *host* is a string, the TCP server is bound to *host* and *port*. - - if *host* is a sequence of strings, the TCP server is bound to all - hosts of the sequence. - - If *host* is an empty string or ``None``, all interfaces are - assumed and a list of multiple sockets will be returned (most likely - one for IPv4 and another one for IPv6). + * The *host* parameter can be set to several types which determine where + the server would be listening: + + - If *host* is a string, the TCP server is bound to a single network + interface specified by *host*. + + - If *host* is a sequence of strings, the TCP server is bound to all + network interfaces specified by the sequence. + + - If *host* is an empty string or ``None``, all interfaces are + assumed and a list of multiple sockets will be returned (most likely + one for IPv4 and another one for IPv6). * *family* can be set to either :data:`socket.AF_INET` or :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. @@ -554,27 +559,26 @@ Creating network servers * *flags* is a bitmask for :meth:`getaddrinfo`. * *sock* can optionally be specified in order to use a preexisting - socket object. If specified, *host* and *port* should be omitted (must be - :const:`None`). + socket object. If specified, *host* and *port* must not be specified. * *backlog* is the maximum number of queued connections passed to :meth:`~socket.socket.listen` (defaults to 100). - * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the - accepted connections. + * *ssl* can be set to an :class:`~ssl.SSLContext` instance to enable + TLS over the accepted connections. * *reuse_address* tells the kernel to reuse a local socket in ``TIME_WAIT`` state, without waiting for its natural timeout to expire. If not specified will automatically be set to ``True`` on - UNIX. + Unix. * *reuse_port* tells the kernel to allow this endpoint to be bound to the same port as other existing endpoints are bound to, so long as they all set this flag when being created. This option is not supported on Windows. - * *ssl_handshake_timeout* is (for an SSL server) the time in seconds to wait - for the SSL handshake to complete before aborting the connection. + * *ssl_handshake_timeout* is (for a TLS server) the time in seconds to wait + for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). * *start_serving* set to ``True`` (the default) causes the created server @@ -594,8 +598,7 @@ Creating network servers .. versionchanged:: 3.5 - Added support for SSL/TLS on Windows with - :class:`ProactorEventLoop`. + Added support for SSL/TLS in :class:`ProactorEventLoop`. .. versionchanged:: 3.5.1 @@ -615,15 +618,15 @@ Creating network servers Similar to :meth:`loop.create_server` but works with the :py:data:`~socket.AF_UNIX` socket family. - *path* is the name of a UNIX domain socket, and is required, - unless a *sock* argument is provided. Abstract UNIX sockets, + *path* is the name of a Unix domain socket, and is required, + unless a *sock* argument is provided. Abstract Unix sockets, :class:`str`, :class:`bytes`, and :class:`~pathlib.Path` paths are supported. See the documentation of the :meth:`loop.create_server` method for information about arguments to this method. - Availability: UNIX. + Availability: Unix. .. versionadded:: 3.7 @@ -680,17 +683,17 @@ Transferring files *offset* tells from where to start reading the file. If specified, *count* is the total number of bytes to transmit as opposed to - sending the file until EOF is reached. File position is updated on - return or also in case of error in which case :meth:`file.tell() - ` can be used to figure out the number of bytes - which were sent. + sending the file until EOF is reached. File position is always updated, + even when this method raises an error, and + :meth:`file.tell() ` can be used to obtain the actual + number of bytes sent. *fallback* set to ``True`` makes asyncio to manually read and send the file when the platform does not support the sendfile system call (e.g. Windows or SSL socket on Unix). Raise :exc:`SendfileNotAvailableError` if the system does not support - *sendfile* syscall and *fallback* is ``False``. + the *sendfile* syscall and *fallback* is ``False``. .. versionadded:: 3.7 @@ -722,8 +725,8 @@ TLS Upgrade * *server_hostname*: sets or overrides the host name that the target server's certificate will be matched against. - * *ssl_handshake_timeout* is (for an SSL connection) the time in seconds to - wait for the SSL handshake to complete before aborting the connection. + * *ssl_handshake_timeout* is (for a TLS connection) the time in seconds to + wait for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). .. versionadded:: 3.7 @@ -734,24 +737,26 @@ Watching file descriptors .. method:: loop.add_reader(fd, callback, \*args) - Start watching the file descriptor *fd* for read availability and - call the *callback* with specified arguments. + Start monitoring the *fd* file descriptor for read availability and + invoke *callback* with the specified arguments once *fd* is available for + reading. .. method:: loop.remove_reader(fd) - Stop watching the file descriptor *fd* for read availability. + Stop monitoring the *fd* file descriptor for read availability. .. method:: loop.add_writer(fd, callback, \*args) - Start watching the file descriptor *fd* for write availability and then - call the *callback* with specified arguments. + Start monitoring the *fd* file descriptor for write availability and + invoke *callback* with the specified arguments once *fd* is available for + writing. Use :func:`functools.partial` :ref:`to pass keywords ` to *func*. .. method:: loop.remove_writer(fd) - Stop watching the file descriptor *fd* for write availability. + Stop monitoring the *fd* file descriptor for write availability. See also :ref:`Platform Support ` section for some limitations of these methods. @@ -769,13 +774,12 @@ convenient. .. coroutinemethod:: loop.sock_recv(sock, nbytes) - Receive data. Asynchronous version of + Receive up to *nbytes* from *sock*. Asynchronous version of :meth:`socket.recv() `. - The received data is returned as a bytes object. The maximum amount - of data to be received is specified by the *nbytes* argument. + Return the received data as a bytes object. - The socket *sock* must be non-blocking. + *sock* must be a non-blocking socket. .. versionchanged:: 3.7 Even though this method was always documented as a coroutine @@ -784,27 +788,27 @@ convenient. .. coroutinemethod:: loop.sock_recv_into(sock, buf) - Receive data into a buffer. Modeled after the blocking + Receive data from *sock* into the *buf* buffer. Modeled after the blocking :meth:`socket.recv_into() ` method. Return the number of bytes written to the buffer. - The socket *sock* must be non-blocking. + *sock* must be a non-blocking socket. .. versionadded:: 3.7 .. coroutinemethod:: loop.sock_sendall(sock, data) - Send data to the socket. Asynchronous version of + Send *data* to the *sock* socket. Asynchronous version of :meth:`socket.sendall() `. - This method continues to send data from *data* to the socket until either - all data in *data* has been sent or an error occurs. ``None`` is returned + This method continues to send to the socket until either all data + in *data* has been sent or an error occurs. ``None`` is returned on success. On error, an exception is raised. Additionally, there is no way to determine how much data, if any, was successfully processed by the receiving end of the connection. - The socket *sock* must be non-blocking. + *sock* must be a non-blocking socket. .. versionchanged:: 3.7 Even though the method was always documented as a coroutine @@ -813,11 +817,11 @@ convenient. .. coroutinemethod:: loop.sock_connect(sock, address) - Connect to a remote socket at *address*. + Connect *sock* to a remote socket at *address*. Asynchronous version of :meth:`socket.connect() `. - The socket *sock* must be non-blocking. + *sock* must be a non-blocking socket. .. versionchanged:: 3.5.2 ``address`` no longer needs to be resolved. ``sock_connect`` @@ -843,7 +847,7 @@ convenient. and *address* is the address bound to the socket on the other end of the connection. - The socket *sock* must be non-blocking. + *sock* must be a non-blocking socket. .. versionchanged:: 3.7 Even though the method was always documented as a coroutine @@ -858,21 +862,21 @@ convenient. \*, fallback=True) Send a file using high-performance :mod:`os.sendfile` if possible. - Return the total number of bytes which were sent. + Return the total number of bytes sent. Asynchronous version of :meth:`socket.sendfile() `. - *sock* must be non-blocking :class:`~socket.socket` of - :const:`socket.SOCK_STREAM` type. + *sock* must be a non-blocking :const:`socket.SOCK_STREAM` + :class:`~socket.socket`. - *file* must be a regular file object opened in binary mode. + *file* must be a regular file object open in binary mode. *offset* tells from where to start reading the file. If specified, *count* is the total number of bytes to transmit as opposed to - sending the file until EOF is reached. File position is updated on - return or also in case of error in which case :meth:`file.tell() - ` can be used to figure out the number of bytes - which were sent. + sending the file until EOF is reached. File position is always updated, + even when this method raises an error, and + :meth:`file.tell() ` can be used to obtain the actual + number of bytes sent. *fallback*, when set to ``True``, makes asyncio manually read and send the file when the platform does not support the sendfile syscall @@ -881,7 +885,7 @@ convenient. Raise :exc:`SendfileNotAvailableError` if the system does not support *sendfile* syscall and *fallback* is ``False``. - The socket *sock* must be non-blocking. + *sock* must be a non-blocking socket. .. versionadded:: 3.7 @@ -910,7 +914,7 @@ Working with pipes .. coroutinemethod:: loop.connect_read_pipe(protocol_factory, pipe) - Register a read-pipe in the event loop. + Register the read end of *pipe* in the event loop. *protocol_factory* must be a callable returning an :ref:`asyncio protocol ` implementation. @@ -926,7 +930,7 @@ Working with pipes .. coroutinemethod:: loop.connect_write_pipe(protocol_factory, pipe) - Register a write-pipe in the event loop. + Register the write end of *pipe* in the event loop. *protocol_factory* must be a callable returning an :ref:`asyncio protocol ` implementation. @@ -951,12 +955,12 @@ Working with pipes :meth:`loop.subprocess_shell` methods. -UNIX signals +Unix signals ^^^^^^^^^^^^ .. method:: loop.add_signal_handler(signum, callback, \*args) - Add a handler for a signal. + Set *callback* as the handler for the *signum* signal. Raise :exc:`ValueError` if the signal number is invalid or uncatchable. Raise :exc:`RuntimeError` if there is a problem setting up the handler. @@ -966,11 +970,12 @@ UNIX signals .. method:: loop.remove_signal_handler(sig) - Remove a handler for a signal. + Remove the handler for the *sig* signal. - Return ``True`` if a signal handler was removed, ``False`` if not. + Return ``True`` if the signal handler was removed, or ``False`` if + no handler was set for the given signal. -Availability: UNIX. +Availability: Unix. .. seealso:: @@ -982,7 +987,7 @@ Executing code in thread or process pools .. coroutinemethod:: loop.run_in_executor(executor, func, \*args) - Arrange for a *func* to be called in the specified executor. + Arrange for *func* to be called in the specified executor. The *executor* argument should be an :class:`concurrent.futures.Executor` instance. The default executor is used if *executor* is ``None``. @@ -1024,18 +1029,17 @@ Allows customizing how exceptions are handled in the event loop. Set *handler* as the new event loop exception handler. If *handler* is ``None``, the default exception handler will - be set. - - If *handler* is a callable object, it should have a - matching signature to ``(loop, context)``, where ``loop`` - will be a reference to the active event loop, ``context`` - will be a ``dict`` object (see :meth:`call_exception_handler` - documentation for details about context). + be set. Otherwise, *handler* must be a callable with the signature + matching ``(loop, context)``, where ``loop`` + is a reference to the active event loop, and ``context`` + is a ``dict`` object containing the details of the exception + (see :meth:`call_exception_handler` documentation for details + about context). .. method:: loop.get_exception_handler() - Return the exception handler, or ``None`` if the default one - is in use. + Return the current exception handler, or ``None`` if no custom + exception handler was set. .. versionadded:: 3.5.2 @@ -1055,7 +1059,7 @@ Allows customizing how exceptions are handled in the event loop. Call the current event loop exception handler. *context* is a ``dict`` object containing the following keys - (new keys may be introduced later): + (new keys may be introduced in future Python versions): * 'message': Error message; * 'exception' (optional): Exception object; @@ -1068,8 +1072,8 @@ Allows customizing how exceptions are handled in the event loop. .. note:: This method should not be overloaded in subclassed - event loops. For any custom exception handling, use - :meth:`set_exception_handler()` method. + event loops. For custom exception handling, use + the :meth:`set_exception_handler()` method. Enabling debug mode ^^^^^^^^^^^^^^^^^^^ @@ -1099,17 +1103,16 @@ Enabling debug mode Running Subprocesses ^^^^^^^^^^^^^^^^^^^^ -Methods described in this subsections are low-level. In an -async/await code consider using high-level convenient +Methods described in this subsections are low-level. In regular +async/await code consider using the high-level :func:`asyncio.create_subprocess_shell` and -:func:`asyncio.create_subprocess_exec` functions instead. +:func:`asyncio.create_subprocess_exec` convenience functions instead. .. note:: - The default event loop that asyncio is pre-configured - to use on **Windows** does not support subprocesses. - See :ref:`Subprocess Support on Windows ` - for details. + The default asyncio event loop on **Windows** does not support + subprocesses. See :ref:`Subprocess Support on Windows + ` for details. .. coroutinemethod:: loop.subprocess_exec(protocol_factory, \*args, \ stdin=subprocess.PIPE, stdout=subprocess.PIPE, \ @@ -1124,7 +1127,7 @@ async/await code consider using high-level convenient * or :class:`bytes`, encoded to the :ref:`filesystem encoding `. - The first string specifies the program to execute, + The first string specifies the program executable, and the remaining strings specify the arguments. Together, string arguments form the ``argv`` of the program. @@ -1134,7 +1137,7 @@ async/await code consider using high-level convenient a single argument which is list of strings, *subprocess_exec* takes multiple string arguments. - The *protocol_factory* must instantiate a subclass of the + The *protocol_factory* must be a callable returning a subclass of the :class:`asyncio.SubprocessProtocol` class. Other parameters: @@ -1185,7 +1188,7 @@ async/await code consider using high-level convenient This is similar to the standard library :class:`subprocess.Popen` class called with ``shell=True``. - The *protocol_factory* must instantiate a subclass of the + The *protocol_factory* must be a callable returning a subclass of the :class:`SubprocessProtocol` class. See :meth:`~loop.subprocess_exec` for more details about @@ -1197,10 +1200,10 @@ async/await code consider using high-level convenient .. note:: It is the application's responsibility to ensure that all whitespace - and metacharacters are quoted appropriately to avoid `shell injection + and special characters are quoted appropriately to avoid `shell injection `_ vulnerabilities. The :func:`shlex.quote` function can be used to - properly escape whitespace and shell metacharacters in strings that + properly escape whitespace and special characters in strings that are going to be used to construct shell commands. @@ -1214,12 +1217,12 @@ Callback Handles .. method:: cancel() - Cancel the call. If the callback is already canceled or executed, - this method has no effect. + Cancel the callback. If the callback has already been canceled + or executed, this method has no effect. .. method:: cancelled() - Return ``True`` if the call was cancelled. + Return ``True`` if the callback was cancelled. .. versionadded:: 3.7 @@ -1228,7 +1231,7 @@ Callback Handles A callback wrapper object returned by :meth:`loop.call_later`, and :meth:`loop.call_at`. - The class is inherited from :class:`Handle`. + This class is a subclass of :class:`Handle`. .. method:: when() @@ -1280,7 +1283,7 @@ Do not instantiate the class directly. .. method:: get_loop() - Gives the event loop associated with the server object. + Return the event loop associated with the server object. .. versionadded:: 3.7 @@ -1291,12 +1294,12 @@ Do not instantiate the class directly. This method is idempotent, so it can be called when the server is already being serving. - The new *start_serving* keyword-only parameter to + The *start_serving* keyword-only parameter to :meth:`loop.create_server` and - :meth:`asyncio.start_server` allows to create a Server object - that is not accepting connections right away. In which case - this method, or :meth:`Server.serve_forever` can be used - to make the Server object to start accepting connections. + :meth:`asyncio.start_server` allows creating a Server object + that is not accepting connections initially. In this case + ``Server.start_serving()``, or :meth:`Server.serve_forever` can be used + to make the Server start accepting connections. .. versionadded:: 3.7 @@ -1338,19 +1341,19 @@ Do not instantiate the class directly. .. attribute:: sockets - List of :class:`socket.socket` objects the server is listening to, + List of :class:`socket.socket` objects the server is listening on, or ``None`` if the server is closed. .. versionchanged:: 3.7 - Prior to Python 3.7 ``Server.sockets`` used to return the - internal list of server's sockets directly. In 3.7 a copy + Prior to Python 3.7 ``Server.sockets`` used to return an + internal list of server sockets directly. In 3.7 a copy of that list is returned. .. _asyncio-event-loops: -Event Loops Implementations -=========================== +Event Loop Implementations +========================== asyncio ships with two different event loop implementations: :class:`SelectorEventLoop` and :class:`ProactorEventLoop`. @@ -1364,8 +1367,8 @@ on all platforms. An event loop based on the :mod:`selectors` module. Uses the most efficient *selector* available for the given - platform. It is also possible to manually configure what - exact selector implementation should be used:: + platform. It is also possible to manually configure the + exact selector implementation to be used:: import asyncio import selectors @@ -1375,7 +1378,7 @@ on all platforms. asyncio.set_event_loop(loop) - Availability: UNIX, Windows. + Availability: Unix, Windows. .. class:: ProactorEventLoop @@ -1412,9 +1415,9 @@ Examples ======== Note that all examples in this section **purposefully** show how -to use low-level event loop APIs such as :meth:`loop.run_forever` +to use the low-level event loop APIs, such as :meth:`loop.run_forever` and :meth:`loop.call_soon`. Modern asyncio applications rarely -need to be written this way; consider using high-level functions +need to be written this way; consider using the high-level functions like :func:`asyncio.run`. @@ -1456,9 +1459,9 @@ event loop:: Display the current date with call_later() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -An example of callback displaying the current date every second. The +An example of a callback displaying the current date every second. The callback uses the :meth:`loop.call_later` method to reschedule itself -during 5 seconds, and then stops the event loop:: +after 5 seconds, and then stops the event loop:: import asyncio import datetime @@ -1545,7 +1548,7 @@ Wait until a file descriptor received some data using the Set signal handlers for SIGINT and SIGTERM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -(This ``signals`` example only works on UNIX.) +(This ``signals`` example only works on Unix.) Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM` using the :meth:`loop.add_signal_handler` method:: diff --git a/Doc/library/asyncio-exceptions.rst b/Doc/library/asyncio-exceptions.rst index 31bc1edf0142..dbd5df720850 100644 --- a/Doc/library/asyncio-exceptions.rst +++ b/Doc/library/asyncio-exceptions.rst @@ -23,12 +23,12 @@ Exceptions This exception can be caught to perform custom operations when asyncio Tasks are cancelled. In almost all situations the - exception must always be re-raised. + exception must be re-raised. .. important:: This exception is a subclass of :exc:`Exception`, so it can be - accidentally suppressed by ``try..except`` block:: + accidentally suppressed by an overly broad ``try..except`` block:: try: await operation @@ -65,27 +65,27 @@ Exceptions .. exception:: IncompleteReadError - Incomplete read error. + The requested read operation did not complete fully. - Raised by :ref:`asyncio streams ` APIs. + Raised by the :ref:`asyncio stream APIs`. This exception is a subclass of :exc:`EOFError`. .. attribute:: expected - Total number (:class:`int`) of expected bytes. + The total number (:class:`int`) of expected bytes. .. attribute:: partial - Read :class:`bytes` string before the end of stream was reached. + A string of :class:`bytes` read before the end of stream was reached. .. exception:: LimitOverrunError - Reached the buffer limit while looking for a separator. + Reached the buffer size limit while looking for a separator. - Raised by :ref:`asyncio streams ` APIs. + Raised by the :ref:`asyncio stream APIs `. .. attribute:: consumed - Total number of to be consumed bytes. + The total number of to be consumed bytes. diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst index 2f0f53495223..f8ecb58d3a01 100644 --- a/Doc/library/asyncio-platforms.rst +++ b/Doc/library/asyncio-platforms.rst @@ -4,11 +4,11 @@ .. _asyncio-platform-support: -================= -Platforms Support -================= +================ +Platform Support +================ -The :mod:`asyncio` module has been designed to be portable, +The :mod:`asyncio` module is designed to be portable, but some platforms have subtle differences and limitations due to the platforms' underlying architecture and capabilities. @@ -17,7 +17,7 @@ All Platforms ============= * :meth:`loop.add_reader` and :meth:`loop.add_writer` - cannot be used to monitor file IO. + cannot be used to monitor file I/O. Windows @@ -27,7 +27,7 @@ All event loops on Windows do not support the following methods: * :meth:`loop.create_unix_connection` and :meth:`loop.create_unix_server` are not supported. - The :data:`socket.AF_UNIX` socket family is specific to UNIX. + The :data:`socket.AF_UNIX` socket family is specific to Unix. * :meth:`loop.add_signal_handler` and :meth:`loop.remove_signal_handler` are not supported. diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst index 61727d40afab..42f936da468e 100644 --- a/Doc/library/asyncio-policy.rst +++ b/Doc/library/asyncio-policy.rst @@ -7,9 +7,9 @@ Policies ======== -An event loop policy, a global per-process object, controls -management of the event loop. Each event loop has a default -policy, which can be changed and customized using the API. +An event loop policy is a global per-process object that controls +the management of the event loop. Each event loop has a default +policy, which can be changed and customized using the policy API. A policy defines the notion of *context* and manages a separate event loop per context. The default policy @@ -20,11 +20,11 @@ By using a custom event loop policy, the behavior of :func:`new_event_loop` functions can be customized. Policy objects should implement the APIs defined -in the abstract base class :class:`AbstractEventLoopPolicy`. +in the :class:`AbstractEventLoopPolicy` abstract base class. -Access the Policy -================= +Getting and Setting the Policy +============================== The following functions can be used to get and set the policy for the current process: @@ -111,14 +111,14 @@ Process Watchers A process watcher allows customization of how an event loop monitors child processes on Unix. Specifically, the event loop needs to know -when a child process has finished its execution. +when a child process has exited. In asyncio, child processes are created with :func:`create_subprocess_exec` and :meth:`loop.subprocess_exec` functions. -asyncio defines an abstract base class :class:`AbstractChildWatcher` -that child watchers should implement, and has two different +asyncio defines the :class:`AbstractChildWatcher` abstract base class, +which child watchers should implement, and has two different implementations: :class:`SafeChildWatcher` (configured to be used by default) and :class:`FastChildWatcher`. @@ -141,8 +141,7 @@ implementation used by the asyncio event loop: .. note:: Third-party event loops implementations might not support custom child watchers. For such event loops, using - :func:`set_child_watcher` might have no effect or even can - be prohibited. + :func:`set_child_watcher` might be prohibited or have no effect. .. class:: AbstractChildWatcher @@ -155,7 +154,7 @@ implementation used by the asyncio event loop: another callback for the same process replaces the previous handler. - *callback* callable must be thread-safe. + The *callback* callable must be thread-safe. .. method:: remove_child_handler(pid) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst index cb16acd58be2..bdfdcf7ddb6e 100644 --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -10,8 +10,8 @@ Transports and Protocols .. rubric:: Preface -Transports and Protocols are used by **low-level** event loop -APIs such as :meth:`loop.create_connection`. They require using +Transports and Protocols are used by the **low-level** event loop +APIs such as :meth:`loop.create_connection`. They use callback-based programming style and enable high-performance implementations of network or IPC protocols (e.g. HTTP). @@ -282,7 +282,7 @@ Write-only Transports .. method:: WriteTransport.get_write_buffer_limits() - Get the *high*- and *low*-water limits for write flow control. Return a + Get the *high* and *low* watermarks for write flow control. Return a tuple ``(low, high)`` where *low* and *high* are positive number of bytes. @@ -292,14 +292,14 @@ Write-only Transports .. method:: WriteTransport.set_write_buffer_limits(high=None, low=None) - Set the *high*- and *low*-water limits for write flow control. + Set the *high* and *low* watermarks for write flow control. These two values (measured in number of bytes) control when the protocol's :meth:`protocol.pause_writing() ` and :meth:`protocol.resume_writing() ` - methods are called. If specified, the low-water limit must be less - than or equal to the high-water limit. Neither *high* nor *low* + methods are called. If specified, the low watermark must be less + than or equal to the high watermark. Neither *high* nor *low* can be negative. :meth:`~BaseProtocol.pause_writing` is called when the buffer size @@ -308,9 +308,9 @@ Write-only Transports the buffer size becomes less than or equal to the *low* value. The defaults are implementation-specific. If only the - high-water limit is given, the low-water limit defaults to an + high watermark is given, the low watermark defaults to an implementation-specific value less than or equal to the - high-water limit. Setting *high* to zero forces *low* to zero as + high watermark. Setting *high* to zero forces *low* to zero as well, and causes :meth:`~BaseProtocol.pause_writing` to be called whenever the buffer becomes non-empty. Setting *low* to zero causes :meth:`~BaseProtocol.resume_writing` to be called only once the @@ -337,11 +337,11 @@ Write-only Transports .. method:: WriteTransport.write_eof() - Close the write end of the transport after flushing buffered data. + Close the write end of the transport after flushing all buffered data. Data may still be received. This method can raise :exc:`NotImplementedError` if the transport - (e.g. SSL) doesn't support half-closes. + (e.g. SSL) doesn't support half-closed connections. Datagram Transports @@ -506,18 +506,18 @@ method for more details. .. method:: BaseProtocol.pause_writing() - Called when the transport's buffer goes over the high-water mark. + Called when the transport's buffer goes over the high watermark. .. method:: BaseProtocol.resume_writing() - Called when the transport's buffer drains below the low-water mark. + Called when the transport's buffer drains below the low watermark. -If the buffer size equals the high-water mark, +If the buffer size equals the high watermark, :meth:`~BaseProtocol.pause_writing` is not called: the buffer size must go strictly over. Conversely, :meth:`~BaseProtocol.resume_writing` is called when the -buffer size is equal or lower than the low-water mark. These end +buffer size is equal or lower than the low watermark. These end conditions are important to ensure that things go as expected when either mark is zero. @@ -541,13 +541,12 @@ accept factories that return streaming protocols. and instead make your parsing generic and flexible. However, data is always received in the correct order. - The method can be called an arbitrary number of times during - a connection. + The method can be called an arbitrary number of times while + a connection is open. However, :meth:`protocol.eof_received() ` - is called at most once and, if called, - :meth:`protocol.data_received() ` - won't be called after it. + is called at most once. Once `eof_received()` is called, + ``data_received()`` is not called anymore. .. method:: Protocol.eof_received() @@ -562,9 +561,9 @@ accept factories that return streaming protocols. Since the default implementation returns ``None``, it implicitly closes the connection. - Some transports such as SSL don't support half-closed connections, - in which case returning true from this method will result in closing - the connection. + Some transports, including SSL, don't support half-closed connections, + in which case returning true from this method will result in the connection + being closed. State machine: @@ -588,12 +587,12 @@ Buffered Streaming Protocols Buffered Protocols can be used with any event loop method that supports `Streaming Protocols`_. -The idea of ``BufferedProtocol`` is that it allows manual allocation +``BufferedProtocol`` implementations allow explicit manual allocation and control of the receive buffer. Event loops can then use the buffer provided by the protocol to avoid unnecessary data copies. This can result in noticeable performance improvement for protocols that -receive big amounts of data. Sophisticated protocols implementations -can allocate the buffer only once at creation time. +receive big amounts of data. Sophisticated protocol implementations +can significantly reduce the number of buffer allocations. The following callbacks are called on :class:`BufferedProtocol` instances: @@ -602,12 +601,12 @@ instances: Called to allocate a new receive buffer. - *sizehint* is a recommended minimal size for the returned - buffer. It is acceptable to return smaller or bigger buffers + *sizehint* is the recommended minimum size for the returned + buffer. It is acceptable to return smaller or larger buffers than what *sizehint* suggests. When set to -1, the buffer size - can be arbitrary. It is an error to return a zero-sized buffer. + can be arbitrary. It is an error to return a buffer with a zero size. - Must return an object that implements the + ``get_buffer()`` must return an object implementing the :ref:`buffer protocol `. .. method:: BufferedProtocol.buffer_updated(nbytes) @@ -658,14 +657,14 @@ factories passed to the :meth:`loop.create_datagram_endpoint` method. :class:`OSError`. *exc* is the :class:`OSError` instance. This method is called in rare conditions, when the transport (e.g. UDP) - detects that a datagram couldn't be delivered to its recipient. + detects that a datagram could not be delivered to its recipient. In many conditions though, undeliverable datagrams will be silently dropped. .. note:: On BSD systems (macOS, FreeBSD, etc.) flow control is not supported - for datagram protocols, because it is difficult to detect easily send + for datagram protocols, because there is no reliable way to detect send failures caused by writing too many packets. The socket always appears 'ready' and excess packets are dropped. An diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index 23d2f3cb1d7c..bd0e70c0d9fc 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -10,7 +10,7 @@ asyncio queues are designed to be similar to classes of the :mod:`queue` module. Although asyncio queues are not thread-safe, they are designed to be used specifically in async/await code. -Note that methods on asyncio queues don't have a *timeout* parameter; +Note that methods of asyncio queues don't have a *timeout* parameter; use :func:`asyncio.wait_for` function to do queue operations with a timeout. @@ -72,7 +72,7 @@ Queue .. coroutinemethod:: put(item) Put an item into the queue. If the queue is full, wait until a - free slot is available before adding item. + free slot is available before adding the item. .. method:: put_nowait(item) @@ -82,7 +82,7 @@ Queue .. method:: qsize() - Number of items in the queue. + Return the number of items in the queue. .. method:: task_done() diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 0489201b4d02..60aae16c6d4f 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -54,7 +54,7 @@ and work with streams: :class:`StreamReader` and :class:`StreamWriter` classes. The *loop* argument is optional and can always be determined - automatically when this method is awaited from a coroutine. + automatically when this function is awaited from a coroutine. *limit* determines the buffer size limit used by the returned :class:`StreamReader` instance. By default the *limit* @@ -84,7 +84,7 @@ and work with streams: *client_connected_cb* can be a plain callable or a :ref:`coroutine function `; if it is a coroutine function, - it will be automatically wrapped into a :class:`Task`. + it will be automatically scheduled as a :class:`Task`. The *loop* argument is optional and can always be determined automatically when this method is awaited from a coroutine. @@ -107,14 +107,14 @@ and work with streams: limit=None, ssl=None, sock=None, \ server_hostname=None, ssl_handshake_timeout=None) - Establish a UNIX socket connection and return a pair of + Establish a Unix socket connection and return a pair of ``(reader, writer)``. - Similar to :func:`open_connection` but operates on UNIX sockets. + Similar to :func:`open_connection` but operates on Unix sockets. See also the documentation of :meth:`loop.create_unix_connection`. - Availability: UNIX. + Availability: Unix. .. versionadded:: 3.7 @@ -130,13 +130,13 @@ and work with streams: backlog=100, ssl=None, ssl_handshake_timeout=None, \ start_serving=True) - Start a UNIX socket server. + Start a Unix socket server. - Similar to :func:`start_server` but works with UNIX sockets. + Similar to :func:`start_server` but works with Unix sockets. See also the documentation of :meth:`loop.create_unix_server`. - Availability: UNIX. + Availability: Unix. .. versionadded:: 3.7 @@ -167,7 +167,7 @@ StreamReader Read up to *n* bytes. If *n* is not provided, or set to ``-1``, read until EOF and return all read bytes. - If an EOF was received and the internal buffer is empty, + If EOF was received and the internal buffer is empty, return an empty ``bytes`` object. .. coroutinemethod:: readline() @@ -175,41 +175,36 @@ StreamReader Read one line, where "line" is a sequence of bytes ending with ``\n``. - If an EOF is received and ``\n`` was not found, the method + If EOF is received and ``\n`` was not found, the method returns partially read data. - If an EOF is received and the internal buffer is empty, + If EOF is received and the internal buffer is empty, return an empty ``bytes`` object. .. coroutinemethod:: readexactly(n) Read exactly *n* bytes. - Raise an :exc:`IncompleteReadError` if an EOF reached before *n* + Raise an :exc:`IncompleteReadError` if EOF is reached before *n* can be read. Use the :attr:`IncompleteReadError.partial` attribute to get the partially read data. .. coroutinemethod:: readuntil(separator=b'\\n') - Read data from the stream until ``separator`` is found. + Read data from the stream until *separator* is found. On success, the data and separator will be removed from the internal buffer (consumed). Returned data will include the separator at the end. - Configured stream limit is used to check result. Limit sets the - maximal length of data that can be returned, not counting the - separator. - - If an EOF occurs and the complete separator is still not found, - an :exc:`IncompleteReadError` exception will be - raised, and the internal buffer will be reset. The - :attr:`IncompleteReadError.partial` attribute may contain the - separator partially. + If the amount of data read exceeds the configured stream limit, a + :exc:`LimitOverrunError` exception is raised, and the data + is left in the internal buffer and can be read again. - If the data cannot be read because of over limit, a - :exc:`LimitOverrunError` exception will be raised, and the data - will be left in the internal buffer, so it can be read again. + If EOF is reached before the complete separator is found, + an :exc:`IncompleteReadError` exception is raised, and the internal + buffer is reset. The :attr:`IncompleteReadError.partial` attribute + may contain a portion of the separator. .. versionadded:: 3.5.2 @@ -235,8 +230,8 @@ StreamWriter Write *data* to the stream. - The method respects control-flow, execution is paused if write - buffer reaches high-water limit. + The method respects flow control, execution is paused if the write + buffer reaches the high watermark. .. versionadded:: 3.8 @@ -244,7 +239,7 @@ StreamWriter Close the stream. - Wait for finishing all closing actions, e.g. SSL shutdown for + Wait until all closing actions are complete, e.g. SSL shutdown for secure sockets. .. versionadded:: 3.8 @@ -272,28 +267,29 @@ StreamWriter Write *data* to the stream. - This method doesn't apply control-flow. The call should be - followed by :meth:`drain`. + This method is not subject to flow control. Calls to ``write()`` should + be followed by :meth:`drain`. The :meth:`awrite` method is a + recommended alternative the applies flow control automatically. .. method:: writelines(data) Write a list (or any iterable) of bytes to the stream. - This method doesn't apply control-flow. The call should be - followed by :meth:`drain`. + This method is not subject to flow control. Calls to ``writelines()`` + should be followed by :meth:`drain`. .. coroutinemethod:: drain() Wait until it is appropriate to resume writing to the stream. - E.g.:: + Example:: writer.write(data) await writer.drain() - This is a flow-control method that interacts with the underlying + This is a flow control method that interacts with the underlying IO write buffer. When the size of the buffer reaches - the high-water limit, *drain()* blocks until the size of the - buffer is drained down to the low-water limit and writing can + the high watermark, *drain()* blocks until the size of the + buffer is drained down to the low watermark and writing can be resumed. When there is nothing to wait for, the :meth:`drain` returns immediately. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 57a7a378b8fd..0bcf66175ce3 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -12,7 +12,7 @@ create and manage subprocesses. .. _asyncio_example_subprocess_shell: Here's an example of how asyncio can run a shell command and -communicate its result back:: +obtain its result:: import asyncio @@ -41,7 +41,7 @@ will print:: Because all asyncio subprocess functions are asynchronous and asyncio provides many tools to work with such functions, it is easy to execute and monitor multiple subprocesses in parallel. It is indeed trivial -to modify the above example to run a few commands at once:: +to modify the above example to run several commands simultaneously:: async def main(): await asyncio.gather( @@ -75,7 +75,7 @@ Creating Subprocesses stdout=None, stderr=None, loop=None, \ limit=None, \*\*kwds) - Run the shell command *cmd*. + Run the *cmd* shell command. The *limit* argument sets the buffer limit for :class:`StreamReader` wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` @@ -89,23 +89,23 @@ Creating Subprocesses .. important:: It is the application's responsibility to ensure that all whitespace and - metacharacters are quoted appropriately to avoid `shell injection + special characters are quoted appropriately to avoid `shell injection `_ vulnerabilities. The :func:`shlex.quote` function can be used to properly - escape whitespace and shell metacharacters in strings that are going to be - used to construct shell commands. + escape whitespace and special shell characters in strings that are going + to be used to construct shell commands. .. note:: - The default event loop that asyncio is pre-configured - to use on **Windows** does not support subprocesses. Subprocesses are - available for Windows if the :class:`ProactorEventLoop` is used. + The default asyncio event loop implementation on **Windows** does not + support subprocesses. Subprocesses are available for Windows if a + :class:`ProactorEventLoop` is used. See :ref:`Subprocess Support on Windows ` for details. .. seealso:: - asyncio has also *low-level* APIs to work with subprocesses: + asyncio also has the following *low-level* APIs to work with subprocesses: :meth:`loop.subprocess_exec`, :meth:`loop.subprocess_shell`, :meth:`loop.connect_read_pipe`, :meth:`loop.connect_write_pipe`, as well as the :ref:`Subprocess Transports ` @@ -130,22 +130,23 @@ Constants .. data:: asyncio.subprocess.STDOUT - Can be passed to the *stderr* parameter to redirect process' - *stderr* to *stdout*. + Special value that can be used as the *stderr* argument and indicates + that standard error should be redirected into standard output. .. data:: asyncio.subprocess.DEVNULL - Can be passed as the *stdin*, *stdout* or *stderr* parameters - to redirect the corresponding subprocess' IO to :data:`os.devnull`. + Special value that can be used as the *stdin*, *stdout* or *stderr* argument + to process creation functions. It indicates that the special file + :data:`os.devnull` will be used for the corresponding subprocess stream. Interacting with Subprocesses ============================= Both :func:`create_subprocess_exec` and :func:`create_subprocess_shell` -functions return instances of the *Process* class. It is a high-level -wrapper that allows to watch for subprocesses completion and -communicate with them. +functions return instances of the *Process* class. *Process* is a high-level +wrapper that allows communicating with subprocesses and watching for +their completion. .. class:: asyncio.subprocess.Process @@ -161,7 +162,7 @@ communicate with them. the :meth:`~subprocess.Popen.poll` method; * the :meth:`~asyncio.subprocess.Process.communicate` and - :meth:`~asyncio.subprocess.Process.wait` methods don't take a + :meth:`~asyncio.subprocess.Process.wait` methods don't have a *timeout* parameter: use the :func:`wait_for` function; * the :meth:`Process.wait() ` method @@ -177,7 +178,7 @@ communicate with them. .. coroutinemethod:: wait() - Wait for child process to terminate. + Wait for the child process to terminate. Set and return the :attr:`returncode` attribute. @@ -229,9 +230,9 @@ communicate with them. .. method:: terminate() - Stop the child. + Stop the child process. - On Posix OSs the method sends :py:data:`signal.SIGTERM` to the + On POSIX systems this method sends :py:data:`signal.SIGTERM` to the child process. On Windows the Win32 API function :c:func:`TerminateProcess` is @@ -241,7 +242,7 @@ communicate with them. Kill the child. - On Posix OSs the function sends :py:data:`SIGKILL` to the child + On POSIX systems this method sends :py:data:`SIGKILL` to the child process. On Windows this method is an alias for :meth:`terminate`. @@ -284,7 +285,7 @@ communicate with them. A ``None`` value indicates that the process has not terminated yet. A negative value ``-N`` indicates that the child was terminated - by signal ``N`` (Unix only). + by signal ``N`` (POSIX only). .. _asyncio-subprocess-threads: @@ -292,17 +293,17 @@ communicate with them. Subprocess and Threads ---------------------- -asyncio built-in event loops support running subprocesses from -different threads, but there are the following limitations: +Standard asyncio event loop supports running subprocesses from +different threads, but there are limitations: * An event loop must run in the main thread. -* The child watcher must be instantiated in the main thread, +* The child watcher must be instantiated in the main thread before executing subprocesses from other threads. Call the :func:`get_child_watcher` function in the main thread to instantiate the child watcher. -Note, that alternative event loop implementations might not share +Note that alternative event loop implementations might not share the above limitations; please refer to their documentation. .. seealso:: @@ -316,7 +317,7 @@ Examples An example using the :class:`~asyncio.subprocess.Process` class to control a subprocess and the :class:`StreamReader` class to read from -the *stdout*. +its standard output. .. _asyncio_example_create_subprocess_exec: diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index f29988554cee..18b562970436 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -10,10 +10,10 @@ asyncio synchronization primitives are designed to be similar to those of the :mod:`threading` module with two important caveats: * asyncio primitives are not thread-safe, therefore they should not - be used for OS threads synchronization (use :mod:`threading` for + be used for OS thread synchronization (use :mod:`threading` for that); -* methods of synchronization primitives do not accept the *timeout* +* methods of these synchronization primitives do not accept the *timeout* argument; use the :func:`asyncio.wait_for` function to perform operations with timeouts. @@ -153,12 +153,12 @@ Condition A Condition object. Not thread-safe. An asyncio condition primitive can be used by a task to wait for - some event to happen and then get an exclusive access to a shared + some event to happen and then get exclusive access to a shared resource. In essence, a Condition object combines the functionality - of :class:`Event` and :class:`Lock`. It is possible to have many - Condition objects sharing one Lock, which allows to coordinate + of an :class:`Event` and a :class:`Lock`. It is possible to have + multiple Condition objects share one Lock, which allows coordinating exclusive access to a shared resource between different tasks interested in particular states of that shared resource. @@ -287,7 +287,7 @@ Semaphore Acquire a semaphore. If the internal counter is greater than zero, decrement - it by one and return ``True`` immediately. If it is zero wait + it by one and return ``True`` immediately. If it is zero, wait until a :meth:`release` is called and return ``True``. .. method:: locked() @@ -300,7 +300,7 @@ Semaphore Can wake up a task waiting to acquire the semaphore. Unlike :class:`BoundedSemaphore`, :class:`Semaphore` allows - to make more ``release()`` calls than ``acquire()`` calls. + making more ``release()`` calls than ``acquire()`` calls. BoundedSemaphore diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 3faaf0cdb9b8..4893a7c75209 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -20,7 +20,7 @@ Coroutines Coroutines declared with async/await syntax is the preferred way of writing asyncio applications. For example, the following snippet -of code prints "hello", waits 1 second, and prints "world":: +of code prints "hello", waits 1 second, and then prints "world":: >>> import asyncio @@ -41,10 +41,10 @@ be executed:: To actually run a coroutine asyncio provides three main mechanisms: -* By using the :func:`asyncio.run` function to run the top-level +* The :func:`asyncio.run` function to run the top-level entry point "main()" function (see the above example.) -* By awaiting on a coroutine. The following snippet of code will +* Awaiting on a coroutine. The following snippet of code will print "hello" after waiting for 1 second, and then print "world" after waiting for *another* 2 seconds:: @@ -72,7 +72,7 @@ To actually run a coroutine asyncio provides three main mechanisms: world finished at 17:13:55 -* By using the :func:`asyncio.create_task` function to run coroutines +* The :func:`asyncio.create_task` function to run coroutines concurrently as asyncio :class:`Tasks `. Let's modify the above example and run two "set_after" coroutines @@ -130,8 +130,8 @@ Running an asyncio Program programs, and should ideally only be called once. .. versionadded:: 3.7 - **Important:** this has been added to asyncio in Python 3.7 - on a :term:`provisional basis `. + **Important:** this function has been added to asyncio in + Python 3.7 on a :term:`provisional basis `. Creating Tasks @@ -139,13 +139,13 @@ Creating Tasks .. function:: create_task(coro, \*, name=None) - Wrap a :ref:`coroutine ` *coro* into a task and schedule + Wrap the *coro* :ref:`coroutine ` into a task and schedule its execution. Return the task object. If *name* is not ``None``, it is set as the name of the task using :meth:`Task.set_name`. - The task is executed in :func:`get_running_loop` context, + The task is executed in the loop returned by :func:`get_running_loop`, :exc:`RuntimeError` is raised if there is no running loop in current thread. @@ -168,7 +168,7 @@ Sleeping .. _asyncio_example_sleep: Example of coroutine displaying the current date every second - during 5 seconds:: + for 5 seconds:: import asyncio import datetime @@ -198,7 +198,7 @@ Running Tasks Concurrently order of the original *fs* sequence. All coroutines in the *fs* list are automatically - wrapped in :class:`Tasks `. + scheduled as :class:`Tasks `. If *return_exceptions* is ``True``, exceptions in the Tasks/Futures are treated the same as successful results, and gathered in the @@ -263,14 +263,14 @@ Shielding Tasks From Cancellation Wait for a Future/Task while protecting it from being cancelled. *fut* can be a coroutine, a Task, or a Future-like object. If - *fut* is a coroutine it is automatically wrapped in a + *fut* is a coroutine it is automatically scheduled as a :class:`Task`. The statement:: res = await shield(something()) - is equivalent to the statement:: + is equivalent to:: res = await something() @@ -278,7 +278,7 @@ Shielding Tasks From Cancellation Task running in ``something()`` is not cancelled. From the point of view of ``something()``, the cancellation did not happen. Although its caller is still cancelled, so the "await" expression - still raises :exc:`CancelledError`. + still raises a :exc:`CancelledError`. If ``something()`` is cancelled by other means (i.e. from within itself) that would also cancel ``shield()``. @@ -298,10 +298,10 @@ Timeouts .. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) - Wait for the coroutine, Task, or Future to complete with timeout. + Wait for a coroutine, Task, or Future to complete with timeout. *fut* can be a coroutine, a Task, or a Future-like object. If - *fut* is a coroutine it is automatically wrapped in a + *fut* is a coroutine it is automatically scheduled as a :class:`Task`. *timeout* can either be ``None`` or a float or int number of seconds @@ -352,10 +352,10 @@ Waiting Primitives .. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\ return_when=ALL_COMPLETED) - Wait for a set of Futures to complete. + Wait for a set of coroutines, Tasks, or Futures to complete. *fs* is a list of coroutines, Futures, and/or Tasks. Coroutines - are automatically wrapped in :class:`Tasks `. + are automatically scheduled as :class:`Tasks `. Returns two sets of Tasks/Futures: ``(done, pending)``. @@ -363,7 +363,7 @@ Waiting Primitives the maximum number of seconds to wait before returning. Note that this function does not raise :exc:`asyncio.TimeoutError`. - Futures or Tasks that aren't done when the timeout occurs are just + Futures or Tasks that aren't done when the timeout occurs are simply returned in the second set. *return_when* indicates when this function should return. It must @@ -397,7 +397,7 @@ Waiting Primitives .. function:: as_completed(fs, \*, loop=None, timeout=None) - Return an iterator which values, when waited for, are + Return an iterator of awaitables which return :class:`Future` instances. Raises :exc:`asyncio.TimeoutError` if the timeout occurs before @@ -500,9 +500,9 @@ Task Object IO operations. Use the high-level :func:`asyncio.create_task` function to create - Tasks, or low-level :meth:`loop.create_task` or - :func:`ensure_future` functions. Manually instantiating Task - objects is discouraged. + Tasks, or the low-level :meth:`loop.create_task` or + :func:`ensure_future` functions. Manual instantiation of Tasks + is discouraged. To cancel a running Task use the :meth:`cancel` method. Calling it will cause the Task to throw a :exc:`CancelledError` exception into @@ -660,7 +660,7 @@ Task Object If *loop* is ``None``, the :func:`get_event_loop` function is used to get the current loop. - This function is **deprecated** and scheduled for removal in + This method is **deprecated** and will be removed in Python 3.9. Use the :func:`all_tasks` function instead. .. classmethod:: current_task(loop=None) @@ -670,7 +670,7 @@ Task Object If *loop* is ``None``, the :func:`get_event_loop` function is used to get the current loop. - This function is **deprecated** and scheduled for removal in + This method is **deprecated** and will be removed in Python 3.9. Use the :func:`current_task` function instead. @@ -682,10 +682,10 @@ Generator-based Coroutines .. note:: Support for generator-based coroutines is **deprecated** and - scheduled for removal in Python 4.0. + is scheduled for removal in Python 4.0. Generator-based coroutines predate async/await syntax. They are -Python generators that use ``yield from`` expression is to await +Python generators that use ``yield from`` expressions to await on Futures and other coroutines. Generator-based coroutines should be decorated with diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 73b0e63a68db..bfc97001bb71 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -20,7 +20,7 @@ asyncio.run(main()) asyncio is a library to write **concurrent** code using -**async/await** syntax. +the **async/await** syntax. asyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, @@ -42,7 +42,8 @@ asyncio provides a set of **high-level** APIs to: * :ref:`synchronize ` concurrent code; -as well as **low-level** APIs for *library and framework developers* to: +Additionally, there are **low-level** APIs for +*library and framework developers* to: * create and manage :ref:`event loops `, which provide asynchronous APIs for :meth:`networking `, From webhook-mailer at python.org Mon Sep 17 19:22:33 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 17 Sep 2018 23:22:33 -0000 Subject: [Python-checkins] bpo-34589: C locale coercion off by default (GH-9073) Message-ID: https://github.com/python/cpython/commit/7a0791b6992d420dc52536257f2f093851ed7215 commit: 7a0791b6992d420dc52536257f2f093851ed7215 branch: master author: Victor Stinner committer: GitHub date: 2018-09-17T16:22:29-07:00 summary: bpo-34589: C locale coercion off by default (GH-9073) Py_Initialize() and Py_Main() cannot enable the C locale coercion (PEP 538) anymore: it is always disabled. It can now only be enabled by the Python program ("python3). test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8 nor PYTHONCOERCECLOCALE, these variables are already set in the parent. files: A Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst M Include/coreconfig.h M Include/pylifecycle.h M Lib/test/test_embed.py M Modules/main.c M Programs/_testembed.c M Programs/python.c M Python/coreconfig.c diff --git a/Include/coreconfig.h b/Include/coreconfig.h index 8944ec23df09..293d8ed16604 100644 --- a/Include/coreconfig.h +++ b/Include/coreconfig.h @@ -301,6 +301,10 @@ typedef struct { variable. The option is also enabled if the LC_CTYPE locale is "C" and a target locale (ex: "C.UTF-8") is supported by the platform. + Py_Initialize() and Py_Main() must not enable C locale coercion: it is + always disabled. The option can only be enabled by the Python program + ("python3). + See also the _coerce_c_locale_warn option. */ int _coerce_c_locale; @@ -308,6 +312,10 @@ typedef struct { Enabled by the PYTHONCOERCECLOCALE=warn environment variable. + Py_Initialize() and Py_Main() must not enable C locale coercion warning: + it is always disabled. The warning can only be enabled by the Python + program ("python3). + See also the _coerce_c_locale option. */ int _coerce_c_locale_warn; @@ -328,7 +336,8 @@ typedef struct { .use_hash_seed = -1, \ .faulthandler = -1, \ .tracemalloc = -1, \ - ._coerce_c_locale = -1, \ + ._coerce_c_locale = 0, \ + ._coerce_c_locale_warn = 0, \ .utf8_mode = -1, \ .argc = -1, \ .nmodule_search_path = -1, \ diff --git a/Include/pylifecycle.h b/Include/pylifecycle.h index 04e672e96e17..f64bae3f6170 100644 --- a/Include/pylifecycle.h +++ b/Include/pylifecycle.h @@ -83,7 +83,11 @@ PyAPI_FUNC(int) Py_FdIsInteractive(FILE *, const char *); /* Bootstrap __main__ (defined in Modules/main.c) */ PyAPI_FUNC(int) Py_Main(int argc, wchar_t **argv); #ifdef Py_BUILD_CORE +# ifdef MS_WINDOWS +PyAPI_FUNC(int) _Py_WindowsMain(int argc, wchar_t **argv); +# else PyAPI_FUNC(int) _Py_UnixMain(int argc, char **argv); +# endif #endif /* In getpath.c */ diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index c00000e68131..e531fd49d5a3 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -324,10 +324,6 @@ def get_filesystem_encoding(self, isolated, env): 'print(sys.getfilesystemencoding(), ' 'sys.getfilesystemencodeerrors())') args = (sys.executable, '-c', code) - env = dict(env) - if not isolated: - env['PYTHONCOERCECLOCALE'] = '0' - env['PYTHONUTF8'] = '0' proc = subprocess.run(args, text=True, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE) diff --git a/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst b/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst new file mode 100644 index 000000000000..27b6a6e0017f --- /dev/null +++ b/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst @@ -0,0 +1,3 @@ +Py_Initialize() and Py_Main() cannot enable the C locale coercion (PEP 538) +anymore: it is always disabled. It can now only be enabled by the Python +program ("python3). diff --git a/Modules/main.c b/Modules/main.c index 6bc2917e2ccc..d4ae4a0ae305 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -1700,7 +1700,8 @@ pymain_cmdline(_PyMain *pymain, _PyCoreConfig *config) static int -pymain_init(_PyMain *pymain, PyInterpreterState **interp_p) +pymain_init(_PyMain *pymain, PyInterpreterState **interp_p, + int use_c_locale_coercion) { /* 754 requires that FP exceptions run in "no stop" mode by default, * and until C vendors implement C99's ways to control FP exceptions, @@ -1713,6 +1714,11 @@ pymain_init(_PyMain *pymain, PyInterpreterState **interp_p) _PyCoreConfig local_config = _PyCoreConfig_INIT; _PyCoreConfig *config = &local_config; + if (use_c_locale_coercion) { + /* set to -1 to be able to enable the feature */ + config->_coerce_c_locale = -1; + config->_coerce_c_locale_warn = -1; + } _PyCoreConfig_GetGlobalConfig(config); @@ -1747,10 +1753,10 @@ pymain_init(_PyMain *pymain, PyInterpreterState **interp_p) static int -pymain_main(_PyMain *pymain) +pymain_main(_PyMain *pymain, int use_c_locale_coercion) { PyInterpreterState *interp; - int res = pymain_init(pymain, &interp); + int res = pymain_init(pymain, &interp, use_c_locale_coercion); if (res != 1) { if (pymain_run_python(pymain, interp) < 0) { _Py_FatalInitError(pymain->err); @@ -1777,10 +1783,22 @@ Py_Main(int argc, wchar_t **argv) pymain.argc = argc; pymain.wchar_argv = argv; - return pymain_main(&pymain); + return pymain_main(&pymain, 0); } +#ifdef MS_WINDOWS +int +_Py_WindowsMain(int argc, wchar_t **argv) +{ + _PyMain pymain = _PyMain_INIT; + pymain.use_bytes_argv = 0; + pymain.argc = argc; + pymain.wchar_argv = argv; + + return pymain_main(&pymain, 1); +} +#else int _Py_UnixMain(int argc, char **argv) { @@ -1789,8 +1807,9 @@ _Py_UnixMain(int argc, char **argv) pymain.argc = argc; pymain.bytes_argv = argv; - return pymain_main(&pymain); + return pymain_main(&pymain, 1); } +#endif /* this is gonna seem *real weird*, but if you put some other code between diff --git a/Programs/_testembed.c b/Programs/_testembed.c index 12880325b254..b9394690c3fc 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -482,8 +482,6 @@ static int test_init_from_config(void) putenv("PYTHONMALLOCSTATS=0"); config.malloc_stats = 1; - /* FIXME: test _coerce_c_locale and _coerce_c_locale_warn */ - putenv("PYTHONUTF8=0"); Py_UTF8Mode = 0; config.utf8_mode = 1; @@ -606,8 +604,7 @@ static int test_init_isolated(void) /* Test _PyCoreConfig.isolated=1 */ _PyCoreConfig config = _PyCoreConfig_INIT; - /* Set _coerce_c_locale and utf8_mode to not depend on the locale */ - config._coerce_c_locale = 0; + /* Set utf8_mode to not depend on the locale */ config.utf8_mode = 0; /* Use path starting with "./" avoids a search along the PATH */ config.program_name = L"./_testembed"; diff --git a/Programs/python.c b/Programs/python.c index 78e48f800c95..c7697facbe3b 100644 --- a/Programs/python.c +++ b/Programs/python.c @@ -6,7 +6,7 @@ int wmain(int argc, wchar_t **argv) { - return Py_Main(argc, argv); + return _Py_WindowsMain(argc, argv); } #else int diff --git a/Python/coreconfig.c b/Python/coreconfig.c index 1e03ce31f3d5..131a043ff280 100644 --- a/Python/coreconfig.c +++ b/Python/coreconfig.c @@ -816,7 +816,9 @@ config_read_env_vars(_PyCoreConfig *config) } } else if (strcmp(env, "warn") == 0) { - config->_coerce_c_locale_warn = 1; + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 1; + } } else { if (config->_coerce_c_locale < 0) { @@ -1324,6 +1326,9 @@ _PyCoreConfig_Read(_PyCoreConfig *config) if (config->_coerce_c_locale < 0) { config->_coerce_c_locale = 0; } + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 0; + } if (config->utf8_mode < 0) { config->utf8_mode = 0; } From webhook-mailer at python.org Mon Sep 17 19:35:35 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Mon, 17 Sep 2018 23:35:35 -0000 Subject: [Python-checkins] bpo-33649: Backport asyncio docs from 'master' to 3.7 (GH-9377) Message-ID: https://github.com/python/cpython/commit/512d7101098b971837cbb406942215244f636547 commit: 512d7101098b971837cbb406942215244f636547 branch: 3.7 author: Yury Selivanov committer: GitHub date: 2018-09-17T19:35:30-04:00 summary: bpo-33649: Backport asyncio docs from 'master' to 3.7 (GH-9377) files: A Doc/library/asyncio-api-index.rst A Doc/library/asyncio-exceptions.rst A Doc/library/asyncio-future.rst A Doc/library/asyncio-llapi-index.rst A Doc/library/asyncio-platforms.rst A Doc/library/asyncio-policy.rst D Doc/library/asyncio-eventloops.rst M Doc/library/asyncio-dev.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-protocol.rst M Doc/library/asyncio-queue.rst M Doc/library/asyncio-stream.rst M Doc/library/asyncio-subprocess.rst M Doc/library/asyncio-sync.rst M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst M Doc/library/ipc.rst M Doc/tools/templates/layout.html M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.7.rst diff --git a/Doc/library/asyncio-api-index.rst b/Doc/library/asyncio-api-index.rst new file mode 100644 index 000000000000..d5b5659abc65 --- /dev/null +++ b/Doc/library/asyncio-api-index.rst @@ -0,0 +1,218 @@ +.. currentmodule:: asyncio + + +==================== +High-level API Index +==================== + +This page lists all high-level async/await enabled asyncio APIs. + + +Tasks +===== + +Utilities to run asyncio programs, create Tasks, and +await on multiple things with timeouts. + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :func:`run` + - Create event loop, run a coroutine, close the loop. + + * - :func:`create_task` + - Start an asyncio Task. + + * - ``await`` :func:`sleep` + - Sleep for a number of seconds. + + * - ``await`` :func:`gather` + - Schedule and wait for things concurrently. + + * - ``await`` :func:`wait_for` + - Run with a timeout. + + * - ``await`` :func:`shield` + - Shield from cancellation. + + * - ``await`` :func:`wait` + - Monitor for completion. + + * - :func:`current_task` + - Return the current Task. + + * - :func:`all_tasks` + - Return all tasks for an event loop. + + * - :class:`Task` + - Task object. + + * - :func:`run_coroutine_threadsafe` + - Schedule a coroutine from another OS thread. + + * - ``for in`` :func:`as_completed` + - Monitor for completion with a ``for`` loop. + + +.. rubric:: Examples + +* :ref:`Using asyncio.gather() to run things in parallel + `. + +* :ref:`Using asyncio.wait_for() to enforce a timeout + `. + +* :ref:`Cancellation `. + +* :ref:`Using asyncio.sleep() `. + +* See also the main :ref:`Tasks documentation page `. + + +Queues +====== + +Queues should be used to distribute work amongst multiple asyncio Tasks, +implement connection pools, and pub/sub patterns. + + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :class:`Queue` + - A FIFO queue. + + * - :class:`PriorityQueue` + - A priority queue. + + * - :class:`LifoQueue` + - A LIFO queue. + + +.. rubric:: Examples + +* :ref:`Using asyncio.Queue to distribute workload between several + Tasks `. + +* See also the :ref:`Queues documentation page `. + + +Subprocesses +============ + +Utilities to spawn subprocesses and run shell commands. + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :func:`create_subprocess_exec` + - Create a subprocess. + + * - ``await`` :func:`create_subprocess_shell` + - Run a shell command. + + +.. rubric:: Examples + +* :ref:`Executing a shell command `. + +* See also the :ref:`subprocess APIs ` + documentation. + + +Streams +======= + +High-level APIs to work with network IO. + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :func:`open_connection` + - Establish a TCP connection. + + * - ``await`` :func:`open_unix_connection` + - Establish a Unix socket connection. + + * - ``await`` :func:`start_server` + - Start a TCP server. + + * - ``await`` :func:`start_unix_server` + - Start a Unix socket server. + + * - :class:`StreamReader` + - High-level async/await object to receive network data. + + * - :class:`StreamWriter` + - High-level async/await object to send network data. + + +.. rubric:: Examples + +* :ref:`Example TCP client `. + +* See also the :ref:`streams APIs ` + documentation. + + +Synchronization +=============== + +Threading-like synchronization primitives that can be used in Tasks. + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :class:`Lock` + - A mutex lock. + + * - :class:`Event` + - An event object. + + * - :class:`Condition` + - A condition object. + + * - :class:`Semaphore` + - A semaphore. + + * - :class:`BoundedSemaphore` + - A bounded semaphore. + + +.. rubric:: Examples + +* :ref:`Using asyncio.Event `. + +* See also the documentation of asyncio + :ref:`synchronization primitives `. + + +Exceptions +========== + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + + * - :exc:`asyncio.TimeoutError` + - Raised on timeout by functions like :func:`wait_for`. + Keep in mind that ``asyncio.TimeoutError`` is **unrelated** + to the built-in :exc:`TimeoutError` exception. + + * - :exc:`asyncio.CancelledError` + - Raised when a Task is cancelled. See also :meth:`Task.cancel`. + + +.. rubric:: Examples + +* :ref:`Handling CancelledError to run code on cancellation request + `. + +* See also the full list of + :ref:`asyncio-specific exceptions `. diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index 100fff561c5b..5f926fceb22d 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -2,415 +2,236 @@ .. _asyncio-dev: -Develop with asyncio -==================== +======================= +Developing with asyncio +======================= -Asynchronous programming is different than classical "sequential" programming. -This page lists common traps and explains how to avoid them. +Asynchronous programming is different from classic "sequential" +programming. +This page lists common mistakes and traps and explains how +to avoid them. -.. _asyncio-debug-mode: - -Debug mode of asyncio ---------------------- - -The implementation of :mod:`asyncio` has been written for performance. -In order to ease the development of asynchronous code, you may wish to -enable *debug mode*. -To enable all debug checks for an application: +.. _asyncio-debug-mode: -* Enable the asyncio debug mode globally by setting the environment variable - :envvar:`PYTHONASYNCIODEBUG` to ``1``, using ``-X dev`` command line option - (see the :option:`-X` option), or by calling - :meth:`AbstractEventLoop.set_debug`. -* Set the log level of the :ref:`asyncio logger ` to - :py:data:`logging.DEBUG`. For example, call - ``logging.basicConfig(level=logging.DEBUG)`` at startup. -* Configure the :mod:`warnings` module to display :exc:`ResourceWarning` - warnings. For example, use the ``-Wdefault`` command line option of Python to - display them. +Debug Mode +========== -Examples debug checks: +By default asyncio runs in production mode. In order to ease +the development asyncio has a *debug mode*. -* Log :ref:`coroutines defined but never "yielded from" - ` -* :meth:`~AbstractEventLoop.call_soon` and :meth:`~AbstractEventLoop.call_at` methods - raise an exception if they are called from the wrong thread. -* Log the execution time of the selector -* Log callbacks taking more than 100 ms to be executed. The - :attr:`AbstractEventLoop.slow_callback_duration` attribute is the minimum - duration in seconds of "slow" callbacks. -* :exc:`ResourceWarning` warnings are emitted when transports and event loops - are :ref:`not closed explicitly `. +There are several ways to enable asyncio debug mode: -.. versionchanged:: 3.7 +* Setting the :envvar:`PYTHONASYNCIODEBUG` environment variable to ``1``. - The new ``-X dev`` command line option can now also be used to enable - the debug mode. +* Using the :option:`-X` ``dev`` Python command line option. -.. seealso:: +* Passing ``debug=True`` to :func:`asyncio.run`. - The :meth:`AbstractEventLoop.set_debug` method and the :ref:`asyncio logger - `. +* Calling :meth:`loop.set_debug`. +In addition to enabling the debug mode, consider also: -Cancellation ------------- +* setting the log level of the :ref:`asyncio logger ` to + :py:data:`logging.DEBUG`, for example the following snippet of code + can be run at startup of the application:: -Cancellation of tasks is not common in classic programming. In asynchronous -programming, not only is it something common, but you have to prepare your -code to handle it. + logging.basicConfig(level=logging.DEBUG) -Futures and tasks can be cancelled explicitly with their :meth:`Future.cancel` -method. The :func:`wait_for` function cancels the waited task when the timeout -occurs. There are many other cases where a task can be cancelled indirectly. +* configuring the :mod:`warnings` module to display + :exc:`ResourceWarning` warnings. One way of doing that is by + using the :option:`-W` ``default`` command line option. -Don't call :meth:`~Future.set_result` or :meth:`~Future.set_exception` method -of :class:`Future` if the future is cancelled: it would fail with an exception. -For example, write:: - if not fut.cancelled(): - fut.set_result('done') +When the debug mode is enabled: -Don't schedule directly a call to the :meth:`~Future.set_result` or the -:meth:`~Future.set_exception` method of a future with -:meth:`AbstractEventLoop.call_soon`: the future can be cancelled before its method -is called. +* asyncio checks for :ref:`coroutines that were not awaited + ` and logs them; this mitigates + the "forgotten await" pitfall. -If you wait for a future, you should check early if the future was cancelled to -avoid useless operations. Example:: +* Many non-treadsafe asyncio APIs (such as :meth:`loop.call_soon` and + :meth:`loop.call_at` methods) raise an exception if they are called + from a wrong thread. - async def slow_operation(fut): - if fut.cancelled(): - return - # ... slow computation ... - await fut - # ... +* The execution time of the I/O selector is logged if it takes too long to + perform an I/O operation. -The :func:`shield` function can also be used to ignore cancellation. +* Callbacks taking longer than 100ms are logged. The + :attr:`loop.slow_callback_duration` attribute can be used to set the + minimum execution duration in seconds that is considered "slow". .. _asyncio-multithreading: -Concurrency and multithreading ------------------------------- +Concurrency and Multithreading +============================== -An event loop runs in a thread and executes all callbacks and tasks in the same -thread. While a task is running in the event loop, no other task is running in -the same thread. But when the task uses ``await``, the task is suspended -and the event loop executes the next task. +An event loop runs in a thread (typically the main thread) and executes +all callbacks and Tasks in its thread. While a Task is running in the +event loop, no other Tasks can run in the same thread. When a Task +executes an ``await`` expression, the running Task gets suspended, and +the event loop executes the next Task. -To schedule a callback from a different thread, the -:meth:`AbstractEventLoop.call_soon_threadsafe` method should be used. Example:: +To schedule a callback from a different OS thread, the +:meth:`loop.call_soon_threadsafe` method should be used. Example:: loop.call_soon_threadsafe(callback, *args) -Most asyncio objects are not thread safe. You should only worry if you access -objects outside the event loop. For example, to cancel a future, don't call -directly its :meth:`Future.cancel` method, but:: +Almost all asyncio objects are not thread safe, which is typically +not a problem unless there is code that works with them from outside +of a Task or a callback. If there's a need for such code to call a +low-level asyncio API, the :meth:`loop.call_soon_threadsafe` method +should be used, e.g.:: loop.call_soon_threadsafe(fut.cancel) -To handle signals and to execute subprocesses, the event loop must be run in -the main thread. - -To schedule a coroutine object from a different thread, the +To schedule a coroutine object from a different OS thread, the :func:`run_coroutine_threadsafe` function should be used. It returns a :class:`concurrent.futures.Future` to access the result:: - future = asyncio.run_coroutine_threadsafe(coro_func(), loop) - result = future.result(timeout) # Wait for the result with a timeout - -The :meth:`AbstractEventLoop.run_in_executor` method can be used with a thread pool -executor to execute a callback in different thread to not block the thread of -the event loop. - -.. seealso:: + async def coro_func(): + return await asyncio.sleep(1, 42) - The :ref:`Synchronization primitives ` section describes ways - to synchronize tasks. + # Later in another OS thread: - The :ref:`Subprocess and threads ` section lists - asyncio limitations to run subprocesses from different threads. + future = asyncio.run_coroutine_threadsafe(coro_func(), loop) + # Wait for the result: + result = future.result() +To handle signals and to execute subprocesses, the event loop must be +run in the main thread. +The :meth:`loop.run_in_executor` method can be used with a +:class:`concurrent.futures.ThreadPoolExecutor` to execute +blocking code in a different OS thread without blocking the OS thread +that the event loop runs in. .. _asyncio-handle-blocking: -Handle blocking functions correctly ------------------------------------ - -Blocking functions should not be called directly. For example, if a function -blocks for 1 second, other tasks are delayed by 1 second which can have an -important impact on reactivity. +Running Blocking Code +===================== -For networking and subprocesses, the :mod:`asyncio` module provides high-level -APIs like :ref:`protocols `. +Blocking (CPU-bound) code should not be called directly. For example, +if a function performs a CPU-intensive calculation for 1 second, +all concurrent asyncio Tasks and IO operations would be delayed +by 1 second. -An executor can be used to run a task in a different thread or even in a -different process, to not block the thread of the event loop. See the -:meth:`AbstractEventLoop.run_in_executor` method. - -.. seealso:: - - The :ref:`Delayed calls ` section details how the - event loop handles time. +An executor can be used to run a task in a different thread or even in +a different process to avoid blocking block the OS thread with the +event loop. See the :meth:`loop.run_in_executor` method for more +details. .. _asyncio-logger: Logging -------- - -The :mod:`asyncio` module logs information with the :mod:`logging` module in -the logger ``'asyncio'``. +======= -The default log level for the :mod:`asyncio` module is :py:data:`logging.INFO`. -For those not wanting such verbosity from :mod:`asyncio` the log level can -be changed. For example, to change the level to :py:data:`logging.WARNING`: +asyncio uses the :mod:`logging` module and all logging is performed +via the ``"asyncio"`` logger. -.. code-block:: none +The default log level is :py:data:`logging.INFO`, which can be easily +adjusted:: - logging.getLogger('asyncio').setLevel(logging.WARNING) + logging.getLogger("asyncio").setLevel(logging.WARNING) .. _asyncio-coroutine-not-scheduled: -Detect coroutine objects never scheduled ----------------------------------------- +Detect never-awaited coroutines +=============================== -When a coroutine function is called and its result is not passed to -:func:`ensure_future` or to the :meth:`AbstractEventLoop.create_task` method, -the execution of the coroutine object will never be scheduled which is -probably a bug. :ref:`Enable the debug mode of asyncio ` -to :ref:`log a warning ` to detect it. - -Example with the bug:: +When a coroutine function is called, but not awaited +(e.g. ``coro()`` instead of ``await coro()``) +or the coroutine is not scheduled with :meth:`asyncio.create_task`, asyncio +will emit a :exc:`RuntimeWarning`:: import asyncio async def test(): print("never scheduled") + async def main(): + test() + + asyncio.run(main()) + +Output:: + + test.py:7: RuntimeWarning: coroutine 'test' was never awaited test() Output in debug mode:: - Coroutine test() at test.py:3 was never yielded from - Coroutine object created at (most recent call last): - File "test.py", line 7, in - test() + test.py:7: RuntimeWarning: coroutine 'test' was never awaited + Coroutine created at (most recent call last) + File "../t.py", line 9, in + asyncio.run(main(), debug=True) + + < .. > -The fix is to call the :func:`ensure_future` function or the -:meth:`AbstractEventLoop.create_task` method with the coroutine object. + File "../t.py", line 7, in main + test() + test() -.. seealso:: +The usual fix is to either await the coroutine or call the +:meth:`asyncio.create_task` function:: - :ref:`Pending task destroyed `. + async def main(): + await test() -Detect exceptions never consumed --------------------------------- +Detect never-retrieved exceptions +================================= -Python usually calls :func:`sys.excepthook` on unhandled exceptions. If -:meth:`Future.set_exception` is called, but the exception is never consumed, -:func:`sys.excepthook` is not called. Instead, :ref:`a log is emitted -` when the future is deleted by the garbage collector, with the -traceback where the exception was raised. +If a :meth:`Future.set_exception` is called but the Future object is +never awaited on, the exception would never be propagated to the +user code. In this case, asyncio would emit a log message when the +Future object is garbage collected. -Example of unhandled exception:: +Example of an unhandled exception:: import asyncio - @asyncio.coroutine - def bug(): + async def bug(): raise Exception("not consumed") - loop = asyncio.get_event_loop() - asyncio.ensure_future(bug()) - loop.run_forever() - loop.close() + async def main(): + asyncio.create_task(bug()) + + asyncio.run(main()) Output:: Task exception was never retrieved - future: exception=Exception('not consumed',)> - Traceback (most recent call last): - File "asyncio/tasks.py", line 237, in _step - result = next(coro) - File "asyncio/coroutines.py", line 141, in coro - res = func(*args, **kw) - File "test.py", line 5, in bug - raise Exception("not consumed") - Exception: not consumed - -:ref:`Enable the debug mode of asyncio ` to get the -traceback where the task was created. Output in debug mode:: + future: + exception=Exception('not consumed')> - Task exception was never retrieved - future: exception=Exception('not consumed',) created at test.py:8> - source_traceback: Object created at (most recent call last): - File "test.py", line 8, in - asyncio.ensure_future(bug()) Traceback (most recent call last): - File "asyncio/tasks.py", line 237, in _step - result = next(coro) - File "asyncio/coroutines.py", line 79, in __next__ - return next(self.gen) - File "asyncio/coroutines.py", line 141, in coro - res = func(*args, **kw) - File "test.py", line 5, in bug + File "test.py", line 4, in bug raise Exception("not consumed") Exception: not consumed -There are different options to fix this issue. The first option is to chain the -coroutine in another coroutine and use classic try/except:: - - async def handle_exception(): - try: - await bug() - except Exception: - print("exception consumed") - - loop = asyncio.get_event_loop() - asyncio.ensure_future(handle_exception()) - loop.run_forever() - loop.close() - -Another option is to use the :meth:`AbstractEventLoop.run_until_complete` -function:: - - task = asyncio.ensure_future(bug()) - try: - loop.run_until_complete(task) - except Exception: - print("exception consumed") - -.. seealso:: - - The :meth:`Future.exception` method. - - -Chain coroutines correctly --------------------------- - -When a coroutine function calls other coroutine functions and tasks, they -should be chained explicitly with ``await``. Otherwise, the execution is -not guaranteed to be sequential. +:ref:`Enable the debug mode ` to get the +traceback where the task was created:: -Example with different bugs using :func:`asyncio.sleep` to simulate slow -operations:: + asyncio.run(main(), debug=True) - import asyncio - - async def create(): - await asyncio.sleep(3.0) - print("(1) create file") - - async def write(): - await asyncio.sleep(1.0) - print("(2) write into file") - - async def close(): - print("(3) close file") - - async def test(): - asyncio.ensure_future(create()) - asyncio.ensure_future(write()) - asyncio.ensure_future(close()) - await asyncio.sleep(2.0) - loop.stop() - - loop = asyncio.get_event_loop() - asyncio.ensure_future(test()) - loop.run_forever() - print("Pending tasks at exit: %s" % asyncio.Task.all_tasks(loop)) - loop.close() - -Expected output: - -.. code-block:: none - - (1) create file - (2) write into file - (3) close file - Pending tasks at exit: set() - -Actual output: - -.. code-block:: none - - (3) close file - (2) write into file - Pending tasks at exit: {>} - Task was destroyed but it is pending! - task: > - -The loop stopped before the ``create()`` finished, ``close()`` has been called -before ``write()``, whereas coroutine functions were called in this order: -``create()``, ``write()``, ``close()``. - -To fix the example, tasks must be marked with ``await``:: - - async def test(): - await asyncio.ensure_future(create()) - await asyncio.ensure_future(write()) - await asyncio.ensure_future(close()) - await asyncio.sleep(2.0) - loop.stop() - -Or without ``asyncio.ensure_future()``:: - - async def test(): - await create() - await write() - await close() - await asyncio.sleep(2.0) - loop.stop() - - -.. _asyncio-pending-task-destroyed: - -Pending task destroyed ----------------------- - -If a pending task is destroyed, the execution of its wrapped :ref:`coroutine -` did not complete. It is probably a bug and so a warning is logged. - -Example of log: - -.. code-block:: none - - Task was destroyed but it is pending! - task: wait_for=> - -:ref:`Enable the debug mode of asyncio ` to get the -traceback where the task was created. Example of log in debug mode: +Output in debug mode:: -.. code-block:: none + Task exception was never retrieved + future: + exception=Exception('not consumed') created at asyncio/tasks.py:321> - Task was destroyed but it is pending! source_traceback: Object created at (most recent call last): - File "test.py", line 15, in - task = asyncio.ensure_future(coro, loop=loop) - task: wait_for= created at test.py:15> - - -.. seealso:: - - :ref:`Detect coroutine objects never scheduled `. - -.. _asyncio-close-transports: + File "../t.py", line 9, in + asyncio.run(main(), debug=True) -Close transports and event loops --------------------------------- + < .. > -When a transport is no more needed, call its ``close()`` method to release -resources. Event loops must also be closed explicitly. - -If a transport or an event loop is not closed explicitly, a -:exc:`ResourceWarning` warning will be emitted in its destructor. By default, -:exc:`ResourceWarning` warnings are ignored. The :ref:`Debug mode of asyncio -` section explains how to display them. + Traceback (most recent call last): + File "../t.py", line 4, in bug + raise Exception("not consumed") + Exception: not consumed diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 317f3fb85c54..3b13a81a5b6a 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1,103 +1,159 @@ .. currentmodule:: asyncio -.. _asyncio-event-loop: -Base Event Loop -=============== +========== +Event Loop +========== -**Source code:** :source:`Lib/asyncio/events.py` -The event loop is the central execution device provided by :mod:`asyncio`. -It provides multiple facilities, including: +.. rubric:: Preface -* Registering, executing and cancelling delayed calls (timeouts). +The event loop is the core of every asyncio application. +Event loops run asynchronous tasks and callbacks, perform network +IO operations, and run subprocesses. -* Creating client and server :ref:`transports ` for various - kinds of communication. +Application developers should typically use the high-level asyncio functions, +such as :func:`asyncio.run`, and should rarely need to reference the loop +object or call its methods. This section is intended mostly for authors +of lower-level code, libraries, and frameworks, who need finer control over +the event loop behavior. -* Launching subprocesses and the associated :ref:`transports - ` for communication with an external program. +.. rubric:: Obtaining the Event Loop -* Delegating costly function calls to a pool of threads. +The following low-level functions can be used to get, set, or create +an event loop: -.. class:: BaseEventLoop +.. function:: get_running_loop() - This class is an implementation detail. It is a subclass of - :class:`AbstractEventLoop` and may be a base class of concrete - event loop implementations found in :mod:`asyncio`. It should not - be used directly; use :class:`AbstractEventLoop` instead. - ``BaseEventLoop`` should not be subclassed by third-party code; the - internal interface is not stable. + Return the running event loop in the current OS thread. -.. class:: AbstractEventLoop + If there is no running event loop a :exc:`RuntimeError` is raised. + This function can only be called from a coroutine or a callback. - Abstract base class of event loops. + .. versionadded:: 3.7 - This class is :ref:`not thread safe `. +.. function:: get_event_loop() -Run an event loop ------------------ + Get the current event loop. If there is no current event loop set + in the current OS thread and :func:`set_event_loop` has not yet + been called, asyncio will create a new event loop and set it as the + current one. -.. method:: AbstractEventLoop.run_forever() + Because this function has rather complex behavior (especially + when custom event loop policies are in use), using the + :func:`get_running_loop` function is preferred to :func:`get_event_loop` + in coroutines and callbacks. - Run until :meth:`stop` is called. If :meth:`stop` is called before - :meth:`run_forever()` is called, this polls the I/O selector once - with a timeout of zero, runs all callbacks scheduled in response to - I/O events (and those that were already scheduled), and then exits. - If :meth:`stop` is called while :meth:`run_forever` is running, - this will run the current batch of callbacks and then exit. Note - that callbacks scheduled by callbacks will not run in that case; - they will run the next time :meth:`run_forever` is called. + Consider also using the :func:`asyncio.run` function instead of using + lower level functions to manually create and close an event loop. - .. versionchanged:: 3.5.1 +.. function:: set_event_loop(loop) -.. method:: AbstractEventLoop.run_until_complete(future) + Set *loop* as a current event loop for the current OS thread. - Run until the :class:`Future` is done. +.. function:: new_event_loop() - If the argument is a :ref:`coroutine object `, it is wrapped by - :func:`ensure_future`. + Create a new event loop object. - Return the Future's result, or raise its exception. +Note that the behaviour of :func:`get_event_loop`, :func:`set_event_loop`, +and :func:`new_event_loop` functions can be altered by +:ref:`setting a custom event loop policy `. -.. method:: AbstractEventLoop.is_running() - Returns running status of event loop. +.. rubric:: Contents -.. method:: AbstractEventLoop.stop() +This documentation page contains the following sections: - Stop running the event loop. +* The `Event Loop Methods`_ section is the reference documentation of + the event loop APIs; - This causes :meth:`run_forever` to exit at the next suitable - opportunity (see there for more details). +* The `Callback Handles`_ section documents the :class:`Handle` and + :class:`TimerHandle` instances which are returned from scheduling + methods such as :meth:`loop.call_soon` and :meth:`loop.call_later`; - .. versionchanged:: 3.5.1 +* The `Server Objects`_ section documents types returned from + event loop methods like :meth:`loop.create_server`; + +* The `Event Loop Implementations`_ section documents the + :class:`SelectorEventLoop` and :class:`ProactorEventLoop` classes; + +* The `Examples`_ section showcases how to work with some event + loop APIs. + + +.. _asyncio-event-loop: + +Event Loop Methods +================== + +Event loops have **low-level** APIs for the following: + +.. contents:: + :depth: 1 + :local: + + +Running and stopping the loop +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. method:: loop.run_until_complete(future) + + Run until the *future* (an instance of :class:`Future`) has + completed. + + If the argument is a :ref:`coroutine object ` it + is implicitly scheduled to run as a :class:`asyncio.Task`. + + Return the Future's result or raise its exception. + +.. method:: loop.run_forever() + + Run the event loop until :meth:`stop` is called. + + If :meth:`stop` is called before :meth:`run_forever()` is called, + the loop will poll the I/O selector once with a timeout of zero, + run all callbacks scheduled in response to I/O events (and + those that were already scheduled), and then exit. + + If :meth:`stop` is called while :meth:`run_forever` is running, + the loop will run the current batch of callbacks and then exit. + Note that new callbacks scheduled by callbacks will not run in this + case; instead, they will run the next time :meth:`run_forever` or + :meth:`run_until_complete` is called. + +.. method:: loop.stop() + + Stop the event loop. -.. method:: AbstractEventLoop.is_closed() +.. method:: loop.is_running() - Returns ``True`` if the event loop was closed. + Return ``True`` if the event loop is currently running. - .. versionadded:: 3.4.2 +.. method:: loop.is_closed() -.. method:: AbstractEventLoop.close() + Return ``True`` if the event loop was closed. - Close the event loop. The loop must not be running. Pending - callbacks will be lost. +.. method:: loop.close() - This clears the queues and shuts down the executor, but does not wait for - the executor to finish. + Close the event loop. - This is idempotent and irreversible. No other methods should be called after - this one. + The loop must be running when this function is called. + Any pending callbacks will be discarded. + This method clears all queues and shuts down the executor, but does + not wait for the executor to finish. -.. coroutinemethod:: AbstractEventLoop.shutdown_asyncgens() + This method is idempotent and irreversible. No other methods + should be called after the event loop is closed. + +.. coroutinemethod:: loop.shutdown_asyncgens() Schedule all currently open :term:`asynchronous generator` objects to close with an :meth:`~agen.aclose()` call. After calling this method, - the event loop will issue a warning whenever a new asynchronous generator - is iterated. Should be used to finalize all scheduled asynchronous - generators reliably. Example:: + the event loop will issue a warning if a new asynchronous generator + is iterated. This should be used to reliably finalize all scheduled + asynchronous generators, e.g.:: + try: loop.run_forever() @@ -108,232 +164,223 @@ Run an event loop .. versionadded:: 3.6 -.. _asyncio-pass-keywords: - -Calls ------ - -Most :mod:`asyncio` functions don't accept keywords. If you want to pass -keywords to your callback, use :func:`functools.partial`. For example, -``loop.call_soon(functools.partial(print, "Hello", flush=True))`` will call -``print("Hello", flush=True)``. - -.. note:: - :func:`functools.partial` is better than ``lambda`` functions, because - :mod:`asyncio` can inspect :func:`functools.partial` object to display - parameters in debug mode, whereas ``lambda`` functions have a poor - representation. +Scheduling callbacks +^^^^^^^^^^^^^^^^^^^^ -.. method:: AbstractEventLoop.call_soon(callback, *args, context=None) +.. method:: loop.call_soon(callback, *args, context=None) - Arrange for a callback to be called as soon as possible. The callback is - called after :meth:`call_soon` returns, when control returns to the event - loop. + Schedule a *callback* to be called with *args* arguments at + the next iteration of the event loop. - This operates as a :abbr:`FIFO (first-in, first-out)` queue, callbacks - are called in the order in which they are registered. Each callback - will be called exactly once. + Callbacks are called in the order in which they are registered. + Each callback will be called exactly once. - Any positional arguments after the callback will be passed to the - callback when it is called. - - An optional keyword-only *context* argument allows specifying a custom - :class:`contextvars.Context` for the *callback* to run in. The current - context is used when no *context* is provided. + An optional keyword-only *context* argument allows specifying a + custom :class:`contextvars.Context` for the *callback* to run in. + The current context is used when no *context* is provided. An instance of :class:`asyncio.Handle` is returned, which can be - used to cancel the callback. - - :ref:`Use functools.partial to pass keywords to the callback - `. + used later to cancel the callback. - .. versionchanged:: 3.7 - The *context* keyword-only parameter was added. See :pep:`567` - for more details. + This method is not thread-safe. -.. method:: AbstractEventLoop.call_soon_threadsafe(callback, *args, context=None) +.. method:: loop.call_soon_threadsafe(callback, *args, context=None) - Like :meth:`call_soon`, but thread safe. + A thread-safe variant of :meth:`call_soon`. Must be used to + schedule callbacks *from another thread*. See the :ref:`concurrency and multithreading ` section of the documentation. - .. versionchanged:: 3.7 - The *context* keyword-only parameter was added. See :pep:`567` - for more details. +.. versionchanged:: 3.7 + The *context* keyword-only parameter was added. See :pep:`567` + for more details. +.. _asyncio-pass-keywords: -.. _asyncio-delayed-calls: +.. note:: -Delayed calls -------------- + Most :mod:`asyncio` scheduling functions don't allow passing + keyword arguments. To do that, use :func:`functools.partial`:: -The event loop has its own internal clock for computing timeouts. -Which clock is used depends on the (platform-specific) event loop -implementation; ideally it is a monotonic clock. This will generally be -a different clock than :func:`time.time`. + # will schedule "print("Hello", flush=True)" + loop.call_soon( + functools.partial(print, "Hello", flush=True)) -.. note:: + Using partial objects is usually more convenient than using lambdas, + as asyncio can render partial objects better in debug and error + messages. - Timeouts (relative *delay* or absolute *when*) should not exceed one day. +.. _asyncio-delayed-calls: + +Scheduling delayed callbacks +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. method:: AbstractEventLoop.call_later(delay, callback, *args, context=None) +Event loop provides mechanisms to schedule callback functions +to be called at some point in the future. Event loop uses monotonic +clocks to track time. - Arrange for the *callback* to be called after the given *delay* - seconds (either an int or float). - An instance of :class:`asyncio.TimerHandle` is returned, which can be - used to cancel the callback. +.. method:: loop.call_later(delay, callback, *args, context=None) - *callback* will be called exactly once per call to :meth:`call_later`. - If two callbacks are scheduled for exactly the same time, it is - undefined which will be called first. + Schedule *callback* to be called after the given *delay* + number of seconds (can be either an int or a float). - The optional positional *args* will be passed to the callback when it - is called. If you want the callback to be called with some named - arguments, use a closure or :func:`functools.partial`. + An instance of :class:`asyncio.TimerHandle` is returned which can + be used to cancel the callback. - An optional keyword-only *context* argument allows specifying a custom - :class:`contextvars.Context` for the *callback* to run in. The current - context is used when no *context* is provided. + *callback* will be called exactly once. If two callbacks are + scheduled for exactly the same time, the order in which they + are called is undefined. - :ref:`Use functools.partial to pass keywords to the callback - `. + The optional positional *args* will be passed to the callback when + it is called. If you want the callback to be called with keyword + arguments use :func:`functools.partial`. + + An optional keyword-only *context* argument allows specifying a + custom :class:`contextvars.Context` for the *callback* to run in. + The current context is used when no *context* is provided. .. versionchanged:: 3.7 The *context* keyword-only parameter was added. See :pep:`567` for more details. -.. method:: AbstractEventLoop.call_at(when, callback, *args, context=None) + .. versionchanged:: 3.7.1 + In Python 3.7.0 and earlier with the default event loop implementation, + the *delay* could not exceed one day. + This has been fixed in Python 3.7.1. - Arrange for the *callback* to be called at the given absolute timestamp - *when* (an int or float), using the same time reference as - :meth:`AbstractEventLoop.time`. +.. method:: loop.call_at(when, callback, *args, context=None) - This method's behavior is the same as :meth:`call_later`. + Schedule *callback* to be called at the given absolute timestamp + *when* (an int or a float), using the same time reference as + :meth:`loop.time`. - An instance of :class:`asyncio.TimerHandle` is returned, which can be - used to cancel the callback. + This method's behavior is the same as :meth:`call_later`. - :ref:`Use functools.partial to pass keywords to the callback - `. + An instance of :class:`asyncio.TimerHandle` is returned which can + be used to cancel the callback. .. versionchanged:: 3.7 The *context* keyword-only parameter was added. See :pep:`567` for more details. -.. method:: AbstractEventLoop.time() + .. versionchanged:: 3.7.1 + In Python 3.7.0 and earlier with the default event loop implementation, + the difference between *when* and the current time could not exceed + one day. This has been fixed in Python 3.7.1. - Return the current time, as a :class:`float` value, according to the - event loop's internal clock. +.. method:: loop.time() -.. seealso:: + Return the current time, as a :class:`float` value, according to + the event loop's internal monotonic clock. - The :func:`asyncio.sleep` function. +.. note:: + Timeouts (relative *delay* or absolute *when*) should not + exceed one day. -Futures -------- +.. seealso:: -.. method:: AbstractEventLoop.create_future() + The :func:`asyncio.sleep` function. - Create an :class:`asyncio.Future` object attached to the loop. - This is a preferred way to create futures in asyncio, as event - loop implementations can provide alternative implementations - of the Future class (with better performance or instrumentation). +Creating Futures and Tasks +^^^^^^^^^^^^^^^^^^^^^^^^^^ - .. versionadded:: 3.5.2 +.. method:: loop.create_future() + Create an :class:`asyncio.Future` object attached to the event loop. -Tasks ------ + This is the preferred way to create Futures in asyncio. This lets + third-party event loops provide alternative implementations of + the Future object (with better performance or instrumentation). -.. method:: AbstractEventLoop.create_task(coro) + .. versionadded:: 3.5.2 - Schedule the execution of a :ref:`coroutine object `: wrap it in - a future. Return a :class:`Task` object. +.. method:: loop.create_task(coro) - Third-party event loops can use their own subclass of :class:`Task` for - interoperability. In this case, the result type is a subclass of - :class:`Task`. + Schedule the execution of a :ref:`coroutine`. + Return a :class:`Task` object. - .. versionadded:: 3.4.2 + Third-party event loops can use their own subclass of :class:`Task` + for interoperability. In this case, the result type is a subclass + of :class:`Task`. -.. method:: AbstractEventLoop.set_task_factory(factory) +.. method:: loop.set_task_factory(factory) Set a task factory that will be used by - :meth:`AbstractEventLoop.create_task`. + :meth:`loop.create_task`. If *factory* is ``None`` the default task factory will be set. + Otherwise, *factory* must be a *callable* with the signature matching + ``(loop, coro)``, where *loop* is a reference to the active + event loop, and *coro* is a coroutine object. The callable + must return a :class:`asyncio.Future`-compatible object. - If *factory* is a *callable*, it should have a signature matching - ``(loop, coro)``, where *loop* will be a reference to the active - event loop, *coro* will be a coroutine object. The callable - must return an :class:`asyncio.Future` compatible object. +.. method:: loop.get_task_factory() - .. versionadded:: 3.4.4 + Return a task factory or ``None`` if the default one is in use. -.. method:: AbstractEventLoop.get_task_factory() - Return a task factory, or ``None`` if the default one is in use. +Opening network connections +^^^^^^^^^^^^^^^^^^^^^^^^^^^ - .. versionadded:: 3.4.4 +.. coroutinemethod:: loop.create_connection(protocol_factory, \ + host=None, port=None, \*, ssl=None, \ + family=0, proto=0, flags=0, sock=None, \ + local_addr=None, server_hostname=None, \ + ssl_handshake_timeout=None) + Open a streaming transport connection to a given + address specified by *host* and *port*. -Creating connections --------------------- + The socket family can be either :py:data:`~socket.AF_INET` or + :py:data:`~socket.AF_INET6` depending on *host* (or the *family* + argument, if provided). -.. coroutinemethod:: AbstractEventLoop.create_connection(protocol_factory, host=None, port=None, \*, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None) + The socket type will be :py:data:`~socket.SOCK_STREAM`. - Create a streaming transport connection to a given Internet *host* and - *port*: socket family :py:data:`~socket.AF_INET` or - :py:data:`~socket.AF_INET6` depending on *host* (or *family* if specified), - socket type :py:data:`~socket.SOCK_STREAM`. *protocol_factory* must be a - callable returning a :ref:`protocol ` instance. + *protocol_factory* must be a callable returning an + :ref:`asyncio protocol ` implementation. This method will try to establish the connection in the background. When successful, it returns a ``(transport, protocol)`` pair. The chronological synopsis of the underlying operation is as follows: - #. The connection is established, and a :ref:`transport ` - is created to represent it. + #. The connection is established and a :ref:`transport ` + is created for it. - #. *protocol_factory* is called without arguments and must return a - :ref:`protocol ` instance. + #. *protocol_factory* is called without arguments and is expected to + return a :ref:`protocol ` instance. - #. The protocol instance is tied to the transport, and its - :meth:`connection_made` method is called. + #. The protocol instance is coupled with the transport by calling its + :meth:`~BaseProtocol.connection_made` method. - #. The coroutine returns successfully with the ``(transport, protocol)`` - pair. + #. A ``(transport, protocol)`` tuple is returned on success. - The created transport is an implementation-dependent bidirectional stream. + The created transport is an implementation-dependent bidirectional + stream. - .. note:: - *protocol_factory* can be any kind of callable, not necessarily - a class. For example, if you want to use a pre-created - protocol instance, you can pass ``lambda: my_protocol``. - - Options that change how the connection is created: + Other arguments: * *ssl*: if given and not false, a SSL/TLS transport is created (by default a plain TCP transport is created). If *ssl* is a :class:`ssl.SSLContext` object, this context is used to create - the transport; if *ssl* is :const:`True`, a context with some - unspecified default settings is used. + the transport; if *ssl* is :const:`True`, a default context returned + from :func:`ssl.create_default_context` is used. .. seealso:: :ref:`SSL/TLS security considerations ` - * *server_hostname*, is only for use together with *ssl*, - and sets or overrides the hostname that the target server's certificate - will be matched against. By default the value of the *host* argument + * *server_hostname* sets or overrides the hostname that the target + server's certificate will be matched against. Should only be passed + if *ssl* is not ``None``. By default the value of the *host* argument is used. If *host* is empty, there is no default and you must pass a value for *server_hostname*. If *server_hostname* is an empty string, hostname matching is disabled (which is a serious security - risk, allowing for man-in-the-middle-attacks). + risk, allowing for potential man-in-the-middle attacks). * *family*, *proto*, *flags* are the optional address family, protocol and flags to be passed through to getaddrinfo() for *host* resolution. @@ -347,38 +394,51 @@ Creating connections * *local_addr*, if given, is a ``(local_host, local_port)`` tuple used to bind the socket to locally. The *local_host* and *local_port* - are looked up using getaddrinfo(), similarly to *host* and *port*. + are looked up using ``getaddrinfo()``, similarly to *host* and *port*. - * *ssl_handshake_timeout* is (for an SSL connection) the time in seconds - to wait for the SSL handshake to complete before aborting the connection. + * *ssl_handshake_timeout* is (for a TLS connection) the time in seconds + to wait for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). .. versionadded:: 3.7 The *ssl_handshake_timeout* parameter. + .. versionchanged:: 3.6 + + The socket option :py:data:`~socket.TCP_NODELAY` is set by default + for all TCP connections. + .. versionchanged:: 3.5 - On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. + Added support for SSL/TLS in :class:`ProactorEventLoop`. .. seealso:: - The :func:`open_connection` function can be used to get a pair of - (:class:`StreamReader`, :class:`StreamWriter`) instead of a protocol. + The :func:`open_connection` function is a high-level alternative + API. It returns a pair of (:class:`StreamReader`, :class:`StreamWriter`) + that can be used directly in async/await code. +.. coroutinemethod:: loop.create_datagram_endpoint(protocol_factory, \ + local_addr=None, remote_addr=None, \*, \ + family=0, proto=0, flags=0, \ + reuse_address=None, reuse_port=None, \ + allow_broadcast=None, sock=None) -.. coroutinemethod:: AbstractEventLoop.create_datagram_endpoint(protocol_factory, local_addr=None, remote_addr=None, \*, family=0, proto=0, flags=0, reuse_address=None, reuse_port=None, allow_broadcast=None, sock=None) + Create a datagram connection. - Create datagram connection: socket family :py:data:`~socket.AF_INET`, - :py:data:`~socket.AF_INET6` or :py:data:`~socket.AF_UNIX` depending on - *host* (or *family* if specified), socket type - :py:data:`~socket.SOCK_DGRAM`. *protocol_factory* must be a - callable returning a :ref:`protocol ` instance. + The socket family can be either :py:data:`~socket.AF_INET`, + :py:data:`~socket.AF_INET6`, or :py:data:`~socket.AF_UNIX`, + depending on *host* (or the *family* argument, if provided). - This method will try to establish the connection in the background. - When successful, it returns a ``(transport, protocol)`` pair. + The socket type will be :py:data:`~socket.SOCK_DGRAM`. + + *protocol_factory* must be a callable returning a + :ref:`protocol ` implementation. - Options changing how the connection is created: + A tuple of ``(transport, protocol)`` is returned on success. + + Other arguments: * *local_addr*, if given, is a ``(local_host, local_port)`` tuple used to bind the socket to locally. The *local_host* and *local_port* @@ -394,14 +454,14 @@ Creating connections corresponding :mod:`socket` module constants. * *reuse_address* tells the kernel to reuse a local socket in - TIME_WAIT state, without waiting for its natural timeout to + ``TIME_WAIT`` state, without waiting for its natural timeout to expire. If not specified will automatically be set to ``True`` on - UNIX. + Unix. * *reuse_port* tells the kernel to allow this endpoint to be bound to the same port as other existing endpoints are bound to, so long as they all set this flag when being created. This option is not supported on Windows - and some UNIX's. If the :py:data:`~socket.SO_REUSEPORT` constant is not + and some Unixes. If the :py:data:`~socket.SO_REUSEPORT` constant is not defined then this capability is unsupported. * *allow_broadcast* tells the kernel to allow this endpoint to send @@ -412,7 +472,7 @@ Creating connections transport. If specified, *local_addr* and *remote_addr* should be omitted (must be :const:`None`). - On Windows with :class:`ProactorEventLoop`, this method is not supported. + On Windows, with :class:`ProactorEventLoop`, this method is not supported. See :ref:`UDP echo client protocol ` and :ref:`UDP echo server protocol ` examples. @@ -421,23 +481,26 @@ Creating connections The *family*, *proto*, *flags*, *reuse_address*, *reuse_port, *allow_broadcast*, and *sock* parameters were added. -.. coroutinemethod:: AbstractEventLoop.create_unix_connection(protocol_factory, path=None, \*, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None) +.. coroutinemethod:: loop.create_unix_connection(protocol_factory, \ + path=None, \*, ssl=None, sock=None, \ + server_hostname=None, ssl_handshake_timeout=None) - Create UNIX connection: socket family :py:data:`~socket.AF_UNIX`, socket - type :py:data:`~socket.SOCK_STREAM`. The :py:data:`~socket.AF_UNIX` socket - family is used to communicate between processes on the same machine - efficiently. + Create a Unix connection. - This method will try to establish the connection in the background. - When successful, it returns a ``(transport, protocol)`` pair. + The socket family will be :py:data:`~socket.AF_UNIX`; socket + type will be :py:data:`~socket.SOCK_STREAM`. + + A tuple of ``(transport, protocol)`` is returned on success. - *path* is the name of a UNIX domain socket, and is required unless a *sock* - parameter is specified. Abstract UNIX sockets, :class:`str`, - :class:`bytes`, and :class:`~pathlib.Path` paths are supported. + *path* is the name of a Unix domain socket and is required, + unless a *sock* parameter is specified. Abstract Unix sockets, + :class:`str`, :class:`bytes`, and :class:`~pathlib.Path` paths are + supported. - See the :meth:`AbstractEventLoop.create_connection` method for parameters. + See the documentation of the :meth:`loop.create_connection` method + for information about arguments to this method. - Availability: UNIX. + Availability: Unix. .. versionadded:: 3.7 @@ -448,55 +511,68 @@ Creating connections The *path* parameter can now be a :term:`path-like object`. -Creating listening connections ------------------------------- +Creating network servers +^^^^^^^^^^^^^^^^^^^^^^^^ -.. coroutinemethod:: AbstractEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, ssl_handshake_timeout=None, start_serving=True) +.. coroutinemethod:: loop.create_server(protocol_factory, \ + host=None, port=None, \*, \ + family=socket.AF_UNSPEC, \ + flags=socket.AI_PASSIVE, \ + sock=None, backlog=100, ssl=None, \ + reuse_address=None, reuse_port=None, \ + ssl_handshake_timeout=None, start_serving=True) - Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) bound to - *host* and *port*. + Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) listening + on *port* of the *host* address. - Return a :class:`Server` object, its :attr:`~Server.sockets` attribute - contains created sockets. Use the :meth:`Server.close` method to stop the - server: close listening sockets. + Returns a :class:`Server` object. - Parameters: + Arguments: + + * *protocol_factory* must be a callable returning a + :ref:`protocol ` implementation. + + * The *host* parameter can be set to several types which determine where + the server would be listening: - * The *host* parameter can be a string, in that case the TCP server is - bound to *host* and *port*. The *host* parameter can also be a sequence - of strings and in that case the TCP server is bound to all hosts of the - sequence. If *host* is an empty string or ``None``, all interfaces are - assumed and a list of multiple sockets will be returned (most likely one - for IPv4 and another one for IPv6). + - If *host* is a string, the TCP server is bound to a single network + interface specified by *host*. + + - If *host* is a sequence of strings, the TCP server is bound to all + network interfaces specified by the sequence. + + - If *host* is an empty string or ``None``, all interfaces are + assumed and a list of multiple sockets will be returned (most likely + one for IPv4 and another one for IPv6). * *family* can be set to either :data:`socket.AF_INET` or - :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set - it will be determined from host (defaults to :data:`socket.AF_UNSPEC`). + :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. + If not set, the *family* will be determined from host name + (defaults to :data:`~socket.AF_UNSPEC`). * *flags* is a bitmask for :meth:`getaddrinfo`. * *sock* can optionally be specified in order to use a preexisting - socket object. If specified, *host* and *port* should be omitted (must be - :const:`None`). + socket object. If specified, *host* and *port* must not be specified. * *backlog* is the maximum number of queued connections passed to :meth:`~socket.socket.listen` (defaults to 100). - * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the - accepted connections. + * *ssl* can be set to an :class:`~ssl.SSLContext` instance to enable + TLS over the accepted connections. * *reuse_address* tells the kernel to reuse a local socket in - TIME_WAIT state, without waiting for its natural timeout to + ``TIME_WAIT`` state, without waiting for its natural timeout to expire. If not specified will automatically be set to ``True`` on - UNIX. + Unix. * *reuse_port* tells the kernel to allow this endpoint to be bound to the same port as other existing endpoints are bound to, so long as they all set this flag when being created. This option is not supported on Windows. - * *ssl_handshake_timeout* is (for an SSL server) the time in seconds to wait - for the SSL handshake to complete before aborting the connection. + * *ssl_handshake_timeout* is (for a TLS server) the time in seconds to wait + for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). * *start_serving* set to ``True`` (the default) causes the created server @@ -507,32 +583,44 @@ Creating listening connections .. versionadded:: 3.7 - *ssl_handshake_timeout* and *start_serving* parameters. + Added *ssl_handshake_timeout* and *start_serving* parameters. - .. versionchanged:: 3.5 + .. versionchanged:: 3.6 - On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. + The socket option :py:data:`~socket.TCP_NODELAY` is set by default + for all TCP connections. - .. seealso:: + .. versionchanged:: 3.5 - The function :func:`start_server` creates a (:class:`StreamReader`, - :class:`StreamWriter`) pair and calls back a function with this pair. + Added support for SSL/TLS in :class:`ProactorEventLoop`. .. versionchanged:: 3.5.1 - The *host* parameter can now be a sequence of strings. + The *host* parameter can be a sequence of strings. + .. seealso:: -.. coroutinemethod:: AbstractEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, start_serving=True) + The :func:`start_server` function is a higher-level alternative API + that returns a pair of :class:`StreamReader` and :class:`StreamWriter` + that can be used in an async/await code. - Similar to :meth:`AbstractEventLoop.create_server`, but specific to the - socket family :py:data:`~socket.AF_UNIX`. - *path* is the name of a UNIX domain socket, and is required unless a *sock* - parameter is specified. Abstract UNIX sockets, :class:`str`, - :class:`bytes`, and :class:`~pathlib.Path` paths are supported. +.. coroutinemethod:: loop.create_unix_server(protocol_factory, path=None, \ + \*, sock=None, backlog=100, ssl=None, \ + ssl_handshake_timeout=None, start_serving=True) - Availability: UNIX. + Similar to :meth:`loop.create_server` but works with the + :py:data:`~socket.AF_UNIX` socket family. + + *path* is the name of a Unix domain socket, and is required, + unless a *sock* argument is provided. Abstract Unix sockets, + :class:`str`, :class:`bytes`, and :class:`~pathlib.Path` paths + are supported. + + See the documentation of the :meth:`loop.create_server` method + for information about arguments to this method. + + Availability: Unix. .. versionadded:: 3.7 @@ -542,26 +630,30 @@ Creating listening connections The *path* parameter can now be a :class:`~pathlib.Path` object. -.. coroutinemethod:: BaseEventLoop.connect_accepted_socket(protocol_factory, sock, \*, ssl=None, ssl_handshake_timeout=None) +.. coroutinemethod:: loop.connect_accepted_socket(protocol_factory, \ + sock, \*, ssl=None, ssl_handshake_timeout=None) - Handle an accepted connection. + Wrap an already accepted connection into a transport/protocol pair. - This is used by servers that accept connections outside of - asyncio but that use asyncio to handle them. + This method can be used by servers that accept connections outside + of asyncio but that use asyncio to handle them. Parameters: - * *sock* is a preexisting socket object returned from an ``accept`` - call. + * *protocol_factory* must be a callable returning a + :ref:`protocol ` implementation. + + * *sock* is a preexisting socket object returned from + :meth:`socket.accept `. - * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the - accepted connections. + * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over + the accepted connections. * *ssl_handshake_timeout* is (for an SSL connection) the time in seconds to wait for the SSL handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). - When completed it returns a ``(transport, protocol)`` pair. + Returns a ``(transport, protocol)`` pair. .. versionadded:: 3.7 @@ -570,15 +662,14 @@ Creating listening connections .. versionadded:: 3.5.3 -File Transferring ------------------ +Transferring files +^^^^^^^^^^^^^^^^^^ -.. coroutinemethod:: AbstractEventLoop.sendfile(transport, file, \ - offset=0, count=None, \ - *, fallback=True) +.. coroutinemethod:: loop.sendfile(transport, file, \ + offset=0, count=None, *, fallback=True) - Send a *file* to *transport*, return the total number of bytes - which were sent. + Send a *file* over a *transport*. Return the total number of bytes + sent. The method uses high-performance :meth:`os.sendfile` if available. @@ -586,167 +677,163 @@ File Transferring *offset* tells from where to start reading the file. If specified, *count* is the total number of bytes to transmit as opposed to - sending the file until EOF is reached. File position is updated on - return or also in case of error in which case :meth:`file.tell() - ` can be used to figure out the number of bytes - which were sent. + sending the file until EOF is reached. File position is always updated, + even when this method raises an error, and + :meth:`file.tell() ` can be used to obtain the actual + number of bytes sent. *fallback* set to ``True`` makes asyncio to manually read and send - the file when the platform does not support the sendfile syscall + the file when the platform does not support the sendfile system call (e.g. Windows or SSL socket on Unix). Raise :exc:`SendfileNotAvailableError` if the system does not support - *sendfile* syscall and *fallback* is ``False``. + the *sendfile* syscall and *fallback* is ``False``. .. versionadded:: 3.7 TLS Upgrade ------------ +^^^^^^^^^^^ -.. coroutinemethod:: AbstractEventLoop.start_tls(transport, protocol, sslcontext, \*, server_side=False, server_hostname=None, ssl_handshake_timeout=None) +.. coroutinemethod:: loop.start_tls(transport, protocol, \ + sslcontext, \*, server_side=False, \ + server_hostname=None, ssl_handshake_timeout=None) - Upgrades an existing connection to TLS. + Upgrade an existing transport-based connection to TLS. - Returns a new transport instance, that the *protocol* must start using + Return a new transport instance, that the *protocol* must start using immediately after the *await*. The *transport* instance passed to the *start_tls* method should never be used again. Parameters: * *transport* and *protocol* instances that methods like - :meth:`~AbstractEventLoop.create_server` and - :meth:`~AbstractEventLoop.create_connection` return. + :meth:`~loop.create_server` and + :meth:`~loop.create_connection` return. * *sslcontext*: a configured instance of :class:`~ssl.SSLContext`. * *server_side* pass ``True`` when a server-side connection is being - upgraded (like the one created by :meth:`~AbstractEventLoop.create_server`). + upgraded (like the one created by :meth:`~loop.create_server`). * *server_hostname*: sets or overrides the host name that the target server's certificate will be matched against. - * *ssl_handshake_timeout* is (for an SSL connection) the time in seconds to - wait for the SSL handshake to complete before aborting the connection. + * *ssl_handshake_timeout* is (for a TLS connection) the time in seconds to + wait for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). .. versionadded:: 3.7 -Watch file descriptors ----------------------- - -On Windows with :class:`SelectorEventLoop`, only socket handles are supported -(ex: pipe file descriptors are not supported). - -On Windows with :class:`ProactorEventLoop`, these methods are not supported. +Watching file descriptors +^^^^^^^^^^^^^^^^^^^^^^^^^ -.. method:: AbstractEventLoop.add_reader(fd, callback, \*args) +.. method:: loop.add_reader(fd, callback, \*args) - Start watching the file descriptor for read availability and then call the - *callback* with specified arguments. + Start monitoring the *fd* file descriptor for read availability and + invoke *callback* with the specified arguments once *fd* is available for + reading. - :ref:`Use functools.partial to pass keywords to the callback - `. +.. method:: loop.remove_reader(fd) -.. method:: AbstractEventLoop.remove_reader(fd) + Stop monitoring the *fd* file descriptor for read availability. - Stop watching the file descriptor for read availability. +.. method:: loop.add_writer(fd, callback, \*args) -.. method:: AbstractEventLoop.add_writer(fd, callback, \*args) + Start monitoring the *fd* file descriptor for write availability and + invoke *callback* with the specified arguments once *fd* is available for + writing. - Start watching the file descriptor for write availability and then call the - *callback* with specified arguments. + Use :func:`functools.partial` :ref:`to pass keywords + ` to *func*. - :ref:`Use functools.partial to pass keywords to the callback - `. +.. method:: loop.remove_writer(fd) -.. method:: AbstractEventLoop.remove_writer(fd) + Stop monitoring the *fd* file descriptor for write availability. - Stop watching the file descriptor for write availability. +See also :ref:`Platform Support ` section +for some limitations of these methods. -The :ref:`watch a file descriptor for read events ` -example uses the low-level :meth:`AbstractEventLoop.add_reader` method to register -the file descriptor of a socket. +Working with socket objects directly +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Low-level socket operations ---------------------------- +In general, protocol implementations that use transport-based APIs +such as :meth:`loop.create_connection` and :meth:`loop.create_server` +are faster than implementations that work with sockets directly. +However, there are some use cases when performance is not critical, and +working with :class:`~socket.socket` objects directly is more +convenient. -.. coroutinemethod:: AbstractEventLoop.sock_recv(sock, nbytes) +.. coroutinemethod:: loop.sock_recv(sock, nbytes) - Receive data from the socket. Modeled after blocking - :meth:`socket.socket.recv` method. + Receive up to *nbytes* from *sock*. Asynchronous version of + :meth:`socket.recv() `. - The return value is a bytes object - representing the data received. The maximum amount of data to be received - at once is specified by *nbytes*. + Return the received data as a bytes object. - With :class:`SelectorEventLoop` event loop, the socket *sock* must be - non-blocking. + *sock* must be a non-blocking socket. .. versionchanged:: 3.7 - Even though the method was always documented as a coroutine - method, before Python 3.7 it returned a :class:`Future`. - Since Python 3.7, this is an ``async def`` method. + Even though this method was always documented as a coroutine + method, releases before Python 3.7 returned a :class:`Future`. + Since Python 3.7 this is an ``async def`` method. -.. coroutinemethod:: AbstractEventLoop.sock_recv_into(sock, buf) +.. coroutinemethod:: loop.sock_recv_into(sock, buf) - Receive data from the socket. Modeled after blocking - :meth:`socket.socket.recv_into` method. + Receive data from *sock* into the *buf* buffer. Modeled after the blocking + :meth:`socket.recv_into() ` method. - The received data is written into *buf* (a writable buffer). - The return value is the number of bytes written. + Return the number of bytes written to the buffer. - With :class:`SelectorEventLoop` event loop, the socket *sock* must be - non-blocking. + *sock* must be a non-blocking socket. .. versionadded:: 3.7 -.. coroutinemethod:: AbstractEventLoop.sock_sendall(sock, data) +.. coroutinemethod:: loop.sock_sendall(sock, data) - Send data to the socket. Modeled after blocking - :meth:`socket.socket.sendall` method. + Send *data* to the *sock* socket. Asynchronous version of + :meth:`socket.sendall() `. - The socket must be connected to a remote socket. - This method continues to send data from *data* until either all data has - been sent or an error occurs. ``None`` is returned on success. On error, - an exception is raised, and there is no way to determine how much data, if - any, was successfully processed by the receiving end of the connection. + This method continues to send to the socket until either all data + in *data* has been sent or an error occurs. ``None`` is returned + on success. On error, an exception is raised. Additionally, there is no way + to determine how much data, if any, was successfully processed by the + receiving end of the connection. - With :class:`SelectorEventLoop` event loop, the socket *sock* must be - non-blocking. + *sock* must be a non-blocking socket. .. versionchanged:: 3.7 Even though the method was always documented as a coroutine method, before Python 3.7 it returned an :class:`Future`. Since Python 3.7, this is an ``async def`` method. -.. coroutinemethod:: AbstractEventLoop.sock_connect(sock, address) +.. coroutinemethod:: loop.sock_connect(sock, address) - Connect to a remote socket at *address*. Modeled after - blocking :meth:`socket.socket.connect` method. + Connect *sock* to a remote socket at *address*. - With :class:`SelectorEventLoop` event loop, the socket *sock* must be - non-blocking. + Asynchronous version of :meth:`socket.connect() `. + + *sock* must be a non-blocking socket. .. versionchanged:: 3.5.2 ``address`` no longer needs to be resolved. ``sock_connect`` will try to check if the *address* is already resolved by calling :func:`socket.inet_pton`. If not, - :meth:`AbstractEventLoop.getaddrinfo` will be used to resolve the + :meth:`loop.getaddrinfo` will be used to resolve the *address*. .. seealso:: - :meth:`AbstractEventLoop.create_connection` + :meth:`loop.create_connection` and :func:`asyncio.open_connection() `. -.. coroutinemethod:: AbstractEventLoop.sock_accept(sock) +.. coroutinemethod:: loop.sock_accept(sock) - Accept a connection. Modeled after blocking - :meth:`socket.socket.accept`. + Accept a connection. Modeled after the blocking + :meth:`socket.accept() ` method. The socket must be bound to an address and listening for connections. The return value is a pair ``(conn, address)`` where *conn* @@ -754,7 +841,7 @@ Low-level socket operations and *address* is the address bound to the socket on the other end of the connection. - The socket *sock* must be non-blocking. + *sock* must be a non-blocking socket. .. versionchanged:: 3.7 Even though the method was always documented as a coroutine @@ -763,51 +850,51 @@ Low-level socket operations .. seealso:: - :meth:`AbstractEventLoop.create_server` and :func:`start_server`. + :meth:`loop.create_server` and :func:`start_server`. -.. coroutinemethod:: AbstractEventLoop.sock_sendfile(sock, file, \ - offset=0, count=None, \ - *, fallback=True) +.. coroutinemethod:: loop.sock_sendfile(sock, file, offset=0, count=None, \ + \*, fallback=True) - Send a file using high-performance :mod:`os.sendfile` if possible - and return the total number of bytes which were sent. + Send a file using high-performance :mod:`os.sendfile` if possible. + Return the total number of bytes sent. - Asynchronous version of :meth:`socket.socket.sendfile`. + Asynchronous version of :meth:`socket.sendfile() `. - *sock* must be non-blocking :class:`~socket.socket` of - :const:`socket.SOCK_STREAM` type. + *sock* must be a non-blocking :const:`socket.SOCK_STREAM` + :class:`~socket.socket`. - *file* must be a regular file object opened in binary mode. + *file* must be a regular file object open in binary mode. *offset* tells from where to start reading the file. If specified, *count* is the total number of bytes to transmit as opposed to - sending the file until EOF is reached. File position is updated on - return or also in case of error in which case :meth:`file.tell() - ` can be used to figure out the number of bytes - which were sent. + sending the file until EOF is reached. File position is always updated, + even when this method raises an error, and + :meth:`file.tell() ` can be used to obtain the actual + number of bytes sent. - *fallback* set to ``True`` makes asyncio to manually read and send + *fallback*, when set to ``True``, makes asyncio manually read and send the file when the platform does not support the sendfile syscall (e.g. Windows or SSL socket on Unix). Raise :exc:`SendfileNotAvailableError` if the system does not support *sendfile* syscall and *fallback* is ``False``. + *sock* must be a non-blocking socket. + .. versionadded:: 3.7 -Resolve host name ------------------ +DNS +^^^ -.. coroutinemethod:: AbstractEventLoop.getaddrinfo(host, port, \*, family=0, type=0, proto=0, flags=0) +.. coroutinemethod:: loop.getaddrinfo(host, port, \*, family=0, \ + type=0, proto=0, flags=0) - This method is a :ref:`coroutine `, similar to - :meth:`socket.getaddrinfo` function but non-blocking. + Asynchronous version of :meth:`socket.getaddrinfo`. -.. coroutinemethod:: AbstractEventLoop.getnameinfo(sockaddr, flags=0) +.. coroutinemethod:: loop.getnameinfo(sockaddr, flags=0) - This method is a :ref:`coroutine `, similar to - :meth:`socket.getnameinfo` function but non-blocking. + Asynchronous version of :meth:`socket.getnameinfo`. .. versionchanged:: 3.7 Both *getaddrinfo* and *getnameinfo* methods were always documented @@ -816,141 +903,157 @@ Resolve host name both methods are coroutines. -Connect pipes -------------- +Working with pipes +^^^^^^^^^^^^^^^^^^ -On Windows with :class:`SelectorEventLoop`, these methods are not supported. -Use :class:`ProactorEventLoop` to support pipes on Windows. +.. coroutinemethod:: loop.connect_read_pipe(protocol_factory, pipe) -.. coroutinemethod:: AbstractEventLoop.connect_read_pipe(protocol_factory, pipe) + Register the read end of *pipe* in the event loop. - Register read pipe in eventloop. + *protocol_factory* must be a callable returning an + :ref:`asyncio protocol ` implementation. - *protocol_factory* should instantiate object with :class:`Protocol` - interface. *pipe* is a :term:`file-like object `. - Return pair ``(transport, protocol)``, where *transport* supports the - :class:`ReadTransport` interface. + *pipe* is a :term:`file-like object `. + + Return pair ``(transport, protocol)``, where *transport* supports + the :class:`ReadTransport` interface and *protocol* is an object + instantiated by the *protocol_factory*. With :class:`SelectorEventLoop` event loop, the *pipe* is set to non-blocking mode. -.. coroutinemethod:: AbstractEventLoop.connect_write_pipe(protocol_factory, pipe) +.. coroutinemethod:: loop.connect_write_pipe(protocol_factory, pipe) + + Register the write end of *pipe* in the event loop. - Register write pipe in eventloop. + *protocol_factory* must be a callable returning an + :ref:`asyncio protocol ` implementation. + + *pipe* is :term:`file-like object `. - *protocol_factory* should instantiate object with :class:`BaseProtocol` - interface. *pipe* is :term:`file-like object `. Return pair ``(transport, protocol)``, where *transport* supports - :class:`WriteTransport` interface. + :class:`WriteTransport` interface and *protocol* is an object + instantiated by the *protocol_factory*. With :class:`SelectorEventLoop` event loop, the *pipe* is set to non-blocking mode. -.. seealso:: +.. note:: - The :meth:`AbstractEventLoop.subprocess_exec` and - :meth:`AbstractEventLoop.subprocess_shell` methods. + :class:`SelectorEventLoop` does not support the above methods on + Windows. Use :class:`ProactorEventLoop` instead for Windows. +.. seealso:: + + The :meth:`loop.subprocess_exec` and + :meth:`loop.subprocess_shell` methods. -UNIX signals ------------- -Availability: UNIX only. +Unix signals +^^^^^^^^^^^^ -.. method:: AbstractEventLoop.add_signal_handler(signum, callback, \*args) +.. method:: loop.add_signal_handler(signum, callback, \*args) - Add a handler for a signal. + Set *callback* as the handler for the *signum* signal. Raise :exc:`ValueError` if the signal number is invalid or uncatchable. Raise :exc:`RuntimeError` if there is a problem setting up the handler. - :ref:`Use functools.partial to pass keywords to the callback - `. + Use :func:`functools.partial` :ref:`to pass keywords + ` to *func*. + +.. method:: loop.remove_signal_handler(sig) -.. method:: AbstractEventLoop.remove_signal_handler(sig) + Remove the handler for the *sig* signal. - Remove a handler for a signal. + Return ``True`` if the signal handler was removed, or ``False`` if + no handler was set for the given signal. - Return ``True`` if a signal handler was removed, ``False`` if not. +Availability: Unix. .. seealso:: The :mod:`signal` module. -Executor --------- +Executing code in thread or process pools +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Call a function in an :class:`~concurrent.futures.Executor` (pool of threads or -pool of processes). By default, an event loop uses a thread pool executor -(:class:`~concurrent.futures.ThreadPoolExecutor`). +.. coroutinemethod:: loop.run_in_executor(executor, func, \*args) -.. method:: AbstractEventLoop.run_in_executor(executor, func, \*args) + Arrange for *func* to be called in the specified executor. - Arrange for a *func* to be called in the specified executor. - - The *executor* argument should be an :class:`~concurrent.futures.Executor` + The *executor* argument should be an :class:`concurrent.futures.Executor` instance. The default executor is used if *executor* is ``None``. - :ref:`Use functools.partial to pass keywords to the *func* - `. + Use :func:`functools.partial` :ref:`to pass keywords + ` to *func*. This method returns a :class:`asyncio.Future` object. .. versionchanged:: 3.5.3 - :meth:`BaseEventLoop.run_in_executor` no longer configures the + :meth:`loop.run_in_executor` no longer configures the ``max_workers`` of the thread pool executor it creates, instead leaving it up to the thread pool executor (:class:`~concurrent.futures.ThreadPoolExecutor`) to set the default. -.. method:: AbstractEventLoop.set_default_executor(executor) +.. method:: loop.set_default_executor(executor) + + Set *executor* as the default executor used by :meth:`run_in_executor`. + *executor* should be an instance of + :class:`~concurrent.futures.ThreadPoolExecutor`. - Set the default executor used by :meth:`run_in_executor`. + .. deprecated:: 3.7 + Using an executor that is not an instance of + :class:`~concurrent.futures.ThreadPoolExecutor` is deprecated and + will trigger an error in Python 3.9. + + *executor* must be an instance of + :class:`concurrent.futures.ThreadPoolExecutor`. Error Handling API ------------------- +^^^^^^^^^^^^^^^^^^ Allows customizing how exceptions are handled in the event loop. -.. method:: AbstractEventLoop.set_exception_handler(handler) +.. method:: loop.set_exception_handler(handler) Set *handler* as the new event loop exception handler. If *handler* is ``None``, the default exception handler will - be set. - - If *handler* is a callable object, it should have a - matching signature to ``(loop, context)``, where ``loop`` - will be a reference to the active event loop, ``context`` - will be a ``dict`` object (see :meth:`call_exception_handler` - documentation for details about context). + be set. Otherwise, *handler* must be a callable with the signature + matching ``(loop, context)``, where ``loop`` + is a reference to the active event loop, and ``context`` + is a ``dict`` object containing the details of the exception + (see :meth:`call_exception_handler` documentation for details + about context). -.. method:: AbstractEventLoop.get_exception_handler() +.. method:: loop.get_exception_handler() - Return the exception handler, or ``None`` if the default one - is in use. + Return the current exception handler, or ``None`` if no custom + exception handler was set. .. versionadded:: 3.5.2 -.. method:: AbstractEventLoop.default_exception_handler(context) +.. method:: loop.default_exception_handler(context) Default exception handler. This is called when an exception occurs and no exception - handler is set, and can be called by a custom exception - handler that wants to defer to the default behavior. + handler is set. This can be called by a custom exception + handler that wants to defer to the default handler behavior. *context* parameter has the same meaning as in :meth:`call_exception_handler`. -.. method:: AbstractEventLoop.call_exception_handler(context) +.. method:: loop.call_exception_handler(context) Call the current event loop exception handler. *context* is a ``dict`` object containing the following keys - (new keys may be introduced later): + (new keys may be introduced in future Python versions): * 'message': Error message; * 'exception' (optional): Exception object; @@ -962,14 +1065,14 @@ Allows customizing how exceptions are handled in the event loop. .. note:: - Note: this method should not be overloaded in subclassed - event loops. For any custom exception handling, use - :meth:`set_exception_handler()` method. + This method should not be overloaded in subclassed + event loops. For custom exception handling, use + the :meth:`set_exception_handler()` method. -Debug mode ----------- +Enabling debug mode +^^^^^^^^^^^^^^^^^^^ -.. method:: AbstractEventLoop.get_debug() +.. method:: loop.get_debug() Get the debug mode (:class:`bool`) of the event loop. @@ -977,29 +1080,173 @@ Debug mode :envvar:`PYTHONASYNCIODEBUG` is set to a non-empty string, ``False`` otherwise. - .. versionadded:: 3.4.2 - -.. method:: AbstractEventLoop.set_debug(enabled: bool) +.. method:: loop.set_debug(enabled: bool) Set the debug mode of the event loop. - .. versionadded:: 3.4.2 + .. versionchanged:: 3.7 + + The new ``-X dev`` command line option can now also be used + to enable the debug mode. .. seealso:: The :ref:`debug mode of asyncio `. -Server ------- -.. class:: Server +Running Subprocesses +^^^^^^^^^^^^^^^^^^^^ + +Methods described in this subsections are low-level. In regular +async/await code consider using the high-level +:func:`asyncio.create_subprocess_shell` and +:func:`asyncio.create_subprocess_exec` convenience functions instead. + +.. note:: + + The default asyncio event loop on **Windows** does not support + subprocesses. See :ref:`Subprocess Support on Windows + ` for details. + +.. coroutinemethod:: loop.subprocess_exec(protocol_factory, \*args, \ + stdin=subprocess.PIPE, stdout=subprocess.PIPE, \ + stderr=subprocess.PIPE, \*\*kwargs) + + Create a subprocess from one or more string arguments specified by + *args*. + + *args* must be a list of strings represented by: + + * :class:`str`; + * or :class:`bytes`, encoded to the + :ref:`filesystem encoding `. + + The first string specifies the program executable, + and the remaining strings specify the arguments. Together, string + arguments form the ``argv`` of the program. + + This is similar to the standard library :class:`subprocess.Popen` + class called with ``shell=False`` and the list of strings passed as + the first argument; however, where :class:`~subprocess.Popen` takes + a single argument which is list of strings, *subprocess_exec* + takes multiple string arguments. + + The *protocol_factory* must be a callable returning a subclass of the + :class:`asyncio.SubprocessProtocol` class. + + Other parameters: + + * *stdin*: either a file-like object representing a pipe to be + connected to the subprocess's standard input stream using + :meth:`~loop.connect_write_pipe`, or the + :const:`subprocess.PIPE` constant (default). By default a new + pipe will be created and connected. + + * *stdout*: either a file-like object representing the pipe to be + connected to the subprocess's standard output stream using + :meth:`~loop.connect_read_pipe`, or the + :const:`subprocess.PIPE` constant (default). By default a new pipe + will be created and connected. + + * *stderr*: either a file-like object representing the pipe to be + connected to the subprocess's standard error stream using + :meth:`~loop.connect_read_pipe`, or one of + :const:`subprocess.PIPE` (default) or :const:`subprocess.STDOUT` + constants. + + By default a new pipe will be created and connected. When + :const:`subprocess.STDOUT` is specified, the subprocess' standard + error stream will be connected to the same pipe as the standard + output stream. + + * All other keyword arguments are passed to :class:`subprocess.Popen` + without interpretation, except for *bufsize*, *universal_newlines* + and *shell*, which should not be specified at all. + + See the constructor of the :class:`subprocess.Popen` class + for documentation on other arguments. + + Returns a pair of ``(transport, protocol)``, where *transport* + conforms to the :class:`asyncio.SubprocessTransport` base class and + *protocol* is an object instantiated by the *protocol_factory*. + +.. coroutinemethod:: loop.subprocess_shell(protocol_factory, cmd, \*, \ + stdin=subprocess.PIPE, stdout=subprocess.PIPE, \ + stderr=subprocess.PIPE, \*\*kwargs) + + Create a subprocess from *cmd*, which can be a :class:`str` or a + :class:`bytes` string encoded to the + :ref:`filesystem encoding `, + using the platform's "shell" syntax. + + This is similar to the standard library :class:`subprocess.Popen` + class called with ``shell=True``. + + The *protocol_factory* must be a callable returning a subclass of the + :class:`SubprocessProtocol` class. + + See :meth:`~loop.subprocess_exec` for more details about + the remaining arguments. + + Returns a pair of ``(transport, protocol)``, where *transport* + conforms to the :class:`SubprocessTransport` base class and + *protocol* is an object instantiated by the *protocol_factory*. + +.. note:: + It is the application's responsibility to ensure that all whitespace + and special characters are quoted appropriately to avoid `shell injection + `_ + vulnerabilities. The :func:`shlex.quote` function can be used to + properly escape whitespace and special characters in strings that + are going to be used to construct shell commands. + + +Callback Handles +================ + +.. class:: Handle + + A callback wrapper object returned by :meth:`loop.call_soon`, + :meth:`loop.call_soon_threadsafe`. + + .. method:: cancel() + + Cancel the callback. If the callback has already been canceled + or executed, this method has no effect. + + .. method:: cancelled() + + Return ``True`` if the callback was cancelled. + + .. versionadded:: 3.7 + +.. class:: TimerHandle + + A callback wrapper object returned by :meth:`loop.call_later`, + and :meth:`loop.call_at`. + + This class is a subclass of :class:`Handle`. + + .. method:: when() + + Return a scheduled callback time as :class:`float` seconds. + + The time is an absolute timestamp, using the same time + reference as :meth:`loop.time`. + + .. versionadded:: 3.7 - Server listening on sockets. - Object created by :meth:`AbstractEventLoop.create_server`, - :meth:`AbstractEventLoop.create_unix_server`, :func:`start_server`, - and :func:`start_unix_server` functions. Don't instantiate the class - directly. +Server Objects +============== + +Server objects are created by :meth:`loop.create_server`, +:meth:`loop.create_unix_server`, :func:`start_server`, +and :func:`start_unix_server` functions. + +Do not instantiate the class directly. + +.. class:: Server *Server* objects are asynchronous context managers. When used in an ``async with`` statement, it's guaranteed that the Server object is @@ -1022,15 +1269,15 @@ Server Stop serving: close listening sockets and set the :attr:`sockets` attribute to ``None``. - The sockets that represent existing incoming client connections are left - open. + The sockets that represent existing incoming client connections + are left open. The server is closed asynchronously, use the :meth:`wait_closed` coroutine to wait until the server is closed. .. method:: get_loop() - Gives the event loop associated with the server object. + Return the event loop associated with the server object. .. versionadded:: 3.7 @@ -1041,12 +1288,12 @@ Server This method is idempotent, so it can be called when the server is already being serving. - The new *start_serving* keyword-only parameter to - :meth:`AbstractEventLoop.create_server` and - :meth:`asyncio.start_server` allows to create a Server object - that is not accepting connections right away. In which case - this method, or :meth:`Server.serve_forever` can be used - to make the Server object to start accepting connections. + The *start_serving* keyword-only parameter to + :meth:`loop.create_server` and + :meth:`asyncio.start_server` allows creating a Server object + that is not accepting connections initially. In this case + ``Server.start_serving()``, or :meth:`Server.serve_forever` can be used + to make the Server start accepting connections. .. versionadded:: 3.7 @@ -1088,78 +1335,99 @@ Server .. attribute:: sockets - List of :class:`socket.socket` objects the server is listening to, or - ``None`` if the server is closed. + List of :class:`socket.socket` objects the server is listening on, + or ``None`` if the server is closed. .. versionchanged:: 3.7 - Prior to Python 3.7 ``Server.sockets`` used to return the - internal list of server's sockets directly. In 3.7 a copy + Prior to Python 3.7 ``Server.sockets`` used to return an + internal list of server sockets directly. In 3.7 a copy of that list is returned. -Handle ------- +.. _asyncio-event-loops: -.. class:: Handle +Event Loop Implementations +========================== - A callback wrapper object returned by :func:`AbstractEventLoop.call_soon`, - :func:`AbstractEventLoop.call_soon_threadsafe`. +asyncio ships with two different event loop implementations: +:class:`SelectorEventLoop` and :class:`ProactorEventLoop`. - .. method:: cancel() +By default asyncio is configured to use :class:`SelectorEventLoop` +on all platforms. - Cancel the call. If the callback is already canceled or executed, - this method has no effect. - .. method:: cancelled() +.. class:: SelectorEventLoop - Return ``True`` if the call was cancelled. + An event loop based on the :mod:`selectors` module. - .. versionadded:: 3.7 + Uses the most efficient *selector* available for the given + platform. It is also possible to manually configure the + exact selector implementation to be used:: -.. class:: TimerHandle + import asyncio + import selectors - A callback wrapper object returned by :func:`AbstractEventLoop.call_later`, - and :func:`AbstractEventLoop.call_at`. + selector = selectors.SelectSelector() + loop = asyncio.SelectorEventLoop(selector) + asyncio.set_event_loop(loop) - The class is inherited from :class:`Handle`. - .. method:: when() + Availability: Unix, Windows. - Return a scheduled callback time as :class:`float` seconds. - The time is an absolute timestamp, using the same time - reference as :meth:`AbstractEventLoop.time`. +.. class:: ProactorEventLoop - .. versionadded:: 3.7 + An event loop for Windows that uses "I/O Completion Ports" (IOCP). + + Availability: Windows. + An example how to use :class:`ProactorEventLoop` on Windows:: -SendfileNotAvailableError -------------------------- + import asyncio + import sys + if sys.platform == 'win32': + loop = asyncio.ProactorEventLoop() + asyncio.set_event_loop(loop) + + .. seealso:: + + `MSDN documentation on I/O Completion Ports + `_. + + +.. class:: AbstractEventLoop -.. exception:: SendfileNotAvailableError + Abstract base class for asyncio-compliant event loops. - Sendfile syscall is not available, subclass of :exc:`RuntimeError`. + The :ref:`Event Loop Methods ` section lists all + methods that an alternative implementation of ``AbstractEventLoop`` + should have defined. - Raised if the OS does not support sendfile syscall for - given socket or file type. +Examples +======== -Event loop examples -------------------- +Note that all examples in this section **purposefully** show how +to use the low-level event loop APIs, such as :meth:`loop.run_forever` +and :meth:`loop.call_soon`. Modern asyncio applications rarely +need to be written this way; consider using the high-level functions +like :func:`asyncio.run`. -.. _asyncio-hello-world-callback: + +.. _asyncio_example_lowlevel_helloworld: Hello World with call_soon() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Example using the :meth:`AbstractEventLoop.call_soon` method to schedule a -callback. The callback displays ``"Hello World"`` and then stops the event -loop:: +An example using the :meth:`loop.call_soon` method to schedule a +callback. The callback displays ``"Hello World"`` and then stops the +event loop:: import asyncio def hello_world(loop): + """A callback to print 'Hello World' and stop the event loop""" print('Hello World') loop.stop() @@ -1169,23 +1437,25 @@ loop:: loop.call_soon(hello_world, loop) # Blocking call interrupted by loop.stop() - loop.run_forever() - loop.close() + try: + loop.run_forever() + finally: + loop.close() .. seealso:: - The :ref:`Hello World coroutine ` example - uses a :ref:`coroutine `. + A similar :ref:`Hello World ` + example created with a coroutine and the :func:`run` function. -.. _asyncio-date-callback: +.. _asyncio_example_call_later: Display the current date with call_later() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Example of callback displaying the current date every second. The callback uses -the :meth:`AbstractEventLoop.call_later` method to reschedule itself during 5 -seconds, and then stops the event loop:: +An example of a callback displaying the current date every second. The +callback uses the :meth:`loop.call_later` method to reschedule itself +after 5 seconds, and then stops the event loop:: import asyncio import datetime @@ -1204,36 +1474,40 @@ seconds, and then stops the event loop:: loop.call_soon(display_date, end_time, loop) # Blocking call interrupted by loop.stop() - loop.run_forever() - loop.close() + try: + loop.run_forever() + finally: + loop.close() .. seealso:: - The :ref:`coroutine displaying the current date - ` example uses a :ref:`coroutine - `. + A similar :ref:`current date ` example + created with a coroutine and the :func:`run` function. -.. _asyncio-watch-read-event: +.. _asyncio_example_watch_fd: Watch a file descriptor for read events ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Wait until a file descriptor received some data using the -:meth:`AbstractEventLoop.add_reader` method and then close the event loop:: +:meth:`loop.add_reader` method and then close the event loop:: import asyncio from socket import socketpair # Create a pair of connected file descriptors rsock, wsock = socketpair() + loop = asyncio.get_event_loop() def reader(): data = rsock.recv(100) print("Received:", data.decode()) + # We are done: unregister the file descriptor loop.remove_reader(rsock) + # Stop the event loop loop.stop() @@ -1243,30 +1517,35 @@ Wait until a file descriptor received some data using the # Simulate the reception of data from the network loop.call_soon(wsock.send, 'abc'.encode()) - # Run the event loop - loop.run_forever() - - # We are done, close sockets and the event loop - rsock.close() - wsock.close() - loop.close() + try: + # Run the event loop + loop.run_forever() + finally: + # We are done. Close sockets and the event loop. + rsock.close() + wsock.close() + loop.close() .. seealso:: - The :ref:`register an open socket to wait for data using a protocol - ` example uses a low-level protocol created by the - :meth:`AbstractEventLoop.create_connection` method. + * A similar :ref:`example ` + using transports, protocols, and the + :meth:`loop.create_connection` method. - The :ref:`register an open socket to wait for data using streams - ` example uses high-level streams - created by the :func:`open_connection` function in a coroutine. + * Another similar :ref:`example ` + using the high-level :func:`asyncio.open_connection` function + and streams. +.. _asyncio_example_unix_signals: + Set signal handlers for SIGINT and SIGTERM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM` using -the :meth:`AbstractEventLoop.add_signal_handler` method:: +(This ``signals`` example only works on Unix.) + +Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM` +using the :meth:`loop.add_signal_handler` method:: import asyncio import functools @@ -1277,16 +1556,17 @@ the :meth:`AbstractEventLoop.add_signal_handler` method:: print("got signal %s: exit" % signame) loop.stop() - loop = asyncio.get_event_loop() - for signame in ('SIGINT', 'SIGTERM'): - loop.add_signal_handler(getattr(signal, signame), - functools.partial(ask_exit, signame)) + async def main(): + loop = asyncio.get_running_loop() - print("Event loop running forever, press Ctrl+C to interrupt.") - print("pid %s: send SIGINT or SIGTERM to exit." % os.getpid()) - try: - loop.run_forever() - finally: - loop.close() + for signame in {'SIGINT', 'SIGTERM'}: + loop.add_signal_handler( + getattr(signal, signame), + functools.partial(ask_exit, signame)) + + await asyncio.sleep(3600) + + print("Event loop running for 1 hour, press Ctrl+C to interrupt.") + print(f"pid {os.getpid()}: send SIGINT or SIGTERM to exit.") -This example only works on UNIX. + asyncio.run(main()) diff --git a/Doc/library/asyncio-eventloops.rst b/Doc/library/asyncio-eventloops.rst deleted file mode 100644 index 7f6e9535a8a5..000000000000 --- a/Doc/library/asyncio-eventloops.rst +++ /dev/null @@ -1,244 +0,0 @@ -.. currentmodule:: asyncio - -Event loops -=========== - -**Source code:** :source:`Lib/asyncio/events.py` - -Event loop functions --------------------- - -The following functions are convenient shortcuts to accessing the methods of the -global policy. Note that this provides access to the default policy, unless an -alternative policy was set by calling :func:`set_event_loop_policy` earlier in -the execution of the process. - -.. function:: get_event_loop() - - Equivalent to calling ``get_event_loop_policy().get_event_loop()``. - -.. function:: set_event_loop(loop) - - Equivalent to calling ``get_event_loop_policy().set_event_loop(loop)``. - -.. function:: new_event_loop() - - Equivalent to calling ``get_event_loop_policy().new_event_loop()``. - -.. function:: get_running_loop() - - Return the running event loop in the current OS thread. If there - is no running event loop a :exc:`RuntimeError` is raised. - - .. versionadded:: 3.7 - - -.. _asyncio-event-loops: - -Available event loops ---------------------- - -asyncio currently provides two implementations of event loops: -:class:`SelectorEventLoop` and :class:`ProactorEventLoop`. - -.. class:: SelectorEventLoop - - Event loop based on the :mod:`selectors` module. Subclass of - :class:`AbstractEventLoop`. - - Use the most efficient selector available on the platform. - - On Windows, only sockets are supported (ex: pipes are not supported): - see the `MSDN documentation of select - `_. - -.. class:: ProactorEventLoop - - Proactor event loop for Windows using "I/O Completion Ports" aka IOCP. - Subclass of :class:`AbstractEventLoop`. - - Availability: Windows. - - .. seealso:: - - `MSDN documentation on I/O Completion Ports - `_. - -Example to use a :class:`ProactorEventLoop` on Windows:: - - import asyncio, sys - - if sys.platform == 'win32': - loop = asyncio.ProactorEventLoop() - asyncio.set_event_loop(loop) - -.. _asyncio-platform-support: - -Platform support ----------------- - -The :mod:`asyncio` module has been designed to be portable, but each platform -still has subtle differences and may not support all :mod:`asyncio` features. - -Windows -^^^^^^^ - -Common limits of Windows event loops: - -- :meth:`~AbstractEventLoop.create_unix_connection` and - :meth:`~AbstractEventLoop.create_unix_server` are not supported: the socket - family :data:`socket.AF_UNIX` is specific to UNIX -- :meth:`~AbstractEventLoop.add_signal_handler` and - :meth:`~AbstractEventLoop.remove_signal_handler` are not supported -- :meth:`EventLoopPolicy.set_child_watcher` is not supported. - :class:`ProactorEventLoop` supports subprocesses. It has only one - implementation to watch child processes, there is no need to configure it. - -:class:`SelectorEventLoop` specific limits: - -- :class:`~selectors.SelectSelector` is used which only supports sockets - and is limited to 512 sockets. -- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` only - accept file descriptors of sockets -- Pipes are not supported - (ex: :meth:`~AbstractEventLoop.connect_read_pipe`, - :meth:`~AbstractEventLoop.connect_write_pipe`) -- :ref:`Subprocesses ` are not supported - (ex: :meth:`~AbstractEventLoop.subprocess_exec`, - :meth:`~AbstractEventLoop.subprocess_shell`) - -:class:`ProactorEventLoop` specific limits: - -- :meth:`~AbstractEventLoop.create_datagram_endpoint` (UDP) is not supported -- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` are - not supported - -The resolution of the monotonic clock on Windows is usually around 15.6 msec. -The best resolution is 0.5 msec. The resolution depends on the hardware -(availability of `HPET -`_) and on the Windows -configuration. See :ref:`asyncio delayed calls `. - -.. versionchanged:: 3.5 - - :class:`ProactorEventLoop` now supports SSL. - - -Mac OS X -^^^^^^^^ - -Character devices like PTY are only well supported since Mavericks (Mac OS -10.9). They are not supported at all on Mac OS 10.5 and older. - -On Mac OS 10.6, 10.7 and 10.8, the default event loop is -:class:`SelectorEventLoop` which uses :class:`selectors.KqueueSelector`. -:class:`selectors.KqueueSelector` does not support character devices on these -versions. The :class:`SelectorEventLoop` can be used with -:class:`~selectors.SelectSelector` or :class:`~selectors.PollSelector` to -support character devices on these versions of Mac OS X. Example:: - - import asyncio - import selectors - - selector = selectors.SelectSelector() - loop = asyncio.SelectorEventLoop(selector) - asyncio.set_event_loop(loop) - - -Event loop policies and the default policy ------------------------------------------- - -Event loop management is abstracted with a *policy* pattern, to provide maximal -flexibility for custom platforms and frameworks. Throughout the execution of a -process, a single global policy object manages the event loops available to the -process based on the calling context. A policy is an object implementing the -:class:`AbstractEventLoopPolicy` interface. - -For most users of :mod:`asyncio`, policies never have to be dealt with -explicitly, since the default global policy is sufficient (see below). - -The module-level functions -:func:`get_event_loop` and :func:`set_event_loop` provide convenient access to -event loops managed by the default policy. - - -Event loop policy interface ---------------------------- - -An event loop policy must implement the following interface: - -.. class:: AbstractEventLoopPolicy - - Event loop policy. - - .. method:: get_event_loop() - - Get the event loop for the current context. - - Returns an event loop object implementing the :class:`AbstractEventLoop` - interface. In case called from coroutine, it returns the currently - running event loop. - - Raises an exception in case no event loop has been set for the current - context and the current policy does not specify to create one. It must - never return ``None``. - - .. versionchanged:: 3.6 - - .. method:: set_event_loop(loop) - - Set the event loop for the current context to *loop*. - - .. method:: new_event_loop() - - Create and return a new event loop object according to this policy's - rules. - - If there's need to set this loop as the event loop for the current - context, :meth:`set_event_loop` must be called explicitly. - - -The default policy defines context as the current thread, and manages an event -loop per thread that interacts with :mod:`asyncio`. An exception to this rule -happens when :meth:`~AbstractEventLoopPolicy.get_event_loop` is called from a -running future/coroutine, in which case it will return the current loop -running that future/coroutine. - -If the current thread doesn't already have an event loop associated with it, -the default policy's :meth:`~AbstractEventLoopPolicy.get_event_loop` method -creates one when called from the main thread, but raises :exc:`RuntimeError` -otherwise. - - -Access to the global loop policy --------------------------------- - -.. function:: get_event_loop_policy() - - Get the current event loop policy. - -.. function:: set_event_loop_policy(policy) - - Set the current event loop policy. If *policy* is ``None``, the default - policy is restored. - - -Customizing the event loop policy ---------------------------------- - -To implement a new event loop policy, it is recommended you subclass the -concrete default event loop policy :class:`DefaultEventLoopPolicy` -and override the methods for which you want to change behavior, for example:: - - class MyEventLoopPolicy(asyncio.DefaultEventLoopPolicy): - - def get_event_loop(self): - """Get the event loop. - - This may be None or an instance of EventLoop. - """ - loop = super().get_event_loop() - # Do something with loop ... - return loop - - asyncio.set_event_loop_policy(MyEventLoopPolicy()) diff --git a/Doc/library/asyncio-exceptions.rst b/Doc/library/asyncio-exceptions.rst new file mode 100644 index 000000000000..dbd5df720850 --- /dev/null +++ b/Doc/library/asyncio-exceptions.rst @@ -0,0 +1,91 @@ +.. currentmodule:: asyncio + + +.. _asyncio-exceptions: + +========== +Exceptions +========== + + +.. exception:: TimeoutError + + The operation has exceeded the given deadline. + + .. important:: + This exception is different from the builtin :exc:`TimeoutError` + exception. + + +.. exception:: CancelledError + + The operation has been cancelled. + + This exception can be caught to perform custom operations + when asyncio Tasks are cancelled. In almost all situations the + exception must be re-raised. + + .. important:: + + This exception is a subclass of :exc:`Exception`, so it can be + accidentally suppressed by an overly broad ``try..except`` block:: + + try: + await operation + except Exception: + # The cancellation is broken because the *except* block + # suppresses the CancelledError exception. + log.log('an error has occurred') + + Instead, the following pattern should be used:: + + try: + await operation + except asyncio.CancelledError: + raise + except Exception: + log.log('an error has occurred') + + +.. exception:: InvalidStateError + + Invalid internal state of :class:`Task` or :class:`Future`. + + Can be raised in situations like setting a result value for a + *Future* object that already has a result value set. + + +.. exception:: SendfileNotAvailableError + + The "sendfile" syscall is not available for the given + socket or file type. + + A subclass of :exc:`RuntimeError`. + + +.. exception:: IncompleteReadError + + The requested read operation did not complete fully. + + Raised by the :ref:`asyncio stream APIs`. + + This exception is a subclass of :exc:`EOFError`. + + .. attribute:: expected + + The total number (:class:`int`) of expected bytes. + + .. attribute:: partial + + A string of :class:`bytes` read before the end of stream was reached. + + +.. exception:: LimitOverrunError + + Reached the buffer size limit while looking for a separator. + + Raised by the :ref:`asyncio stream APIs `. + + .. attribute:: consumed + + The total number of to be consumed bytes. diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst new file mode 100644 index 000000000000..d6c5335c0e18 --- /dev/null +++ b/Doc/library/asyncio-future.rst @@ -0,0 +1,250 @@ +.. currentmodule:: asyncio + + +.. _asyncio-futures: + +======= +Futures +======= + +*Future* objects are used to bridge low-level callback-based code +with high-level async/await code. + + +Future Functions +================ + +.. function:: isfuture(obj) + + Return ``True`` if *obj* is either of: + + * an instance of :class:`asyncio.Future`, + * an instance of :class:`asyncio.Task`, + * a Future-like object with a ``_asyncio_future_blocking`` + attribute. + + .. versionadded:: 3.5 + + +.. function:: ensure_future(obj, \*, loop=None) + + Return: + + * *obj* argument as is, if *obj* is a :class:`Future`, + a :class:`Task`, or a Future-like object (:func:`isfuture` + is used for the test.) + + * a :class:`Task` object wrapping *obj*, if *obj* is a + coroutine (:func:`iscoroutine` is used for the test.) + + * a :class:`Task` object that would await on *obj*, if *obj* is an + awaitable (:func:`inspect.isawaitable` is used for the test.) + + If *obj* is neither of the above a :exc:`TypeError` is raised. + + .. important:: + + See also the :func:`create_task` function which is the + preferred way for creating new Tasks. + + .. versionchanged:: 3.5.1 + The function accepts any :term:`awaitable` object. + + +.. function:: wrap_future(future, \*, loop=None) + + Wrap a :class:`concurrent.futures.Future` object in a + :class:`asyncio.Future` object. + + +Future Object +============= + +.. class:: Future(\*, loop=None) + + A Future represents an eventual result of an asynchronous + operation. Not thread-safe. + + Future is an :term:`awaitable` object. Coroutines can await on + Future objects until they either have a result or an exception + set, or until they are cancelled. + + Typically Futures are used to enable low-level + callback-based code (e.g. in protocols implemented using asyncio + :ref:`transports `) + to interoperate with high-level async/await code. + + The rule of thumb is to never expose Future objects in user-facing + APIs, and the recommended way to create a Future object is to call + :meth:`loop.create_future`. This way alternative event loop + implementations can inject their own optimized implementations + of a Future object. + + .. versionchanged:: 3.7 + Added support for the :mod:`contextvars` module. + + .. method:: result() + + Return the result of the Future. + + If the Future is *done* and has a result set by the + :meth:`set_result` method, the result value is returned. + + If the Future is *done* and has an exception set by the + :meth:`set_exception` method, this method raises the exception. + + If the Future has been *cancelled*, this method raises + a :exc:`CancelledError` exception. + + If the Future's result isn't yet available, this method raises + a :exc:`InvalidStateError` exception. + + .. method:: set_result(result) + + Mark the Future as *done* and set its result. + + Raises a :exc:`InvalidStateError` error if the Future is + already *done*. + + .. method:: set_exception(exception) + + Mark the Future as *done* and set an exception. + + Raises a :exc:`InvalidStateError` error if the Future is + already *done*. + + .. method:: done() + + Return ``True`` if the Future is *done*. + + A Future is *done* if it was *cancelled* or if it has a result + or an exception set with :meth:`set_result` or + :meth:`set_exception` calls. + + .. method:: cancelled() + + Return ``True`` if the Future was *cancelled*. + + The method is usually used to check if a Future is not + *cancelled* before setting a result or an exception for it:: + + if not fut.cancelled(): + fut.set_result(42) + + .. method:: add_done_callback(callback, *, context=None) + + Add a callback to be run when the Future is *done*. + + The *callback* is called with the Future object as its only + argument. + + If the Future is already *done* when this method is called, + the callback is scheduled with :meth:`loop.call_soon`. + + An optional keyword-only *context* argument allows specifying a + custom :class:`contextvars.Context` for the *callback* to run in. + The current context is used when no *context* is provided. + + :func:`functools.partial` can be used to pass parameters + to the callback, e.g.:: + + # Call 'print("Future:", fut)' when "fut" is done. + fut.add_done_callback( + functools.partial(print, "Future:")) + + .. versionchanged:: 3.7 + The *context* keyword-only parameter was added. + See :pep:`567` for more details. + + .. method:: remove_done_callback(callback) + + Remove *callback* from the callbacks list. + + Returns the number of callbacks removed, which is typically 1, + unless a callback was added more than once. + + .. method:: cancel() + + Cancel the Future and schedule callbacks. + + If the Future is already *done* or *cancelled*, return ``False``. + Otherwise, change the Future's state to *cancelled*, + schedule the callbacks, and return ``True``. + + .. method:: exception() + + Return the exception that was set on this Future. + + The exception (or ``None`` if no exception was set) is + returned only if the Future is *done*. + + If the Future has been *cancelled*, this method raises a + :exc:`CancelledError` exception. + + If the Future isn't *done* yet, this method raises an + :exc:`InvalidStateError` exception. + + .. method:: get_loop() + + Return the event loop the Future object is bound to. + + .. versionadded:: 3.7 + + +.. _asyncio_example_future: + +This example creates a Future object, creates and schedules an +asynchronous Task to set result for the Future, and waits until +the Future has a result:: + + async def set_after(fut, delay, value): + # Sleep for *delay* seconds. + await asyncio.sleep(delay) + + # Set *value* as a result of *fut* Future. + fut.set_result(value) + + async def main(): + # Get the current event loop. + loop = asyncio.get_running_loop() + + # Create a new Future object. + fut = loop.create_future() + + # Run "set_after()" coroutine in a parallel Task. + # We are using the low-level "loop.create_task()" API here because + # we already have a reference to the event loop at hand. + # Otherwise we could have just used "asyncio.create_task()". + loop.create_task( + set_after(fut, 1, '... world')) + + print('hello ...') + + # Wait until *fut* has a result (1 second) and print it. + print(await fut) + + asyncio.run(main()) + + +.. important:: + + The Future object was designed to mimic + :class:`concurrent.futures.Future`. Key differences include: + + - unlike asyncio Futures, :class:`concurrent.futures.Future` + instances cannot be awaited. + + - :meth:`asyncio.Future.result` and :meth:`asyncio.Future.exception` + do not accept the *timeout* argument. + + - :meth:`asyncio.Future.result` and :meth:`asyncio.Future.exception` + raise an :exc:`InvalidStateError` exception when the Future is not + *done*. + + - Callbacks registered with :meth:`asyncio.Future.add_done_callback` + are not called immediately. They are scheduled with + :meth:`loop.call_soon` instead. + + - asyncio Future is not compatible with the + :func:`concurrent.futures.wait` and + :func:`concurrent.futures.as_completed` functions. diff --git a/Doc/library/asyncio-llapi-index.rst b/Doc/library/asyncio-llapi-index.rst new file mode 100644 index 000000000000..7fb1e60f50de --- /dev/null +++ b/Doc/library/asyncio-llapi-index.rst @@ -0,0 +1,510 @@ +.. currentmodule:: asyncio + + +=================== +Low-level API Index +=================== + +This page lists all low-level asyncio APIs. + + +Obtaining the Event Loop +======================== + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :func:`asyncio.get_running_loop` + - The **preferred** function to get the running event loop. + + * - :func:`asyncio.get_event_loop` + - Get an event loop instance (current or via the policy). + + * - :func:`asyncio.set_event_loop` + - Set the event loop as current via the current policy. + + * - :func:`asyncio.new_event_loop` + - Create a new event loop. + + +.. rubric:: Examples + +* :ref:`Using asyncio.get_running_loop() `. + + +Event Loop Methods +================== + +See also the main documentation section about the +:ref:`event loop methods `. + +.. rubric:: Lifecycle +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.run_until_complete` + - Run a Future/Task/awaitable until complete. + + * - :meth:`loop.run_forever` + - Run the event loop forever. + + * - :meth:`loop.stop` + - Stop the event loop. + + * - :meth:`loop.stop` + - Close the event loop. + + * - :meth:`loop.is_running()` + - Return ``True`` if the event loop is running. + + * - :meth:`loop.is_closed()` + - Return ``True`` if the event loop is closed. + + * - ``await`` :meth:`loop.shutdown_asyncgens` + - Close asynchronous generators. + + +.. rubric:: Debugging +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.set_debug` + - Enable or disable the debug mode. + + * - :meth:`loop.get_debug` + - Get the current debug mode. + + +.. rubric:: Scheduling Callbacks +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.call_soon` + - Invoke a callback soon. + + * - :meth:`loop.call_soon_threadsafe` + - A thread-safe variant of :meth:`loop.call_soon`. + + * - :meth:`loop.call_later` + - Invoke a callback *after* the given time. + + * - :meth:`loop.call_at` + - Invoke a callback *at* the given time. + + +.. rubric:: Thread/Process Pool +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :meth:`loop.run_in_executor` + - Run a CPU-bound or other blocking function in + a :mod:`concurrent.futures` executor. + + * - :meth:`loop.set_default_executor` + - Set the default executor for :meth:`loop.run_in_executor`. + + +.. rubric:: Tasks and Futures +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.create_future` + - Create a :class:`Future` object. + + * - :meth:`loop.create_task` + - Schedule coroutine as a :class:`Task`. + + * - :meth:`loop.set_task_factory` + - Set a factory used by :meth:`loop.create_task` to + create :class:`Tasks `. + + * - :meth:`loop.get_task_factory` + - Get the factory :meth:`loop.create_task` uses + to create :class:`Tasks `. + + +.. rubric:: DNS +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :meth:`loop.getaddrinfo` + - Asynchronous version of :meth:`socket.getaddrinfo`. + + * - ``await`` :meth:`loop.getnameinfo` + - Asynchronous version of :meth:`socket.getnameinfo`. + + +.. rubric:: Networking and IPC +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :meth:`loop.create_connection` + - Open a TCP connection. + + * - ``await`` :meth:`loop.create_server` + - Create a TCP server. + + * - ``await`` :meth:`loop.create_unix_connection` + - Open a Unix socket connection. + + * - ``await`` :meth:`loop.create_unix_server` + - Create a Unix socket server. + + * - ``await`` :meth:`loop.connect_accepted_socket` + - Wrap a :class:`~socket.socket` into a ``(transport, protocol)`` + pair. + + * - ``await`` :meth:`loop.create_datagram_endpoint` + - Open a datagram (UDP) connection. + + * - ``await`` :meth:`loop.sendfile` + - Send a file over a transport. + + * - ``await`` :meth:`loop.start_tls` + - Upgrade an existing connection to TLS. + + * - ``await`` :meth:`loop.connect_read_pipe` + - Wrap a read end of a pipe into a ``(transport, protocol)`` pair. + + * - ``await`` :meth:`loop.connect_write_pipe` + - Wrap a write end of a pipe into a ``(transport, protocol)`` pair. + + +.. rubric:: Sockets +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``await`` :meth:`loop.sock_recv` + - Receive data from the :class:`~socket.socket`. + + * - ``await`` :meth:`loop.sock_recv_into` + - Receive data from the :class:`~socket.socket` into a buffer. + + * - ``await`` :meth:`loop.sock_sendall` + - Send data to the :class:`~socket.socket`. + + * - ``await`` :meth:`loop.sock_connect` + - Connect the :class:`~socket.socket`. + + * - ``await`` :meth:`loop.sock_accept` + - Accept a :class:`~socket.socket` connection. + + * - ``await`` :meth:`loop.sock_sendfile` + - Send a file over the :class:`~socket.socket`. + + * - :meth:`loop.add_reader` + - Start watching a file descriptor for read availability. + + * - :meth:`loop.remove_reader` + - Stop watching a file descriptor for read availability. + + * - :meth:`loop.add_writer` + - Start watching a file descriptor for write availability. + + * - :meth:`loop.remove_writer` + - Stop watching a file descriptor for write availability. + + +.. rubric:: Unix Signals +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.add_signal_handler` + - Add a handler for a :mod:`signal`. + + * - :meth:`loop.remove_signal_handler` + - Remove a handler for a :mod:`signal`. + + +.. rubric:: Subprocesses +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.subprocess_exec` + - Spawn a subprocess. + + * - :meth:`loop.subprocess_shell` + - Spawn a subprocess from a shell command. + + +.. rubric:: Error Handling +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`loop.call_exception_handler` + - Call the exception handler. + + * - :meth:`loop.set_exception_handler` + - Set a new exception handler. + + * - :meth:`loop.get_exception_handler` + - Get the current exception handler. + + * - :meth:`loop.default_exception_handler` + - The default exception handler implementation. + + +.. rubric:: Examples + +* :ref:`Using asyncio.get_event_loop() and loop.run_forever() + `. + +* :ref:`Using loop.call_later() `. + +* Using ``loop.create_connection()`` to implement + :ref:`an echo-client `. + +* Using ``loop.create_connection()`` to + :ref:`connect a socket `. + +* :ref:`Using add_reader() to watch an FD for read events + `. + +* :ref:`Using loop.add_signal_handler() `. + +* :ref:`Using loop.subprocess_exec() `. + + +Transports +========== + +All transports implement the following methods: + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.close() ` + - Close the transport. + + * - :meth:`transport.is_closing() ` + - Return ``True`` if the transport is closing or is closed. + + * - :meth:`transport.get_extra_info() ` + - Request for information about the transport. + + * - :meth:`transport.set_protocol() ` + - Set a new protocol. + + * - :meth:`transport.get_protocol() ` + - Return the current protocol. + + +Transports that can receive data (TCP and Unix connections, +pipes, etc). Returned from methods like +:meth:`loop.create_connection`, :meth:`loop.create_unix_connection`, +:meth:`loop.connect_read_pipe`, etc: + +.. rubric:: Read Transports +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.is_reading() ` + - Return ``True`` if the transport is receiving. + + * - :meth:`transport.pause_reading() ` + - Pause receiving. + + * - :meth:`transport.resume_reading() ` + - Resume receiving. + + +Transports that can Send data (TCP and Unix connections, +pipes, etc). Returned from methods like +:meth:`loop.create_connection`, :meth:`loop.create_unix_connection`, +:meth:`loop.connect_write_pipe`, etc: + +.. rubric:: Write Transports +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.write() ` + - Write data to the transport. + + * - :meth:`transport.writelines() ` + - Write buffers to the transport. + + * - :meth:`transport.can_write_eof() ` + - Return :const:`True` if the transport supports sending EOF. + + * - :meth:`transport.write_eof() ` + - Close and send EOF after flushing buffered data. + + * - :meth:`transport.abort() ` + - Close the transport immediately. + + * - :meth:`transport.get_write_buffer_size() + ` + - Return high and low water marks for write flow control. + + * - :meth:`transport.set_write_buffer_limits() + ` + - Set new high and low water marks for write flow control. + + +Transports returned by :meth:`loop.create_datagram_endpoint`: + +.. rubric:: Datagram Transports +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.sendto() ` + - Send data to the remote peer. + + * - :meth:`transport.abort() ` + - Close the transport immediately. + + +Low-level transport abstraction over subprocesses. +Returned by :meth:`loop.subprocess_exec` and +:meth:`loop.subprocess_shell`: + +.. rubric:: Subprocess Transports +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`transport.get_pid() ` + - Return the subprocess process id. + + * - :meth:`transport.get_pipe_transport() + ` + - Return the transport for the requested communication pipe + (*stdin*, *stdout*, or *stderr*). + + * - :meth:`transport.get_returncode() ` + - Return the subprocess return code. + + * - :meth:`transport.kill() ` + - Kill the subprocess. + + * - :meth:`transport.send_signal() ` + - Send a signal to the subprocess. + + * - :meth:`transport.terminate() ` + - Stop the subprocess. + + * - :meth:`transport.close() ` + - Kill the subprocess and close all pipes. + + +Protocols +========= + +Protocol classes can implement the following **callback methods**: + +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`connection_made() ` + - Called when a connection is made. + + * - ``callback`` :meth:`connection_lost() ` + - Called when the connection is lost or closed. + + * - ``callback`` :meth:`pause_writing() ` + - Called when the transport's buffer goes over the high water mark. + + * - ``callback`` :meth:`resume_writing() ` + - Called when the transport's buffer drains below the low water mark. + + +.. rubric:: Streaming Protocols (TCP, Unix Sockets, Pipes) +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`data_received() ` + - Called when some data is received. + + * - ``callback`` :meth:`eof_received() ` + - Called when an EOF is received. + + +.. rubric:: Buffered Streaming Protocols +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`get_buffer() ` + - Called to allocate a new receive buffer. + + * - ``callback`` :meth:`buffer_updated() ` + - Called when the buffer was updated with the received data. + + * - ``callback`` :meth:`eof_received() ` + - Called when an EOF is received. + + +.. rubric:: Datagram Protocols +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`datagram_received() + ` + - Called when a datagram is received. + + * - ``callback`` :meth:`error_received() ` + - Called when a previous send or receive operation raises an + :class:`OSError`. + + +.. rubric:: Subprocess Protocols +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - ``callback`` :meth:`pipe_data_received() + ` + - Called when the child process writes data into its + *stdout* or *stderr* pipe. + + * - ``callback`` :meth:`pipe_connection_lost() + ` + - Called when one of the pipes communicating with + the child process is closed. + + * - ``callback`` :meth:`process_exited() + ` + - Called when the child process has exited. + + +Event Loop Policies +=================== + +Policies is a low-level mechanism to alter the behavior of +functions like :func:`asyncio.get_event_loop`. See also +the main :ref:`policies section ` for more +details. + + +.. rubric:: Accessing Policies +.. list-table:: + :widths: 50 50 + :class: full-width-table + + * - :meth:`asyncio.get_event_loop_policy` + - Return the current process-wide policy. + + * - :meth:`asyncio.set_event_loop_policy` + - Set a new process-wide policy. + + * - :class:`AbstractEventLoopPolicy` + - Base class for policy objects. diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst new file mode 100644 index 000000000000..f8ecb58d3a01 --- /dev/null +++ b/Doc/library/asyncio-platforms.rst @@ -0,0 +1,106 @@ +.. currentmodule:: asyncio + + +.. _asyncio-platform-support: + + +================ +Platform Support +================ + +The :mod:`asyncio` module is designed to be portable, +but some platforms have subtle differences and limitations +due to the platforms' underlying architecture and capabilities. + + +All Platforms +============= + +* :meth:`loop.add_reader` and :meth:`loop.add_writer` + cannot be used to monitor file I/O. + + +Windows +======= + +All event loops on Windows do not support the following methods: + +* :meth:`loop.create_unix_connection` and + :meth:`loop.create_unix_server` are not supported. + The :data:`socket.AF_UNIX` socket family is specific to Unix. + +* :meth:`loop.add_signal_handler` and + :meth:`loop.remove_signal_handler` are not supported. + +:class:`SelectorEventLoop` has the following limitations: + +* :class:`~selectors.SelectSelector` is used to wait on socket events: + it supports sockets and is limited to 512 sockets. + +* :meth:`loop.add_reader` and :meth:`loop.add_writer` only accept + socket handles (e.g. pipe file descriptors are not supported). + +* Pipes are not supported, so the :meth:`loop.connect_read_pipe` + and :meth:`loop.connect_write_pipe` methods are not implemented. + +* :ref:`Subprocesses ` are not supported, i.e. + :meth:`loop.subprocess_exec` and :meth:`loop.subprocess_shell` + methods are not implemented. + +:class:`ProactorEventLoop` has the following limitations: + +* The :meth:`loop.create_datagram_endpoint` method + is not supported. + +* The :meth:`loop.add_reader` and :meth:`loop.add_writer` + methods are not supported. + +The resolution of the monotonic clock on Windows is usually around 15.6 +msec. The best resolution is 0.5 msec. The resolution depends on the +hardware (availability of `HPET +`_) and on the +Windows configuration. + + +.. _asyncio-windows-subprocess: + +Subprocess Support on Windows +----------------------------- + +:class:`SelectorEventLoop` on Windows does not support subproceses. +On Windows, :class:`ProactorEventLoop` should be used instead:: + + import asyncio + + asyncio.set_event_loop_policy( + asyncio.WindowsProactorEventLoopPolicy()) + + asyncio.run(your_code()) + + +The :meth:`policy.set_child_watcher() +` function is also +not supported, as :class:`ProactorEventLoop` has a different mechanism +to watch child processes. + + +macOS +===== + +Modern macOS versions are fully supported. + +.. rubric:: macOS <= 10.8 + +On macOS 10.6, 10.7 and 10.8, the default event loop +uses :class:`selectors.KqueueSelector`, which does not support +character devices on these versions. The :class:`SelectorEventLoop` +can be manually configured to use :class:`~selectors.SelectSelector` +or :class:`~selectors.PollSelector` to support character devices on +these older versions of macOS. Example:: + + import asyncio + import selectors + + selector = selectors.SelectSelector() + loop = asyncio.SelectorEventLoop(selector) + asyncio.set_event_loop(loop) diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst new file mode 100644 index 000000000000..42f936da468e --- /dev/null +++ b/Doc/library/asyncio-policy.rst @@ -0,0 +1,221 @@ +.. currentmodule:: asyncio + + +.. _asyncio-policies: + +======== +Policies +======== + +An event loop policy is a global per-process object that controls +the management of the event loop. Each event loop has a default +policy, which can be changed and customized using the policy API. + +A policy defines the notion of *context* and manages a +separate event loop per context. The default policy +defines *context* to be the current thread. + +By using a custom event loop policy, the behavior of +:func:`get_event_loop`, :func:`set_event_loop`, and +:func:`new_event_loop` functions can be customized. + +Policy objects should implement the APIs defined +in the :class:`AbstractEventLoopPolicy` abstract base class. + + +Getting and Setting the Policy +============================== + +The following functions can be used to get and set the policy +for the current process: + +.. function:: get_event_loop_policy() + + Return the current process-wide policy. + +.. function:: set_event_loop_policy(policy) + + Set the current process-wide policy to *policy*. + + If *policy* is set to ``None``, the default policy is restored. + + +Policy Objects +============== + +The abstract event loop policy base class is defined as follows: + +.. class:: AbstractEventLoopPolicy + + An abstract base class for asyncio policies. + + .. method:: get_event_loop() + + Get the event loop for the current context. + + Return an event loop object implementing the + :class:`AbstractEventLoop` interface. + + This method should never return ``None``. + + .. versionchanged:: 3.6 + + .. method:: set_event_loop(loop) + + Set the event loop for the current context to *loop*. + + .. method:: new_event_loop() + + Create and return a new event loop object. + + This method should never return ``None``. + + .. method:: get_child_watcher() + + Get a child process watcher object. + + Return a watcher object implementing the + :class:`AbstractChildWatcher` interface. + + This function is Unix specific. + + .. method:: set_child_watcher(watcher) + + Get the current child process watcher to *watcher*. + + This function is Unix specific. + + +asyncio ships with the following built-in policies: + + +.. class:: DefaultEventLoopPolicy + + The default asyncio policy. Uses :class:`SelectorEventLoop` + on both Unix and Windows platforms. + + There is no need to install the default policy manually. asyncio + is configured to use the default policy automatically. + + +.. class:: WindowsProactorEventLoopPolicy + + An alternative event loop policy that uses the + :class:`ProactorEventLoop` event loop implementation. + + Availability: Windows. + + +Process Watchers +================ + +A process watcher allows customization of how an event loop monitors +child processes on Unix. Specifically, the event loop needs to know +when a child process has exited. + +In asyncio, child processes are created with +:func:`create_subprocess_exec` and :meth:`loop.subprocess_exec` +functions. + +asyncio defines the :class:`AbstractChildWatcher` abstract base class, +which child watchers should implement, and has two different +implementations: :class:`SafeChildWatcher` (configured to be used +by default) and :class:`FastChildWatcher`. + +See also the :ref:`Subprocess and Threads ` +section. + +The following two functions can be used to customize the child process watcher +implementation used by the asyncio event loop: + +.. function:: get_child_watcher() + + Return the current child watcher for the current policy. + +.. function:: set_child_watcher(watcher) + + Set the current child watcher to *watcher* for the current + policy. *watcher* must implement methods defined in the + :class:`AbstractChildWatcher` base class. + +.. note:: + Third-party event loops implementations might not support + custom child watchers. For such event loops, using + :func:`set_child_watcher` might be prohibited or have no effect. + +.. class:: AbstractChildWatcher + + .. method:: add_child_handler(pid, callback, \*args) + + Register a new child handler. + + Arrange for ``callback(pid, returncode, *args)`` to be called + when a process with PID equal to *pid* terminates. Specifying + another callback for the same process replaces the previous + handler. + + The *callback* callable must be thread-safe. + + .. method:: remove_child_handler(pid) + + Removes the handler for process with PID equal to *pid*. + + The function returns ``True`` if the handler was successfully + removed, ``False`` if there was nothing to remove. + + .. method:: attach_loop(loop) + + Attach the watcher to an event loop. + + If the watcher was previously attached to an event loop, then + it is first detached before attaching to the new loop. + + Note: loop may be ``None``. + + .. method:: close() + + Close the watcher. + + This method has to be called to ensure that underlying + resources are cleaned-up. + +.. class:: SafeChildWatcher + + This implementation avoids disrupting other code spawning processes + by polling every process explicitly on a :py:data:`SIGCHLD` signal. + + This is a safe solution but it has a significant overhead when + handling a big number of processes (*O(n)* each time a + :py:data:`SIGCHLD` is received). + + asyncio uses this safe implementation by default. + +.. class:: FastChildWatcher + + This implementation reaps every terminated processes by calling + ``os.waitpid(-1)`` directly, possibly breaking other code spawning + processes and waiting for their termination. + + There is no noticeable overhead when handling a big number of + children (*O(1)* each time a child terminates). + + +Custom Policies +=============== + +To implement a new event loop policy, it is recommended to subclass +:class:`DefaultEventLoopPolicy` and override the methods for which +custom behavior is wanted, e.g.:: + + class MyEventLoopPolicy(asyncio.DefaultEventLoopPolicy): + + def get_event_loop(self): + """Get the event loop. + + This may be None or an instance of EventLoop. + """ + loop = super().get_event_loop() + # Do something with loop ... + return loop + + asyncio.set_event_loop_policy(MyEventLoopPolicy()) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst index 9a08a4a49021..bdfdcf7ddb6e 100644 --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -1,12 +1,68 @@ .. currentmodule:: asyncio -+++++++++++++++++++++++++++++++++++++++++++++ -Transports and protocols (callback based API) -+++++++++++++++++++++++++++++++++++++++++++++ -**Source code:** :source:`Lib/asyncio/transports.py` +.. _asyncio-transports-protocols: + + +======================== +Transports and Protocols +======================== + +.. rubric:: Preface + +Transports and Protocols are used by the **low-level** event loop +APIs such as :meth:`loop.create_connection`. They use +callback-based programming style and enable high-performance +implementations of network or IPC protocols (e.g. HTTP). + +Essentially, transports and protocols should only be used in +libraries and frameworks and never in high-level asyncio +applications. + +This documentation page covers both `Transports`_ and `Protocols`_. + +.. rubric:: Introduction + +At the highest level, the transport is concerned with *how* bytes +are transmitted, while the protocol determines *which* bytes to +transmit (and to some extent when). + +A different way of saying the same thing: a transport is an +abstraction for a socket (or similar I/O endpoint) while a protocol +is an abstraction for an application, from the transport's point +of view. + +Yet another view is the transport and protocol interfaces +together define an abstract interface for using network I/O and +interprocess I/O. + +There is always a 1:1 relationship between transport and protocol +objects: the protocol calls transport methods to send data, +while the transport calls protocol methods to pass it data that +has been received. + +Most of connection oriented event loop methods +(such as :meth:`loop.create_connection`) usually accept a +*protocol_factory* argument used to create a *Protocol* object +for an accepted connection, represented by a *Transport* object. +Such methods usually return a tuple of ``(transport, protocol)``. + +.. rubric:: Contents + +This documentation page contains the following sections: + +* The `Transports`_ section documents asyncio :class:`BaseTransport`, + :class:`ReadTransport`, :class:`WriteTransport`, :class:`Transport`, + :class:`DatagramTransport`, and :class:`SubprocessTransport` + classes. + +* The `Protocols`_ section documents asyncio :class:`BaseProtocol`, + :class:`Protocol`, :class:`BufferedProtocol`, + :class:`DatagramProtocol`, and :class:`SubprocessProtocol` classes. + +* The `Examples`_ section showcases how to work with transports, + protocols, and low-level event loop APIs. -**Source code:** :source:`Lib/asyncio/protocols.py` .. _asyncio-transport: @@ -14,293 +70,360 @@ Transports ========== Transports are classes provided by :mod:`asyncio` in order to abstract -various kinds of communication channels. You generally won't instantiate -a transport yourself; instead, you will call an :class:`AbstractEventLoop` method -which will create the transport and try to initiate the underlying -communication channel, calling you back when it succeeds. +various kinds of communication channels. -Once the communication channel is established, a transport is always -paired with a :ref:`protocol ` instance. The protocol can -then call the transport's methods for various purposes. +Transport objects are always instantiated by an +ref:`asyncio event loop `. -:mod:`asyncio` currently implements transports for TCP, UDP, SSL, and -subprocess pipes. The methods available on a transport depend on -the transport's kind. +asyncio implements transports for TCP, UDP, SSL, and subprocess pipes. +The methods available on a transport depend on the transport's kind. The transport classes are :ref:`not thread safe `. -.. versionchanged:: 3.6 - The socket option ``TCP_NODELAY`` is now set by default. - -BaseTransport -------------- +Transports Hierarchy +-------------------- .. class:: BaseTransport - Base class for transports. + Base class for all transports. Contains methods that all + asyncio transports share. - .. method:: close() +.. class:: WriteTransport(BaseTransport) - Close the transport. If the transport has a buffer for outgoing - data, buffered data will be flushed asynchronously. No more data - will be received. After all buffered data is flushed, the - protocol's :meth:`connection_lost` method will be called with - :const:`None` as its argument. + A base transport for write-only connections. - .. method:: is_closing() + Instances of the *WriteTransport* class are returned from + the :meth:`loop.connect_write_pipe` event loop method and + are also used by subprocess-related methods like + :meth:`loop.subprocess_exec`. - Return ``True`` if the transport is closing or is closed. +.. class:: ReadTransport(BaseTransport) - .. versionadded:: 3.5.1 + A base transport for read-only connections. - .. method:: get_extra_info(name, default=None) + Instances of the *ReadTransport* class are returned from + the :meth:`loop.connect_read_pipe` event loop method and + are also used by subprocess-related methods like + :meth:`loop.subprocess_exec`. - Return optional transport information. *name* is a string representing - the piece of transport-specific information to get, *default* is the - value to return if the information doesn't exist. +.. class:: Transport(WriteTransport, ReadTransport) - This method allows transport implementations to easily expose - channel-specific information. + Interface representing a bidirectional transport, such as a + TCP connection. - * socket: + The user does not instantiate a transport directly; they call a + utility function, passing it a protocol factory and other + information necessary to create the transport and protocol. - - ``'peername'``: the remote address to which the socket is connected, - result of :meth:`socket.socket.getpeername` (``None`` on error) - - ``'socket'``: :class:`socket.socket` instance - - ``'sockname'``: the socket's own address, - result of :meth:`socket.socket.getsockname` + Instances of the *Transport* class are returned from or used by + event loop methods like :meth:`loop.create_connection`, + :meth:`loop.create_unix_connection`, + :meth:`loop.create_server`, :meth:`loop.sendfile`, etc. - * SSL socket: - - ``'compression'``: the compression algorithm being used as a string, - or ``None`` if the connection isn't compressed; result of - :meth:`ssl.SSLSocket.compression` - - ``'cipher'``: a three-value tuple containing the name of the cipher - being used, the version of the SSL protocol that defines its use, and - the number of secret bits being used; result of - :meth:`ssl.SSLSocket.cipher` - - ``'peercert'``: peer certificate; result of - :meth:`ssl.SSLSocket.getpeercert` - - ``'sslcontext'``: :class:`ssl.SSLContext` instance - - ``'ssl_object'``: :class:`ssl.SSLObject` or :class:`ssl.SSLSocket` - instance +.. class:: DatagramTransport(BaseTransport) - * pipe: + A transport for datagram (UDP) connections. - - ``'pipe'``: pipe object + Instances of the *DatagramTransport* class are returned from + the :meth:`loop.create_datagram_endpoint` event loop method. - * subprocess: - - ``'subprocess'``: :class:`subprocess.Popen` instance +.. class:: SubprocessTransport(BaseTransport) - .. method:: set_protocol(protocol) + An abstraction to represent a connection between a parent and its + child OS process. - Set a new protocol. Switching protocol should only be done when both - protocols are documented to support the switch. + Instances of the *SubprocessTransport* class are returned from + event loop methods :meth:`loop.subprocess_shell` and + :meth:`loop.subprocess_exec`. - .. versionadded:: 3.5.3 - .. method:: get_protocol +Base Transport +-------------- - Return the current protocol. +.. method:: BaseTransport.close() - .. versionadded:: 3.5.3 + Close the transport. - .. versionchanged:: 3.5.1 - ``'ssl_object'`` info was added to SSL sockets. + If the transport has a buffer for outgoing + data, buffered data will be flushed asynchronously. No more data + will be received. After all buffered data is flushed, the + protocol's :meth:`protocol.connection_lost() + ` method will be called with + :const:`None` as its argument. +.. method:: BaseTransport.is_closing() -ReadTransport -------------- + Return ``True`` if the transport is closing or is closed. -.. class:: ReadTransport +.. method:: BaseTransport.get_extra_info(name, default=None) - Interface for read-only transports. + Return information about the transport or underlying resources + it uses. - .. method:: is_reading() + *name* is a string representing the piece of transport-specific + information to get. - Return ``True`` if the transport is receiving new data. + *default* is the value to return if the information is not + available, or if the transport does not support querying it + with the given third-party event loop implementation or on the + current platform. - .. versionadded:: 3.7 + For example, the following code attempts to get the underlying + socket object of the transport:: - .. method:: pause_reading() + sock = transport.get_extra_info('socket') + if sock is not None: + print(sock.getsockopt(...)) - Pause the receiving end of the transport. No data will be passed to - the protocol's :meth:`data_received` method until :meth:`resume_reading` - is called. + Categories of information that can be queried on some transports: - .. versionchanged:: 3.7 - The method is idempotent, i.e. it can be called when the - transport is already paused or closed. + * socket: - .. method:: resume_reading() + - ``'peername'``: the remote address to which the socket is + connected, result of :meth:`socket.socket.getpeername` + (``None`` on error) - Resume the receiving end. The protocol's :meth:`data_received` method - will be called once again if some data is available for reading. + - ``'socket'``: :class:`socket.socket` instance - .. versionchanged:: 3.7 - The method is idempotent, i.e. it can be called when the - transport is already reading. + - ``'sockname'``: the socket's own address, + result of :meth:`socket.socket.getsockname` + * SSL socket: -WriteTransport --------------- + - ``'compression'``: the compression algorithm being used as a + string, or ``None`` if the connection isn't compressed; result + of :meth:`ssl.SSLSocket.compression` + + - ``'cipher'``: a three-value tuple containing the name of the + cipher being used, the version of the SSL protocol that defines + its use, and the number of secret bits being used; result of + :meth:`ssl.SSLSocket.cipher` + + - ``'peercert'``: peer certificate; result of + :meth:`ssl.SSLSocket.getpeercert` -.. class:: WriteTransport + - ``'sslcontext'``: :class:`ssl.SSLContext` instance - Interface for write-only transports. + - ``'ssl_object'``: :class:`ssl.SSLObject` or + :class:`ssl.SSLSocket` instance - .. method:: abort() + * pipe: - Close the transport immediately, without waiting for pending operations - to complete. Buffered data will be lost. No more data will be received. - The protocol's :meth:`connection_lost` method will eventually be - called with :const:`None` as its argument. + - ``'pipe'``: pipe object - .. method:: can_write_eof() + * subprocess: - Return :const:`True` if the transport supports :meth:`write_eof`, - :const:`False` if not. + - ``'subprocess'``: :class:`subprocess.Popen` instance - .. method:: get_write_buffer_size() +.. method:: BaseTransport.set_protocol(protocol) - Return the current size of the output buffer used by the transport. + Set a new protocol. - .. method:: get_write_buffer_limits() + Switching protocol should only be done when both + protocols are documented to support the switch. - Get the *high*- and *low*-water limits for write flow control. Return a - tuple ``(low, high)`` where *low* and *high* are positive number of - bytes. +.. method:: BaseTransport.get_protocol() - Use :meth:`set_write_buffer_limits` to set the limits. + Return the current protocol. - .. versionadded:: 3.4.2 - .. method:: set_write_buffer_limits(high=None, low=None) +Read-only Transports +-------------------- - Set the *high*- and *low*-water limits for write flow control. +.. method:: ReadTransport.is_reading() - These two values (measured in number of - bytes) control when the protocol's - :meth:`pause_writing` and :meth:`resume_writing` methods are called. - If specified, the low-water limit must be less than or equal to the - high-water limit. Neither *high* nor *low* can be negative. + Return ``True`` if the transport is receiving new data. - :meth:`pause_writing` is called when the buffer size becomes greater - than or equal to the *high* value. If writing has been paused, - :meth:`resume_writing` is called when the buffer size becomes less - than or equal to the *low* value. + .. versionadded:: 3.7 - The defaults are implementation-specific. If only the - high-water limit is given, the low-water limit defaults to an - implementation-specific value less than or equal to the - high-water limit. Setting *high* to zero forces *low* to zero as - well, and causes :meth:`pause_writing` to be called whenever the - buffer becomes non-empty. Setting *low* to zero causes - :meth:`resume_writing` to be called only once the buffer is empty. - Use of zero for either limit is generally sub-optimal as it - reduces opportunities for doing I/O and computation - concurrently. +.. method:: ReadTransport.pause_reading() - Use :meth:`get_write_buffer_limits` to get the limits. + Pause the receiving end of the transport. No data will be passed to + the protocol's :meth:`protocol.data_received() ` + method until :meth:`resume_reading` is called. - .. method:: write(data) + .. versionchanged:: 3.7 + The method is idempotent, i.e. it can be called when the + transport is already paused or closed. - Write some *data* bytes to the transport. +.. method:: ReadTransport.resume_reading() - This method does not block; it buffers the data and arranges for it - to be sent out asynchronously. + Resume the receiving end. The protocol's + :meth:`protocol.data_received() ` method + will be called once again if some data is available for reading. - .. method:: writelines(list_of_data) + .. versionchanged:: 3.7 + The method is idempotent, i.e. it can be called when the + transport is already reading. - Write a list (or any iterable) of data bytes to the transport. - This is functionally equivalent to calling :meth:`write` on each - element yielded by the iterable, but may be implemented more efficiently. - .. method:: write_eof() +Write-only Transports +--------------------- - Close the write end of the transport after flushing buffered data. - Data may still be received. +.. method:: WriteTransport.abort() - This method can raise :exc:`NotImplementedError` if the transport - (e.g. SSL) doesn't support half-closes. + Close the transport immediately, without waiting for pending operations + to complete. Buffered data will be lost. No more data will be received. + The protocol's :meth:`protocol.connection_lost() + ` method will eventually be + called with :const:`None` as its argument. +.. method:: WriteTransport.can_write_eof() -DatagramTransport ------------------ + Return :const:`True` if the transport supports + :meth:`~WriteTransport.write_eof`, :const:`False` if not. -.. method:: DatagramTransport.sendto(data, addr=None) +.. method:: WriteTransport.get_write_buffer_size() - Send the *data* bytes to the remote peer given by *addr* (a - transport-dependent target address). If *addr* is :const:`None`, the - data is sent to the target address given on transport creation. + Return the current size of the output buffer used by the transport. + +.. method:: WriteTransport.get_write_buffer_limits() + + Get the *high* and *low* watermarks for write flow control. Return a + tuple ``(low, high)`` where *low* and *high* are positive number of + bytes. + + Use :meth:`set_write_buffer_limits` to set the limits. + + .. versionadded:: 3.4.2 + +.. method:: WriteTransport.set_write_buffer_limits(high=None, low=None) + + Set the *high* and *low* watermarks for write flow control. + + These two values (measured in number of + bytes) control when the protocol's + :meth:`protocol.pause_writing() ` + and :meth:`protocol.resume_writing() ` + methods are called. If specified, the low watermark must be less + than or equal to the high watermark. Neither *high* nor *low* + can be negative. + + :meth:`~BaseProtocol.pause_writing` is called when the buffer size + becomes greater than or equal to the *high* value. If writing has + been paused, :meth:`~BaseProtocol.resume_writing` is called when + the buffer size becomes less than or equal to the *low* value. + + The defaults are implementation-specific. If only the + high watermark is given, the low watermark defaults to an + implementation-specific value less than or equal to the + high watermark. Setting *high* to zero forces *low* to zero as + well, and causes :meth:`~BaseProtocol.pause_writing` to be called + whenever the buffer becomes non-empty. Setting *low* to zero causes + :meth:`~BaseProtocol.resume_writing` to be called only once the + buffer is empty. Use of zero for either limit is generally + sub-optimal as it reduces opportunities for doing I/O and + computation concurrently. + + Use :meth:`~WriteTransport.get_write_buffer_limits` + to get the limits. + +.. method:: WriteTransport.write(data) + + Write some *data* bytes to the transport. This method does not block; it buffers the data and arranges for it to be sent out asynchronously. +.. method:: WriteTransport.writelines(list_of_data) + + Write a list (or any iterable) of data bytes to the transport. + This is functionally equivalent to calling :meth:`write` on each + element yielded by the iterable, but may be implemented more + efficiently. + +.. method:: WriteTransport.write_eof() + + Close the write end of the transport after flushing all buffered data. + Data may still be received. + + This method can raise :exc:`NotImplementedError` if the transport + (e.g. SSL) doesn't support half-closed connections. + + +Datagram Transports +------------------- + +.. method:: DatagramTransport.sendto(data, addr=None) + + Send the *data* bytes to the remote peer given by *addr* (a + transport-dependent target address). If *addr* is :const:`None`, + the data is sent to the target address given on transport + creation. + + This method does not block; it buffers the data and arranges + for it to be sent out asynchronously. + .. method:: DatagramTransport.abort() - Close the transport immediately, without waiting for pending operations - to complete. Buffered data will be lost. No more data will be received. - The protocol's :meth:`connection_lost` method will eventually be - called with :const:`None` as its argument. + Close the transport immediately, without waiting for pending + operations to complete. Buffered data will be lost. + No more data will be received. The protocol's + :meth:`protocol.connection_lost() ` + method will eventually be called with :const:`None` as its argument. + + +.. _asyncio-subprocess-transports: + +Subprocess Transports +--------------------- +.. method:: SubprocessTransport.get_pid() -BaseSubprocessTransport ------------------------ + Return the subprocess process id as an integer. -.. class:: BaseSubprocessTransport +.. method:: SubprocessTransport.get_pipe_transport(fd) - .. method:: get_pid() + Return the transport for the communication pipe corresponding to the + integer file descriptor *fd*: - Return the subprocess process id as an integer. + * ``0``: readable streaming transport of the standard input (*stdin*), + or :const:`None` if the subprocess was not created with ``stdin=PIPE`` + * ``1``: writable streaming transport of the standard output (*stdout*), + or :const:`None` if the subprocess was not created with ``stdout=PIPE`` + * ``2``: writable streaming transport of the standard error (*stderr*), + or :const:`None` if the subprocess was not created with ``stderr=PIPE`` + * other *fd*: :const:`None` - .. method:: get_pipe_transport(fd) +.. method:: SubprocessTransport.get_returncode() - Return the transport for the communication pipe corresponding to the - integer file descriptor *fd*: + Return the subprocess return code as an integer or :const:`None` + if it hasn't returned, which is similar to the + :attr:`subprocess.Popen.returncode` attribute. - * ``0``: readable streaming transport of the standard input (*stdin*), - or :const:`None` if the subprocess was not created with ``stdin=PIPE`` - * ``1``: writable streaming transport of the standard output (*stdout*), - or :const:`None` if the subprocess was not created with ``stdout=PIPE`` - * ``2``: writable streaming transport of the standard error (*stderr*), - or :const:`None` if the subprocess was not created with ``stderr=PIPE`` - * other *fd*: :const:`None` +.. method:: SubprocessTransport.kill() - .. method:: get_returncode() + Kill the subprocess. - Return the subprocess returncode as an integer or :const:`None` - if it hasn't returned, similarly to the - :attr:`subprocess.Popen.returncode` attribute. + On POSIX systems, the function sends SIGKILL to the subprocess. + On Windows, this method is an alias for :meth:`terminate`. - .. method:: kill() + See also :meth:`subprocess.Popen.kill`. - Kill the subprocess, as in :meth:`subprocess.Popen.kill`. +.. method:: SubprocessTransport.send_signal(signal) - On POSIX systems, the function sends SIGKILL to the subprocess. - On Windows, this method is an alias for :meth:`terminate`. + Send the *signal* number to the subprocess, as in + :meth:`subprocess.Popen.send_signal`. - .. method:: send_signal(signal) +.. method:: SubprocessTransport.terminate() - Send the *signal* number to the subprocess, as in - :meth:`subprocess.Popen.send_signal`. + Stop the subprocess. - .. method:: terminate() + On POSIX systems, this method sends SIGTERM to the subprocess. + On Windows, the Windows API function TerminateProcess() is called to + stop the subprocess. - Ask the subprocess to stop, as in :meth:`subprocess.Popen.terminate`. - This method is an alias for the :meth:`close` method. + See also :meth:`subprocess.Popen.terminate`. - On POSIX systems, this method sends SIGTERM to the subprocess. - On Windows, the Windows API function TerminateProcess() is called to - stop the subprocess. +.. method:: SubprocessTransport.close() - .. method:: close() + Kill the subprocess by calling the :meth:`kill` method. - Ask the subprocess to stop by calling the :meth:`terminate` method if the - subprocess hasn't returned yet, and close transports of all pipes - (*stdin*, *stdout* and *stderr*). + If the subprocess hasn't returned yet, and close transports of + *stdin*, *stdout*, and *stderr* pipes. .. _asyncio-protocol: @@ -308,65 +431,61 @@ BaseSubprocessTransport Protocols ========= -:mod:`asyncio` provides base classes that you can subclass to implement -your network protocols. Those classes are used in conjunction with -:ref:`transports ` (see below): the protocol parses incoming -data and asks for the writing of outgoing data, while the transport is -responsible for the actual I/O and buffering. +asyncio provides a set of abstract base classes that should be used +to implement network protocols. Those classes are meant to be used +together with :ref:`transports `. -When subclassing a protocol class, it is recommended you override certain -methods. Those methods are callbacks: they will be called by the transport -on certain events (for example when some data is received); you shouldn't -call them yourself, unless you are implementing a transport. +Subclasses of abstract base protocol classes may implement some or +all methods. All these methods are callbacks: they are called by +transports on certain events, for example when some data is received. +A base protocol method should be called by the corresponding transport. -.. note:: - All callbacks have default implementations, which are empty. Therefore, - you only need to implement the callbacks for the events in which you - are interested. +Base Protocols +-------------- + +.. class:: BaseProtocol -Protocol classes ----------------- + Base protocol with methods that all protocols share. -.. class:: Protocol +.. class:: Protocol(BaseProtocol) - The base class for implementing streaming protocols (for use with - e.g. TCP and SSL transports). + The base class for implementing streaming protocols + (TCP, Unix sockets, etc). -.. class:: BufferedProtocol +.. class:: BufferedProtocol(BaseProtocol) A base class for implementing streaming protocols with manual control of the receive buffer. - .. versionadded:: 3.7 - **Important:** this has been added to asyncio in Python 3.7 - *on a provisional basis*! Treat it as an experimental API that - might be changed or removed in Python 3.8. - -.. class:: DatagramProtocol +.. class:: DatagramProtocol(BaseProtocol) - The base class for implementing datagram protocols (for use with - e.g. UDP transports). + The base class for implementing datagram (UDP) protocols. -.. class:: SubprocessProtocol +.. class:: SubprocessProtocol(BaseProtocol) The base class for implementing protocols communicating with child - processes (through a set of unidirectional pipes). + processes (unidirectional pipes). -Connection callbacks --------------------- +Base Protocol +------------- + +All asyncio protocols can implement Base Protocol callbacks. + +.. rubric:: Connection Callbacks -These callbacks may be called on :class:`Protocol`, :class:`DatagramProtocol` -and :class:`SubprocessProtocol` instances: +Connection callbacks are called on all protocols, exactly once per +a successful connection. All other protocol callbacks can only be +called between those two methods. .. method:: BaseProtocol.connection_made(transport) Called when a connection is made. The *transport* argument is the transport representing the - connection. You are responsible for storing it somewhere - (e.g. as an attribute) if you need to. + connection. The protocol is responsible for storing the reference + to its transport. .. method:: BaseProtocol.connection_lost(exc) @@ -376,65 +495,76 @@ and :class:`SubprocessProtocol` instances: The latter means a regular EOF is received, or the connection was aborted or closed by this side of the connection. -:meth:`~BaseProtocol.connection_made` and :meth:`~BaseProtocol.connection_lost` -are called exactly once per successful connection. All other callbacks will be -called between those two methods, which allows for easier resource management -in your protocol implementation. -The following callbacks may be called only on :class:`SubprocessProtocol` -instances: +.. rubric:: Flow Control Callbacks -.. method:: SubprocessProtocol.pipe_data_received(fd, data) +Flow control callbacks can be called by transports to pause or +resume writing performed by the protocol. - Called when the child process writes data into its stdout or stderr pipe. - *fd* is the integer file descriptor of the pipe. *data* is a non-empty - bytes object containing the data. +See the documentation of the :meth:`~WriteTransport.set_write_buffer_limits` +method for more details. -.. method:: SubprocessProtocol.pipe_connection_lost(fd, exc) +.. method:: BaseProtocol.pause_writing() - Called when one of the pipes communicating with the child process - is closed. *fd* is the integer file descriptor that was closed. + Called when the transport's buffer goes over the high watermark. -.. method:: SubprocessProtocol.process_exited() +.. method:: BaseProtocol.resume_writing() - Called when the child process has exited. + Called when the transport's buffer drains below the low watermark. + +If the buffer size equals the high watermark, +:meth:`~BaseProtocol.pause_writing` is not called: the buffer size must +go strictly over. + +Conversely, :meth:`~BaseProtocol.resume_writing` is called when the +buffer size is equal or lower than the low watermark. These end +conditions are important to ensure that things go as expected when +either mark is zero. -Streaming protocols +Streaming Protocols ------------------- -The following callbacks are called on :class:`Protocol` instances: +Event methods, such as :meth:`loop.create_server`, +:meth:`loop.create_unix_server`, :meth:`loop.create_connection`, +:meth:`loop.create_unix_connection`, :meth:`loop.connect_accepted_socket`, +:meth:`loop.connect_read_pipe`, and :meth:`loop.connect_write_pipe` +accept factories that return streaming protocols. .. method:: Protocol.data_received(data) - Called when some data is received. *data* is a non-empty bytes object - containing the incoming data. + Called when some data is received. *data* is a non-empty bytes + object containing the incoming data. - .. note:: - Whether the data is buffered, chunked or reassembled depends on - the transport. In general, you shouldn't rely on specific semantics - and instead make your parsing generic and flexible enough. However, - data is always received in the correct order. + Whether the data is buffered, chunked or reassembled depends on + the transport. In general, you shouldn't rely on specific semantics + and instead make your parsing generic and flexible. However, + data is always received in the correct order. + + The method can be called an arbitrary number of times while + a connection is open. + + However, :meth:`protocol.eof_received() ` + is called at most once. Once `eof_received()` is called, + ``data_received()`` is not called anymore. .. method:: Protocol.eof_received() Called when the other end signals it won't send any more data - (for example by calling :meth:`write_eof`, if the other end also uses + (for example by calling :meth:`transport.write_eof() + `, if the other end also uses asyncio). This method may return a false value (including ``None``), in which case the transport will close itself. Conversely, if this method returns a - true value, closing the transport is up to the protocol. Since the - default implementation returns ``None``, it implicitly closes the connection. + true value, the protocol used determines whether to close the transport. + Since the default implementation returns ``None``, it implicitly closes the + connection. - .. note:: - Some transports such as SSL don't support half-closed connections, - in which case returning true from this method will not prevent closing - the connection. + Some transports, including SSL, don't support half-closed connections, + in which case returning true from this method will result in the connection + being closed. -:meth:`data_received` can be called an arbitrary number of times during -a connection. However, :meth:`eof_received` is called at most once -and, if called, :meth:`data_received` won't be called after it. State machine: @@ -446,25 +576,23 @@ State machine: -> connection_lost -> end -Streaming protocols with manual receive buffer control ------------------------------------------------------- +Buffered Streaming Protocols +---------------------------- .. versionadded:: 3.7 - **Important:** :class:`BufferedProtocol` has been added to - asyncio in Python 3.7 *on a provisional basis*! Consider it as an - experimental API that might be changed or removed in Python 3.8. - + **Important:** this has been added to asyncio in Python 3.7 + *on a provisional basis*! This is as an experimental API that + might be changed or removed completely in Python 3.8. -Event methods, such as :meth:`AbstractEventLoop.create_server` and -:meth:`AbstractEventLoop.create_connection`, accept factories that -return protocols that implement this interface. +Buffered Protocols can be used with any event loop method +that supports `Streaming Protocols`_. -The idea of BufferedProtocol is that it allows to manually allocate -and control the receive buffer. Event loops can then use the buffer +``BufferedProtocol`` implementations allow explicit manual allocation +and control of the receive buffer. Event loops can then use the buffer provided by the protocol to avoid unnecessary data copies. This can result in noticeable performance improvement for protocols that -receive big amounts of data. Sophisticated protocols implementations -can allocate the buffer only once at creation time. +receive big amounts of data. Sophisticated protocol implementations +can significantly reduce the number of buffer allocations. The following callbacks are called on :class:`BufferedProtocol` instances: @@ -473,12 +601,12 @@ instances: Called to allocate a new receive buffer. - *sizehint* is a recommended minimal size for the returned - buffer. It is acceptable to return smaller or bigger buffers + *sizehint* is the recommended minimum size for the returned + buffer. It is acceptable to return smaller or larger buffers than what *sizehint* suggests. When set to -1, the buffer size - can be arbitrary. It is an error to return a zero-sized buffer. + can be arbitrary. It is an error to return a buffer with a zero size. - Must return an object that implements the + ``get_buffer()`` must return an object implementing the :ref:`buffer protocol `. .. method:: BufferedProtocol.buffer_updated(nbytes) @@ -489,13 +617,15 @@ instances: .. method:: BufferedProtocol.eof_received() - See the documentation of the :meth:`Protocol.eof_received` method. + See the documentation of the :meth:`protocol.eof_received() + ` method. -:meth:`get_buffer` can be called an arbitrary number of times during -a connection. However, :meth:`eof_received` is called at most once -and, if called, :meth:`get_buffer` and :meth:`buffer_updated` -won't be called after it. +:meth:`~BufferedProtocol.get_buffer` can be called an arbitrary number +of times during a connection. However, :meth:`protocol.eof_received() +` is called at most once +and, if called, :meth:`~BufferedProtocol.get_buffer` and +:meth:`~BufferedProtocol.buffer_updated` won't be called after it. State machine: @@ -509,10 +639,11 @@ State machine: -> connection_lost -> end -Datagram protocols +Datagram Protocols ------------------ -The following callbacks are called on :class:`DatagramProtocol` instances. +Datagram Protocol instances should be constructed by protocol +factories passed to the :meth:`loop.create_datagram_endpoint` method. .. method:: DatagramProtocol.datagram_received(data, addr) @@ -526,80 +657,120 @@ The following callbacks are called on :class:`DatagramProtocol` instances. :class:`OSError`. *exc* is the :class:`OSError` instance. This method is called in rare conditions, when the transport (e.g. UDP) - detects that a datagram couldn't be delivered to its recipient. + detects that a datagram could not be delivered to its recipient. In many conditions though, undeliverable datagrams will be silently dropped. +.. note:: -Flow control callbacks ----------------------- + On BSD systems (macOS, FreeBSD, etc.) flow control is not supported + for datagram protocols, because there is no reliable way to detect send + failures caused by writing too many packets. -These callbacks may be called on :class:`Protocol`, -:class:`DatagramProtocol` and :class:`SubprocessProtocol` instances: + The socket always appears 'ready' and excess packets are dropped. An + :class:`OSError` with ``errno`` set to :const:`errno.ENOBUFS` may + or may not be raised; if it is raised, it will be reported to + :meth:`DatagramProtocol.error_received` but otherwise ignored. -.. method:: BaseProtocol.pause_writing() - Called when the transport's buffer goes over the high-water mark. +.. _asyncio-subprocess-protocols: -.. method:: BaseProtocol.resume_writing() +Subprocess Protocols +-------------------- - Called when the transport's buffer drains below the low-water mark. +Datagram Protocol instances should be constructed by protocol +factories passed to the :meth:`loop.subprocess_exec` and +:meth:`loop.subprocess_shell` methods. +.. method:: SubprocessProtocol.pipe_data_received(fd, data) -:meth:`pause_writing` and :meth:`resume_writing` calls are paired -- -:meth:`pause_writing` is called once when the buffer goes strictly over -the high-water mark (even if subsequent writes increases the buffer size -even more), and eventually :meth:`resume_writing` is called once when the -buffer size reaches the low-water mark. + Called when the child process writes data into its stdout or stderr + pipe. -.. note:: - If the buffer size equals the high-water mark, - :meth:`pause_writing` is not called -- it must go strictly over. - Conversely, :meth:`resume_writing` is called when the buffer size is - equal or lower than the low-water mark. These end conditions - are important to ensure that things go as expected when either - mark is zero. + *fd* is the integer file descriptor of the pipe. -.. note:: - On BSD systems (OS X, FreeBSD, etc.) flow control is not supported - for :class:`DatagramProtocol`, because send failures caused by - writing too many packets cannot be detected easily. The socket - always appears 'ready' and excess packets are dropped; an - :class:`OSError` with errno set to :const:`errno.ENOBUFS` may or - may not be raised; if it is raised, it will be reported to - :meth:`DatagramProtocol.error_received` but otherwise ignored. + *data* is a non-empty bytes object containing the received data. + +.. method:: SubprocessProtocol.pipe_connection_lost(fd, exc) + + Called when one of the pipes communicating with the child process + is closed. + + *fd* is the integer file descriptor that was closed. + +.. method:: SubprocessProtocol.process_exited() + + Called when the child process has exited. + + +Examples +======== + +.. _asyncio_example_tcp_echo_server_protocol: + +TCP Echo Server +--------------- + +Create a TCP echo server using the :meth:`loop.create_server` method, send back +received data, and close the connection:: + + import asyncio + + + class EchoServerClientProtocol(asyncio.Protocol): + def connection_made(self, transport): + peername = transport.get_extra_info('peername') + print('Connection from {}'.format(peername)) + self.transport = transport + + def data_received(self, data): + message = data.decode() + print('Data received: {!r}'.format(message)) + + print('Send: {!r}'.format(message)) + self.transport.write(data) + + print('Close the client socket') + self.transport.close() + + + async def main(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() + server = await loop.create_server( + lambda: EchoServerClientProtocol(), + '127.0.0.1', 8888) -Coroutines and protocols ------------------------- + async with server: + await server.serve_forever() -Coroutines can be scheduled in a protocol method using :func:`ensure_future`, -but there is no guarantee made about the execution order. Protocols are not -aware of coroutines created in protocol methods and so will not wait for them. -To have a reliable execution order, -use :ref:`stream objects ` in a -coroutine with ``await``. For example, the :meth:`StreamWriter.drain` -coroutine can be used to wait until the write buffer is flushed. + asyncio.run(main()) -Protocol examples -================= +.. seealso:: + + The :ref:`TCP echo server using streams ` + example uses the high-level :func:`asyncio.start_server` function. -.. _asyncio-tcp-echo-client-protocol: +.. _asyncio_example_tcp_echo_client_protocol: -TCP echo client protocol ------------------------- +TCP Echo Client +--------------- -TCP echo client using the :meth:`AbstractEventLoop.create_connection` method, send -data and wait until the connection is closed:: +A TCP echo client using the :meth:`loop.create_connection` method, sends +data, and waits until the connection is closed:: import asyncio + class EchoClientProtocol(asyncio.Protocol): - def __init__(self, message, loop): + def __init__(self, message, on_con_lost, loop): self.message = message self.loop = loop + self.on_con_lost = on_con_lost def connection_made(self, transport): transport.write(self.message.encode()) @@ -610,99 +781,99 @@ data and wait until the connection is closed:: def connection_lost(self, exc): print('The server closed the connection') - print('Stop the event loop') - self.loop.stop() - - loop = asyncio.get_event_loop() - message = 'Hello World!' - coro = loop.create_connection(lambda: EchoClientProtocol(message, loop), - '127.0.0.1', 8888) - loop.run_until_complete(coro) - loop.run_forever() - loop.close() - -The event loop is running twice. The -:meth:`~AbstractEventLoop.run_until_complete` method is preferred in this short -example to raise an exception if the server is not listening, instead of -having to write a short coroutine to handle the exception and stop the -running loop. At :meth:`~AbstractEventLoop.run_until_complete` exit, the loop is -no longer running, so there is no need to stop the loop in case of an error. + self.on_con_lost.set_result(True) + + + async def main(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() + + on_con_lost = loop.create_future() + message = 'Hello World!' + + transport, protocol = await loop.create_connection( + lambda: EchoClientProtocol(message, on_con_lost, loop), + '127.0.0.1', 8888) + + # Wait until the protocol signals that the connection + # is lost and close the transport. + try: + await on_con_lost + finally: + transport.close() + + + asyncio.run(main()) + .. seealso:: The :ref:`TCP echo client using streams ` - example uses the :func:`asyncio.open_connection` function. + example uses the high-level :func:`asyncio.open_connection` function. -.. _asyncio-tcp-echo-server-protocol: +.. _asyncio-udp-echo-server-protocol: -TCP echo server protocol ------------------------- +UDP Echo Server +--------------- -TCP echo server using the :meth:`AbstractEventLoop.create_server` method, send back -received data and close the connection:: +A UDP echo server, using the :meth:`loop.create_datagram_endpoint` +method, sends back received data:: import asyncio - class EchoServerClientProtocol(asyncio.Protocol): + + class EchoServerProtocol: def connection_made(self, transport): - peername = transport.get_extra_info('peername') - print('Connection from {}'.format(peername)) self.transport = transport - def data_received(self, data): + def datagram_received(self, data, addr): message = data.decode() - print('Data received: {!r}'.format(message)) + print('Received %r from %s' % (message, addr)) + print('Send %r to %s' % (message, addr)) + self.transport.sendto(data, addr) - print('Send: {!r}'.format(message)) - self.transport.write(data) - print('Close the client socket') - self.transport.close() - - loop = asyncio.get_event_loop() - # Each client connection will create a new protocol instance - coro = loop.create_server(EchoServerClientProtocol, '127.0.0.1', 8888) - server = loop.run_until_complete(coro) + async def main(): + print("Starting UDP server") - # Serve requests until Ctrl+C is pressed - print('Serving on {}'.format(server.sockets[0].getsockname())) - try: - loop.run_forever() - except KeyboardInterrupt: - pass + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() - # Close the server - server.close() - loop.run_until_complete(server.wait_closed()) - loop.close() + # One protocol instance will be created to serve all + # client requests. + transport, protocol = await loop.create_datagram_endpoint( + lambda: EchoServerProtocol(), + local_addr=('127.0.0.1', 9999)) -:meth:`Transport.close` can be called immediately after -:meth:`WriteTransport.write` even if data are not sent yet on the socket: both -methods are asynchronous. ``await`` is not needed because these transport -methods are not coroutines. + try: + await asyncio.sleep(3600) # Serve for 1 hour. + finally: + transport.close() -.. seealso:: - The :ref:`TCP echo server using streams ` - example uses the :func:`asyncio.start_server` function. + asyncio.run(main()) .. _asyncio-udp-echo-client-protocol: -UDP echo client protocol ------------------------- +UDP Echo Client +--------------- -UDP echo client using the :meth:`AbstractEventLoop.create_datagram_endpoint` -method, send data and close the transport when we received the answer:: +A UDP echo client, using the :meth:`loop.create_datagram_endpoint` +method, sends data and closes the transport when it receives the answer:: import asyncio + class EchoClientProtocol: def __init__(self, message, loop): self.message = message self.loop = loop self.transport = None + self.on_con_lost = loop.create_future() def connection_made(self, transport): self.transport = transport @@ -719,75 +890,46 @@ method, send data and close the transport when we received the answer:: print('Error received:', exc) def connection_lost(self, exc): - print("Socket closed, stop the event loop") - loop = asyncio.get_event_loop() - loop.stop() + print("Connection closed") + self.on_con_lost.set_result(True) - loop = asyncio.get_event_loop() - message = "Hello World!" - connect = loop.create_datagram_endpoint( - lambda: EchoClientProtocol(message, loop), - remote_addr=('127.0.0.1', 9999)) - transport, protocol = loop.run_until_complete(connect) - loop.run_forever() - transport.close() - loop.close() + async def main(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() -.. _asyncio-udp-echo-server-protocol: - -UDP echo server protocol ------------------------- - -UDP echo server using the :meth:`AbstractEventLoop.create_datagram_endpoint` -method, send back received data:: - - import asyncio - - class EchoServerProtocol: - def connection_made(self, transport): - self.transport = transport - - def datagram_received(self, data, addr): - message = data.decode() - print('Received %r from %s' % (message, addr)) - print('Send %r to %s' % (message, addr)) - self.transport.sendto(data, addr) + message = "Hello World!" + transport, protocol = await loop.create_datagram_endpoint( + lambda: EchoClientProtocol(message, loop), + remote_addr=('127.0.0.1', 9999)) - loop = asyncio.get_event_loop() - print("Starting UDP server") - # One protocol instance will be created to serve all client requests - listen = loop.create_datagram_endpoint( - EchoServerProtocol, local_addr=('127.0.0.1', 9999)) - transport, protocol = loop.run_until_complete(listen) + try: + await protocol.on_con_lost + finally: + transport.close() - try: - loop.run_forever() - except KeyboardInterrupt: - pass - transport.close() - loop.close() + asyncio.run(main()) -.. _asyncio-register-socket: +.. _asyncio_example_create_connection: -Register an open socket to wait for data using a protocol ---------------------------------------------------------- +Connecting Existing Sockets +--------------------------- Wait until a socket receives data using the -:meth:`AbstractEventLoop.create_connection` method with a protocol, and then close -the event loop :: +:meth:`loop.create_connection` method with a protocol:: import asyncio - from socket import socketpair + import socket - # Create a pair of connected sockets - rsock, wsock = socketpair() - loop = asyncio.get_event_loop() class MyProtocol(asyncio.Protocol): - transport = None + + def __init__(self, loop): + self.transport = None + self.on_con_lost = loop.create_future() def connection_made(self, transport): self.transport = transport @@ -795,35 +937,105 @@ the event loop :: def data_received(self, data): print("Received:", data.decode()) - # We are done: close the transport (it will call connection_lost()) + # We are done: close the transport; + # connection_lost() will be called automatically. self.transport.close() def connection_lost(self, exc): - # The socket has been closed, stop the event loop - loop.stop() + # The socket has been closed + self.on_con_lost.set_result(True) + + + async def main(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() + + # Create a pair of connected sockets + rsock, wsock = socket.socketpair() - # Register the socket to wait for data - connect_coro = loop.create_connection(MyProtocol, sock=rsock) - transport, protocol = loop.run_until_complete(connect_coro) + # Register the socket to wait for data. + transport, protocol = await loop.create_connection( + lambda: MyProtocol(loop), sock=rsock) - # Simulate the reception of data from the network - loop.call_soon(wsock.send, 'abc'.encode()) + # Simulate the reception of data from the network. + loop.call_soon(wsock.send, 'abc'.encode()) - # Run the event loop - loop.run_forever() + try: + await protocol.on_con_lost + finally: + transport.close() + wsock.close() - # We are done, close sockets and the event loop - rsock.close() - wsock.close() - loop.close() + asyncio.run(main()) .. seealso:: The :ref:`watch a file descriptor for read events - ` example uses the low-level - :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a - socket. + ` example uses the low-level + :meth:`loop.add_reader` method to register an FD. The :ref:`register an open socket to wait for data using streams - ` example uses high-level streams + ` example uses high-level streams created by the :func:`open_connection` function in a coroutine. + +.. _asyncio_example_subprocess_proto: + +loop.subprocess_exec() and SubprocessProtocol +--------------------------------------------- + +An example of a subprocess protocol used to get the output of a +subprocess and to wait for the subprocess exit. + +The subprocess is created by th :meth:`loop.subprocess_exec` method:: + + import asyncio + import sys + + class DateProtocol(asyncio.SubprocessProtocol): + def __init__(self, exit_future): + self.exit_future = exit_future + self.output = bytearray() + + def pipe_data_received(self, fd, data): + self.output.extend(data) + + def process_exited(self): + self.exit_future.set_result(True) + + async def get_date(): + # Get a reference to the event loop as we plan to use + # low-level APIs. + loop = asyncio.get_running_loop() + + code = 'import datetime; print(datetime.datetime.now())' + exit_future = asyncio.Future(loop=loop) + + # Create the subprocess controlled by DateProtocol; + # redirect the standard output into a pipe. + transport, protocol = await loop.subprocess_exec( + lambda: DateProtocol(exit_future), + sys.executable, '-c', code, + stdin=None, stderr=None) + + # Wait for the subprocess exit using the process_exited() + # method of the protocol. + await exit_future + + # Close the stdout pipe. + transport.close() + + # Read the output which was collected by the + # pipe_data_received() method of the protocol. + data = bytes(protocol.output) + return data.decode('ascii').rstrip() + + if sys.platform == "win32": + asyncio.set_event_loop_policy( + asyncio.WindowsProactorEventLoopPolicy()) + + date = asyncio.run(get_date()) + print(f"Current date: {date}") + +See also the :ref:`same example ` +written using high-level APIs. diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index 65497f29d895..bd0e70c0d9fc 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -1,41 +1,42 @@ .. currentmodule:: asyncio +.. _asyncio-queues: + +====== Queues ====== -**Source code:** :source:`Lib/asyncio/queues.py` - -Queues: +asyncio queues are designed to be similar to classes of the +:mod:`queue` module. Although asyncio queues are not thread-safe, +they are designed to be used specifically in async/await code. -* :class:`Queue` -* :class:`PriorityQueue` -* :class:`LifoQueue` +Note that methods of asyncio queues don't have a *timeout* parameter; +use :func:`asyncio.wait_for` function to do queue operations with a +timeout. -asyncio queue API was designed to be close to classes of the :mod:`queue` -module (:class:`~queue.Queue`, :class:`~queue.PriorityQueue`, -:class:`~queue.LifoQueue`), but it has no *timeout* parameter. The -:func:`asyncio.wait_for` function can be used to cancel a task after a timeout. +See also the `Examples`_ section below. Queue ------ +===== .. class:: Queue(maxsize=0, \*, loop=None) - A queue, useful for coordinating producer and consumer coroutines. + A first in, first out (FIFO) queue. - If *maxsize* is less than or equal to zero, the queue size is infinite. If - it is an integer greater than ``0``, then ``await put()`` will block - when the queue reaches *maxsize*, until an item is removed by :meth:`get`. + If *maxsize* is less than or equal to zero, the queue size is + infinite. If it is an integer greater than ``0``, then + ``await put()`` blocks when the queue reaches *maxsize* + until an item is removed by :meth:`get`. - Unlike the standard library :mod:`queue`, you can reliably know this Queue's - size with :meth:`qsize`, since your single-threaded asyncio application won't - be interrupted between calling :meth:`qsize` and doing an operation on the - Queue. + Unlike the standard library threading :mod:`queue`, the size of + the queue is always known and can be returned by calling the + :meth:`qsize` method. This class is :ref:`not thread safe `. - .. versionchanged:: 3.4.4 - New :meth:`join` and :meth:`task_done` methods. + .. attribute:: maxsize + + Number of items allowed in the queue. .. method:: empty() @@ -45,53 +46,33 @@ Queue Return ``True`` if there are :attr:`maxsize` items in the queue. - .. note:: - - If the Queue was initialized with ``maxsize=0`` (the default), then - :meth:`full()` is never ``True``. + If the queue was initialized with ``maxsize=0`` (the default), + then :meth:`full()` never returns ``True``. .. coroutinemethod:: get() - Remove and return an item from the queue. If queue is empty, wait until - an item is available. - - This method is a :ref:`coroutine `. - - .. seealso:: - - The :meth:`empty` method. + Remove and return an item from the queue. If queue is empty, + wait until an item is available. .. method:: get_nowait() - Remove and return an item from the queue. - Return an item if one is immediately available, else raise :exc:`QueueEmpty`. .. coroutinemethod:: join() - Block until all items in the queue have been gotten and processed. - - The count of unfinished tasks goes up whenever an item is added to the - queue. The count goes down whenever a consumer thread calls - :meth:`task_done` to indicate that the item was retrieved and all work on - it is complete. When the count of unfinished tasks drops to zero, - :meth:`join` unblocks. - - This method is a :ref:`coroutine `. + Block until all items in the queue have been received and processed. - .. versionadded:: 3.4.4 + The count of unfinished tasks goes up whenever an item is added + to the queue. The count goes down whenever a consumer thread calls + :meth:`task_done` to indicate that the item was retrieved and all + work on it is complete. When the count of unfinished tasks drops + to zero, :meth:`join` unblocks. .. coroutinemethod:: put(item) - Put an item into the queue. If the queue is full, wait until a free slot - is available before adding item. - - This method is a :ref:`coroutine `. - - .. seealso:: - - The :meth:`full` method. + Put an item into the queue. If the queue is full, wait until a + free slot is available before adding the item. .. method:: put_nowait(item) @@ -101,60 +82,119 @@ Queue .. method:: qsize() - Number of items in the queue. + Return the number of items in the queue. .. method:: task_done() Indicate that a formerly enqueued task is complete. - Used by queue consumers. For each :meth:`~Queue.get` used to fetch a task, a - subsequent call to :meth:`task_done` tells the queue that the processing - on the task is complete. - - If a :meth:`join` is currently blocking, it will resume when all items - have been processed (meaning that a :meth:`task_done` call was received - for every item that had been :meth:`~Queue.put` into the queue). - - Raises :exc:`ValueError` if called more times than there were items - placed in the queue. + Used by queue consumers. For each :meth:`~Queue.get` used to + fetch a task, a subsequent call to :meth:`task_done` tells the + queue that the processing on the task is complete. - .. versionadded:: 3.4.4 - - .. attribute:: maxsize + If a :meth:`join` is currently blocking, it will resume when all + items have been processed (meaning that a :meth:`task_done` + call was received for every item that had been :meth:`~Queue.put` + into the queue). - Number of items allowed in the queue. + Raises :exc:`ValueError` if called more times than there were + items placed in the queue. -PriorityQueue -------------- +Priority Queue +============== .. class:: PriorityQueue - A subclass of :class:`Queue`; retrieves entries in priority order (lowest - first). + A variant of :class:`Queue`; retrieves entries in priority order + (lowest first). - Entries are typically tuples of the form: (priority number, data). + Entries are typically tuples of the form + ``(priority_number, data)``. -LifoQueue ---------- +LIFO Queue +========== .. class:: LifoQueue - A subclass of :class:`Queue` that retrieves most recently added entries - first. + A variant of :class:`Queue` that retrieves most recently added + entries first (last in, first out). Exceptions -^^^^^^^^^^ +========== .. exception:: QueueEmpty - Exception raised when the :meth:`~Queue.get_nowait` method is called on a - :class:`Queue` object which is empty. + This exception is raised when the :meth:`~Queue.get_nowait` method + is called on an empty queue. .. exception:: QueueFull - Exception raised when the :meth:`~Queue.put_nowait` method is called on a - :class:`Queue` object which is full. + Exception raised when the :meth:`~Queue.put_nowait` method is called + on a queue that has reached its *maxsize*. + + +Examples +======== + +.. _asyncio_example_queue_dist: + +Queues can be used to distribute workload between several +concurrent tasks:: + + import asyncio + import random + import time + + + async def worker(name, queue): + while True: + # Get a "work item" out of the queue. + sleep_for = await queue.get() + + # Sleep for the "sleep_for" seconds. + await asyncio.sleep(sleep_for) + + # Notify the queue that the "work item" has been processed. + queue.task_done() + + print(f'{name} has slept for {sleep_for:.2f} seconds') + + + async def main(): + # Create a queue that we will use to store our "workload". + queue = asyncio.Queue() + + # Generate random timings and put them into the queue. + total_sleep_time = 0 + for _ in range(20): + sleep_for = random.uniform(0.05, 1.0) + total_sleep_time += sleep_for + queue.put_nowait(sleep_for) + + # Create three worker tasks to process the queue concurrently. + tasks = [] + for i in range(3): + task = asyncio.create_task(worker(f'worker-{i}', queue)) + tasks.append(task) + + # Wait until the queue is fully processed. + started_at = time.monotonic() + await queue.join() + total_slept_for = time.monotonic() - started_at + + # Cancel our worker tasks. + for task in tasks: + task.cancel() + # Wait until all worker tasks are cancelled. + await asyncio.gather(*tasks, return_exceptions=True) + + print('====') + print(f'3 workers slept in parallel for {total_slept_for:.2f} seconds') + print(f'total expected sleep time: {total_sleep_time:.2f} seconds') + + + asyncio.run(main()) diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index ca7daabdadd5..c543aa6d4179 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -2,85 +2,120 @@ .. _asyncio-streams: -+++++++++++++++++++++++++++++ -Streams (coroutine based API) -+++++++++++++++++++++++++++++ +======= +Streams +======= -**Source code:** :source:`Lib/asyncio/streams.py` +Streams are high-level async/await-ready primitives to work with +network connections. Streams allow sending and receiving data without +using callbacks or low-level protocols and transports. -Stream functions -================ +.. _asyncio_example_stream: -.. note:: +Here is an example of a TCP echo client written using asyncio +streams:: - The top-level functions in this module are meant as convenience wrappers - only; there's really nothing special there, and if they don't do - exactly what you want, feel free to copy their code. + import asyncio + + async def tcp_echo_client(message): + reader, writer = await asyncio.open_connection( + '127.0.0.1', 8888) + print(f'Send: {message!r}') + writer.write(message.encode()) -.. coroutinefunction:: open_connection(host=None, port=None, \*, loop=None, limit=None, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None) + data = await reader.read(100) + print(f'Received: {data.decode()!r}') - A wrapper for :meth:`~AbstractEventLoop.create_connection()` returning a (reader, - writer) pair. + print('Close the connection') + writer.close() + await writer.wait_closed() - The reader returned is a :class:`StreamReader` instance; the writer is - a :class:`StreamWriter` instance. + asyncio.run(tcp_echo_client('Hello World!')) - When specified, the *loop* argument determines which event loop to use, - and the *limit* argument determines the buffer size limit used by the - returned :class:`StreamReader` instance. - The rest of the arguments are passed directly to - :meth:`AbstractEventLoop.create_connection`. +See also the `Examples`_ section below. + + +.. rubric:: Stream Functions + +The following top-level asyncio functions can be used to create +and work with streams: + + +.. coroutinefunction:: open_connection(host=None, port=None, \*, \ + loop=None, limit=None, ssl=None, family=0, \ + proto=0, flags=0, sock=None, local_addr=None, \ + server_hostname=None, ssl_handshake_timeout=None) + + Establish a network connection and return a pair of + ``(reader, writer)`` objects. - This function is a :ref:`coroutine `. + The returned *reader* and *writer* objects are instances of + :class:`StreamReader` and :class:`StreamWriter` classes. + + The *loop* argument is optional and can always be determined + automatically when this function is awaited from a coroutine. + + *limit* determines the buffer size limit used by the + returned :class:`StreamReader` instance. By default the *limit* + is set to 64 KiB. + + The rest of the arguments are passed directly to + :meth:`loop.create_connection`. .. versionadded:: 3.7 The *ssl_handshake_timeout* parameter. -.. coroutinefunction:: start_server(client_connected_cb, host=None, port=None, \*, loop=None, limit=None, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, ssl_handshake_timeout=None, start_serving=True) +.. coroutinefunction:: start_server(client_connected_cb, host=None, \ + port=None, \*, loop=None, limit=None, \ + family=socket.AF_UNSPEC, \ + flags=socket.AI_PASSIVE, sock=None, \ + backlog=100, ssl=None, reuse_address=None, \ + reuse_port=None, ssl_handshake_timeout=None, \ + start_serving=True) - Start a socket server, with a callback for each client connected. The return - value is the same as :meth:`~AbstractEventLoop.create_server()`. + Start a socket server. The *client_connected_cb* callback is called whenever a new client - connection is established. It receives a reader/writer pair as two - arguments, the first is a :class:`StreamReader` instance, - and the second is a :class:`StreamWriter` instance. + connection is established. It receives a ``(reader, writer)`` pair + as two arguments, instances of the :class:`StreamReader` and + :class:`StreamWriter` classes. - *client_connected_cb* accepts a plain callable or a + *client_connected_cb* can be a plain callable or a :ref:`coroutine function `; if it is a coroutine function, - it will be automatically converted into a :class:`Task`. + it will be automatically scheduled as a :class:`Task`. - When specified, the *loop* argument determines which event loop to use, - and the *limit* argument determines the buffer size limit used by the - :class:`StreamReader` instance passed to *client_connected_cb*. + The *loop* argument is optional and can always be determined + automatically when this method is awaited from a coroutine. - The rest of the arguments are passed directly to - :meth:`~AbstractEventLoop.create_server()`. + *limit* determines the buffer size limit used by the + returned :class:`StreamReader` instance. By default the *limit* + is set to 64 KiB. - This function is a :ref:`coroutine `. + The rest of the arguments are passed directly to + :meth:`loop.create_server`. .. versionadded:: 3.7 The *ssl_handshake_timeout* and *start_serving* parameters. -.. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, limit=None, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None) - A wrapper for :meth:`~AbstractEventLoop.create_unix_connection()` returning - a (reader, writer) pair. +.. rubric:: Unix Sockets - When specified, the *loop* argument determines which event loop to use, - and the *limit* argument determines the buffer size limit used by the - returned :class:`StreamReader` instance. +.. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, \ + limit=None, ssl=None, sock=None, \ + server_hostname=None, ssl_handshake_timeout=None) - The rest of the arguments are passed directly to - :meth:`~AbstractEventLoop.create_unix_connection()`. + Establish a Unix socket connection and return a pair of + ``(reader, writer)``. + + Similar to :func:`open_connection` but operates on Unix sockets. - This function is a :ref:`coroutine `. + See also the documentation of :meth:`loop.create_unix_connection`. - Availability: UNIX. + Availability: Unix. .. versionadded:: 3.7 @@ -90,29 +125,19 @@ Stream functions The *path* parameter can now be a :term:`path-like object` -.. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \*, loop=None, limit=None, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, start_serving=True) - Start a UNIX Domain Socket server, with a callback for each client connected. +.. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \ + \*, loop=None, limit=None, sock=None, \ + backlog=100, ssl=None, ssl_handshake_timeout=None, \ + start_serving=True) - The *client_connected_cb* callback is called whenever a new client - connection is established. It receives a reader/writer pair as two - arguments, the first is a :class:`StreamReader` instance, - and the second is a :class:`StreamWriter` instance. + Start a Unix socket server. - *client_connected_cb* accepts a plain callable or a - :ref:`coroutine function `; if it is a coroutine function, - it will be automatically converted into a :class:`Task`. + Similar to :func:`start_server` but works with Unix sockets. - When specified, the *loop* argument determines which event loop to use, - and the *limit* argument determines the buffer size limit used by the - :class:`StreamReader` instance passed to *client_connected_cb*. + See also the documentation of :meth:`loop.create_unix_server`. - The rest of the arguments are passed directly to - :meth:`~AbstractEventLoop.create_unix_server()`. - - This function is a :ref:`coroutine `. - - Availability: UNIX. + Availability: Unix. .. versionadded:: 3.7 @@ -123,229 +148,156 @@ Stream functions The *path* parameter can now be a :term:`path-like object`. -StreamReader -============ - -.. class:: StreamReader(limit=_DEFAULT_LIMIT, loop=None) - - This class is :ref:`not thread safe `. - - The *limit* argument's default value is set to _DEFAULT_LIMIT which is 2**16 (64 KiB) - - .. method:: exception() - - Get the exception. - - .. method:: feed_eof() - - Acknowledge the EOF. +--------- - .. method:: feed_data(data) - Feed *data* bytes in the internal buffer. Any operations waiting - for the data will be resumed. - - .. method:: set_exception(exc) +StreamReader +============ - Set the exception. +.. class:: StreamReader - .. method:: set_transport(transport) + Represents a reader object that provides APIs to read data + from the IO stream. - Set the transport. + It is not recommended to instantiate *StreamReader* objects + directly; use :func:`open_connection` and :func:`start_server` + instead. .. coroutinemethod:: read(n=-1) Read up to *n* bytes. If *n* is not provided, or set to ``-1``, read until EOF and return all read bytes. - If the EOF was received and the internal buffer is empty, + If EOF was received and the internal buffer is empty, return an empty ``bytes`` object. - This method is a :ref:`coroutine `. - .. coroutinemethod:: readline() - Read one line, where "line" is a sequence of bytes ending with ``\n``. + Read one line, where "line" is a sequence of bytes + ending with ``\n``. - If EOF is received, and ``\n`` was not found, the method will - return the partial read bytes. + If EOF is received and ``\n`` was not found, the method + returns partially read data. - If the EOF was received and the internal buffer is empty, + If EOF is received and the internal buffer is empty, return an empty ``bytes`` object. - This method is a :ref:`coroutine `. - .. coroutinemethod:: readexactly(n) - Read exactly *n* bytes. Raise an :exc:`IncompleteReadError` if the end of - the stream is reached before *n* can be read, the - :attr:`IncompleteReadError.partial` attribute of the exception contains - the partial read bytes. + Read exactly *n* bytes. - This method is a :ref:`coroutine `. + Raise an :exc:`IncompleteReadError` if EOF is reached before *n* + can be read. Use the :attr:`IncompleteReadError.partial` + attribute to get the partially read data. .. coroutinemethod:: readuntil(separator=b'\\n') - Read data from the stream until ``separator`` is found. + Read data from the stream until *separator* is found. On success, the data and separator will be removed from the internal buffer (consumed). Returned data will include the separator at the end. - Configured stream limit is used to check result. Limit sets the - maximal length of data that can be returned, not counting the - separator. - - If an EOF occurs and the complete separator is still not found, - an :exc:`IncompleteReadError` exception will be - raised, and the internal buffer will be reset. The - :attr:`IncompleteReadError.partial` attribute may contain the - separator partially. + If the amount of data read exceeds the configured stream limit, a + :exc:`LimitOverrunError` exception is raised, and the data + is left in the internal buffer and can be read again. - If the data cannot be read because of over limit, a - :exc:`LimitOverrunError` exception will be raised, and the data - will be left in the internal buffer, so it can be read again. + If EOF is reached before the complete separator is found, + an :exc:`IncompleteReadError` exception is raised, and the internal + buffer is reset. The :attr:`IncompleteReadError.partial` attribute + may contain a portion of the separator. .. versionadded:: 3.5.2 .. method:: at_eof() - Return ``True`` if the buffer is empty and :meth:`feed_eof` was called. + Return ``True`` if the buffer is empty and :meth:`feed_eof` + was called. StreamWriter ============ -.. class:: StreamWriter(transport, protocol, reader, loop) +.. class:: StreamWriter - Wraps a Transport. + Represents a writer object that provides APIs to write data + to the IO stream. - This exposes :meth:`write`, :meth:`writelines`, :meth:`can_write_eof()`, - :meth:`write_eof`, :meth:`get_extra_info` and :meth:`close`. It adds - :meth:`drain` which returns an optional :class:`Future` on which you can - wait for flow control. It also adds a transport attribute which references - the :class:`Transport` directly. - - This class is :ref:`not thread safe `. - - .. attribute:: transport - - Transport. + It is not recommended to instantiate *StreamWriter* objects + directly; use :func:`open_connection` and :func:`start_server` + instead. .. method:: can_write_eof() - Return :const:`True` if the transport supports :meth:`write_eof`, - :const:`False` if not. See :meth:`WriteTransport.can_write_eof`. - - .. method:: close() - - Close the transport: see :meth:`BaseTransport.close`. - - .. method:: is_closing() - - Return ``True`` if the writer is closing or is closed. - - .. versionadded:: 3.7 - - .. coroutinemethod:: wait_closed() + Return *True* if the underlying transport supports + the :meth:`write_eof` method, *False* otherwise. - Wait until the writer is closed. - - Should be called after :meth:`close` to wait until the underlying - connection (and the associated transport/protocol pair) is closed. - - .. versionadded:: 3.7 - - .. coroutinemethod:: drain() - - Let the write buffer of the underlying transport a chance to be flushed. - - The intended use is to write:: - - w.write(data) - await w.drain() + .. method:: write_eof() - When the size of the transport buffer reaches the high-water limit (the - protocol is paused), block until the size of the buffer is drained down - to the low-water limit and the protocol is resumed. When there is nothing - to wait for, the yield-from continues immediately. + Close the write end of the stream after the buffered write + data is flushed. - Yielding from :meth:`drain` gives the opportunity for the loop to - schedule the write operation and flush the buffer. It should especially - be used when a possibly large amount of data is written to the transport, - and the coroutine does not yield-from between calls to :meth:`write`. + .. attribute:: transport - This method is a :ref:`coroutine `. + Return the underlying asyncio transport. .. method:: get_extra_info(name, default=None) - Return optional transport information: see - :meth:`BaseTransport.get_extra_info`. + Access optional transport information; see + :meth:`BaseTransport.get_extra_info` for details. .. method:: write(data) - Write some *data* bytes to the transport: see - :meth:`WriteTransport.write`. - - .. method:: writelines(data) + Write *data* to the stream. - Write a list (or any iterable) of data bytes to the transport: - see :meth:`WriteTransport.writelines`. - - .. method:: write_eof() + This method is not subject to flow control. Calls to ``write()`` should + be followed by :meth:`drain`. - Close the write end of the transport after flushing buffered data: - see :meth:`WriteTransport.write_eof`. - - -StreamReaderProtocol -==================== - -.. class:: StreamReaderProtocol(stream_reader, client_connected_cb=None, loop=None) - - Trivial helper class to adapt between :class:`Protocol` and - :class:`StreamReader`. Subclass of :class:`Protocol`. - - *stream_reader* is a :class:`StreamReader` instance, *client_connected_cb* - is an optional function called with (stream_reader, stream_writer) when a - connection is made, *loop* is the event loop instance to use. + .. method:: writelines(data) - (This is a helper class instead of making :class:`StreamReader` itself a - :class:`Protocol` subclass, because the :class:`StreamReader` has other - potential uses, and to prevent the user of the :class:`StreamReader` from - accidentally calling inappropriate methods of the protocol.) + Write a list (or any iterable) of bytes to the stream. + This method is not subject to flow control. Calls to ``writelines()`` + should be followed by :meth:`drain`. -IncompleteReadError -=================== + .. coroutinemethod:: drain() -.. exception:: IncompleteReadError + Wait until it is appropriate to resume writing to the stream. + Example:: - Incomplete read error, subclass of :exc:`EOFError`. + writer.write(data) + await writer.drain() - .. attribute:: expected + This is a flow control method that interacts with the underlying + IO write buffer. When the size of the buffer reaches + the high watermark, *drain()* blocks until the size of the + buffer is drained down to the low watermark and writing can + be resumed. When there is nothing to wait for, the :meth:`drain` + returns immediately. - Total number of expected bytes (:class:`int`). + .. method:: close() - .. attribute:: partial + Close the stream. - Read bytes string before the end of stream was reached (:class:`bytes`). + .. method:: is_closing() + Return ``True`` if the stream is closed or in the process of + being closed. -LimitOverrunError -================= + .. versionadded:: 3.7 -.. exception:: LimitOverrunError + .. coroutinemethod:: wait_closed() - Reached the buffer limit while looking for a separator. + Wait until the stream is closed. - .. attribute:: consumed + Should be called after :meth:`close` to wait until the underlying + connection is closed. - Total number of to be consumed bytes. + .. versionadded:: 3.7 -Stream examples -=============== +Examples +======== .. _asyncio-tcp-echo-client-streams: @@ -356,28 +308,26 @@ TCP echo client using the :func:`asyncio.open_connection` function:: import asyncio - async def tcp_echo_client(message, loop): - reader, writer = await asyncio.open_connection('127.0.0.1', 8888, - loop=loop) + async def tcp_echo_client(message): + reader, writer = await asyncio.open_connection( + '127.0.0.1', 8888) - print('Send: %r' % message) + print(f'Send: {message!r}') writer.write(message.encode()) data = await reader.read(100) - print('Received: %r' % data.decode()) + print(f'Received: {data.decode()!r}') - print('Close the socket') + print('Close the connection') writer.close() - message = 'Hello World!' - loop = asyncio.get_event_loop() - loop.run_until_complete(tcp_echo_client(message, loop)) - loop.close() + asyncio.run(tcp_echo_client('Hello World!')) + .. seealso:: - The :ref:`TCP echo client protocol ` - example uses the :meth:`AbstractEventLoop.create_connection` method. + The :ref:`TCP echo client protocol ` + example uses the low-level :meth:`loop.create_connection` method. .. _asyncio-tcp-echo-server-streams: @@ -393,35 +343,33 @@ TCP echo server using the :func:`asyncio.start_server` function:: data = await reader.read(100) message = data.decode() addr = writer.get_extra_info('peername') - print("Received %r from %r" % (message, addr)) - print("Send: %r" % message) + print(f"Received {message!r} from {addr!r}") + + print(f"Send: {message!r}") writer.write(data) await writer.drain() - print("Close the client socket") + print("Close the connection") writer.close() - loop = asyncio.get_event_loop() - coro = asyncio.start_server(handle_echo, '127.0.0.1', 8888, loop=loop) - server = loop.run_until_complete(coro) + async def main(): + server = await asyncio.start_server( + handle_echo, '127.0.0.1', 8888) - # Serve requests until Ctrl+C is pressed - print('Serving on {}'.format(server.sockets[0].getsockname())) - try: - loop.run_forever() - except KeyboardInterrupt: - pass + addr = server.sockets[0].getsockname() + print(f'Serving on {addr}') + + async with server: + await server.serve_forever() + + asyncio.run(main()) - # Close the server - server.close() - loop.run_until_complete(server.wait_closed()) - loop.close() .. seealso:: - The :ref:`TCP echo server protocol ` - example uses the :meth:`AbstractEventLoop.create_server` method. + The :ref:`TCP echo server protocol ` + example uses the :meth:`loop.create_server` method. Get HTTP headers @@ -436,30 +384,34 @@ Simple example querying HTTP headers of the URL passed on the command line:: async def print_http_headers(url): url = urllib.parse.urlsplit(url) if url.scheme == 'https': - connect = asyncio.open_connection(url.hostname, 443, ssl=True) + reader, writer = await asyncio.open_connection( + url.hostname, 443, ssl=True) else: - connect = asyncio.open_connection(url.hostname, 80) - reader, writer = await connect - query = ('HEAD {path} HTTP/1.0\r\n' - 'Host: {hostname}\r\n' - '\r\n').format(path=url.path or '/', hostname=url.hostname) + reader, writer = await asyncio.open_connection( + url.hostname, 80) + + query = ( + f"HEAD {url.path or '/'} HTTP/1.0\r\n" + f"Host: {url.hostname}\r\n" + f"\r\n" + ) + writer.write(query.encode('latin-1')) while True: line = await reader.readline() if not line: break + line = line.decode('latin1').rstrip() if line: - print('HTTP header> %s' % line) + print(f'HTTP header> {line}') # Ignore the body, close the socket writer.close() url = sys.argv[1] - loop = asyncio.get_event_loop() - task = asyncio.ensure_future(print_http_headers(url)) - loop.run_until_complete(task) - loop.close() + asyncio.run(print_http_headers(url)) + Usage:: @@ -469,7 +421,8 @@ or with HTTPS:: python example.py https://example.com/path/page.html -.. _asyncio-register-socket-streams: + +.. _asyncio_example_create_connection-streams: Register an open socket to wait for data using streams ------------------------------------------------------ @@ -478,14 +431,18 @@ Coroutine waiting until a socket receives data using the :func:`open_connection` function:: import asyncio - from socket import socketpair + import socket + + async def wait_for_data(): + # Get a reference to the current event loop because + # we want to access low-level APIs. + loop = asyncio.get_running_loop() - async def wait_for_data(loop): - # Create a pair of connected sockets - rsock, wsock = socketpair() + # Create a pair of connected sockets. + rsock, wsock = socket.socketpair() - # Register the open socket to wait for data - reader, writer = await asyncio.open_connection(sock=rsock, loop=loop) + # Register the open socket to wait for data. + reader, writer = await asyncio.open_connection(sock=rsock) # Simulate the reception of data from the network loop.call_soon(wsock.send, 'abc'.encode()) @@ -500,17 +457,14 @@ Coroutine waiting until a socket receives data using the # Close the second socket wsock.close() - loop = asyncio.get_event_loop() - loop.run_until_complete(wait_for_data(loop)) - loop.close() + asyncio.run(wait_for_data()) .. seealso:: The :ref:`register an open socket to wait for data using a protocol - ` example uses a low-level protocol created by the - :meth:`AbstractEventLoop.create_connection` method. + ` example uses a low-level protocol and + the :meth:`loop.create_connection` method. The :ref:`watch a file descriptor for read events - ` example uses the low-level - :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a - socket. + ` example uses the low-level + :meth:`loop.add_reader` method to watch a file descriptor. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 60e174574b04..0bcf66175ce3 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -2,245 +2,220 @@ .. _asyncio-subprocess: -Subprocess -========== +============ +Subprocesses +============ -**Source code:** :source:`Lib/asyncio/subprocess.py` +This section describes high-level async/await asyncio APIs to +create and manage subprocesses. -Windows event loop ------------------- +.. _asyncio_example_subprocess_shell: -On Windows, the default event loop is :class:`SelectorEventLoop` which does not -support subprocesses. :class:`ProactorEventLoop` should be used instead. -Example to use it on Windows:: +Here's an example of how asyncio can run a shell command and +obtain its result:: - import asyncio, sys - - if sys.platform == 'win32': - loop = asyncio.ProactorEventLoop() - asyncio.set_event_loop(loop) - -.. seealso:: - - :ref:`Available event loops ` and :ref:`Platform - support `. - - -Create a subprocess: high-level API using Process -------------------------------------------------- - -.. coroutinefunction:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) - - Create a subprocess. - - The *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_exec` for other - parameters. - - Return a :class:`~asyncio.subprocess.Process` instance. - - This function is a :ref:`coroutine `. - -.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) - - Run the shell command *cmd*. - - The *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_shell` for other - parameters. + import asyncio - Return a :class:`~asyncio.subprocess.Process` instance. + async def run(cmd): + proc = await asyncio.create_subprocess_shell( + cmd, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE) - It is the application's responsibility to ensure that all whitespace and - metacharacters are quoted appropriately to avoid `shell injection - `_ - vulnerabilities. The :func:`shlex.quote` function can be used to properly - escape whitespace and shell metacharacters in strings that are going to be - used to construct shell commands. + stdout, stderr = await proc.communicate() - This function is a :ref:`coroutine `. + print(f'[{cmd!r} exited with {proc.returncode}]') + if stdout: + print(f'[stdout]\n{stdout.decode()}') + if stderr: + print(f'[stderr]\n{stderr.decode()}') -Use the :meth:`AbstractEventLoop.connect_read_pipe` and -:meth:`AbstractEventLoop.connect_write_pipe` methods to connect pipes. + asyncio.run(run('ls /zzz')) +will print:: -Create a subprocess: low-level API using subprocess.Popen ---------------------------------------------------------- + ['ls /zzz' exited with 1] + [stderr] + ls: /zzz: No such file or directory -Run subprocesses asynchronously using the :mod:`subprocess` module. +Because all asyncio subprocess functions are asynchronous and asyncio +provides many tools to work with such functions, it is easy to execute +and monitor multiple subprocesses in parallel. It is indeed trivial +to modify the above example to run several commands simultaneously:: -.. coroutinemethod:: AbstractEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) + async def main(): + await asyncio.gather( + run('ls /zzz'), + run('sleep 1; echo "hello"')) - Create a subprocess from one or more string arguments (character strings or - bytes strings encoded to the :ref:`filesystem encoding - `), where the first string - specifies the program to execute, and the remaining strings specify the - program's arguments. (Thus, together the string arguments form the - ``sys.argv`` value of the program, assuming it is a Python script.) This is - similar to the standard library :class:`subprocess.Popen` class called with - shell=False and the list of strings passed as the first argument; - however, where :class:`~subprocess.Popen` takes a single argument which is - list of strings, :func:`subprocess_exec` takes multiple string arguments. + asyncio.run(main()) - The *protocol_factory* must instantiate a subclass of the - :class:`asyncio.SubprocessProtocol` class. +See also the `Examples`_ subsection. - Other parameters: - * *stdin*: Either a file-like object representing the pipe to be connected - to the subprocess's standard input stream using - :meth:`~AbstractEventLoop.connect_write_pipe`, or the constant - :const:`subprocess.PIPE` (the default). By default a new pipe will be - created and connected. +Creating Subprocesses +===================== - * *stdout*: Either a file-like object representing the pipe to be connected - to the subprocess's standard output stream using - :meth:`~AbstractEventLoop.connect_read_pipe`, or the constant - :const:`subprocess.PIPE` (the default). By default a new pipe will be - created and connected. +.. coroutinefunction:: create_subprocess_exec(\*args, stdin=None, \ + stdout=None, stderr=None, loop=None, \ + limit=None, \*\*kwds) - * *stderr*: Either a file-like object representing the pipe to be connected - to the subprocess's standard error stream using - :meth:`~AbstractEventLoop.connect_read_pipe`, or one of the constants - :const:`subprocess.PIPE` (the default) or :const:`subprocess.STDOUT`. - By default a new pipe will be created and connected. When - :const:`subprocess.STDOUT` is specified, the subprocess's standard error - stream will be connected to the same pipe as the standard output stream. + Create a subprocess. - * All other keyword arguments are passed to :class:`subprocess.Popen` - without interpretation, except for *bufsize*, *universal_newlines* and - *shell*, which should not be specified at all. + The *limit* argument sets the buffer limit for :class:`StreamReader` + wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` + (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). - Returns a pair of ``(transport, protocol)``, where *transport* is an - instance of :class:`BaseSubprocessTransport`. + Return a :class:`~asyncio.subprocess.Process` instance. - This method is a :ref:`coroutine `. + See the documentation of :meth:`loop.subprocess_exec` for other + parameters. - See the constructor of the :class:`subprocess.Popen` class for parameters. +.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \ + stdout=None, stderr=None, loop=None, \ + limit=None, \*\*kwds) -.. coroutinemethod:: AbstractEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) + Run the *cmd* shell command. - Create a subprocess from *cmd*, which is a character string or a bytes - string encoded to the :ref:`filesystem encoding `, - using the platform's "shell" syntax. This is similar to the standard library - :class:`subprocess.Popen` class called with ``shell=True``. + The *limit* argument sets the buffer limit for :class:`StreamReader` + wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` + (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). - The *protocol_factory* must instantiate a subclass of the - :class:`asyncio.SubprocessProtocol` class. + Return a :class:`~asyncio.subprocess.Process` instance. - See :meth:`~AbstractEventLoop.subprocess_exec` for more details about - the remaining arguments. + See the documentation of :meth:`loop.subprocess_shell` for other + parameters. - Returns a pair of ``(transport, protocol)``, where *transport* is an - instance of :class:`BaseSubprocessTransport`. +.. important:: It is the application's responsibility to ensure that all whitespace and - metacharacters are quoted appropriately to avoid `shell injection + special characters are quoted appropriately to avoid `shell injection `_ vulnerabilities. The :func:`shlex.quote` function can be used to properly - escape whitespace and shell metacharacters in strings that are going to be - used to construct shell commands. + escape whitespace and special shell characters in strings that are going + to be used to construct shell commands. - This method is a :ref:`coroutine `. +.. note:: + + The default asyncio event loop implementation on **Windows** does not + support subprocesses. Subprocesses are available for Windows if a + :class:`ProactorEventLoop` is used. + See :ref:`Subprocess Support on Windows ` + for details. .. seealso:: - The :meth:`AbstractEventLoop.connect_read_pipe` and - :meth:`AbstractEventLoop.connect_write_pipe` methods. + asyncio also has the following *low-level* APIs to work with subprocesses: + :meth:`loop.subprocess_exec`, :meth:`loop.subprocess_shell`, + :meth:`loop.connect_read_pipe`, :meth:`loop.connect_write_pipe`, + as well as the :ref:`Subprocess Transports ` + and :ref:`Subprocess Protocols `. Constants ---------- +========= .. data:: asyncio.subprocess.PIPE - Special value that can be used as the *stdin*, *stdout* or *stderr* argument - to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and - indicates that a pipe to the standard stream should be opened. + Can be passed to the *stdin*, *stdout* or *stderr* parameters. + + If *PIPE* is passed to *stdin* argument, the + :attr:`Process.stdin ` attribute + will point to a :class:`StreamWriter` instance. + + If *PIPE* is passed to *stdout* or *stderr* arguments, the + :attr:`Process.stdout ` and + :attr:`Process.stderr ` + attributes will point to :class:`StreamReader` instances. .. data:: asyncio.subprocess.STDOUT - Special value that can be used as the *stderr* argument to - :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and - indicates that standard error should go into the same handle as standard - output. + Special value that can be used as the *stderr* argument and indicates + that standard error should be redirected into standard output. .. data:: asyncio.subprocess.DEVNULL Special value that can be used as the *stdin*, *stdout* or *stderr* argument - to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and - indicates that the special file :data:`os.devnull` will be used. + to process creation functions. It indicates that the special file + :data:`os.devnull` will be used for the corresponding subprocess stream. -Process -------- +Interacting with Subprocesses +============================= + +Both :func:`create_subprocess_exec` and :func:`create_subprocess_shell` +functions return instances of the *Process* class. *Process* is a high-level +wrapper that allows communicating with subprocesses and watching for +their completion. .. class:: asyncio.subprocess.Process - A subprocess created by the :func:`create_subprocess_exec` or the - :func:`create_subprocess_shell` function. + An object that wraps OS processes created by the + :func:`create_subprocess_exec` and :func:`create_subprocess_shell` + functions. + + This class is designed to have a similar API to the + :class:`subprocess.Popen` class, but there are some + notable differences: - The API of the :class:`~asyncio.subprocess.Process` class was designed to be - close to the API of the :class:`subprocess.Popen` class, but there are some - differences: + * unlike Popen, Process instances do not have an equivalent to + the :meth:`~subprocess.Popen.poll` method; - * There is no explicit :meth:`~subprocess.Popen.poll` method - * The :meth:`~subprocess.Popen.communicate` and - :meth:`~subprocess.Popen.wait` methods don't take a *timeout* parameter: - use the :func:`wait_for` function - * The *universal_newlines* parameter is not supported (only bytes strings - are supported) - * The :meth:`~asyncio.subprocess.Process.wait` method of - the :class:`~asyncio.subprocess.Process` class is asynchronous whereas the - :meth:`~subprocess.Popen.wait` method of the :class:`~subprocess.Popen` - class is implemented as a busy loop. + * the :meth:`~asyncio.subprocess.Process.communicate` and + :meth:`~asyncio.subprocess.Process.wait` methods don't have a + *timeout* parameter: use the :func:`wait_for` function; - This class is :ref:`not thread safe `. See also the - :ref:`Subprocess and threads ` section. + * the :meth:`Process.wait() ` method + is asynchronous, whereas :meth:`subprocess.Popen.wait` method + is implemented as a blocking busy loop; + + * the *universal_newlines* parameter is not supported. + + This class is :ref:`not thread safe `. + + See also the :ref:`Subprocess and Threads ` + section. .. coroutinemethod:: wait() - Wait for child process to terminate. Set and return :attr:`returncode` - attribute. + Wait for the child process to terminate. - This method is a :ref:`coroutine `. + Set and return the :attr:`returncode` attribute. .. note:: - This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE`` and - the child process generates enough output to a pipe such that it - blocks waiting for the OS pipe buffer to accept more data. Use the - :meth:`communicate` method when using pipes to avoid that. + This method can deadlock when using ``stdout=PIPE`` or + ``stderr=PIPE`` and the child process generates so much output + that it blocks waiting for the OS pipe buffer to accept + more data. Use the :meth:`communicate` method when using pipes + to avoid this condition. .. coroutinemethod:: communicate(input=None) - Interact with process: Send data to stdin. Read data from stdout and - stderr, until end-of-file is reached. Wait for process to terminate. - The optional *input* argument should be data to be sent to the child - process, or ``None``, if no data should be sent to the child. The type - of *input* must be bytes. + Interact with process: - :meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``. + 1. send data to *stdin* (if *input* is not ``None``); + 2. read data from *stdout* and *stderr*, until EOF is reached; + 3. wait for process to terminate. - If a :exc:`BrokenPipeError` or :exc:`ConnectionResetError` exception is - raised when writing *input* into stdin, the exception is ignored. It - occurs when the process exits before all data are written into stdin. + The optional *input* argument is the data (:class:`bytes` object) + that will be sent to the child process. - Note that if you want to send data to the process's stdin, you need to - create the Process object with ``stdin=PIPE``. Similarly, to get anything - other than ``None`` in the result tuple, you need to give ``stdout=PIPE`` - and/or ``stderr=PIPE`` too. + Return a tuple ``(stdout_data, stderr_data)``. - This method is a :ref:`coroutine `. - - .. note:: + If either :exc:`BrokenPipeError` or :exc:`ConnectionResetError` + exception is raised when writing *input* into *stdin*, the + exception is ignored. This condition occurs when the process + exits before all data are written into *stdin*. - The data read is buffered in memory, so do not use this method if the - data size is large or unlimited. + If it is desired to send data to the process' *stdin*, + the process needs to be created with ``stdin=PIPE``. Similarly, + to get anything other than ``None`` in the result tuple, the + process has to be created with ``stdout=PIPE`` and/or + ``stderr=PIPE`` arguments. - .. versionchanged:: 3.4.2 - The method now ignores :exc:`BrokenPipeError` and - :exc:`ConnectionResetError`. + Note, that the data read is buffered in memory, so do not use + this method if the data size is large or unlimited. .. method:: send_signal(signal) @@ -255,67 +230,81 @@ Process .. method:: terminate() - Stop the child. On Posix OSs the method sends :py:data:`signal.SIGTERM` - to the child. On Windows the Win32 API function - :c:func:`TerminateProcess` is called to stop the child. + Stop the child process. + + On POSIX systems this method sends :py:data:`signal.SIGTERM` to the + child process. + + On Windows the Win32 API function :c:func:`TerminateProcess` is + called to stop the child process. .. method:: kill() - Kills the child. On Posix OSs the function sends :py:data:`SIGKILL` to - the child. On Windows :meth:`kill` is an alias for :meth:`terminate`. + Kill the child. + + On POSIX systems this method sends :py:data:`SIGKILL` to the child + process. + + On Windows this method is an alias for :meth:`terminate`. .. attribute:: stdin - Standard input stream (:class:`StreamWriter`), ``None`` if the process - was created with ``stdin=None``. + Standard input stream (:class:`StreamWriter`) or ``None`` + if the process was created with ``stdin=None``. .. attribute:: stdout - Standard output stream (:class:`StreamReader`), ``None`` if the process - was created with ``stdout=None``. + Standard output stream (:class:`StreamReader`) or ``None`` + if the process was created with ``stdout=None``. .. attribute:: stderr - Standard error stream (:class:`StreamReader`), ``None`` if the process - was created with ``stderr=None``. + Standard error stream (:class:`StreamReader`) or ``None`` + if the process was created with ``stderr=None``. .. warning:: - Use the :meth:`communicate` method rather than :attr:`.stdin.write - `, :attr:`.stdout.read ` or :attr:`.stderr.read ` - to avoid deadlocks due to streams pausing reading or writing and blocking - the child process. + Use the :meth:`communicate` method rather than + :attr:`process.stdin.write() `, + :attr:`await process.stdout.read() ` or + :attr:`await process.stderr.read `. + This avoids deadlocks due to streams pausing reading or writing + and blocking the child process. .. attribute:: pid - The identifier of the process. + Process identification number (PID). Note that for processes created by the :func:`create_subprocess_shell` - function, this attribute is the process identifier of the spawned shell. + function, this attribute is the PID of the spawned shell. .. attribute:: returncode - Return code of the process when it exited. A ``None`` value indicates - that the process has not terminated yet. + Return code of the process when it exits. - A negative value ``-N`` indicates that the child was terminated by signal - ``N`` (Unix only). + A ``None`` value indicates that the process has not terminated yet. + + A negative value ``-N`` indicates that the child was terminated + by signal ``N`` (POSIX only). .. _asyncio-subprocess-threads: -Subprocess and threads +Subprocess and Threads ---------------------- -asyncio supports running subprocesses from different threads, but there -are limits: +Standard asyncio event loop supports running subprocesses from +different threads, but there are limitations: + +* An event loop must run in the main thread. -* An event loop must run in the main thread -* The child watcher must be instantiated in the main thread, before executing - subprocesses from other threads. Call the :func:`get_child_watcher` - function in the main thread to instantiate the child watcher. +* The child watcher must be instantiated in the main thread + before executing subprocesses from other threads. Call the + :func:`get_child_watcher` function in the main thread to instantiate + the child watcher. -The :class:`asyncio.subprocess.Process` class is not thread safe. +Note that alternative event loop implementations might not share +the above limitations; please refer to their documentation. .. seealso:: @@ -323,97 +312,45 @@ The :class:`asyncio.subprocess.Process` class is not thread safe. ` section. -Subprocess examples -------------------- - -Subprocess using transport and protocol -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Example of a subprocess protocol using to get the output of a subprocess and to -wait for the subprocess exit. The subprocess is created by the -:meth:`AbstractEventLoop.subprocess_exec` method:: - - import asyncio - import sys - - class DateProtocol(asyncio.SubprocessProtocol): - def __init__(self, exit_future): - self.exit_future = exit_future - self.output = bytearray() +Examples +-------- - def pipe_data_received(self, fd, data): - self.output.extend(data) +An example using the :class:`~asyncio.subprocess.Process` class to +control a subprocess and the :class:`StreamReader` class to read from +its standard output. - def process_exited(self): - self.exit_future.set_result(True) +.. _asyncio_example_create_subprocess_exec: - async def get_date(loop): - code = 'import datetime; print(datetime.datetime.now())' - exit_future = asyncio.Future(loop=loop) - - # Create the subprocess controlled by the protocol DateProtocol, - # redirect the standard output into a pipe - transport, protocol = await loop.subprocess_exec( - lambda: DateProtocol(exit_future), - sys.executable, '-c', code, - stdin=None, stderr=None) - - # Wait for the subprocess exit using the process_exited() method - # of the protocol - await exit_future - - # Close the stdout pipe - transport.close() - - # Read the output which was collected by the pipe_data_received() - # method of the protocol - data = bytes(protocol.output) - return data.decode('ascii').rstrip() - - if sys.platform == "win32": - loop = asyncio.ProactorEventLoop() - asyncio.set_event_loop(loop) - else: - loop = asyncio.get_event_loop() - - date = loop.run_until_complete(get_date(loop)) - print("Current date: %s" % date) - loop.close() - - -Subprocess using streams -^^^^^^^^^^^^^^^^^^^^^^^^ - -Example using the :class:`~asyncio.subprocess.Process` class to control the -subprocess and the :class:`StreamReader` class to read from the standard -output. The subprocess is created by the :func:`create_subprocess_exec` +The subprocess is created by the :func:`create_subprocess_exec` function:: - import asyncio.subprocess + import asyncio import sys async def get_date(): code = 'import datetime; print(datetime.datetime.now())' - # Create the subprocess, redirect the standard output into a pipe + # Create the subprocess; redirect the standard output + # into a pipe. proc = await asyncio.create_subprocess_exec( sys.executable, '-c', code, stdout=asyncio.subprocess.PIPE) - # Read one line of output + # Read one line of output. data = await proc.stdout.readline() line = data.decode('ascii').rstrip() - # Wait for the subprocess exit + # Wait for the subprocess exit. await proc.wait() return line if sys.platform == "win32": - loop = asyncio.ProactorEventLoop() - asyncio.set_event_loop(loop) - else: - loop = asyncio.get_event_loop() - - date = loop.run_until_complete(get_date()) - print("Current date: %s" % date) - loop.close() + asyncio.set_event_loop_policy( + asyncio.WindowsProactorEventLoopPolicy()) + + date = asyncio.run(get_date()) + print(f"Current date: {date}") + + +See also the :ref:`same example ` +written using low-level APIs. diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index 574f70f069b3..18b562970436 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -1,172 +1,206 @@ .. currentmodule:: asyncio + .. _asyncio-sync: -Synchronization primitives ========================== +Synchronization Primitives +========================== + +asyncio synchronization primitives are designed to be similar to +those of the :mod:`threading` module with two important caveats: + +* asyncio primitives are not thread-safe, therefore they should not + be used for OS thread synchronization (use :mod:`threading` for + that); -**Source code:** :source:`Lib/asyncio/locks.py` +* methods of these synchronization primitives do not accept the *timeout* + argument; use the :func:`asyncio.wait_for` function to perform + operations with timeouts. -Locks: +asyncio has the following basic sychronization primitives: * :class:`Lock` * :class:`Event` * :class:`Condition` - -Semaphores: - * :class:`Semaphore` * :class:`BoundedSemaphore` -asyncio lock API was designed to be close to classes of the :mod:`threading` -module (:class:`~threading.Lock`, :class:`~threading.Event`, -:class:`~threading.Condition`, :class:`~threading.Semaphore`, -:class:`~threading.BoundedSemaphore`), but it has no *timeout* parameter. The -:func:`asyncio.wait_for` function can be used to cancel a task after a timeout. + +--------- Lock ----- +==== .. class:: Lock(\*, loop=None) - Primitive lock objects. + Implements a mutex lock for asyncio tasks. Not thread-safe. - A primitive lock is a synchronization primitive that is not owned by a - particular coroutine when locked. A primitive lock is in one of two states, - 'locked' or 'unlocked'. + An asyncio lock can be used to guarantee exclusive access to a + shared resource. - The lock is created in the unlocked state. - It has two basic methods, :meth:`acquire` and :meth:`release`. - When the state is unlocked, acquire() changes the state to - locked and returns immediately. When the state is locked, acquire() blocks - until a call to release() in another coroutine changes it to unlocked, then - the acquire() call resets it to locked and returns. The release() method - should only be called in the locked state; it changes the state to unlocked - and returns immediately. If an attempt is made to release an unlocked lock, - a :exc:`RuntimeError` will be raised. + The preferred way to use a Lock is an :keyword:`async with` + statement:: - When more than one coroutine is blocked in acquire() waiting for the state - to turn to unlocked, only one coroutine proceeds when a release() call - resets the state to unlocked; first coroutine which is blocked in acquire() - is being processed. + lock = asyncio.Lock() - :meth:`acquire` is a coroutine and should be called with ``await``. + # ... later + async with lock: + # access shared state - Locks support the :ref:`context management protocol `. + which is equivalent to:: - This class is :ref:`not thread safe `. + lock = asyncio.Lock() - .. method:: locked() - - Return ``True`` if the lock is acquired. + # ... later + await lock.acquire() + try: + # access shared state + finally: + lock.release() .. coroutinemethod:: acquire() - Acquire a lock. - - This method blocks until the lock is unlocked, then sets it to locked and - returns ``True``. + Acquire the lock. - This method is a :ref:`coroutine `. + This method waits until the lock is *unlocked*, sets it to + *locked* and returns ``True``. .. method:: release() - Release a lock. + Release the lock. - When the lock is locked, reset it to unlocked, and return. If any other - coroutines are blocked waiting for the lock to become unlocked, allow - exactly one of them to proceed. + When the lock is *locked*, reset it to *unlocked* and return. - When invoked on an unlocked lock, a :exc:`RuntimeError` is raised. + If the lock is *unlocked*, a :exc:`RuntimeError` is raised. - There is no return value. + .. method:: locked() + + Return ``True`` if the lock is *locked*. Event ------ +===== .. class:: Event(\*, loop=None) - An Event implementation, asynchronous equivalent to :class:`threading.Event`. + An event object. Not thread-safe. - Class implementing event objects. An event manages a flag that can be set to - true with the :meth:`set` method and reset to false with the :meth:`clear` - method. The :meth:`wait` method blocks until the flag is true. The flag is - initially false. + An asyncio event can be used to notify multiple asyncio tasks + that some event has happened. - This class is :ref:`not thread safe `. + An Event object manages an internal flag that can be set to *true* + with the :meth:`set` method and reset to *false* with the + :meth:`clear` method. The :meth:`wait` method blocks until the + flag is set to *true*. The flag is set to *false* initially. - .. method:: clear() + .. _asyncio_example_sync_event: - Reset the internal flag to false. Subsequently, coroutines calling - :meth:`wait` will block until :meth:`set` is called to set the internal - flag to true again. + Example:: - .. method:: is_set() + async def waiter(event): + print('waiting for it ...') + await event.wait() + print('... got it!') - Return ``True`` if and only if the internal flag is true. + async def main(): + # Create an Event object. + event = asyncio.Event() - .. method:: set() + # Spawn a Task to wait until 'event' is set. + waiter_task = asyncio.create_task(waiter(event)) + + # Sleep for 1 second and set the event. + await asyncio.sleep(1) + event.set() - Set the internal flag to true. All coroutines waiting for it to become - true are awakened. Coroutine that call :meth:`wait` once the flag is true - will not block at all. + # Wait until the waiter task is finished. + await waiter_task + + asyncio.run(main()) .. coroutinemethod:: wait() - Block until the internal flag is true. + Wait until the event is set. + + If the event is set, return ``True`` immediately. + Otherwise block until another task calls :meth:`set`. - If the internal flag is true on entry, return ``True`` immediately. - Otherwise, block until another coroutine calls :meth:`set` to set the - flag to true, then return ``True``. + .. method:: set() + + Set the event. - This method is a :ref:`coroutine `. + All tasks waiting for event to be set will be immediately + awakened. + + .. method:: clear() + + Clear (unset) the event. + + Tasks awaiting on :meth:`wait` will now block until the + :meth:`set` method is called again. + + .. method:: is_set() + + Return ``True`` if the event is set. Condition ---------- +========= .. class:: Condition(lock=None, \*, loop=None) - A Condition implementation, asynchronous equivalent to - :class:`threading.Condition`. + A Condition object. Not thread-safe. - This class implements condition variable objects. A condition variable - allows one or more coroutines to wait until they are notified by another - coroutine. + An asyncio condition primitive can be used by a task to wait for + some event to happen and then get exclusive access to a shared + resource. - If the *lock* argument is given and not ``None``, it must be a :class:`Lock` - object, and it is used as the underlying lock. Otherwise, - a new :class:`Lock` object is created and used as the underlying lock. + In essence, a Condition object combines the functionality + of an :class:`Event` and a :class:`Lock`. It is possible to have + multiple Condition objects share one Lock, which allows coordinating + exclusive access to a shared resource between different tasks + interested in particular states of that shared resource. - Conditions support the :ref:`context management protocol - `. + The optional *lock* argument must be a :class:`Lock` object or + ``None``. In the latter case a new Lock object is created + automatically. - This class is :ref:`not thread safe `. + The preferred way to use a Condition is an :keyword:`async with` + statement:: - .. coroutinemethod:: acquire() + cond = asyncio.Condition() - Acquire the underlying lock. + # ... later + async with cond: + await cond.wait() - This method blocks until the lock is unlocked, then sets it to locked and - returns ``True``. + which is equivalent to:: - This method is a :ref:`coroutine `. + cond = asyncio.Condition() - .. method:: notify(n=1) + # ... later + await lock.acquire() + try: + await cond.wait() + finally: + lock.release() - By default, wake up one coroutine waiting on this condition, if any. - If the calling coroutine has not acquired the lock when this method is - called, a :exc:`RuntimeError` is raised. + .. coroutinemethod:: acquire() + + Acquire the underlying lock. + + This method waits until the underlying lock is *unlocked*, + sets it to *locked* and returns ``True``. - This method wakes up at most *n* of the coroutines waiting for the - condition variable; it is a no-op if no coroutines are waiting. + .. method:: notify(n=1) - .. note:: + Wake up at most *n* tasks (1 by default) waiting on this + condition. The method is no-op if no tasks are waiting. - An awakened coroutine does not actually return from its :meth:`wait` - call until it can reacquire the lock. Since :meth:`notify` does not - release the lock, its caller should. + The lock must be acquired before this method is called and + released shortly after. If called with an *unlocked* lock + a :exc:`RuntimeError` error is raised. .. method:: locked() @@ -174,78 +208,87 @@ Condition .. method:: notify_all() - Wake up all coroutines waiting on this condition. This method acts like - :meth:`notify`, but wakes up all waiting coroutines instead of one. If the - calling coroutine has not acquired the lock when this method is called, a - :exc:`RuntimeError` is raised. + Wake up all tasks waiting on this condition. - .. method:: release() + This method acts like :meth:`notify`, but wakes up all waiting + tasks. - Release the underlying lock. + The lock must be acquired before this method is called and + released shortly after. If called with an *unlocked* lock + a :exc:`RuntimeError` error is raised. - When the lock is locked, reset it to unlocked, and return. If any other - coroutines are blocked waiting for the lock to become unlocked, allow - exactly one of them to proceed. + .. method:: release() - When invoked on an unlocked lock, a :exc:`RuntimeError` is raised. + Release the underlying lock. - There is no return value. + When invoked on an unlocked lock, a :exc:`RuntimeError` is + raised. .. coroutinemethod:: wait() Wait until notified. - If the calling coroutine has not acquired the lock when this method is + If the calling task has not acquired the lock when this method is called, a :exc:`RuntimeError` is raised. - This method releases the underlying lock, and then blocks until it is - awakened by a :meth:`notify` or :meth:`notify_all` call for the same - condition variable in another coroutine. Once awakened, it re-acquires - the lock and returns ``True``. - - This method is a :ref:`coroutine `. + This method releases the underlying lock, and then blocks until + it is awakened by a :meth:`notify` or :meth:`notify_all` call. + Once awakened, the Condition re-acquires its lock and this method + returns ``True``. .. coroutinemethod:: wait_for(predicate) - Wait until a predicate becomes true. - - The predicate should be a callable which result will be interpreted as a - boolean value. The final predicate value is the return value. + Wait until a predicate becomes *true*. - This method is a :ref:`coroutine `. + The predicate must be a callable which result will be + interpreted as a boolean value. The final value is the + return value. Semaphore ---------- +========= .. class:: Semaphore(value=1, \*, loop=None) - A Semaphore implementation. + A Semaphore object. Not thread-safe. A semaphore manages an internal counter which is decremented by each - :meth:`acquire` call and incremented by each :meth:`release` call. The - counter can never go below zero; when :meth:`acquire` finds that it is zero, - it blocks, waiting until some other coroutine calls :meth:`release`. + :meth:`acquire` call and incremented by each :meth:`release` call. + The counter can never go below zero; when :meth:`acquire` finds + that it is zero, it blocks, waiting until some task calls + :meth:`release`. + + The optional *value* argument gives the initial value for the + internal counter (``1`` by default). If the given value is + less than ``0`` a :exc:`ValueError` is raised. + + The preferred way to use a Semaphore is an :keyword:`async with` + statement:: + + sem = asyncio.Semaphore(10) - The optional argument gives the initial value for the internal counter; it - defaults to ``1``. If the value given is less than ``0``, :exc:`ValueError` - is raised. + # ... later + async with sem: + # work with shared resource - Semaphores support the :ref:`context management protocol - `. + which is equivalent to:: - This class is :ref:`not thread safe `. + sem = asyncio.Semaphore(10) + + # ... later + await sem.acquire() + try: + # work with shared resource + finally: + sem.release() .. coroutinemethod:: acquire() Acquire a semaphore. - If the internal counter is larger than zero on entry, decrement it by one - and return ``True`` immediately. If it is zero on entry, block, waiting - until some other coroutine has called :meth:`release` to make it larger - than ``0``, and then return ``True``. - - This method is a :ref:`coroutine `. + If the internal counter is greater than zero, decrement + it by one and return ``True`` immediately. If it is zero, wait + until a :meth:`release` is called and return ``True``. .. method:: locked() @@ -253,53 +296,30 @@ Semaphore .. method:: release() - Release a semaphore, incrementing the internal counter by one. When it - was zero on entry and another coroutine is waiting for it to become - larger than zero again, wake up that coroutine. + Release a semaphore, incrementing the internal counter by one. + Can wake up a task waiting to acquire the semaphore. + + Unlike :class:`BoundedSemaphore`, :class:`Semaphore` allows + making more ``release()`` calls than ``acquire()`` calls. BoundedSemaphore ----------------- +================ .. class:: BoundedSemaphore(value=1, \*, loop=None) - A bounded semaphore implementation. Inherit from :class:`Semaphore`. - - This raises :exc:`ValueError` in :meth:`~Semaphore.release` if it would - increase the value above the initial value. + A bounded semaphore object. Not thread-safe. - Bounded semaphores support the :ref:`context management - protocol `. + Bounded Semaphore is a version of :class:`Semaphore` that raises + a :exc:`ValueError` in :meth:`~Semaphore.release` if it + increases the internal counter above the initial *value*. - This class is :ref:`not thread safe `. +--------- -.. _async-with-locks: - -Using locks, conditions and semaphores in the :keyword:`async with` statement ------------------------------------------------------------------------------ - -:class:`Lock`, :class:`Condition`, :class:`Semaphore`, and -:class:`BoundedSemaphore` objects can be used in :keyword:`async with` -statements. - -The :meth:`acquire` method will be called when the block is entered, -and :meth:`release` will be called when the block is exited. Hence, -the following snippet:: - - async with lock: - # do something... - -is equivalent to:: - - await lock.acquire() - try: - # do something... - finally: - lock.release() .. deprecated:: 3.7 - Lock acquiring using ``await lock`` or ``yield from lock`` and + Acquiring a lock using ``await lock`` or ``yield from lock`` and/or :keyword:`with` statement (``with await lock``, ``with (yield from - lock)``) are deprecated. + lock)``) is deprecated. Use ``async with lock`` instead. diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 2b480d4be3fb..670e4a5fbe04 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -1,97 +1,119 @@ .. currentmodule:: asyncio -Tasks and coroutines + +==================== +Coroutines and Tasks ==================== -**Source code:** :source:`Lib/asyncio/tasks.py` +This section outlines high-level asyncio APIs to work with coroutines +and Tasks. + +.. contents:: + :depth: 1 + :local: -**Source code:** :source:`Lib/asyncio/coroutines.py` .. _coroutine: Coroutines ----------- +========== -Coroutines used with :mod:`asyncio` may be implemented using the -:keyword:`async def` statement, or by using :term:`generators `. -The :keyword:`async def` type of coroutine was added in Python 3.5, and -is recommended if there is no need to support older Python versions. +Coroutines declared with async/await syntax is the preferred way of +writing asyncio applications. For example, the following snippet +of code prints "hello", waits 1 second, and then prints "world":: -Generator-based coroutines should be decorated with :func:`@asyncio.coroutine -`, although this is not strictly enforced. -The decorator enables compatibility with :keyword:`async def` coroutines, -and also serves as documentation. Generator-based -coroutines use the ``yield from`` syntax introduced in :pep:`380`, -instead of the original ``yield`` syntax. + >>> import asyncio -The word "coroutine", like the word "generator", is used for two -different (though related) concepts: + >>> async def main(): + ... print('hello') + ... await asyncio.sleep(1) + ... print('world') -- The function that defines a coroutine - (a function definition using :keyword:`async def` or - decorated with ``@asyncio.coroutine``). If disambiguation is needed - we will call this a *coroutine function* (:func:`iscoroutinefunction` - returns ``True``). + >>> asyncio.run(main()) + hello + world -- The object obtained by calling a coroutine function. This object - represents a computation or an I/O operation (usually a combination) - that will complete eventually. If disambiguation is needed we will - call it a *coroutine object* (:func:`iscoroutine` returns ``True``). +Note that simply calling a coroutine will not schedule it to +be executed:: -Things a coroutine can do: + >>> main() + -- ``result = await future`` or ``result = yield from future`` -- - suspends the coroutine until the - future is done, then returns the future's result, or raises an - exception, which will be propagated. (If the future is cancelled, - it will raise a ``CancelledError`` exception.) Note that tasks are - futures, and everything said about futures also applies to tasks. +To actually run a coroutine asyncio provides three main mechanisms: -- ``result = await coroutine`` or ``result = yield from coroutine`` -- - wait for another coroutine to - produce a result (or raise an exception, which will be propagated). - The ``coroutine`` expression must be a *call* to another coroutine. +* The :func:`asyncio.run` function to run the top-level + entry point "main()" function (see the above example.) -- ``return expression`` -- produce a result to the coroutine that is - waiting for this one using :keyword:`await` or ``yield from``. +* Awaiting on a coroutine. The following snippet of code will + print "hello" after waiting for 1 second, and then print "world" + after waiting for *another* 2 seconds:: -- ``raise exception`` -- raise an exception in the coroutine that is - waiting for this one using :keyword:`await` or ``yield from``. + import asyncio + import time -Calling a coroutine does not start its code running -- -the coroutine object returned by the call doesn't do anything until you -schedule its execution. There are two basic ways to start it running: -call ``await coroutine`` or ``yield from coroutine`` from another coroutine -(assuming the other coroutine is already running!), or schedule its execution -using the :func:`ensure_future` function or the :meth:`AbstractEventLoop.create_task` -method. + async def say_after(delay, what): + await asyncio.sleep(delay) + print(what) + async def main(): + print('started at', time.strftime('%X')) -Coroutines (and tasks) can only run when the event loop is running. + await say_after(1, 'hello') + await say_after(2, 'world') -.. decorator:: coroutine + print('finished at', time.strftime('%X')) - Decorator to mark generator-based coroutines. This enables - the generator use :keyword:`!yield from` to call :keyword:`async - def` coroutines, and also enables the generator to be called by - :keyword:`async def` coroutines, for instance using an - :keyword:`await` expression. + asyncio.run(main()) - There is no need to decorate :keyword:`async def` coroutines themselves. + Expected output:: - If the generator is not yielded from before it is destroyed, an error - message is logged. See :ref:`Detect coroutines never scheduled - `. + started at 17:13:52 + hello + world + finished at 17:13:55 -.. note:: +* The :func:`asyncio.create_task` function to run coroutines + concurrently as asyncio :class:`Tasks `. + + Let's modify the above example and run two "set_after" coroutines + *concurrently*:: + + async def main(): + task1 = asyncio.create_task( + say_after(1, 'hello')) + + task2 = asyncio.create_task( + say_after(2, 'world')) + + print('started at', time.strftime('%X')) + + # Wait until both tasks are completed (should take + # around 2 seconds.) + await task1 + await task2 + + print('finished at', time.strftime('%X')) + + Note that expected output now shows that the snippet runs + 1 second faster than before:: + + started at 17:14:32 + hello + world + finished at 17:14:34 + +Note that in this documentation the term "coroutine" can be used for +two closely related concepts: + +* a *coroutine function*: an :keyword:`async def` function; - In this documentation, some methods are documented as coroutines, - even if they are plain Python functions returning a :class:`Future`. - This is intentional to have a freedom of tweaking the implementation - of these functions in the future. If such a function is needed to be - used in a callback-style code, wrap its result with :func:`ensure_future`. +* a *coroutine object*: object returned by calling a + *coroutine function*. +Running an asyncio Program +========================== + .. function:: run(coro, \*, debug=False) This function runs the passed coroutine, taking care of @@ -101,45 +123,46 @@ Coroutines (and tasks) can only run when the event loop is running. This function cannot be called when another asyncio event loop is running in the same thread. - If debug is True, the event loop will be run in debug mode. + If *debug* is ``True``, the event loop will be run in debug mode. This function always creates a new event loop and closes it at the end. It should be used as a main entry point for asyncio programs, and should ideally only be called once. .. versionadded:: 3.7 - **Important:** this has been been added to asyncio in Python 3.7 - on a :term:`provisional basis `. + **Important:** this function has been added to asyncio in + Python 3.7 on a :term:`provisional basis `. -.. _asyncio-hello-world-coroutine: +Creating Tasks +============== -Example: Hello World coroutine -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. function:: create_task(coro) -Example of coroutine displaying ``"Hello World"``:: + Wrap the *coro* :ref:`coroutine ` into a task and schedule + its execution. Return the task object. - import asyncio + The task is executed in the loop returned by :func:`get_running_loop`, + :exc:`RuntimeError` is raised if there is no running loop in + current thread. - async def hello_world(): - print("Hello World!") + .. versionadded:: 3.7 - asyncio.run(hello_world()) -.. seealso:: +Sleeping +======== - The :ref:`Hello World with call_soon() ` - example uses the :meth:`AbstractEventLoop.call_soon` method to schedule a - callback. +.. coroutinefunction:: sleep(delay, result=None, \*, loop=None) + Block for *delay* seconds. -.. _asyncio-date-coroutine: + If *result* is provided, it is returned to the caller + when the coroutine completes. -Example: Coroutine displaying the current date -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + .. _asyncio_example_sleep: -Example of coroutine displaying the current date every second during 5 seconds -using the :meth:`sleep` function:: + Example of coroutine displaying the current date every second + for 5 seconds:: import asyncio import datetime @@ -155,408 +178,283 @@ using the :meth:`sleep` function:: asyncio.run(display_date()) -.. seealso:: - - The :ref:`display the current date with call_later() - ` example uses a callback with the - :meth:`AbstractEventLoop.call_later` method. - - -Example: Chain coroutines -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Example chaining coroutines:: - - import asyncio - - async def compute(x, y): - print("Compute %s + %s ..." % (x, y)) - await asyncio.sleep(1.0) - return x + y - - async def print_sum(x, y): - result = await compute(x, y) - print("%s + %s = %s" % (x, y, result)) - - loop = asyncio.get_event_loop() - loop.run_until_complete(print_sum(1, 2)) - loop.close() - -``compute()`` is chained to ``print_sum()``: ``print_sum()`` coroutine waits -until ``compute()`` is completed before returning its result. - -Sequence diagram of the example: - -.. image:: tulip_coro.png - :align: center - -The "Task" is created by the :meth:`AbstractEventLoop.run_until_complete` method -when it gets a coroutine object instead of a task. - -The diagram shows the control flow, it does not describe exactly how things -work internally. For example, the sleep coroutine creates an internal future -which uses :meth:`AbstractEventLoop.call_later` to wake up the task in 1 second. - - -InvalidStateError ------------------ - -.. exception:: InvalidStateError - - The operation is not allowed in this state. - - -TimeoutError ------------- - -.. exception:: TimeoutError - - The operation exceeded the given deadline. - -.. note:: - - This exception is different from the builtin :exc:`TimeoutError` exception! - - -Future ------- - -.. class:: Future(\*, loop=None) - - This class is *almost* compatible with :class:`concurrent.futures.Future`. - - Differences: - - - :meth:`result` and :meth:`exception` do not take a timeout argument and - raise an exception when the future isn't done yet. - - - Callbacks registered with :meth:`add_done_callback` are always called - via the event loop's :meth:`~AbstractEventLoop.call_soon`. - - - This class is not compatible with the :func:`~concurrent.futures.wait` and - :func:`~concurrent.futures.as_completed` functions in the - :mod:`concurrent.futures` package. - - This class is :ref:`not thread safe `. - - .. method:: cancel() - - Cancel the future and schedule callbacks. - - If the future is already done or cancelled, return ``False``. Otherwise, - change the future's state to cancelled, schedule the callbacks and return - ``True``. - - .. method:: cancelled() - - Return ``True`` if the future was cancelled. - - .. method:: done() - - Return ``True`` if the future is done. - - Done means either that a result / exception are available, or that the - future was cancelled. - - .. method:: result() - - Return the result this future represents. - - If the future has been cancelled, raises :exc:`CancelledError`. If the - future's result isn't yet available, raises :exc:`InvalidStateError`. If - the future is done and has an exception set, this exception is raised. - - .. method:: exception() - - Return the exception that was set on this future. - - The exception (or ``None`` if no exception was set) is returned only if - the future is done. If the future has been cancelled, raises - :exc:`CancelledError`. If the future isn't done yet, raises - :exc:`InvalidStateError`. - .. method:: add_done_callback(callback, *, context=None) +Running Tasks Concurrently +========================== - Add a callback to be run when the future becomes done. +.. function:: gather(\*fs, loop=None, return_exceptions=False) - The *callback* is called with a single argument - the future object. If the - future is already done when this is called, the callback is scheduled - with :meth:`~AbstractEventLoop.call_soon`. + Return a Future aggregating results from the given coroutine objects, + Tasks, or Futures. - An optional keyword-only *context* argument allows specifying a custom - :class:`contextvars.Context` for the *callback* to run in. The current - context is used when no *context* is provided. + If all Tasks/Futures are completed successfully, the result is an + aggregate list of returned values. The result values are in the + order of the original *fs* sequence. - :ref:`Use functools.partial to pass parameters to the callback - `. For example, - ``fut.add_done_callback(functools.partial(print, "Future:", - flush=True))`` will call ``print("Future:", fut, flush=True)``. + All coroutines in the *fs* list are automatically + scheduled as :class:`Tasks `. - .. versionchanged:: 3.7 - The *context* keyword-only parameter was added. See :pep:`567` - for more details. + If *return_exceptions* is ``True``, exceptions in the Tasks/Futures + are treated the same as successful results, and gathered in the + result list. Otherwise, the first raised exception is immediately + propagated to the returned Future. - .. method:: remove_done_callback(fn) + If the outer Future is *cancelled*, all submitted Tasks/Futures + (that have not completed yet) are also *cancelled*. - Remove all instances of a callback from the "call when done" list. + If any child is *cancelled*, it is treated as if it raised + :exc:`CancelledError` -- the outer Future is **not** cancelled in + this case. This is to prevent the cancellation of one submitted + Task/Future to cause other Tasks/Futures to be cancelled. - Returns the number of callbacks removed. + All futures must share the same event loop. - .. method:: set_result(result) + .. versionchanged:: 3.7 + If the *gather* itself is cancelled, the cancellation is + propagated regardless of *return_exceptions*. - Mark the future done and set its result. + .. _asyncio_example_gather: - If the future is already done when this method is called, raises - :exc:`InvalidStateError`. + Example:: - .. method:: set_exception(exception) + import asyncio - Mark the future done and set an exception. + async def factorial(name, number): + f = 1 + for i in range(2, number + 1): + print(f"Task {name}: Compute factorial({i})...") + await asyncio.sleep(1) + f *= i + print(f"Task {name}: factorial({number}) = {f}") - If the future is already done when this method is called, raises - :exc:`InvalidStateError`. + async def main(): + await asyncio.gather( + factorial("A", 2), + factorial("B", 3), + factorial("C", 4), + )) - .. method:: get_loop() + asyncio.run(main()) - Return the event loop the future object is bound to. + # Expected output: + # + # Task A: Compute factorial(2)... + # Task B: Compute factorial(2)... + # Task C: Compute factorial(2)... + # Task A: factorial(2) = 2 + # Task B: Compute factorial(3)... + # Task C: Compute factorial(3)... + # Task B: factorial(3) = 6 + # Task C: Compute factorial(4)... + # Task C: factorial(4) = 24 - .. versionadded:: 3.7 +Shielding Tasks From Cancellation +================================= -Example: Future with run_until_complete() -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. coroutinefunction:: shield(fut, \*, loop=None) -Example combining a :class:`Future` and a :ref:`coroutine function -`:: + Wait for a Future/Task while protecting it from being cancelled. - import asyncio + *fut* can be a coroutine, a Task, or a Future-like object. If + *fut* is a coroutine it is automatically scheduled as a + :class:`Task`. - async def slow_operation(future): - await asyncio.sleep(1) - future.set_result('Future is done!') + The statement:: - loop = asyncio.get_event_loop() - future = asyncio.Future() - asyncio.ensure_future(slow_operation(future)) - loop.run_until_complete(future) - print(future.result()) - loop.close() + res = await shield(something()) -The coroutine function is responsible for the computation (which takes 1 second) -and it stores the result into the future. The -:meth:`~AbstractEventLoop.run_until_complete` method waits for the completion of -the future. + is equivalent to:: -.. note:: - The :meth:`~AbstractEventLoop.run_until_complete` method uses internally the - :meth:`~Future.add_done_callback` method to be notified when the future is - done. + res = await something() + *except* that if the coroutine containing it is cancelled, the + Task running in ``something()`` is not cancelled. From the point + of view of ``something()``, the cancellation did not happen. + Although its caller is still cancelled, so the "await" expression + still raises a :exc:`CancelledError`. -Example: Future with run_forever() -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + If ``something()`` is cancelled by other means (i.e. from within + itself) that would also cancel ``shield()``. -The previous example can be written differently using the -:meth:`Future.add_done_callback` method to describe explicitly the control -flow:: + If it is desired to completely ignore cancellation (not recommended) + the ``shield()`` function should be combined with a try/except + clause, as follows:: - import asyncio - - async def slow_operation(future): - await asyncio.sleep(1) - future.set_result('Future is done!') + try: + res = await shield(something()) + except CancelledError: + res = None - def got_result(future): - print(future.result()) - loop.stop() - loop = asyncio.get_event_loop() - future = asyncio.Future() - asyncio.ensure_future(slow_operation(future)) - future.add_done_callback(got_result) - try: - loop.run_forever() - finally: - loop.close() +Timeouts +======== -In this example, the future is used to link ``slow_operation()`` to -``got_result()``: when ``slow_operation()`` is done, ``got_result()`` is called -with the result. +.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) + Wait for a coroutine, Task, or Future to complete with timeout. -Task ----- + *fut* can be a coroutine, a Task, or a Future-like object. If + *fut* is a coroutine it is automatically scheduled as a + :class:`Task`. -.. function:: create_task(coro) + *timeout* can either be ``None`` or a float or int number of seconds + to wait for. If *timeout* is ``None``, block until the future + completes. - Wrap a :ref:`coroutine ` *coro* into a task and schedule - its execution. Return the task object. + If a timeout occurs, it cancels the task and raises + :exc:`asyncio.TimeoutError`. - The task is executed in :func:`get_running_loop` context, - :exc:`RuntimeError` is raised if there is no running loop in - current thread. + To avoid the task cancellation, wrap it in :func:`shield`. - .. versionadded:: 3.7 + The function will wait until the future is actually cancelled, + so the total wait time may exceed the *timeout*. -.. class:: Task(coro, \*, loop=None) + If the wait is cancelled, the future *fut* is also cancelled. - A unit for concurrent running of :ref:`coroutines `, - subclass of :class:`Future`. + .. _asyncio_example_waitfor: - A task is responsible for executing a coroutine object in an event loop. If - the wrapped coroutine yields from a future, the task suspends the execution - of the wrapped coroutine and waits for the completion of the future. When - the future is done, the execution of the wrapped coroutine restarts with the - result or the exception of the future. + Example:: - Event loops use cooperative scheduling: an event loop only runs one task at - a time. Other tasks may run in parallel if other event loops are - running in different threads. While a task waits for the completion of a - future, the event loop executes a new task. + async def eternity(): + # Sleep for one hour + await asyncio.sleep(3600) + print('yay!') - The cancellation of a task is different from the cancellation of a - future. Calling :meth:`cancel` will throw a - :exc:`~concurrent.futures.CancelledError` to the wrapped - coroutine. :meth:`~Future.cancelled` only returns ``True`` if the - wrapped coroutine did not catch the - :exc:`~concurrent.futures.CancelledError` exception, or raised a - :exc:`~concurrent.futures.CancelledError` exception. + async def main(): + # Wait for at most 1 second + try: + await asyncio.wait_for(eternity(), timeout=1.0) + except asyncio.TimeoutError: + print('timeout!') - If a pending task is destroyed, the execution of its wrapped :ref:`coroutine - ` did not complete. It is probably a bug and a warning is - logged: see :ref:`Pending task destroyed `. + asyncio.run(main()) - Don't directly create :class:`Task` instances: use the :func:`create_task` - function or the :meth:`AbstractEventLoop.create_task` method. + # Expected output: + # + # timeout! - Tasks support the :mod:`contextvars` module. When a Task - is created it copies the current context and later runs its coroutine - in the copied context. See :pep:`567` for more details. + .. versionchanged:: 3.7 + When *fut* is cancelled due to a timeout, ``wait_for`` waits + for *fut* to be cancelled. Previously, it raised + :exc:`asyncio.TimeoutError` immediately. - This class is :ref:`not thread safe `. - .. versionchanged:: 3.7 - Added support for the :mod:`contextvars` module. +Waiting Primitives +================== - .. classmethod:: all_tasks(loop=None) +.. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\ + return_when=ALL_COMPLETED) - Return a set of all tasks for an event loop. + Wait for a set of coroutines, Tasks, or Futures to complete. - By default all tasks for the current event loop are returned. - If *loop* is ``None``, :func:`get_event_loop` function - is used to get the current loop. + *fs* is a list of coroutines, Futures, and/or Tasks. Coroutines + are automatically scheduled as :class:`Tasks `. - .. classmethod:: current_task(loop=None) + Returns two sets of Tasks/Futures: ``(done, pending)``. - Return the currently running task in an event loop or ``None``. + *timeout* (a float or int), if specified, can be used to control + the maximum number of seconds to wait before returning. - By default the current task for the current event loop is returned. + Note that this function does not raise :exc:`asyncio.TimeoutError`. + Futures or Tasks that aren't done when the timeout occurs are simply + returned in the second set. - ``None`` is returned when called not in the context of a :class:`Task`. + *return_when* indicates when this function should return. It must + be one of the following constants: - .. method:: cancel() + .. tabularcolumns:: |l|L| - Request that this task cancel itself. + +-----------------------------+----------------------------------------+ + | Constant | Description | + +=============================+========================================+ + | :const:`FIRST_COMPLETED` | The function will return when any | + | | future finishes or is cancelled. | + +-----------------------------+----------------------------------------+ + | :const:`FIRST_EXCEPTION` | The function will return when any | + | | future finishes by raising an | + | | exception. If no future raises an | + | | exception then it is equivalent to | + | | :const:`ALL_COMPLETED`. | + +-----------------------------+----------------------------------------+ + | :const:`ALL_COMPLETED` | The function will return when all | + | | futures finish or are cancelled. | + +-----------------------------+----------------------------------------+ - This arranges for a :exc:`~concurrent.futures.CancelledError` to be - thrown into the wrapped coroutine on the next cycle through the event - loop. The coroutine then has a chance to clean up or even deny the - request using try/except/finally. + Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the + futures when a timeout occurs. - Unlike :meth:`Future.cancel`, this does not guarantee that the task - will be cancelled: the exception might be caught and acted upon, delaying - cancellation of the task or preventing cancellation completely. The task - may also return a value or raise a different exception. + Usage:: - Immediately after this method is called, :meth:`~Future.cancelled` will - not return ``True`` (unless the task was already cancelled). A task will - be marked as cancelled when the wrapped coroutine terminates with a - :exc:`~concurrent.futures.CancelledError` exception (even if - :meth:`cancel` was not called). + done, pending = await asyncio.wait(fs) - .. method:: get_stack(\*, limit=None) - Return the list of stack frames for this task's coroutine. +.. function:: as_completed(fs, \*, loop=None, timeout=None) - If the coroutine is not done, this returns the stack where it is - suspended. If the coroutine has completed successfully or was - cancelled, this returns an empty list. If the coroutine was - terminated by an exception, this returns the list of traceback - frames. + Return an iterator of awaitables which return + :class:`Future` instances. - The frames are always ordered from oldest to newest. + Raises :exc:`asyncio.TimeoutError` if the timeout occurs before + all Futures are done. - The optional limit gives the maximum number of frames to return; by - default all available frames are returned. Its meaning differs depending - on whether a stack or a traceback is returned: the newest frames of a - stack are returned, but the oldest frames of a traceback are returned. - (This matches the behavior of the traceback module.) + Example:: - For reasons beyond our control, only one stack frame is returned for a - suspended coroutine. + for f in as_completed(fs): + result = await f + # ... - .. method:: print_stack(\*, limit=None, file=None) - Print the stack or traceback for this task's coroutine. +Scheduling From Other Threads +============================= - This produces output similar to that of the traceback module, for the - frames retrieved by get_stack(). The limit argument is passed to - get_stack(). The file argument is an I/O stream to which the output - is written; by default output is written to sys.stderr. +.. function:: run_coroutine_threadsafe(coro, loop) + Submit a coroutine to the given event loop. Thread-safe. -Example: Parallel execution of tasks -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Return a :class:`concurrent.futures.Future` to access the result. -Example executing 3 tasks (A, B, C) in parallel:: + This function is meant to be called from a different OS thread + than the one where the event loop is running. Example:: - import asyncio + # Create a coroutine + coro = asyncio.sleep(1, result=3) - async def factorial(name, number): - f = 1 - for i in range(2, number+1): - print("Task %s: Compute factorial(%s)..." % (name, i)) - await asyncio.sleep(1) - f *= i - print("Task %s: factorial(%s) = %s" % (name, number, f)) + # Submit the coroutine to a given loop + future = asyncio.run_coroutine_threadsafe(coro, loop) - loop = asyncio.get_event_loop() - loop.run_until_complete(asyncio.gather( - factorial("A", 2), - factorial("B", 3), - factorial("C", 4), - )) - loop.close() + # Wait for the result with an optional timeout argument + assert future.result(timeout) == 3 -Output:: + If an exception is raised in the coroutine, the returned Future + will be notified. It can also be used to cancel the task in + the event loop:: - Task A: Compute factorial(2)... - Task B: Compute factorial(2)... - Task C: Compute factorial(2)... - Task A: factorial(2) = 2 - Task B: Compute factorial(3)... - Task C: Compute factorial(3)... - Task B: factorial(3) = 6 - Task C: Compute factorial(4)... - Task C: factorial(4) = 24 + try: + result = future.result(timeout) + except asyncio.TimeoutError: + print('The coroutine took too long, cancelling the task...') + future.cancel() + except Exception as exc: + print('The coroutine raised an exception: {!r}'.format(exc)) + else: + print('The coroutine returned: {!r}'.format(result)) -A task is automatically scheduled for execution when it is created. The event -loop stops when all tasks are done. + See the :ref:`concurrency and multithreading ` + section of the documentation. + Unlike other asyncio functions this functions requires the *loop* + argument to be passed explicitly. -Task functions --------------- + .. versionadded:: 3.5.1 -.. note:: - In the functions below, the optional *loop* argument allows explicitly setting - the event loop object used by the underlying task or coroutine. If it's - not provided, the default event loop is used. +Introspection +============= .. function:: current_task(loop=None) - Return the current running :class:`Task` instance or ``None``, if + Return the currently running :class:`Task` instance, or ``None`` if no task is running. If *loop* is ``None`` :func:`get_running_loop` is used to get @@ -567,246 +465,236 @@ Task functions .. function:: all_tasks(loop=None) - Return a set of :class:`Task` objects created for the loop. + Return a set of not yet finished :class:`Task` objects run by + the loop. If *loop* is ``None``, :func:`get_running_loop` is used for getting - current loop (contrary to the deprecated :meth:`Task.all_tasks` method - that uses :func:`get_event_loop`.) + current loop. .. versionadded:: 3.7 -.. function:: as_completed(fs, \*, loop=None, timeout=None) - - Return an iterator whose values, when waited for, are :class:`Future` - instances. +Task Object +=========== - Raises :exc:`asyncio.TimeoutError` if the timeout occurs before all Futures - are done. - - Example:: - - for f in as_completed(fs): - result = await f # The 'await' may raise - # Use result - - .. note:: - - The futures ``f`` are not necessarily members of fs. +.. class:: Task(coro, \*, loop=None) -.. function:: ensure_future(coro_or_future, \*, loop=None) + A :class:`Future`-like object that wraps a Python + :ref:`coroutine `. Not thread-safe. - Schedule the execution of a :ref:`coroutine object `: wrap it in - a future. Return a :class:`Task` object. + Tasks are used to run coroutines in event loops. + If a coroutine awaits on a Future, the Task suspends + the execution of the coroutine and waits for the completion + of the Future. When the Future is *done*, the execution of + the wrapped coroutine resumes. - If the argument is a :class:`Future`, it is returned directly. + Event loops use cooperative scheduling: an event loop runs + one Task at a time. While a Task awaits for the completion of a + Future, the event loop runs other Tasks, callbacks, or performs + IO operations. - .. versionadded:: 3.4.4 + Use the high-level :func:`asyncio.create_task` function to create + Tasks, or the low-level :meth:`loop.create_task` or + :func:`ensure_future` functions. Manual instantiation of Tasks + is discouraged. - .. versionchanged:: 3.5.1 - The function accepts any :term:`awaitable` object. + To cancel a running Task use the :meth:`cancel` method. Calling it + will cause the Task to throw a :exc:`CancelledError` exception into + the wrapped coroutine. If a coroutine is awaiting on a Future + object during cancellation, the Future object will be cancelled. - .. note:: + :meth:`cancelled` can be used to check if the Task was cancelled. + The method returns ``True`` if the wrapped coroutine did not + suppress the :exc:`CancelledError` exception and was actually + cancelled. - :func:`create_task` (added in Python 3.7) is the preferable way - for spawning new tasks. + :class:`asyncio.Task` inherits from :class:`Future` all of its + APIs except :meth:`Future.set_result` and + :meth:`Future.set_exception`. - .. seealso:: + Tasks support the :mod:`contextvars` module. When a Task + is created it copies the current context and later runs its + coroutine in the copied context. - The :func:`create_task` function and - :meth:`AbstractEventLoop.create_task` method. + .. versionchanged:: 3.7 + Added support for the :mod:`contextvars` module. -.. function:: wrap_future(future, \*, loop=None) + .. method:: cancel() - Wrap a :class:`concurrent.futures.Future` object in a :class:`Future` - object. + Request the Task to be cancelled. -.. function:: gather(\*coros_or_futures, loop=None, return_exceptions=False) + This arranges for a :exc:`CancelledError` exception to be thrown + into the wrapped coroutine on the next cycle of the event loop. - Return a future aggregating results from the given coroutine objects or - futures. + The coroutine then has a chance to clean up or even deny the + request by suppressing the exception with a :keyword:`try` ... + ... ``except CancelledError`` ... :keyword:`finally` block. + Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does + not guarantee that the Task will be cancelled, although + suppressing cancellation completely is not common and is actively + discouraged. - All futures must share the same event loop. If all the tasks are done - successfully, the returned future's result is the list of results (in the - order of the original sequence, not necessarily the order of results - arrival). If *return_exceptions* is true, exceptions in the tasks are - treated the same as successful results, and gathered in the result list; - otherwise, the first raised exception will be immediately propagated to the - returned future. + .. _asyncio_example_task_cancel: - Cancellation: if the outer Future is cancelled, all children (that have not - completed yet) are also cancelled. If any child is cancelled, this is - treated as if it raised :exc:`~concurrent.futures.CancelledError` -- the - outer Future is *not* cancelled in this case. (This is to prevent the - cancellation of one child to cause other children to be cancelled.) + The following example illustrates how coroutines can intercept + the cancellation request:: - .. versionchanged:: 3.7.0 - If the *gather* itself is cancelled, the cancellation is propagated - regardless of *return_exceptions*. + async def cancel_me(): + print('cancel_me(): before sleep') -.. function:: iscoroutine(obj) + try: + # Wait for 1 hour + await asyncio.sleep(3600) + except asyncio.CancelledError: + print('cancel_me(): cancel sleep') + raise + finally: + print('cancel_me(): after sleep') - Return ``True`` if *obj* is a :ref:`coroutine object `, - which may be based on a generator or an :keyword:`async def` coroutine. + async def main(): + # Create a "cancel_me" Task + task = asyncio.create_task(cancel_me()) -.. function:: iscoroutinefunction(func) + # Wait for 1 second + await asyncio.sleep(1) - Return ``True`` if *func* is determined to be a :ref:`coroutine function - `, which may be a decorated generator function or an - :keyword:`async def` function. + task.cancel() + try: + await task + except asyncio.CancelledError: + print("main(): cancel_me is cancelled now") -.. function:: run_coroutine_threadsafe(coro, loop) + asyncio.run(main()) - Submit a :ref:`coroutine object ` to a given event loop. + # Expected output: + # + # cancel_me(): before sleep + # cancel_me(): cancel sleep + # cancel_me(): after sleep + # main(): cancel_me is cancelled now - Return a :class:`concurrent.futures.Future` to access the result. + .. method:: cancelled() - This function is meant to be called from a different thread than the one - where the event loop is running. Usage:: + Return ``True`` if the Task is *cancelled*. - # Create a coroutine - coro = asyncio.sleep(1, result=3) - # Submit the coroutine to a given loop - future = asyncio.run_coroutine_threadsafe(coro, loop) - # Wait for the result with an optional timeout argument - assert future.result(timeout) == 3 + The Task is *cancelled* when the cancellation was requested with + :meth:`cancel` and the wrapped coroutine propagated the + :exc:`CancelledError` exception thrown into it. - If an exception is raised in the coroutine, the returned future will be - notified. It can also be used to cancel the task in the event loop:: + .. method:: done() - try: - result = future.result(timeout) - except asyncio.TimeoutError: - print('The coroutine took too long, cancelling the task...') - future.cancel() - except Exception as exc: - print('The coroutine raised an exception: {!r}'.format(exc)) - else: - print('The coroutine returned: {!r}'.format(result)) + Return ``True`` if the Task is *done*. - See the :ref:`concurrency and multithreading ` - section of the documentation. + A Task is *done* when the wrapped coroutine either returned + a value, raised an exception, or the Task was cancelled. - .. note:: + .. method:: get_stack(\*, limit=None) - Unlike other functions from the module, - :func:`run_coroutine_threadsafe` requires the *loop* argument to - be passed explicitly. + Return the list of stack frames for this Task. - .. versionadded:: 3.5.1 + If the wrapped coroutine is not done, this returns the stack + where it is suspended. If the coroutine has completed + successfully or was cancelled, this returns an empty list. + If the coroutine was terminated by an exception, this returns + the list of traceback frames. -.. coroutinefunction:: sleep(delay, result=None, \*, loop=None) + The frames are always ordered from oldest to newest. - Create a :ref:`coroutine ` that completes after a given - time (in seconds). If *result* is provided, it is produced to the caller - when the coroutine completes. + Only one stack frame is returned for a suspended coroutine. - The resolution of the sleep depends on the :ref:`granularity of the event - loop `. + The optional *limit* argument sets the maximum number of frames + to return; by default all available frames are returned. + The ordering of the returned list differs depending on whether + a stack or a traceback is returned: the newest frames of a + stack are returned, but the oldest frames of a traceback are + returned. (This matches the behavior of the traceback module.) - This function is a :ref:`coroutine `. + .. method:: print_stack(\*, limit=None, file=None) -.. coroutinefunction:: shield(arg, \*, loop=None) + Print the stack or traceback for this Task. - Wait for a future, shielding it from cancellation. + This produces output similar to that of the traceback module + for the frames retrieved by :meth:`get_stack`. - The statement:: + The *limit* argument is passed to :meth:`get_stack` directly. - res = await shield(something()) + The *file* argument is an I/O stream to which the output + is written; by default output is written to :data:`sys.stderr`. - is exactly equivalent to the statement:: + .. classmethod:: all_tasks(loop=None) - res = await something() + Return a set of all tasks for an event loop. - *except* that if the coroutine containing it is cancelled, the task running - in ``something()`` is not cancelled. From the point of view of - ``something()``, the cancellation did not happen. But its caller is still - cancelled, so the yield-from expression still raises - :exc:`~concurrent.futures.CancelledError`. Note: If ``something()`` is - cancelled by other means this will still cancel ``shield()``. + By default all tasks for the current event loop are returned. + If *loop* is ``None``, the :func:`get_event_loop` function + is used to get the current loop. - If you want to completely ignore cancellation (not recommended) you can - combine ``shield()`` with a try/except clause, as follows:: + This method is **deprecated** and will be removed in + Python 3.9. Use the :func:`all_tasks` function instead. - try: - res = await shield(something()) - except CancelledError: - res = None + .. classmethod:: current_task(loop=None) + Return the currently running task or ``None``. -.. coroutinefunction:: wait(futures, \*, loop=None, timeout=None,\ - return_when=ALL_COMPLETED) + If *loop* is ``None``, the :func:`get_event_loop` function + is used to get the current loop. - Wait for the Futures and coroutine objects given by the sequence *futures* - to complete. Coroutines will be wrapped in Tasks. Returns two sets of - :class:`Future`: (done, pending). + This method is **deprecated** and will be removed in + Python 3.9. Use the :func:`current_task` function instead. - The sequence *futures* must not be empty. - *timeout* can be used to control the maximum number of seconds to wait before - returning. *timeout* can be an int or float. If *timeout* is not specified - or ``None``, there is no limit to the wait time. +.. _asyncio_generator_based_coro: - *return_when* indicates when this function should return. It must be one of - the following constants of the :mod:`concurrent.futures` module: +Generator-based Coroutines +========================== - .. tabularcolumns:: |l|L| +.. note:: - +-----------------------------+----------------------------------------+ - | Constant | Description | - +=============================+========================================+ - | :const:`FIRST_COMPLETED` | The function will return when any | - | | future finishes or is cancelled. | - +-----------------------------+----------------------------------------+ - | :const:`FIRST_EXCEPTION` | The function will return when any | - | | future finishes by raising an | - | | exception. If no future raises an | - | | exception then it is equivalent to | - | | :const:`ALL_COMPLETED`. | - +-----------------------------+----------------------------------------+ - | :const:`ALL_COMPLETED` | The function will return when all | - | | futures finish or are cancelled. | - +-----------------------------+----------------------------------------+ + Support for generator-based coroutines is **deprecated** and + is scheduled for removal in Python 4.0. - Unlike :func:`~asyncio.wait_for`, ``wait()`` will not cancel the futures - when a timeout occurs. +Generator-based coroutines predate async/await syntax. They are +Python generators that use ``yield from`` expressions to await +on Futures and other coroutines. - This function is a :ref:`coroutine `. +Generator-based coroutines should be decorated with +:func:`@asyncio.coroutine `, although this is not +enforced. - Usage:: - done, pending = await asyncio.wait(fs) +.. decorator:: coroutine - .. note:: + Decorator to mark generator-based coroutines. - This does not raise :exc:`asyncio.TimeoutError`! Futures that aren't done - when the timeout occurs are returned in the second set. + This decorator enables legacy generator-based coroutines to be + compatible with async/await code:: + @asyncio.coroutine + def old_style_coroutine(): + yield from asyncio.sleep(1) -.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) + async def main(): + await old_style_coroutine() - Wait for the single :class:`Future` or :ref:`coroutine object ` - to complete with timeout. If *timeout* is ``None``, block until the future - completes. + This decorator is **deprecated** and is scheduled for removal in + Python 4.0. - Coroutine will be wrapped in :class:`Task`. + This decorator should not be used for :keyword:`async def` + coroutines. - Returns result of the Future or coroutine. When a timeout occurs, it - cancels the task and raises :exc:`asyncio.TimeoutError`. To avoid the task - cancellation, wrap it in :func:`shield`. The function will wait until - the future is actually cancelled, so the total wait time may exceed - the *timeout*. +.. function:: iscoroutine(obj) - If the wait is cancelled, the future *fut* is also cancelled. + Return ``True`` if *obj* is a :ref:`coroutine object `. - This function is a :ref:`coroutine `, usage:: + This method is different from :func:`inspect.iscoroutine` because + it returns ``True`` for generator-based coroutines decorated with + :func:`@coroutine `. - result = await asyncio.wait_for(fut, 60.0) +.. function:: iscoroutinefunction(func) - .. versionchanged:: 3.4.3 - If the wait is cancelled, the future *fut* is now also cancelled. + Return ``True`` if *func* is a :ref:`coroutine function + `. - .. versionchanged:: 3.7 - When *fut* is cancelled due to a timeout, ``wait_for`` now waits - for *fut* to be cancelled. Previously, - it raised :exc:`~asyncio.TimeoutError` immediately. + This method is different from :func:`inspect.iscoroutinefunction` + because it returns ``True`` for generator-based coroutine functions + decorated with :func:`@coroutine `. diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index b076b7d0093b..bfc97001bb71 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -1,66 +1,92 @@ -:mod:`asyncio` --- Asynchronous I/O, event loop, coroutines and tasks -===================================================================== +:mod:`asyncio` --- Asynchronous I/O +=================================== .. module:: asyncio - :synopsis: Asynchronous I/O, event loop, coroutines and tasks. + :synopsis: Asynchronous I/O. -.. versionadded:: 3.4 +-------------- -**Source code:** :source:`Lib/asyncio/` +.. sidebar:: Hello World! --------------- + .. code-block:: python + + import asyncio + + async def main(): + print('Hello ...') + await asyncio.sleep(1) + print('... World!') + + asyncio.run(main()) + +asyncio is a library to write **concurrent** code using +the **async/await** syntax. -This module provides infrastructure for writing single-threaded concurrent -code using coroutines, multiplexing I/O access over sockets and other -resources, running network clients and servers, and other related primitives. -Here is a more detailed list of the package contents: +asyncio is used as a foundation for multiple Python asynchronous +frameworks that provide high-performance network and web-servers, +database connection libraries, distributed task queues, etc. -* a pluggable :ref:`event loop ` with various system-specific - implementations; +asyncio is often a perfect fit for IO-bound and high-level +**structured** network code. -* :ref:`transport ` and :ref:`protocol ` abstractions - (similar to those in `Twisted `_); +asyncio provides a set of **high-level** APIs to: -* concrete support for TCP, UDP, SSL, subprocess pipes, delayed calls, and - others (some may be system-dependent); +* :ref:`run Python coroutines ` concurrently and + have full control over their execution; -* a :class:`Future` class that mimics the one in the :mod:`concurrent.futures` - module, but adapted for use with the event loop; +* perform :ref:`network IO and IPC `; -* coroutines and tasks based on ``yield from`` (:PEP:`380`), to help write - concurrent code in a sequential fashion; +* control :ref:`subprocesses `; -* cancellation support for :class:`Future`\s and coroutines; +* distribute tasks via :ref:`queues `; -* :ref:`synchronization primitives ` for use between coroutines in - a single thread, mimicking those in the :mod:`threading` module; +* :ref:`synchronize ` concurrent code; -* an interface for passing work off to a threadpool, for times when - you absolutely, positively have to use a library that makes blocking - I/O calls. +Additionally, there are **low-level** APIs for +*library and framework developers* to: -Asynchronous programming is more complex than classical "sequential" -programming: see the :ref:`Develop with asyncio ` page which lists -common traps and explains how to avoid them. :ref:`Enable the debug mode -` during development to detect common issues. +* create and manage :ref:`event loops `, which + provide asynchronous APIs for :meth:`networking `, + running :meth:`subprocesses `, + handling :meth:`OS signals `, etc; -Table of contents: +* implement efficient protocols using + :ref:`transports `; + +* :ref:`bridge ` callback-based libraries and code + with async/await syntax. + + +.. We use the "rubric" directive here to avoid creating + the "Reference" subsection in the TOC. + +.. rubric:: Reference .. toctree:: - :maxdepth: 3 + :caption: High-level APIs + :maxdepth: 1 - asyncio-eventloop.rst - asyncio-eventloops.rst asyncio-task.rst - asyncio-protocol.rst asyncio-stream.rst - asyncio-subprocess.rst asyncio-sync.rst + asyncio-subprocess.rst asyncio-queue.rst - asyncio-dev.rst + asyncio-exceptions.rst + +.. toctree:: + :caption: Low-level APIs + :maxdepth: 1 -.. seealso:: + asyncio-eventloop.rst + asyncio-future.rst + asyncio-protocol.rst + asyncio-policy.rst + asyncio-platforms.rst - The :mod:`asyncio` module was designed in :PEP:`3156`. For a - motivational primer on transports and protocols, see :PEP:`3153`. +.. toctree:: + :caption: Guides and Tutorials + :maxdepth: 1 + asyncio-api-index.rst + asyncio-llapi-index.rst + asyncio-dev.rst diff --git a/Doc/library/ipc.rst b/Doc/library/ipc.rst index 6b1756331ec0..b88a174eb97f 100644 --- a/Doc/library/ipc.rst +++ b/Doc/library/ipc.rst @@ -1,11 +1,11 @@ .. _ipc: ***************************************** -Interprocess Communication and Networking +Networking and Interprocess Communication ***************************************** -The modules described in this chapter provide mechanisms for different processes -to communicate. +The modules described in this chapter provide mechanisms for +networking and inter-processes communication. Some modules only work for two processes that are on the same machine, e.g. :mod:`signal` and :mod:`mmap`. Other modules support networking protocols @@ -15,12 +15,13 @@ The list of modules described in this chapter is: .. toctree:: + :maxdepth: 1 + asyncio.rst socket.rst ssl.rst select.rst selectors.rst - asyncio.rst asyncore.rst asynchat.rst signal.rst diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index c2106678ac60..c39922456140 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -1,118 +1,34 @@ {% extends "!layout.html" %} + {% block rootrellink %} -
    • -
  • Python{{ reldelim1 }}
    • -
  • - {%- if switchers is defined %} - {{ language or 'en' }} - {{ release }} - {% trans %}Documentation {% endtrans %}{{ reldelim1 }} - {%- else %} - {{ shorttitle }}{{ reldelim1 }} - {%- endif %} -
    • -{% endblock %} -{%- macro searchbox() %} -{# modified from sphinx/themes/basic/searchbox.html #} - {%- if builder != "htmlhelp" %} - - - {%- endif %} -{%- endmacro %} -{% block relbar1 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} -{% block relbar2 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} -{% block relbaritems %} - {%- if pagename != "search" and builder != "singlehtml" and builder != "htmlhelp" %} -
  • - {{ searchbox() }} - {{ reldelim2 }} +{{ super() }} +
  • + {%- if switchers is defined %} + {{ language or 'en' }} + {{ release }} + {% trans %}Documentation {% endtrans %}{{ reldelim1 }} + {%- else %} + {{ shorttitle }}{{ reldelim1 }} + {%- endif %}
    • - {%- endif %} {% endblock %} + {% block extrahead %} - {% if builder != "htmlhelp" %} - {% if not embedded %}{% endif %} - {% if switchers is defined and not embedded %}{% endif %} - {% if pagename == 'whatsnew/changelog' and not embedded %} - - {% endif %} + {% if switchers is defined and not embedded %} + {% endif %} + {% if pagename == 'whatsnew/changelog' and not embedded %} + {% endif %} {% endif %} + + {# custom CSS; used in asyncio docs! #} + {{ super() }} {% endblock %} -{% block footer %} -
    - © {% trans %}Copyright{% endtrans %} {{ copyright|e }}. -
    - {% trans %}The Python Software Foundation is a non-profit corporation.{% endtrans %} - {% trans %}Please donate.{% endtrans %} -
    - {% trans last_updated=last_updated|e %}Last updated on {{ last_updated }}.{% endtrans %} - {% trans pathto_bugs=pathto('bugs') %}Found a bug?{% endtrans %} -
    - {% trans sphinx_version=sphinx_version|e %}Created using Sphinx {{ sphinx_version }}.{% endtrans %} -
    -{% endblock %} diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 96f86e6c6142..d3aed84d250e 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -777,18 +777,18 @@ all changes introduced in Python 3.5 have also been backported to Python 3.4.x. Notable changes in the :mod:`asyncio` module since Python 3.4.0: -* New debugging APIs: :meth:`loop.set_debug() ` - and :meth:`loop.get_debug() ` methods. +* New debugging APIs: :meth:`loop.set_debug() ` + and :meth:`loop.get_debug() ` methods. (Contributed by Victor Stinner.) * The proactor event loop now supports SSL. (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) -* A new :meth:`loop.is_closed() ` method to +* A new :meth:`loop.is_closed() ` method to check if the event loop is closed. (Contributed by Victor Stinner in :issue:`21326`.) -* A new :meth:`loop.create_task() ` +* A new :meth:`loop.create_task() ` to conveniently create and schedule a new :class:`~asyncio.Task` for a coroutine. The ``create_task`` method is also used by all asyncio functions that wrap coroutines into tasks, such as @@ -805,10 +805,10 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0: (Contributed by Yury Selivanov.) * New :meth:`loop.set_task_factory() - ` and - :meth:`loop.get_task_factory() ` + ` and + :meth:`loop.get_task_factory() ` methods to customize the task factory that :meth:`loop.create_task() - ` method uses. (Contributed by Yury + ` method uses. (Contributed by Yury Selivanov.) * New :meth:`Queue.join() ` and @@ -822,7 +822,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0: Updates in 3.5.1: * The :func:`~asyncio.ensure_future` function and all functions that - use it, such as :meth:`loop.run_until_complete() `, + use it, such as :meth:`loop.run_until_complete() `, now accept all kinds of :term:`awaitable objects `. (Contributed by Yury Selivanov.) @@ -834,20 +834,20 @@ Updates in 3.5.1: method to check if the transport is closing or closed. (Contributed by Yury Selivanov.) -* The :meth:`loop.create_server() ` +* The :meth:`loop.create_server() ` method can now accept a list of hosts. (Contributed by Yann Sionneau.) Updates in 3.5.2: -* New :meth:`loop.create_future() ` +* New :meth:`loop.create_future() ` method to create Future objects. This allows alternative event loop implementations, such as `uvloop `_, to provide a faster :class:`asyncio.Future` implementation. (Contributed by Yury Selivanov.) -* New :meth:`loop.get_exception_handler() ` +* New :meth:`loop.get_exception_handler() ` method to get the current exception handler. (Contributed by Yury Selivanov.) @@ -856,13 +856,13 @@ Updates in 3.5.2: sequence appears. (Contributed by Mark Korenberg.) -* The :meth:`loop.create_connection() ` - and :meth:`loop.create_server() ` +* The :meth:`loop.create_connection() ` + and :meth:`loop.create_server() ` methods are optimized to avoid calling the system ``getaddrinfo`` function if the address is already resolved. (Contributed by A. Jesse Jiryu Davis.) -* The :meth:`loop.sock_connect(sock, address) ` +* The :meth:`loop.sock_connect(sock, address) ` no longer requires the *address* to be resolved prior to the call. (Contributed by A. Jesse Jiryu Davis.) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 1360af79d073..b413b5a65d52 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -824,7 +824,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 (Contributed by Yury Selivanov in :issue:`28613`.) * The :func:`~asyncio.ensure_future` function and all functions that - use it, such as :meth:`loop.run_until_complete() `, + use it, such as :meth:`loop.run_until_complete() `, now accept all kinds of :term:`awaitable objects `. (Contributed by Yury Selivanov.) @@ -836,18 +836,18 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 method to check if the transport is closing or closed. (Contributed by Yury Selivanov.) -* The :meth:`loop.create_server() ` +* The :meth:`loop.create_server() ` method can now accept a list of hosts. (Contributed by Yann Sionneau.) -* New :meth:`loop.create_future() ` +* New :meth:`loop.create_future() ` method to create Future objects. This allows alternative event loop implementations, such as `uvloop `_, to provide a faster :class:`asyncio.Future` implementation. (Contributed by Yury Selivanov in :issue:`27041`.) -* New :meth:`loop.get_exception_handler() ` +* New :meth:`loop.get_exception_handler() ` method to get the current exception handler. (Contributed by Yury Selivanov in :issue:`27040`.) @@ -860,12 +860,12 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 has been improved. (Contributed by Mark Korenberg in :issue:`28370`.) -* The :meth:`loop.getaddrinfo() ` +* The :meth:`loop.getaddrinfo() ` method is optimized to avoid calling the system ``getaddrinfo`` function if the address is already resolved. (Contributed by A. Jesse Jiryu Davis.) -* The :meth:`loop.stop() ` +* The :meth:`loop.stop() ` method has been changed to stop the loop immediately after the current iteration. Any new callbacks scheduled as a result of the last iteration will be discarded. @@ -876,7 +876,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 the :exc:`StopIteration` exception. (Contributed by Chris Angelico in :issue:`26221`.) -* New :meth:`loop.connect_accepted_socket() ` +* New :meth:`loop.connect_accepted_socket() ` method to be used by servers that accept connections outside of asyncio, but that use asyncio to handle them. (Contributed by Jim Fulton in :issue:`27392`.) @@ -884,7 +884,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 * ``TCP_NODELAY`` flag is now set for all TCP transports by default. (Contributed by Yury Selivanov in :issue:`27456`.) -* New :meth:`loop.shutdown_asyncgens() ` +* New :meth:`loop.shutdown_asyncgens() ` to properly close pending asynchronous generators before closing the loop. (Contributed by Yury Selivanov in :issue:`28003`.) diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 2a2e23cc6f91..f53a0268738a 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -637,10 +637,10 @@ include: (Contributed by Yury Selivanov in :issue:`32314`.) * asyncio gained support for :mod:`contextvars`. - :meth:`loop.call_soon() `, - :meth:`loop.call_soon_threadsafe() `, - :meth:`loop.call_later() `, - :meth:`loop.call_at() `, and + :meth:`loop.call_soon() `, + :meth:`loop.call_soon_threadsafe() `, + :meth:`loop.call_later() `, + :meth:`loop.call_at() `, and :meth:`Future.add_done_callback() ` have a new optional keyword-only *context* parameter. :class:`Tasks ` now track their context automatically. @@ -651,11 +651,11 @@ include: to ``asyncio.get_event_loop().create_task()``. (Contributed by Andrew Svetlov in :issue:`32311`.) -* The new :meth:`loop.start_tls() ` +* The new :meth:`loop.start_tls() ` method can be used to upgrade an existing connection to TLS. (Contributed by Yury Selivanov in :issue:`23749`.) -* The new :meth:`loop.sock_recv_into() ` +* The new :meth:`loop.sock_recv_into() ` method allows reading data from a socket directly into a provided buffer making it possible to reduce data copies. (Contributed by Antoine Pitrou in :issue:`31819`.) @@ -683,13 +683,13 @@ include: can be used to determine if the writer is closing. (Contributed by Andrew Svetlov in :issue:`32391`.) -* The new :meth:`loop.sock_sendfile() ` +* The new :meth:`loop.sock_sendfile() ` coroutine method allows sending files using :mod:`os.sendfile` when possible. (Contributed by Andrew Svetlov in :issue:`32410`.) -* The new :meth:`Task.get_loop() ` and - :meth:`Future.get_loop() ` methods - return the instance of the loop on which a task or a future were created. +* The new :meth:`Future.get_loop() ` and + ``Task.get_loop()`` methods return the instance of the loop on which a task or + a future were created. :meth:`Server.get_loop() ` allows doing the same for :class:`asyncio.Server` objects. (Contributed by Yury Selivanov in :issue:`32415` and @@ -698,8 +698,8 @@ include: * It is now possible to control how instances of :class:`asyncio.Server` begin serving. Previously, the server would start serving immediately when created. The new *start_serving* keyword argument to - :meth:`loop.create_server() ` and - :meth:`loop.create_unix_server() `, + :meth:`loop.create_server() ` and + :meth:`loop.create_unix_server() `, as well as :meth:`Server.start_serving() `, and :meth:`Server.serve_forever() ` can be used to decouple server instantiation and serving. The new @@ -717,20 +717,20 @@ include: (Contributed by Yury Selivanov in :issue:`32662`.) * Callback objects returned by - :func:`loop.call_later() ` + :func:`loop.call_later() ` gained the new :meth:`when() ` method which returns an absolute scheduled callback timestamp. (Contributed by Andrew Svetlov in :issue:`32741`.) * The :meth:`loop.create_datagram_endpoint() \ - ` method + ` method gained support for Unix sockets. (Contributed by Quentin Dawans in :issue:`31245`.) * The :func:`asyncio.open_connection`, :func:`asyncio.start_server` functions, - :meth:`loop.create_connection() `, - :meth:`loop.create_server() `, - :meth:`loop.create_accepted_socket() ` + :meth:`loop.create_connection() `, + :meth:`loop.create_server() `, + :meth:`loop.create_accepted_socket() ` methods and their corresponding UNIX socket variants now accept the *ssl_handshake_timeout* keyword argument. (Contributed by Neil Aspinall in :issue:`29970`.) @@ -1923,8 +1923,7 @@ asyncio Support for directly ``await``-ing instances of :class:`asyncio.Lock` and other asyncio synchronization primitives has been deprecated. An asynchronous context manager must be used in order to acquire and release -the synchronization resource. See :ref:`async-with-locks` for more -information. +the synchronization resource. (Contributed by Andrew Svetlov in :issue:`32253`.) The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` @@ -2360,11 +2359,11 @@ Changes in the Python API (Contributed by Brett Cannon in :issue:`33169`.) * In :mod:`asyncio`, - :meth:`loop.sock_recv() `, - :meth:`loop.sock_sendall() `, - :meth:`loop.sock_accept() `, - :meth:`loop.getaddrinfo() `, - :meth:`loop.getnameinfo() ` + :meth:`loop.sock_recv() `, + :meth:`loop.sock_sendall() `, + :meth:`loop.sock_accept() `, + :meth:`loop.getaddrinfo() `, + :meth:`loop.getnameinfo() ` have been changed to be proper coroutine methods to match their documentation. Previously, these methods returned :class:`asyncio.Future` instances. From webhook-mailer at python.org Mon Sep 17 20:19:31 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 18 Sep 2018 00:19:31 -0000 Subject: [Python-checkins] bpo-34589: Add -X coerce_c_locale command line option (GH-9378) Message-ID: https://github.com/python/cpython/commit/dbdee0073cf0b88fe541980ace1f650900f455cc commit: dbdee0073cf0b88fe541980ace1f650900f455cc branch: master author: Victor Stinner committer: GitHub date: 2018-09-17T17:19:26-07:00 summary: bpo-34589: Add -X coerce_c_locale command line option (GH-9378) Add a new -X coerce_c_locale command line option to control C locale coercion (PEP 538). files: A Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst M Doc/using/cmdline.rst M Doc/whatsnew/3.7.rst M Lib/test/test_c_locale_coercion.py M Lib/test/test_cmd_line.py M Lib/test/test_sys.py M Lib/test/test_utf8_mode.py M Python/coreconfig.c diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index b61df8a4b77d..cd3b2410c84d 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -438,13 +438,22 @@ Miscellaneous options * Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to ``True`` - * ``-X utf8`` enables UTF-8 mode for operating system interfaces, overriding + * ``-X utf8`` enables UTF-8 mode (:pep:`540`) for operating system interfaces, overriding the default locale-aware mode. ``-X utf8=0`` explicitly disables UTF-8 mode (even when it would otherwise activate automatically). See :envvar:`PYTHONUTF8` for more details. * ``-X pycache_prefix=PATH`` enables writing ``.pyc`` files to a parallel tree rooted at the given directory instead of to the code tree. See also :envvar:`PYTHONPYCACHEPREFIX`. + * ``-X coerce_c_locale`` or ``-X coerce_c_locale=1`` tries to coerce the C + locale (:pep:`538`). + ``-X coerce_c_locale=0`` skips coercing the legacy ASCII-based C and POSIX + locales to a more capable UTF-8 based alternative. + ``-X coerce_c_locale=warn`` will cause Python to emit warning messages on + ``stderr`` if either the locale coercion activates, or else if a locale + that *would* have triggered coercion is still active when the Python + runtime is initialized. + See :envvar:`PYTHONCOERCECLOCALE` for more details. It also allows passing arbitrary values and retrieving them through the :data:`sys._xoptions` dictionary. @@ -464,6 +473,9 @@ Miscellaneous options .. versionadded:: 3.7 The ``-X importtime``, ``-X dev`` and ``-X utf8`` options. + .. versionadded:: 3.7.1 + The ``-X coerce_c_locale`` option. + .. versionadded:: 3.8 The ``-X pycache_prefix`` option. @@ -850,6 +862,8 @@ conflict. order to force the interpreter to use ``ASCII`` instead of ``UTF-8`` for system interfaces. + Also available as the :option:`-X` ``coerce_c_locale`` option. + Availability: \*nix .. versionadded:: 3.7 diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index f53a0268738a..6cd9d46a42b0 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2494,3 +2494,10 @@ versions, it respected an ill-defined subset of those environment variables, while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before calling :c:func:`Py_Initialize`. + +:c:func:`Py_Initialize` and :c:func:`Py_Main` cannot enable the C locale +coercion (:pep:`538`) anymore: it is always disabled. It can now only be +enabled by the Python program ("python3). + +New :option:`-X` ``coerce_c_locale`` command line option to control C locale +coercion (:pep:`538`). diff --git a/Lib/test/test_c_locale_coercion.py b/Lib/test/test_c_locale_coercion.py index 1db293b9c373..f62208ab2006 100644 --- a/Lib/test/test_c_locale_coercion.py +++ b/Lib/test/test_c_locale_coercion.py @@ -139,7 +139,7 @@ def _handle_output_variations(data): return data @classmethod - def get_child_details(cls, env_vars): + def get_child_details(cls, env_vars, xoption=None): """Retrieves fsencoding and standard stream details from a child process Returns (encoding_details, stderr_lines): @@ -150,10 +150,11 @@ def get_child_details(cls, env_vars): The child is run in isolated mode if the current interpreter supports that. """ - result, py_cmd = run_python_until_end( - "-X", "utf8=0", "-c", cls.CHILD_PROCESS_SCRIPT, - **env_vars - ) + args = [] + if xoption: + args.extend(("-X", f"coerce_c_locale={xoption}")) + args.extend(("-X", "utf8=0", "-c", cls.CHILD_PROCESS_SCRIPT)) + result, py_cmd = run_python_until_end(*args, **env_vars) if not result.rc == 0: result.fail(py_cmd) # All subprocess outputs in this test case should be pure ASCII @@ -212,7 +213,8 @@ def _check_child_encoding_details(self, expected_fs_encoding, expected_stream_encoding, expected_warnings, - coercion_expected): + coercion_expected, + xoption=None): """Check the C locale handling for the given process environment Parameters: @@ -220,7 +222,7 @@ def _check_child_encoding_details(self, expected_stream_encoding: expected encoding for standard streams expected_warning: stderr output to expect (if any) """ - result = EncodingDetails.get_child_details(env_vars) + result = EncodingDetails.get_child_details(env_vars, xoption) encoding_details, stderr_lines = result expected_details = EncodingDetails.get_expected_details( coercion_expected, @@ -290,6 +292,7 @@ def _check_c_locale_coercion(self, coerce_c_locale, expected_warnings=None, coercion_expected=True, + use_xoption=False, **extra_vars): """Check the C locale handling for various configurations @@ -319,8 +322,12 @@ def _check_c_locale_coercion(self, "PYTHONCOERCECLOCALE": "", } base_var_dict.update(extra_vars) + xoption = None if coerce_c_locale is not None: - base_var_dict["PYTHONCOERCECLOCALE"] = coerce_c_locale + if use_xoption: + xoption = coerce_c_locale + else: + base_var_dict["PYTHONCOERCECLOCALE"] = coerce_c_locale # Check behaviour for the default locale with self.subTest(default_locale=True, @@ -342,7 +349,8 @@ def _check_c_locale_coercion(self, fs_encoding, stream_encoding, _expected_warnings, - _coercion_expected) + _coercion_expected, + xoption=xoption) # Check behaviour for explicitly configured locales for locale_to_set in EXPECTED_C_LOCALE_EQUIVALENTS: @@ -357,7 +365,8 @@ def _check_c_locale_coercion(self, fs_encoding, stream_encoding, expected_warnings, - coercion_expected) + coercion_expected, + xoption=xoption) def test_PYTHONCOERCECLOCALE_not_set(self): # This should coerce to the first available target locale by default @@ -404,6 +413,32 @@ def test_LC_ALL_set_to_C(self): expected_warnings=[LEGACY_LOCALE_WARNING], coercion_expected=False) + def test_xoption_set_to_1(self): + self._check_c_locale_coercion("utf-8", "utf-8", coerce_c_locale="1", + use_xoption=True) + + def test_xoption_set_to_zero(self): + # The setting "0" should result in the locale coercion being disabled + self._check_c_locale_coercion(EXPECTED_C_LOCALE_FS_ENCODING, + EXPECTED_C_LOCALE_STREAM_ENCODING, + coerce_c_locale="0", + coercion_expected=False, + use_xoption=True) + # Setting LC_ALL=C shouldn't make any difference to the behaviour + self._check_c_locale_coercion(EXPECTED_C_LOCALE_FS_ENCODING, + EXPECTED_C_LOCALE_STREAM_ENCODING, + coerce_c_locale="0", + LC_ALL="C", + coercion_expected=False, + use_xoption=True) + + def test_xoption_set_to_warn(self): + # -X coerce_c_locale=warn enables runtime warnings for legacy locales + self._check_c_locale_coercion("utf-8", "utf-8", + coerce_c_locale="warn", + expected_warnings=[CLI_COERCION_WARNING], + use_xoption=True) + def test_main(): test.support.run_unittest( LocaleConfigurationTests, diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index 21511b896cad..7e967b20ab88 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -159,13 +159,16 @@ def test_undecodable_code(self): env = os.environ.copy() # Use C locale to get ascii for the locale encoding env['LC_ALL'] = 'C' - env['PYTHONCOERCECLOCALE'] = '0' code = ( b'import locale; ' b'print(ascii("' + undecodable + b'"), ' b'locale.getpreferredencoding())') p = subprocess.Popen( - [sys.executable, "-c", code], + [sys.executable, + # Disable C locale coercion and UTF-8 Mode to not use UTF-8 + "-X", "coerce_c_locale=0", + "-X", "utf8=0", + "-c", code], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=env) stdout, stderr = p.communicate() diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index b90366d81445..a7f292827130 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -656,9 +656,8 @@ def test_getfilesystemencoding(self): def c_locale_get_error_handler(self, locale, isolated=False, encoding=None): # Force the POSIX locale - env = os.environ.copy() + env = dict(os.environ) env["LC_ALL"] = locale - env["PYTHONCOERCECLOCALE"] = "0" code = '\n'.join(( 'import sys', 'def dump(name):', @@ -668,7 +667,10 @@ def c_locale_get_error_handler(self, locale, isolated=False, encoding=None): 'dump("stdout")', 'dump("stderr")', )) - args = [sys.executable, "-X", "utf8=0", "-c", code] + args = [sys.executable, + "-X", "utf8=0", + "-X", "coerce_c_locale=0", + "-c", code] if isolated: args.append("-I") if encoding is not None: diff --git a/Lib/test/test_utf8_mode.py b/Lib/test/test_utf8_mode.py index 7280ce77ef82..c3cbb49060e7 100644 --- a/Lib/test/test_utf8_mode.py +++ b/Lib/test/test_utf8_mode.py @@ -27,6 +27,8 @@ def posix_locale(self): return (loc in POSIX_LOCALES) def get_output(self, *args, failure=False, **kw): + # Always disable the C locale coercion (PEP 538) + args = ('-X', 'coerce_c_locale=0', *args) kw = dict(self.DEFAULT_ENV, **kw) if failure: out = assert_python_failure(*args, **kw) @@ -116,7 +118,6 @@ def test_filesystemencoding(self): # PYTHONLEGACYWINDOWSFSENCODING disables the UTF-8 mode # and has the priority over -X utf8 and PYTHONUTF8 out = self.get_output('-X', 'utf8', '-c', code, - PYTHONUTF8='strict', PYTHONLEGACYWINDOWSFSENCODING='1') self.assertEqual(out, 'mbcs/replace') diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst new file mode 100644 index 000000000000..618092d192c4 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst @@ -0,0 +1,2 @@ +Add a new :option:`-X` ``coerce_c_locale`` command line option to control C +locale coercion (:pep:`538`). diff --git a/Python/coreconfig.c b/Python/coreconfig.c index 131a043ff280..b2459dca57b0 100644 --- a/Python/coreconfig.c +++ b/Python/coreconfig.c @@ -705,6 +705,17 @@ config_init_utf8_mode(_PyCoreConfig *config) return _Py_INIT_OK(); } +#ifndef MS_WINDOWS + /* The C locale and the POSIX locale enable the UTF-8 Mode (PEP 540) */ + const char *ctype_loc = setlocale(LC_CTYPE, NULL); + if (ctype_loc != NULL + && (strcmp(ctype_loc, "C") == 0 || strcmp(ctype_loc, "POSIX") == 0)) + { + config->utf8_mode = 1; + return _Py_INIT_OK(); + } +#endif + return _Py_INIT_OK(); } @@ -808,25 +819,6 @@ config_read_env_vars(_PyCoreConfig *config) config->malloc_stats = 1; } - const char *env = _PyCoreConfig_GetEnv(config, "PYTHONCOERCECLOCALE"); - if (env) { - if (strcmp(env, "0") == 0) { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 0; - } - } - else if (strcmp(env, "warn") == 0) { - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 1; - } - } - else { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 1; - } - } - } - wchar_t *path; int res = _PyCoreConfig_GetEnvDup(config, &path, L"PYTHONPATH", "PYTHONPATH"); @@ -966,28 +958,76 @@ config_read_complex_options(_PyCoreConfig *config) } -static void -config_init_locale(_PyCoreConfig *config) +static _PyInitError +config_init_coerce_c_locale(_PyCoreConfig *config) { + const wchar_t *xopt = config_get_xoption(config, L"coerce_c_locale"); + if (xopt) { + wchar_t *sep = wcschr(xopt, L'='); + if (sep) { + xopt = sep + 1; + if (wcscmp(xopt, L"1") == 0) { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 1; + } + } + else if (wcscmp(xopt, L"0") == 0) { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 0; + } + } + else if (wcscmp(xopt, L"warn") == 0) { + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 1; + } + } + else { + return _Py_INIT_USER_ERR("invalid -X coerce_c_locale option value"); + } + } + else { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 1; + } + } + + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 0; + } + } + + const char *env = _PyCoreConfig_GetEnv(config, "PYTHONCOERCECLOCALE"); + if (env) { + if (strcmp(env, "0") == 0) { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 0; + } + } + else if (strcmp(env, "warn") == 0) { + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 1; + } + } + else { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 1; + } + } + + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 0; + } + } + if (config->_coerce_c_locale < 0) { /* The C locale enables the C locale coercion (PEP 538) */ if (_Py_LegacyLocaleDetected()) { config->_coerce_c_locale = 1; + return _Py_INIT_OK(); } } -#ifndef MS_WINDOWS - if (config->utf8_mode < 0) { - /* The C locale and the POSIX locale enable the UTF-8 Mode (PEP 540) */ - const char *ctype_loc = setlocale(LC_CTYPE, NULL); - if (ctype_loc != NULL - && (strcmp(ctype_loc, "C") == 0 - || strcmp(ctype_loc, "POSIX") == 0)) - { - config->utf8_mode = 1; - } - } -#endif + return _Py_INIT_OK(); } @@ -1293,8 +1333,11 @@ _PyCoreConfig_Read(_PyCoreConfig *config) } } - if (config->utf8_mode < 0 || config->_coerce_c_locale < 0) { - config_init_locale(config); + if (config->_coerce_c_locale < 0 || config->_coerce_c_locale_warn < 0) { + err = config_init_coerce_c_locale(config); + if (_Py_INIT_FAILED(err)) { + return err; + } } if (config->_install_importlib) { @@ -1349,6 +1392,7 @@ _PyCoreConfig_Read(_PyCoreConfig *config) } assert(config->_coerce_c_locale >= 0); + assert(config->_coerce_c_locale_warn >= 0); assert(config->use_environment >= 0); assert(config->filesystem_encoding != NULL); assert(config->filesystem_errors != NULL); From webhook-mailer at python.org Mon Sep 17 21:01:44 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 18 Sep 2018 01:01:44 -0000 Subject: [Python-checkins] [3.7] bpo-34589: Add -X coerce_c_locale option; C locale coercion off by default (GH-9379) Message-ID: https://github.com/python/cpython/commit/144f1e2c6f4a24bd288c045986842c65cc289684 commit: 144f1e2c6f4a24bd288c045986842c65cc289684 branch: 3.7 author: Victor Stinner committer: GitHub date: 2018-09-17T18:01:39-07:00 summary: [3.7] bpo-34589: Add -X coerce_c_locale option; C locale coercion off by default (GH-9379) * bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371) _PyCoreConfig: * Rename coerce_c_locale to _coerce_c_locale * Rename coerce_c_locale_warn to _coerce_c_locale_warn These fields are now private (name prefixed by "_"). (cherry picked from commit 188ebfa475a6f6aa2d0ea14ca8e1fbe7865b6d27) * bpo-34589: C locale coercion off by default (GH-9073) Py_Initialize() and Py_Main() cannot enable the C locale coercion (PEP 538) anymore: it is always disabled. It can now only be enabled by the Python program ("python3). test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8 nor PYTHONCOERCECLOCALE, these variables are already set in the parent. (cherry picked from commit 7a0791b6992d420dc52536257f2f093851ed7215) * bpo-34589: Add -X coerce_c_locale command line option (GH-9378) Add a new -X coerce_c_locale command line option to control C locale coercion (PEP 538). (cherry picked from commit dbdee0073cf0b88fe541980ace1f650900f455cc) files: A Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst A Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst M Doc/using/cmdline.rst M Doc/whatsnew/3.7.rst M Include/pylifecycle.h M Include/pystate.h M Lib/test/test_c_locale_coercion.py M Lib/test/test_cmd_line.py M Lib/test/test_embed.py M Lib/test/test_sys.py M Lib/test/test_utf8_mode.py M Modules/main.c M Programs/_testembed.c M Programs/python.c M Python/pylifecycle.c diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 70e8d1afb42a..29de155f2d1e 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -438,10 +438,19 @@ Miscellaneous options * Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to ``True`` - * ``-X utf8`` enables UTF-8 mode for operating system interfaces, overriding + * ``-X utf8`` enables UTF-8 mode (:pep:`540`) for operating system interfaces, overriding the default locale-aware mode. ``-X utf8=0`` explicitly disables UTF-8 mode (even when it would otherwise activate automatically). See :envvar:`PYTHONUTF8` for more details. + * ``-X coerce_c_locale`` or ``-X coerce_c_locale=1`` tries to coerce the C + locale (:pep:`538`). + ``-X coerce_c_locale=0`` skips coercing the legacy ASCII-based C and POSIX + locales to a more capable UTF-8 based alternative. + ``-X coerce_c_locale=warn`` will cause Python to emit warning messages on + ``stderr`` if either the locale coercion activates, or else if a locale + that *would* have triggered coercion is still active when the Python + runtime is initialized. + See :envvar:`PYTHONCOERCECLOCALE` for more details. It also allows passing arbitrary values and retrieving them through the :data:`sys._xoptions` dictionary. @@ -461,6 +470,9 @@ Miscellaneous options .. versionadded:: 3.7 The ``-X importtime``, ``-X dev`` and ``-X utf8`` options. + .. versionadded:: 3.7.1 + The ``-X coerce_c_locale`` option. + Options you shouldn't use ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -834,6 +846,8 @@ conflict. order to force the interpreter to use ``ASCII`` instead of ``UTF-8`` for system interfaces. + Also available as the :option:`-X` ``coerce_c_locale`` option. + Availability: \*nix .. versionadded:: 3.7 diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index f53a0268738a..6cd9d46a42b0 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2494,3 +2494,10 @@ versions, it respected an ill-defined subset of those environment variables, while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before calling :c:func:`Py_Initialize`. + +:c:func:`Py_Initialize` and :c:func:`Py_Main` cannot enable the C locale +coercion (:pep:`538`) anymore: it is always disabled. It can now only be +enabled by the Python program ("python3). + +New :option:`-X` ``coerce_c_locale`` command line option to control C locale +coercion (:pep:`538`). diff --git a/Include/pylifecycle.h b/Include/pylifecycle.h index 119296194934..68fc036479fd 100644 --- a/Include/pylifecycle.h +++ b/Include/pylifecycle.h @@ -119,7 +119,11 @@ PyAPI_FUNC(int) Py_FdIsInteractive(FILE *, const char *); /* Bootstrap __main__ (defined in Modules/main.c) */ PyAPI_FUNC(int) Py_Main(int argc, wchar_t **argv); #ifdef Py_BUILD_CORE +# ifdef MS_WINDOWS +PyAPI_FUNC(int) _Py_WindowsMain(int argc, wchar_t **argv); +# else PyAPI_FUNC(int) _Py_UnixMain(int argc, char **argv); +# endif #endif /* In getpath.c */ diff --git a/Include/pystate.h b/Include/pystate.h index f16ffb8fd2ae..c2ccb203db20 100644 --- a/Include/pystate.h +++ b/Include/pystate.h @@ -41,8 +41,6 @@ typedef struct { int show_alloc_count; /* -X showalloccount */ int dump_refs; /* PYTHONDUMPREFS */ int malloc_stats; /* PYTHONMALLOCSTATS */ - int coerce_c_locale; /* PYTHONCOERCECLOCALE, -1 means unknown */ - int coerce_c_locale_warn; /* PYTHONCOERCECLOCALE=warn */ int utf8_mode; /* PYTHONUTF8, -X utf8; -1 means unknown */ wchar_t *program_name; /* Program name, see also Py_GetProgramName() */ @@ -74,6 +72,8 @@ typedef struct { /* Private fields */ int _disable_importlib; /* Needed by freeze_importlib */ + int _coerce_c_locale; /* PYTHONCOERCECLOCALE, -1 means unknown */ + int _coerce_c_locale_warn; /* PYTHONCOERCECLOCALE=warn */ } _PyCoreConfig; #define _PyCoreConfig_INIT \ @@ -81,7 +81,8 @@ typedef struct { .install_signal_handlers = -1, \ .ignore_environment = -1, \ .use_hash_seed = -1, \ - .coerce_c_locale = -1, \ + ._coerce_c_locale = 0, \ + ._coerce_c_locale_warn = 0, \ .faulthandler = -1, \ .tracemalloc = -1, \ .utf8_mode = -1, \ diff --git a/Lib/test/test_c_locale_coercion.py b/Lib/test/test_c_locale_coercion.py index 1db293b9c373..f62208ab2006 100644 --- a/Lib/test/test_c_locale_coercion.py +++ b/Lib/test/test_c_locale_coercion.py @@ -139,7 +139,7 @@ def _handle_output_variations(data): return data @classmethod - def get_child_details(cls, env_vars): + def get_child_details(cls, env_vars, xoption=None): """Retrieves fsencoding and standard stream details from a child process Returns (encoding_details, stderr_lines): @@ -150,10 +150,11 @@ def get_child_details(cls, env_vars): The child is run in isolated mode if the current interpreter supports that. """ - result, py_cmd = run_python_until_end( - "-X", "utf8=0", "-c", cls.CHILD_PROCESS_SCRIPT, - **env_vars - ) + args = [] + if xoption: + args.extend(("-X", f"coerce_c_locale={xoption}")) + args.extend(("-X", "utf8=0", "-c", cls.CHILD_PROCESS_SCRIPT)) + result, py_cmd = run_python_until_end(*args, **env_vars) if not result.rc == 0: result.fail(py_cmd) # All subprocess outputs in this test case should be pure ASCII @@ -212,7 +213,8 @@ def _check_child_encoding_details(self, expected_fs_encoding, expected_stream_encoding, expected_warnings, - coercion_expected): + coercion_expected, + xoption=None): """Check the C locale handling for the given process environment Parameters: @@ -220,7 +222,7 @@ def _check_child_encoding_details(self, expected_stream_encoding: expected encoding for standard streams expected_warning: stderr output to expect (if any) """ - result = EncodingDetails.get_child_details(env_vars) + result = EncodingDetails.get_child_details(env_vars, xoption) encoding_details, stderr_lines = result expected_details = EncodingDetails.get_expected_details( coercion_expected, @@ -290,6 +292,7 @@ def _check_c_locale_coercion(self, coerce_c_locale, expected_warnings=None, coercion_expected=True, + use_xoption=False, **extra_vars): """Check the C locale handling for various configurations @@ -319,8 +322,12 @@ def _check_c_locale_coercion(self, "PYTHONCOERCECLOCALE": "", } base_var_dict.update(extra_vars) + xoption = None if coerce_c_locale is not None: - base_var_dict["PYTHONCOERCECLOCALE"] = coerce_c_locale + if use_xoption: + xoption = coerce_c_locale + else: + base_var_dict["PYTHONCOERCECLOCALE"] = coerce_c_locale # Check behaviour for the default locale with self.subTest(default_locale=True, @@ -342,7 +349,8 @@ def _check_c_locale_coercion(self, fs_encoding, stream_encoding, _expected_warnings, - _coercion_expected) + _coercion_expected, + xoption=xoption) # Check behaviour for explicitly configured locales for locale_to_set in EXPECTED_C_LOCALE_EQUIVALENTS: @@ -357,7 +365,8 @@ def _check_c_locale_coercion(self, fs_encoding, stream_encoding, expected_warnings, - coercion_expected) + coercion_expected, + xoption=xoption) def test_PYTHONCOERCECLOCALE_not_set(self): # This should coerce to the first available target locale by default @@ -404,6 +413,32 @@ def test_LC_ALL_set_to_C(self): expected_warnings=[LEGACY_LOCALE_WARNING], coercion_expected=False) + def test_xoption_set_to_1(self): + self._check_c_locale_coercion("utf-8", "utf-8", coerce_c_locale="1", + use_xoption=True) + + def test_xoption_set_to_zero(self): + # The setting "0" should result in the locale coercion being disabled + self._check_c_locale_coercion(EXPECTED_C_LOCALE_FS_ENCODING, + EXPECTED_C_LOCALE_STREAM_ENCODING, + coerce_c_locale="0", + coercion_expected=False, + use_xoption=True) + # Setting LC_ALL=C shouldn't make any difference to the behaviour + self._check_c_locale_coercion(EXPECTED_C_LOCALE_FS_ENCODING, + EXPECTED_C_LOCALE_STREAM_ENCODING, + coerce_c_locale="0", + LC_ALL="C", + coercion_expected=False, + use_xoption=True) + + def test_xoption_set_to_warn(self): + # -X coerce_c_locale=warn enables runtime warnings for legacy locales + self._check_c_locale_coercion("utf-8", "utf-8", + coerce_c_locale="warn", + expected_warnings=[CLI_COERCION_WARNING], + use_xoption=True) + def test_main(): test.support.run_unittest( LocaleConfigurationTests, diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index 95cdc8db7efb..0424f75b3b8c 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -159,13 +159,16 @@ def test_undecodable_code(self): env = os.environ.copy() # Use C locale to get ascii for the locale encoding env['LC_ALL'] = 'C' - env['PYTHONCOERCECLOCALE'] = '0' code = ( b'import locale; ' b'print(ascii("' + undecodable + b'"), ' b'locale.getpreferredencoding())') p = subprocess.Popen( - [sys.executable, "-c", code], + [sys.executable, + # Disable C locale coercion and UTF-8 Mode to not use UTF-8 + "-X", "coerce_c_locale=0", + "-X", "utf8=0", + "-c", code], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=env) stdout, stderr = p.communicate() diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 29274580339f..79c34ec591a0 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -267,9 +267,6 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): 'malloc_stats': 0, 'utf8_mode': 0, - 'coerce_c_locale': 0, - 'coerce_c_locale_warn': 0, - 'program_name': './_testembed', 'argc': 0, 'argv': '[]', @@ -290,6 +287,8 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): '_disable_importlib': 0, 'Py_FrozenFlag': 0, + '_coerce_c_locale': 0, + '_coerce_c_locale_warn': 0, } def check_config(self, testname, expected): diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index 27f75901c63f..9155afc069e1 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -656,9 +656,8 @@ def test_getfilesystemencoding(self): def c_locale_get_error_handler(self, locale, isolated=False, encoding=None): # Force the POSIX locale - env = os.environ.copy() + env = dict(os.environ) env["LC_ALL"] = locale - env["PYTHONCOERCECLOCALE"] = "0" code = '\n'.join(( 'import sys', 'def dump(name):', @@ -668,7 +667,10 @@ def c_locale_get_error_handler(self, locale, isolated=False, encoding=None): 'dump("stdout")', 'dump("stderr")', )) - args = [sys.executable, "-c", code] + args = [sys.executable, + "-X", "utf8=0", + "-X", "coerce_c_locale=0", + "-c", code] if isolated: args.append("-I") if encoding is not None: diff --git a/Lib/test/test_utf8_mode.py b/Lib/test/test_utf8_mode.py index 554abfab3163..8c64276bcf68 100644 --- a/Lib/test/test_utf8_mode.py +++ b/Lib/test/test_utf8_mode.py @@ -27,6 +27,8 @@ def posix_locale(self): return (loc in POSIX_LOCALES) def get_output(self, *args, failure=False, **kw): + # Always disable the C locale coercion (PEP 538) + args = ('-X', 'coerce_c_locale=0', *args) kw = dict(self.DEFAULT_ENV, **kw) if failure: out = assert_python_failure(*args, **kw) @@ -116,7 +118,6 @@ def test_filesystemencoding(self): # PYTHONLEGACYWINDOWSFSENCODING disables the UTF-8 mode # and has the priority over -X utf8 and PYTHONUTF8 out = self.get_output('-X', 'utf8', '-c', code, - PYTHONUTF8='strict', PYTHONLEGACYWINDOWSFSENCODING='1') self.assertEqual(out, 'mbcs/replace') diff --git a/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst b/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst new file mode 100644 index 000000000000..27b6a6e0017f --- /dev/null +++ b/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst @@ -0,0 +1,3 @@ +Py_Initialize() and Py_Main() cannot enable the C locale coercion (PEP 538) +anymore: it is always disabled. It can now only be enabled by the Python +program ("python3). diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst new file mode 100644 index 000000000000..618092d192c4 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst @@ -0,0 +1,2 @@ +Add a new :option:`-X` ``coerce_c_locale`` command line option to control C +locale coercion (:pep:`538`). diff --git a/Modules/main.c b/Modules/main.c index 0f7498d6104f..c6ffb15d9f99 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -1834,6 +1834,17 @@ config_init_utf8_mode(_PyCoreConfig *config) return _Py_INIT_OK(); } +#ifndef MS_WINDOWS + /* The C locale and the POSIX locale enable the UTF-8 Mode (PEP 540) */ + const char *ctype_loc = setlocale(LC_CTYPE, NULL); + if (ctype_loc != NULL + && (strcmp(ctype_loc, "C") == 0 || strcmp(ctype_loc, "POSIX") == 0)) + { + config->utf8_mode = 1; + return _Py_INIT_OK(); + } +#endif + return _Py_INIT_OK(); } @@ -1857,16 +1868,18 @@ config_read_env_vars(_PyCoreConfig *config) const char *env = config_get_env_var("PYTHONCOERCECLOCALE"); if (env) { if (strcmp(env, "0") == 0) { - if (config->coerce_c_locale < 0) { - config->coerce_c_locale = 0; + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 0; } } else if (strcmp(env, "warn") == 0) { - config->coerce_c_locale_warn = 1; + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 1; + } } else { - if (config->coerce_c_locale < 0) { - config->coerce_c_locale = 1; + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 1; } } } @@ -2046,7 +2059,7 @@ pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) * See the documentation of the PYTHONCOERCECLOCALE setting for more * details. */ - if (config->coerce_c_locale && !locale_coerced) { + if (config->_coerce_c_locale && !locale_coerced) { locale_coerced = 1; _Py_CoerceLegacyLocale(config); encoding_changed = 1; @@ -2073,7 +2086,7 @@ pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) pymain_read_conf_impl(). Reset Py_IsolatedFlag and Py_NoSiteFlag modified by _PyCoreConfig_Read(). */ int new_utf8_mode = config->utf8_mode; - int new_coerce_c_locale = config->coerce_c_locale; + int new_coerce_c_locale = config->_coerce_c_locale; Py_IgnoreEnvironmentFlag = init_ignore_env; if (_PyCoreConfig_Copy(config, &save_config) < 0) { pymain->err = _Py_INIT_NO_MEMORY(); @@ -2085,7 +2098,7 @@ pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) cmdline_get_global_config(cmdline); _PyCoreConfig_GetGlobalConfig(config); config->utf8_mode = new_utf8_mode; - config->coerce_c_locale = new_coerce_c_locale; + config->_coerce_c_locale = new_coerce_c_locale; /* The encoding changed: read again the configuration with the new encoding */ @@ -2103,28 +2116,76 @@ pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) } -static void -config_init_locale(_PyCoreConfig *config) +static _PyInitError +config_init_coerce_c_locale(_PyCoreConfig *config) { - if (config->coerce_c_locale < 0) { - /* The C locale enables the C locale coercion (PEP 538) */ - if (_Py_LegacyLocaleDetected()) { - config->coerce_c_locale = 1; + const wchar_t *xopt = config_get_xoption(config, L"coerce_c_locale"); + if (xopt) { + wchar_t *sep = wcschr(xopt, L'='); + if (sep) { + xopt = sep + 1; + if (wcscmp(xopt, L"1") == 0) { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 1; + } + } + else if (wcscmp(xopt, L"0") == 0) { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 0; + } + } + else if (wcscmp(xopt, L"warn") == 0) { + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 1; + } + } + else { + return _Py_INIT_USER_ERR("invalid -X coerce_c_locale option value"); + } + } + else { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 1; + } + } + + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 0; } } -#ifndef MS_WINDOWS - if (config->utf8_mode < 0) { - /* The C locale and the POSIX locale enable the UTF-8 Mode (PEP 540) */ - const char *ctype_loc = setlocale(LC_CTYPE, NULL); - if (ctype_loc != NULL - && (strcmp(ctype_loc, "C") == 0 - || strcmp(ctype_loc, "POSIX") == 0)) - { - config->utf8_mode = 1; + const char *env = config_get_env_var("PYTHONCOERCECLOCALE"); + if (env) { + if (strcmp(env, "0") == 0) { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 0; + } + } + else if (strcmp(env, "warn") == 0) { + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 1; + } + } + else { + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 1; + } + } + + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 0; } } -#endif + + if (config->_coerce_c_locale < 0) { + /* The C locale enables the C locale coercion (PEP 538) */ + if (_Py_LegacyLocaleDetected()) { + config->_coerce_c_locale = 1; + return _Py_INIT_OK(); + } + } + + return _Py_INIT_OK(); } @@ -2284,8 +2345,11 @@ _PyCoreConfig_Read(_PyCoreConfig *config) } } - if (config->utf8_mode < 0 || config->coerce_c_locale < 0) { - config_init_locale(config); + if (config->_coerce_c_locale < 0 || config->_coerce_c_locale_warn < 0) { + err = config_init_coerce_c_locale(config); + if (_Py_INIT_FAILED(err)) { + return err; + } } if (!config->_disable_importlib) { @@ -2317,8 +2381,11 @@ _PyCoreConfig_Read(_PyCoreConfig *config) if (config->tracemalloc < 0) { config->tracemalloc = 0; } - if (config->coerce_c_locale < 0) { - config->coerce_c_locale = 0; + if (config->_coerce_c_locale < 0) { + config->_coerce_c_locale = 0; + } + if (config->_coerce_c_locale_warn < 0) { + config->_coerce_c_locale_warn = 0; } if (config->utf8_mode < 0) { config->utf8_mode = 0; @@ -2327,6 +2394,10 @@ _PyCoreConfig_Read(_PyCoreConfig *config) config->argc = 0; } + assert(config->_coerce_c_locale >= 0); + assert(config->_coerce_c_locale_warn >= 0); + assert(config->ignore_environment >= 0); + return _Py_INIT_OK(); } @@ -2410,8 +2481,8 @@ _PyCoreConfig_Copy(_PyCoreConfig *config, const _PyCoreConfig *config2) COPY_ATTR(dump_refs); COPY_ATTR(malloc_stats); - COPY_ATTR(coerce_c_locale); - COPY_ATTR(coerce_c_locale_warn); + COPY_ATTR(_coerce_c_locale); + COPY_ATTR(_coerce_c_locale_warn); COPY_ATTR(utf8_mode); COPY_STR_ATTR(module_search_path_env); @@ -2638,7 +2709,7 @@ pymain_run_python(_PyMain *pymain) static void -pymain_init(_PyMain *pymain) +pymain_init(_PyMain *pymain, int use_c_locale_coercion) { /* 754 requires that FP exceptions run in "no stop" mode by default, * and until C vendors implement C99's ways to control FP exceptions, @@ -2651,6 +2722,11 @@ pymain_init(_PyMain *pymain) pymain->config._disable_importlib = 0; pymain->config.install_signal_handlers = 1; + if (use_c_locale_coercion) { + /* set to -1 to be able to enable the feature */ + pymain->config._coerce_c_locale = -1; + pymain->config._coerce_c_locale_warn = -1; + } } @@ -2751,9 +2827,9 @@ pymain_cmdline(_PyMain *pymain) static int -pymain_main(_PyMain *pymain) +pymain_main(_PyMain *pymain, int use_c_locale_coercion) { - pymain_init(pymain); + pymain_init(pymain, use_c_locale_coercion); int res = pymain_cmdline(pymain); if (res < 0) { @@ -2802,10 +2878,22 @@ Py_Main(int argc, wchar_t **argv) pymain.argc = argc; pymain.wchar_argv = argv; - return pymain_main(&pymain); + return pymain_main(&pymain, 0); } +#ifdef MS_WINDOWS +int +_Py_WindowsMain(int argc, wchar_t **argv) +{ + _PyMain pymain = _PyMain_INIT; + pymain.use_bytes_argv = 0; + pymain.argc = argc; + pymain.wchar_argv = argv; + + return pymain_main(&pymain, 1); +} +#else int _Py_UnixMain(int argc, char **argv) { @@ -2814,8 +2902,9 @@ _Py_UnixMain(int argc, char **argv) pymain.argc = argc; pymain.bytes_argv = argv; - return pymain_main(&pymain); + return pymain_main(&pymain, 1); } +#endif /* this is gonna seem *real weird*, but if you put some other code between diff --git a/Programs/_testembed.c b/Programs/_testembed.c index 6c35f9586bfe..029f6ad4b680 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -328,8 +328,8 @@ dump_config(void) printf("dump_refs = %i\n", config->dump_refs); printf("malloc_stats = %i\n", config->malloc_stats); - printf("coerce_c_locale = %i\n", config->coerce_c_locale); - printf("coerce_c_locale_warn = %i\n", config->coerce_c_locale_warn); + printf("_coerce_c_locale = %i\n", config->_coerce_c_locale); + printf("_coerce_c_locale_warn = %i\n", config->_coerce_c_locale_warn); printf("utf8_mode = %i\n", config->utf8_mode); printf("program_name = %ls\n", config->program_name); @@ -473,8 +473,6 @@ static int test_init_from_config(void) putenv("PYTHONMALLOCSTATS=0"); config.malloc_stats = 1; - /* FIXME: test coerce_c_locale and coerce_c_locale_warn */ - putenv("PYTHONUTF8=0"); Py_UTF8Mode = 0; config.utf8_mode = 1; @@ -544,8 +542,7 @@ static int test_init_isolated(void) /* Test _PyCoreConfig.isolated=1 */ _PyCoreConfig config = _PyCoreConfig_INIT; - /* Set coerce_c_locale and utf8_mode to not depend on the locale */ - config.coerce_c_locale = 0; + /* Set utf8_mode to not depend on the locale */ config.utf8_mode = 0; /* Use path starting with "./" avoids a search along the PATH */ config.program_name = L"./_testembed"; diff --git a/Programs/python.c b/Programs/python.c index 78e48f800c95..c7697facbe3b 100644 --- a/Programs/python.c +++ b/Programs/python.c @@ -6,7 +6,7 @@ int wmain(int argc, wchar_t **argv) { - return Py_Main(argc, argv); + return _Py_WindowsMain(argc, argv); } #else int diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index ba4b54864fd8..ef742c16c014 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -400,7 +400,7 @@ static const char *_C_LOCALE_WARNING = static void _emit_stderr_warning_for_legacy_locale(const _PyCoreConfig *core_config) { - if (core_config->coerce_c_locale_warn) { + if (core_config->_coerce_c_locale_warn) { if (_Py_LegacyLocaleDetected()) { fprintf(stderr, "%s", _C_LOCALE_WARNING); } @@ -462,7 +462,7 @@ _coerce_default_locale_settings(const _PyCoreConfig *config, const _LocaleCoerci "Error setting LC_CTYPE, skipping C locale coercion\n"); return; } - if (config->coerce_c_locale_warn) { + if (config->_coerce_c_locale_warn) { fprintf(stderr, C_LOCALE_COERCION_WARNING, newloc); } From webhook-mailer at python.org Mon Sep 17 23:58:06 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 18 Sep 2018 03:58:06 -0000 Subject: [Python-checkins] bpo-33649: Note that asyncio.run() calls shutdown_asyncgens() (GH-9380) Message-ID: https://github.com/python/cpython/commit/ac94e38d076aebc56c2ff96a249b5e40f32633ea commit: ac94e38d076aebc56c2ff96a249b5e40f32633ea branch: master author: Yury Selivanov committer: GitHub date: 2018-09-17T23:58:00-04:00 summary: bpo-33649: Note that asyncio.run() calls shutdown_asyncgens() (GH-9380) files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 0320d798c1e5..20d18c02976b 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -152,8 +152,12 @@ Running and stopping the loop close with an :meth:`~agen.aclose()` call. After calling this method, the event loop will issue a warning if a new asynchronous generator is iterated. This should be used to reliably finalize all scheduled - asynchronous generators, e.g.:: + asynchronous generators. + Note that there is no need to call this function when + :func:`asyncio.run` is used. + + Example:: try: loop.run_forever() From webhook-mailer at python.org Tue Sep 18 00:12:08 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 18 Sep 2018 04:12:08 -0000 Subject: [Python-checkins] bpo-33649: Note that asyncio.run() calls shutdown_asyncgens() (GH-9380) Message-ID: https://github.com/python/cpython/commit/1f4ea580675d7b30d3906d78c500997d7d96995d commit: 1f4ea580675d7b30d3906d78c500997d7d96995d branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-17T21:12:05-07:00 summary: bpo-33649: Note that asyncio.run() calls shutdown_asyncgens() (GH-9380) (cherry picked from commit ac94e38d076aebc56c2ff96a249b5e40f32633ea) Co-authored-by: Yury Selivanov files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 3b13a81a5b6a..252224218572 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -152,8 +152,12 @@ Running and stopping the loop close with an :meth:`~agen.aclose()` call. After calling this method, the event loop will issue a warning if a new asynchronous generator is iterated. This should be used to reliably finalize all scheduled - asynchronous generators, e.g.:: + asynchronous generators. + Note that there is no need to call this function when + :func:`asyncio.run` is used. + + Example:: try: loop.run_forever() From webhook-mailer at python.org Tue Sep 18 00:49:24 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 18 Sep 2018 04:49:24 -0000 Subject: [Python-checkins] Change "set_after" reference to `say_after`. (GH-9384) Message-ID: https://github.com/python/cpython/commit/7bfbda46f49c0b2c43e128835106cf13315b6ae8 commit: 7bfbda46f49c0b2c43e128835106cf13315b6ae8 branch: master author: Danny Hermes committer: Yury Selivanov date: 2018-09-18T00:49:21-04:00 summary: Change "set_after" reference to `say_after`. (GH-9384) files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 4893a7c75209..25a0768aa8da 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -75,7 +75,7 @@ To actually run a coroutine asyncio provides three main mechanisms: * The :func:`asyncio.create_task` function to run coroutines concurrently as asyncio :class:`Tasks `. - Let's modify the above example and run two "set_after" coroutines + Let's modify the above example and run two ``say_after`` coroutines *concurrently*:: async def main(): From webhook-mailer at python.org Tue Sep 18 02:01:31 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 18 Sep 2018 06:01:31 -0000 Subject: [Python-checkins] Fix syntax error on Asyncio example in doc (GH-9387) Message-ID: https://github.com/python/cpython/commit/9c53fa6ad9cd23fb03867b4a1f74264c426c1772 commit: 9c53fa6ad9cd23fb03867b4a1f74264c426c1772 branch: master author: Miguel ?ngel Garc?a committer: Yury Selivanov date: 2018-09-18T02:01:26-04:00 summary: Fix syntax error on Asyncio example in doc (GH-9387) The `gather` method requires to close the parenthesis, but it is being closed twice. files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 25a0768aa8da..d597234383db 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -238,7 +238,7 @@ Running Tasks Concurrently factorial("A", 2), factorial("B", 3), factorial("C", 4), - )) + ) asyncio.run(main()) From webhook-mailer at python.org Tue Sep 18 02:16:31 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 18 Sep 2018 06:16:31 -0000 Subject: [Python-checkins] bpo-34681: Rename class Pattern in sre_parse to State. (GH-9310) Message-ID: https://github.com/python/cpython/commit/e0c19ddc661e56cc7e694be52d3e47f1dfe5af24 commit: e0c19ddc661e56cc7e694be52d3e47f1dfe5af24 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-18T09:16:26+03:00 summary: bpo-34681: Rename class Pattern in sre_parse to State. (GH-9310) Also rename corresponding attributes, parameters and variables. files: M Lib/re.py M Lib/sre_compile.py M Lib/sre_parse.py diff --git a/Lib/re.py b/Lib/re.py index 94d486579e08..68d62dc2a93b 100644 --- a/Lib/re.py +++ b/Lib/re.py @@ -334,7 +334,7 @@ def __init__(self, lexicon, flags=0): self.lexicon = lexicon # combine phrases into a compound pattern p = [] - s = sre_parse.Pattern() + s = sre_parse.State() s.flags = flags for phrase, action in lexicon: gid = s.opengroup() diff --git a/Lib/sre_compile.py b/Lib/sre_compile.py index e5216b792f6b..36670427db76 100644 --- a/Lib/sre_compile.py +++ b/Lib/sre_compile.py @@ -597,7 +597,7 @@ def isstring(obj): def _code(p, flags): - flags = p.pattern.flags | flags + flags = p.state.flags | flags code = [] # compile info block @@ -772,13 +772,13 @@ def compile(p, flags=0): dis(code) # map in either direction - groupindex = p.pattern.groupdict - indexgroup = [None] * p.pattern.groups + groupindex = p.state.groupdict + indexgroup = [None] * p.state.groups for k, i in groupindex.items(): indexgroup[i] = k return _sre.compile( - pattern, flags | p.pattern.flags, code, - p.pattern.groups-1, + pattern, flags | p.state.flags, code, + p.state.groups-1, groupindex, tuple(indexgroup) ) diff --git a/Lib/sre_parse.py b/Lib/sre_parse.py index f414402f9379..84c912573e7f 100644 --- a/Lib/sre_parse.py +++ b/Lib/sre_parse.py @@ -71,8 +71,8 @@ class Verbose(Exception): pass -class Pattern: - # main pattern object. keeps track of global attributes +class State: + # keeps track of state for parsing def __init__(self): self.flags = 0 self.groupdict = {} @@ -108,8 +108,8 @@ def checklookbehindgroup(self, gid, source): class SubPattern: # a subpattern, in intermediate form - def __init__(self, pattern, data=None): - self.pattern = pattern + def __init__(self, state, data=None): + self.state = state if data is None: data = [] self.data = data @@ -163,7 +163,7 @@ def __delitem__(self, index): del self.data[index] def __getitem__(self, index): if isinstance(index, slice): - return SubPattern(self.pattern, self.data[index]) + return SubPattern(self.state, self.data[index]) return self.data[index] def __setitem__(self, index, code): self.data[index] = code @@ -202,7 +202,7 @@ def getwidth(self): lo = lo + 1 hi = hi + 1 elif op is GROUPREF: - i, j = self.pattern.groupwidths[av] + i, j = self.state.groupwidths[av] lo = lo + i hi = hi + j elif op is GROUPREF_EXISTS: @@ -940,28 +940,28 @@ def fix_flags(src, flags): raise ValueError("ASCII and LOCALE flags are incompatible") return flags -def parse(str, flags=0, pattern=None): +def parse(str, flags=0, state=None): # parse 're' pattern into list of (opcode, argument) tuples source = Tokenizer(str) - if pattern is None: - pattern = Pattern() - pattern.flags = flags - pattern.str = str + if state is None: + state = State() + state.flags = flags + state.str = str try: - p = _parse_sub(source, pattern, flags & SRE_FLAG_VERBOSE, 0) + p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0) except Verbose: # the VERBOSE flag was switched on inside the pattern. to be # on the safe side, we'll parse the whole thing again... - pattern = Pattern() - pattern.flags = flags | SRE_FLAG_VERBOSE - pattern.str = str + state = State() + state.flags = flags | SRE_FLAG_VERBOSE + state.str = str source.seek(0) - p = _parse_sub(source, pattern, True, 0) + p = _parse_sub(source, state, True, 0) - p.pattern.flags = fix_flags(str, p.pattern.flags) + p.state.flags = fix_flags(str, p.state.flags) if source.next is not None: assert source.next == ")" @@ -972,7 +972,7 @@ def parse(str, flags=0, pattern=None): return p -def parse_template(source, pattern): +def parse_template(source, state): # parse 're' replacement string into list of literals and # group references s = Tokenizer(source) @@ -982,14 +982,14 @@ def parse_template(source, pattern): literal = [] lappend = literal.append def addgroup(index, pos): - if index > pattern.groups: + if index > state.groups: raise s.error("invalid group reference %d" % index, pos) if literal: literals.append(''.join(literal)) del literal[:] groups.append((len(literals), index)) literals.append(None) - groupindex = pattern.groupindex + groupindex = state.groupindex while True: this = sget() if this is None: From webhook-mailer at python.org Tue Sep 18 02:27:11 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 18 Sep 2018 06:27:11 -0000 Subject: [Python-checkins] Change "set_after" reference to `say_after`. (GH-9384) (GH-9386) Message-ID: https://github.com/python/cpython/commit/9a89fd688445d10c1fd1aed589643d8266f08b1e commit: 9a89fd688445d10c1fd1aed589643d8266f08b1e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Yury Selivanov date: 2018-09-18T02:27:07-04:00 summary: Change "set_after" reference to `say_after`. (GH-9384) (GH-9386) (cherry picked from commit 7bfbda46f49c0b2c43e128835106cf13315b6ae8) Co-authored-by: Danny Hermes files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 670e4a5fbe04..9ca0dcd7f57c 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -75,7 +75,7 @@ To actually run a coroutine asyncio provides three main mechanisms: * The :func:`asyncio.create_task` function to run coroutines concurrently as asyncio :class:`Tasks `. - Let's modify the above example and run two "set_after" coroutines + Let's modify the above example and run two ``say_after`` coroutines *concurrently*:: async def main(): From webhook-mailer at python.org Tue Sep 18 02:27:29 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 18 Sep 2018 06:27:29 -0000 Subject: [Python-checkins] Fix syntax error on Asyncio example in doc (GH-9387) (GH-9388) Message-ID: https://github.com/python/cpython/commit/ee2ff1a335b0a59f445959178c5d2a9802c5b3e4 commit: ee2ff1a335b0a59f445959178c5d2a9802c5b3e4 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Yury Selivanov date: 2018-09-18T02:27:27-04:00 summary: Fix syntax error on Asyncio example in doc (GH-9387) (GH-9388) The `gather` method requires to close the parenthesis, but it is being closed twice. (cherry picked from commit 9c53fa6ad9cd23fb03867b4a1f74264c426c1772) Co-authored-by: Miguel ?ngel Garc?a files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 9ca0dcd7f57c..2753998745ef 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -232,7 +232,7 @@ Running Tasks Concurrently factorial("A", 2), factorial("B", 3), factorial("C", 4), - )) + ) asyncio.run(main()) From webhook-mailer at python.org Tue Sep 18 02:47:59 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 18 Sep 2018 06:47:59 -0000 Subject: [Python-checkins] bpo-33649: Fix markup; add another note that asyncio.run is 3.7+ (GH-9389) Message-ID: https://github.com/python/cpython/commit/b042cf10c6084d14279c55a7e0d2d7595ff4e694 commit: b042cf10c6084d14279c55a7e0d2d7595ff4e694 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-18T02:47:54-04:00 summary: bpo-33649: Fix markup; add another note that asyncio.run is 3.7+ (GH-9389) files: M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index d597234383db..4f37296f4a3e 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -20,7 +20,8 @@ Coroutines Coroutines declared with async/await syntax is the preferred way of writing asyncio applications. For example, the following snippet -of code prints "hello", waits 1 second, and then prints "world":: +of code (requires Python 3.7+) prints "hello", waits 1 second, +and then prints "world":: >>> import asyncio diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index bfc97001bb71..1511b2f75a89 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -8,7 +8,7 @@ .. sidebar:: Hello World! - .. code-block:: python + :: import asyncio From webhook-mailer at python.org Tue Sep 18 02:54:29 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 18 Sep 2018 06:54:29 -0000 Subject: [Python-checkins] bpo-32455: Add jump parameter to dis.stack_effect(). (GH-6610) Message-ID: https://github.com/python/cpython/commit/7bdf28265aa371b39f82dfc6562635801aff15a5 commit: 7bdf28265aa371b39f82dfc6562635801aff15a5 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-18T09:54:26+03:00 summary: bpo-32455: Add jump parameter to dis.stack_effect(). (GH-6610) Add C API function PyCompile_OpcodeStackEffectWithJump(). files: A Misc/NEWS.d/next/C API/2018-07-08-12-06-18.bpo-32455.KVHlkz.rst A Misc/NEWS.d/next/Library/2018-04-26-13-31-10.bpo-32455.KPWg3K.rst M Doc/library/dis.rst M Include/compile.h M Lib/test/test__opcode.py M Modules/_opcode.c M Modules/clinic/_opcode.c.h M Python/compile.c diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 95649379ac6d..fe9979db0eaa 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -248,12 +248,21 @@ operation is being performed, so the intermediate analysis object isn't useful: return a list of these offsets. -.. function:: stack_effect(opcode, [oparg]) +.. function:: stack_effect(opcode, oparg=None, *, jump=None) Compute the stack effect of *opcode* with argument *oparg*. + If the code has a jump target and *jump* is ``True``, :func:`~stack_effect` + will return the stack effect of jumping. If *jump* is ``False``, + it will return the stack effect of not jumping. And if *jump* is + ``None`` (default), it will return the maximal stack effect of both cases. + .. versionadded:: 3.4 + .. versionchanged:: 3.8 + Added *jump* parameter. + + .. _bytecodes: Python Bytecode Instructions diff --git a/Include/compile.h b/Include/compile.h index edb961f4d720..2dacfff37f8c 100644 --- a/Include/compile.h +++ b/Include/compile.h @@ -75,6 +75,7 @@ PyAPI_FUNC(PyObject*) _Py_Mangle(PyObject *p, PyObject *name); #define PY_INVALID_STACK_EFFECT INT_MAX PyAPI_FUNC(int) PyCompile_OpcodeStackEffect(int opcode, int oparg); +PyAPI_FUNC(int) PyCompile_OpcodeStackEffectWithJump(int opcode, int oparg, int jump); PyAPI_FUNC(int) _PyAST_Optimize(struct _mod *, PyArena *arena, int optimize); diff --git a/Lib/test/test__opcode.py b/Lib/test/test__opcode.py index 2af1ee35bff0..0fb39eed6064 100644 --- a/Lib/test/test__opcode.py +++ b/Lib/test/test__opcode.py @@ -3,32 +3,65 @@ import unittest _opcode = import_module("_opcode") +from _opcode import stack_effect + class OpcodeTests(unittest.TestCase): def test_stack_effect(self): - self.assertEqual(_opcode.stack_effect(dis.opmap['POP_TOP']), -1) - self.assertEqual(_opcode.stack_effect(dis.opmap['DUP_TOP_TWO']), 2) - self.assertEqual(_opcode.stack_effect(dis.opmap['BUILD_SLICE'], 0), -1) - self.assertEqual(_opcode.stack_effect(dis.opmap['BUILD_SLICE'], 1), -1) - self.assertEqual(_opcode.stack_effect(dis.opmap['BUILD_SLICE'], 3), -2) - self.assertRaises(ValueError, _opcode.stack_effect, 30000) - self.assertRaises(ValueError, _opcode.stack_effect, dis.opmap['BUILD_SLICE']) - self.assertRaises(ValueError, _opcode.stack_effect, dis.opmap['POP_TOP'], 0) + self.assertEqual(stack_effect(dis.opmap['POP_TOP']), -1) + self.assertEqual(stack_effect(dis.opmap['DUP_TOP_TWO']), 2) + self.assertEqual(stack_effect(dis.opmap['BUILD_SLICE'], 0), -1) + self.assertEqual(stack_effect(dis.opmap['BUILD_SLICE'], 1), -1) + self.assertEqual(stack_effect(dis.opmap['BUILD_SLICE'], 3), -2) + self.assertRaises(ValueError, stack_effect, 30000) + self.assertRaises(ValueError, stack_effect, dis.opmap['BUILD_SLICE']) + self.assertRaises(ValueError, stack_effect, dis.opmap['POP_TOP'], 0) # All defined opcodes for name, code in dis.opmap.items(): with self.subTest(opname=name): if code < dis.HAVE_ARGUMENT: - _opcode.stack_effect(code) - self.assertRaises(ValueError, _opcode.stack_effect, code, 0) + stack_effect(code) + self.assertRaises(ValueError, stack_effect, code, 0) else: - _opcode.stack_effect(code, 0) - self.assertRaises(ValueError, _opcode.stack_effect, code) + stack_effect(code, 0) + self.assertRaises(ValueError, stack_effect, code) # All not defined opcodes for code in set(range(256)) - set(dis.opmap.values()): with self.subTest(opcode=code): - self.assertRaises(ValueError, _opcode.stack_effect, code) - self.assertRaises(ValueError, _opcode.stack_effect, code, 0) + self.assertRaises(ValueError, stack_effect, code) + self.assertRaises(ValueError, stack_effect, code, 0) + + def test_stack_effect_jump(self): + JUMP_IF_TRUE_OR_POP = dis.opmap['JUMP_IF_TRUE_OR_POP'] + self.assertEqual(stack_effect(JUMP_IF_TRUE_OR_POP, 0), 0) + self.assertEqual(stack_effect(JUMP_IF_TRUE_OR_POP, 0, jump=True), 0) + self.assertEqual(stack_effect(JUMP_IF_TRUE_OR_POP, 0, jump=False), -1) + FOR_ITER = dis.opmap['FOR_ITER'] + self.assertEqual(stack_effect(FOR_ITER, 0), 1) + self.assertEqual(stack_effect(FOR_ITER, 0, jump=True), -1) + self.assertEqual(stack_effect(FOR_ITER, 0, jump=False), 1) + JUMP_FORWARD = dis.opmap['JUMP_FORWARD'] + self.assertEqual(stack_effect(JUMP_FORWARD, 0), 0) + self.assertEqual(stack_effect(JUMP_FORWARD, 0, jump=True), 0) + self.assertEqual(stack_effect(JUMP_FORWARD, 0, jump=False), 0) + # All defined opcodes + has_jump = dis.hasjabs + dis.hasjrel + for name, code in dis.opmap.items(): + with self.subTest(opname=name): + if code < dis.HAVE_ARGUMENT: + common = stack_effect(code) + jump = stack_effect(code, jump=True) + nojump = stack_effect(code, jump=False) + else: + common = stack_effect(code, 0) + jump = stack_effect(code, 0, jump=True) + nojump = stack_effect(code, 0, jump=False) + if code in has_jump: + self.assertEqual(common, max(jump, nojump)) + else: + self.assertEqual(jump, common) + self.assertEqual(nojump, common) if __name__ == "__main__": diff --git a/Misc/NEWS.d/next/C API/2018-07-08-12-06-18.bpo-32455.KVHlkz.rst b/Misc/NEWS.d/next/C API/2018-07-08-12-06-18.bpo-32455.KVHlkz.rst new file mode 100644 index 000000000000..f28be876cf39 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2018-07-08-12-06-18.bpo-32455.KVHlkz.rst @@ -0,0 +1 @@ +Added :c:func:`PyCompile_OpcodeStackEffectWithJump`. diff --git a/Misc/NEWS.d/next/Library/2018-04-26-13-31-10.bpo-32455.KPWg3K.rst b/Misc/NEWS.d/next/Library/2018-04-26-13-31-10.bpo-32455.KPWg3K.rst new file mode 100644 index 000000000000..dd873f77bbd4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-04-26-13-31-10.bpo-32455.KPWg3K.rst @@ -0,0 +1 @@ +Added *jump* parameter to :func:`dis.stack_effect`. diff --git a/Modules/_opcode.c b/Modules/_opcode.c index f9c1c0108d6a..42a8732694af 100644 --- a/Modules/_opcode.c +++ b/Modules/_opcode.c @@ -15,16 +15,20 @@ _opcode.stack_effect -> int opcode: int oparg: object = None / + * + jump: object = None Compute the stack effect of the opcode. [clinic start generated code]*/ static int -_opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg) -/*[clinic end generated code: output=ad39467fa3ad22ce input=2d0a9ee53c0418f5]*/ +_opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg, + PyObject *jump) +/*[clinic end generated code: output=64a18f2ead954dbb input=461c9d4a44851898]*/ { int effect; int oparg_int = 0; + int jump_int; if (HAS_ARG(opcode)) { if (oparg == Py_None) { PyErr_SetString(PyExc_ValueError, @@ -40,7 +44,21 @@ _opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg) "stack_effect: opcode does not permit oparg but oparg was specified"); return -1; } - effect = PyCompile_OpcodeStackEffect(opcode, oparg_int); + if (jump == Py_None) { + jump_int = -1; + } + else if (jump == Py_True) { + jump_int = 1; + } + else if (jump == Py_False) { + jump_int = 0; + } + else { + PyErr_SetString(PyExc_ValueError, + "stack_effect: jump must be False, True or None"); + return -1; + } + effect = PyCompile_OpcodeStackEffectWithJump(opcode, oparg_int, jump_int); if (effect == PY_INVALID_STACK_EFFECT) { PyErr_SetString(PyExc_ValueError, "invalid opcode or oparg"); diff --git a/Modules/clinic/_opcode.c.h b/Modules/clinic/_opcode.c.h index 4d593edfac08..b162d84e5db4 100644 --- a/Modules/clinic/_opcode.c.h +++ b/Modules/clinic/_opcode.c.h @@ -3,30 +3,34 @@ preserve [clinic start generated code]*/ PyDoc_STRVAR(_opcode_stack_effect__doc__, -"stack_effect($module, opcode, oparg=None, /)\n" +"stack_effect($module, opcode, oparg=None, /, *, jump=None)\n" "--\n" "\n" "Compute the stack effect of the opcode."); #define _OPCODE_STACK_EFFECT_METHODDEF \ - {"stack_effect", (PyCFunction)_opcode_stack_effect, METH_FASTCALL, _opcode_stack_effect__doc__}, + {"stack_effect", (PyCFunction)_opcode_stack_effect, METH_FASTCALL|METH_KEYWORDS, _opcode_stack_effect__doc__}, static int -_opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg); +_opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg, + PyObject *jump); static PyObject * -_opcode_stack_effect(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +_opcode_stack_effect(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", "", "jump", NULL}; + static _PyArg_Parser _parser = {"i|O$O:stack_effect", _keywords, 0}; int opcode; PyObject *oparg = Py_None; + PyObject *jump = Py_None; int _return_value; - if (!_PyArg_ParseStack(args, nargs, "i|O:stack_effect", - &opcode, &oparg)) { + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &opcode, &oparg, &jump)) { goto exit; } - _return_value = _opcode_stack_effect_impl(module, opcode, oparg); + _return_value = _opcode_stack_effect_impl(module, opcode, oparg, jump); if ((_return_value == -1) && PyErr_Occurred()) { goto exit; } @@ -35,4 +39,4 @@ _opcode_stack_effect(PyObject *module, PyObject *const *args, Py_ssize_t nargs) exit: return return_value; } -/*[clinic end generated code: output=577a91c9aa5559a9 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=bbf6c4cfc91edc29 input=a9049054013a1b77]*/ diff --git a/Python/compile.c b/Python/compile.c index ebd73fb72319..707da79ab662 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -1116,6 +1116,12 @@ stack_effect(int opcode, int oparg, int jump) return PY_INVALID_STACK_EFFECT; /* not reachable */ } +int +PyCompile_OpcodeStackEffectWithJump(int opcode, int oparg, int jump) +{ + return stack_effect(opcode, oparg, jump); +} + int PyCompile_OpcodeStackEffect(int opcode, int oparg) { From webhook-mailer at python.org Tue Sep 18 03:01:03 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 18 Sep 2018 07:01:03 -0000 Subject: [Python-checkins] bpo-33649: Fix markup; add another note that asyncio.run is 3.7+ (GH-9389) Message-ID: https://github.com/python/cpython/commit/45452b738b7f94221a94e903fb5975222fbb7a8f commit: 45452b738b7f94221a94e903fb5975222fbb7a8f branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-18T00:00:58-07:00 summary: bpo-33649: Fix markup; add another note that asyncio.run is 3.7+ (GH-9389) (cherry picked from commit b042cf10c6084d14279c55a7e0d2d7595ff4e694) Co-authored-by: Yury Selivanov files: M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 2753998745ef..45b8b604200e 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -20,7 +20,8 @@ Coroutines Coroutines declared with async/await syntax is the preferred way of writing asyncio applications. For example, the following snippet -of code prints "hello", waits 1 second, and then prints "world":: +of code (requires Python 3.7+) prints "hello", waits 1 second, +and then prints "world":: >>> import asyncio diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index bfc97001bb71..1511b2f75a89 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -8,7 +8,7 @@ .. sidebar:: Hello World! - .. code-block:: python + :: import asyncio From webhook-mailer at python.org Tue Sep 18 04:29:04 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 18 Sep 2018 08:29:04 -0000 Subject: [Python-checkins] bpo-33721: Make some os.path functions and pathlib.Path methods be tolerant to invalid paths. (#7695) Message-ID: https://github.com/python/cpython/commit/0185f34ddcf07b78feb6ac666fbfd4615d26b028 commit: 0185f34ddcf07b78feb6ac666fbfd4615d26b028 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-18T11:28:51+03:00 summary: bpo-33721: Make some os.path functions and pathlib.Path methods be tolerant to invalid paths. (#7695) Such functions as os.path.exists(), os.path.lexists(), os.path.isdir(), os.path.isfile(), os.path.islink(), and os.path.ismount() now return False instead of raising ValueError or its subclasses UnicodeEncodeError and UnicodeDecodeError for paths that contain characters or bytes unrepresentative at the OS level. files: A Misc/NEWS.d/next/Library/2018-06-14-17-53-30.bpo-33721.8i9_9A.rst M Doc/library/os.path.rst M Doc/library/pathlib.rst M Doc/whatsnew/3.8.rst M Lib/genericpath.py M Lib/macpath.py M Lib/ntpath.py M Lib/pathlib.py M Lib/posixpath.py M Lib/test/test_genericpath.py M Lib/test/test_pathlib.py M Lib/test/test_posixpath.py M Lib/test/test_site.py M Modules/clinic/posixmodule.c.h M Modules/posixmodule.c diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst index 36bb21c222ed..5a0b178b9d04 100644 --- a/Doc/library/os.path.rst +++ b/Doc/library/os.path.rst @@ -55,6 +55,14 @@ the :mod:`glob` module.) * :mod:`macpath` for old-style MacOS paths +.. versionchanged:: 3.8 + + :func:`exists`, :func:`lexists`, :func:`isdir`, :func:`isfile`, + :func:`islink`, and :func:`ismount` now return ``False`` instead of + raising an exception for paths that contain characters or bytes + unrepresentable at the OS level. + + .. function:: abspath(path) Return a normalized absolutized version of the pathname *path*. On most diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst index ec604f681593..fc193000ba69 100644 --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -638,7 +638,17 @@ Methods Concrete paths provide the following methods in addition to pure paths methods. Many of these methods can raise an :exc:`OSError` if a system -call fails (for example because the path doesn't exist): +call fails (for example because the path doesn't exist). + +.. versionchanged:: 3.8 + + :meth:`~Path.exists()`, :meth:`~Path.is_dir()`, :meth:`~Path.is_file()`, + :meth:`~Path.is_mount()`, :meth:`~Path.is_symlink()`, + :meth:`~Path.is_block_device()`, :meth:`~Path.is_char_device()`, + :meth:`~Path.is_fifo()`, :meth:`~Path.is_socket()` now return ``False`` + instead of raising an exception for paths that contain characters + unrepresentable at the OS level. + .. classmethod:: Path.cwd() diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 38b8623dddd2..1c129a704429 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -112,6 +112,31 @@ New Modules Improved Modules ================ +os.path +------- + +:mod:`os.path` functions that return a boolean result like +:func:`~os.path.exists`, :func:`~os.path.lexists`, :func:`~os.path.isdir`, +:func:`~os.path.isfile`, :func:`~os.path.islink`, and :func:`~os.path.ismount` +now return ``False`` instead of raising :exc:`ValueError` or its subclasses +:exc:`UnicodeEncodeError` and :exc:`UnicodeDecodeError` for paths that contain +characters or bytes unrepresentable at the OS level. +(Contributed by Serhiy Storchaka in :issue:`33721`.) + +pathlib +------- + +:mod:`pathlib.Path` methods that return a boolean result like +:meth:`~pathlib.Path.exists()`, :meth:`~pathlib.Path.is_dir()`, +:meth:`~pathlib.Path.is_file()`, :meth:`~pathlib.Path.is_mount()`, +:meth:`~pathlib.Path.is_symlink()`, :meth:`~pathlib.Path.is_block_device()`, +:meth:`~pathlib.Path.is_char_device()`, :meth:`~pathlib.Path.is_fifo()`, +:meth:`~pathlib.Path.is_socket()` now return ``False`` instead of raising +:exc:`ValueError` or its subclass :exc:`UnicodeEncodeError` for paths that +contain characters unrepresentable at the OS level. +(Contributed by Serhiy Storchaka in :issue:`33721`.) + + Optimizations ============= diff --git a/Lib/genericpath.py b/Lib/genericpath.py index 303b3b349a9f..5dd703d736c5 100644 --- a/Lib/genericpath.py +++ b/Lib/genericpath.py @@ -17,7 +17,7 @@ def exists(path): """Test whether a path exists. Returns False for broken symbolic links""" try: os.stat(path) - except OSError: + except (OSError, ValueError): return False return True @@ -28,7 +28,7 @@ def isfile(path): """Test whether a path is a regular file""" try: st = os.stat(path) - except OSError: + except (OSError, ValueError): return False return stat.S_ISREG(st.st_mode) @@ -40,7 +40,7 @@ def isdir(s): """Return true if the pathname refers to an existing directory.""" try: st = os.stat(s) - except OSError: + except (OSError, ValueError): return False return stat.S_ISDIR(st.st_mode) diff --git a/Lib/macpath.py b/Lib/macpath.py index aacf7235b011..9a12d2feee35 100644 --- a/Lib/macpath.py +++ b/Lib/macpath.py @@ -138,7 +138,7 @@ def lexists(path): try: st = os.lstat(path) - except OSError: + except (OSError, ValueError): return False return True diff --git a/Lib/ntpath.py b/Lib/ntpath.py index f0e03a2f496a..0e6de2829f32 100644 --- a/Lib/ntpath.py +++ b/Lib/ntpath.py @@ -229,7 +229,7 @@ def islink(path): """ try: st = os.lstat(path) - except (OSError, AttributeError): + except (OSError, ValueError, AttributeError): return False return stat.S_ISLNK(st.st_mode) @@ -239,7 +239,7 @@ def lexists(path): """Test whether a path exists. Returns True for broken symbolic links""" try: st = os.lstat(path) - except OSError: + except (OSError, ValueError): return False return True @@ -524,7 +524,7 @@ def abspath(path): """Return the absolute version of a path.""" try: return _getfullpathname(path) - except OSError: + except (OSError, ValueError): return _abspath_fallback(path) # realpath is a no-op on systems without islink support diff --git a/Lib/pathlib.py b/Lib/pathlib.py index c2986bd022d0..89dffa5561a7 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -1331,6 +1331,9 @@ def exists(self): if e.errno not in _IGNORED_ERROS: raise return False + except ValueError: + # Non-encodable path + return False return True def is_dir(self): @@ -1345,6 +1348,9 @@ def is_dir(self): # Path doesn't exist or is a broken symlink # (see https://bitbucket.org/pitrou/pathlib/issue/12/) return False + except ValueError: + # Non-encodable path + return False def is_file(self): """ @@ -1359,6 +1365,9 @@ def is_file(self): # Path doesn't exist or is a broken symlink # (see https://bitbucket.org/pitrou/pathlib/issue/12/) return False + except ValueError: + # Non-encodable path + return False def is_mount(self): """ @@ -1392,6 +1401,9 @@ def is_symlink(self): raise # Path doesn't exist return False + except ValueError: + # Non-encodable path + return False def is_block_device(self): """ @@ -1405,6 +1417,9 @@ def is_block_device(self): # Path doesn't exist or is a broken symlink # (see https://bitbucket.org/pitrou/pathlib/issue/12/) return False + except ValueError: + # Non-encodable path + return False def is_char_device(self): """ @@ -1418,6 +1433,9 @@ def is_char_device(self): # Path doesn't exist or is a broken symlink # (see https://bitbucket.org/pitrou/pathlib/issue/12/) return False + except ValueError: + # Non-encodable path + return False def is_fifo(self): """ @@ -1431,6 +1449,9 @@ def is_fifo(self): # Path doesn't exist or is a broken symlink # (see https://bitbucket.org/pitrou/pathlib/issue/12/) return False + except ValueError: + # Non-encodable path + return False def is_socket(self): """ @@ -1444,6 +1465,9 @@ def is_socket(self): # Path doesn't exist or is a broken symlink # (see https://bitbucket.org/pitrou/pathlib/issue/12/) return False + except ValueError: + # Non-encodable path + return False def expanduser(self): """ Return a new path with expanded ~ and ~user constructs diff --git a/Lib/posixpath.py b/Lib/posixpath.py index e92186c64e0d..7e3f3db4b6d0 100644 --- a/Lib/posixpath.py +++ b/Lib/posixpath.py @@ -169,7 +169,7 @@ def islink(path): """Test whether a path is a symbolic link""" try: st = os.lstat(path) - except (OSError, AttributeError): + except (OSError, ValueError, AttributeError): return False return stat.S_ISLNK(st.st_mode) @@ -179,7 +179,7 @@ def lexists(path): """Test whether a path exists. Returns True for broken symbolic links""" try: os.lstat(path) - except OSError: + except (OSError, ValueError): return False return True @@ -191,7 +191,7 @@ def ismount(path): """Test whether a path is a mount point""" try: s1 = os.lstat(path) - except OSError: + except (OSError, ValueError): # It doesn't exist -- so not a mount point. :-) return False else: @@ -206,7 +206,7 @@ def ismount(path): parent = realpath(parent) try: s2 = os.lstat(parent) - except OSError: + except (OSError, ValueError): return False dev1 = s1.st_dev diff --git a/Lib/test/test_genericpath.py b/Lib/test/test_genericpath.py index 8b291cda689c..08e1c12101c7 100644 --- a/Lib/test/test_genericpath.py +++ b/Lib/test/test_genericpath.py @@ -138,10 +138,20 @@ def test_exists(self): self.assertIs(self.pathmodule.exists(filename), True) self.assertIs(self.pathmodule.exists(bfilename), True) + self.assertIs(self.pathmodule.exists(filename + '\udfff'), False) + self.assertIs(self.pathmodule.exists(bfilename + b'\xff'), False) + self.assertIs(self.pathmodule.exists(filename + '\x00'), False) + self.assertIs(self.pathmodule.exists(bfilename + b'\x00'), False) + if self.pathmodule is not genericpath: self.assertIs(self.pathmodule.lexists(filename), True) self.assertIs(self.pathmodule.lexists(bfilename), True) + self.assertIs(self.pathmodule.lexists(filename + '\udfff'), False) + self.assertIs(self.pathmodule.lexists(bfilename + b'\xff'), False) + self.assertIs(self.pathmodule.lexists(filename + '\x00'), False) + self.assertIs(self.pathmodule.lexists(bfilename + b'\x00'), False) + @unittest.skipUnless(hasattr(os, "pipe"), "requires os.pipe()") def test_exists_fd(self): r, w = os.pipe() @@ -158,6 +168,11 @@ def test_isdir(self): self.assertIs(self.pathmodule.isdir(filename), False) self.assertIs(self.pathmodule.isdir(bfilename), False) + self.assertIs(self.pathmodule.isdir(filename + '\udfff'), False) + self.assertIs(self.pathmodule.isdir(bfilename + b'\xff'), False) + self.assertIs(self.pathmodule.isdir(filename + '\x00'), False) + self.assertIs(self.pathmodule.isdir(bfilename + b'\x00'), False) + try: create_file(filename) self.assertIs(self.pathmodule.isdir(filename), False) @@ -178,6 +193,11 @@ def test_isfile(self): self.assertIs(self.pathmodule.isfile(filename), False) self.assertIs(self.pathmodule.isfile(bfilename), False) + self.assertIs(self.pathmodule.isfile(filename + '\udfff'), False) + self.assertIs(self.pathmodule.isfile(bfilename + b'\xff'), False) + self.assertIs(self.pathmodule.isfile(filename + '\x00'), False) + self.assertIs(self.pathmodule.isfile(bfilename + b'\x00'), False) + try: create_file(filename) self.assertIs(self.pathmodule.isfile(filename), True) @@ -298,18 +318,20 @@ def test_invalid_paths(self): continue func = getattr(self.pathmodule, attr) with self.subTest(attr=attr): - try: + if attr in ('exists', 'isdir', 'isfile'): func('/tmp\udfffabcds') - except (OSError, UnicodeEncodeError): - pass - try: func(b'/tmp\xffabcds') - except (OSError, UnicodeDecodeError): - pass - with self.assertRaisesRegex(ValueError, 'embedded null'): func('/tmp\x00abcds') - with self.assertRaisesRegex(ValueError, 'embedded null'): func(b'/tmp\x00abcds') + else: + with self.assertRaises((OSError, UnicodeEncodeError)): + func('/tmp\udfffabcds') + with self.assertRaises((OSError, UnicodeDecodeError)): + func(b'/tmp\xffabcds') + with self.assertRaisesRegex(ValueError, 'embedded null'): + func('/tmp\x00abcds') + with self.assertRaisesRegex(ValueError, 'embedded null'): + func(b'/tmp\x00abcds') # Following TestCase is not supposed to be run from test_genericpath. # It is inherited by other test modules (macpath, ntpath, posixpath). diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index e436db995ce4..876eecccfd5f 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1342,6 +1342,8 @@ def test_exists(self): self.assertIs(False, (p / 'linkA' / 'bah').exists()) self.assertIs(False, (p / 'foo').exists()) self.assertIs(False, P('/xyzzy').exists()) + self.assertIs(False, P(BASE + '\udfff').exists()) + self.assertIs(False, P(BASE + '\x00').exists()) def test_open_common(self): p = self.cls(BASE) @@ -1866,7 +1868,9 @@ def test_is_dir(self): if support.can_symlink(): self.assertFalse((P / 'linkA').is_dir()) self.assertTrue((P / 'linkB').is_dir()) - self.assertFalse((P/ 'brokenLink').is_dir()) + self.assertFalse((P/ 'brokenLink').is_dir(), False) + self.assertIs((P / 'dirA\udfff').is_dir(), False) + self.assertIs((P / 'dirA\x00').is_dir(), False) def test_is_file(self): P = self.cls(BASE) @@ -1878,6 +1882,8 @@ def test_is_file(self): self.assertTrue((P / 'linkA').is_file()) self.assertFalse((P / 'linkB').is_file()) self.assertFalse((P/ 'brokenLink').is_file()) + self.assertIs((P / 'fileA\udfff').is_file(), False) + self.assertIs((P / 'fileA\x00').is_file(), False) @only_posix def test_is_mount(self): @@ -1890,6 +1896,8 @@ def test_is_mount(self): self.assertTrue(R.is_mount()) if support.can_symlink(): self.assertFalse((P / 'linkA').is_mount()) + self.assertIs(self.cls('/\udfff').is_mount(), False) + self.assertIs(self.cls('/\x00').is_mount(), False) def test_is_symlink(self): P = self.cls(BASE) @@ -1901,6 +1909,11 @@ def test_is_symlink(self): self.assertTrue((P / 'linkA').is_symlink()) self.assertTrue((P / 'linkB').is_symlink()) self.assertTrue((P/ 'brokenLink').is_symlink()) + self.assertIs((P / 'fileA\udfff').is_file(), False) + self.assertIs((P / 'fileA\x00').is_file(), False) + if support.can_symlink(): + self.assertIs((P / 'linkA\udfff').is_file(), False) + self.assertIs((P / 'linkA\x00').is_file(), False) def test_is_fifo_false(self): P = self.cls(BASE) @@ -1908,6 +1921,8 @@ def test_is_fifo_false(self): self.assertFalse((P / 'dirA').is_fifo()) self.assertFalse((P / 'non-existing').is_fifo()) self.assertFalse((P / 'fileA' / 'bah').is_fifo()) + self.assertIs((P / 'fileA\udfff').is_fifo(), False) + self.assertIs((P / 'fileA\x00').is_fifo(), False) @unittest.skipUnless(hasattr(os, "mkfifo"), "os.mkfifo() required") def test_is_fifo_true(self): @@ -1919,6 +1934,8 @@ def test_is_fifo_true(self): self.assertTrue(P.is_fifo()) self.assertFalse(P.is_socket()) self.assertFalse(P.is_file()) + self.assertIs(self.cls(BASE, 'myfifo\udfff').is_fifo(), False) + self.assertIs(self.cls(BASE, 'myfifo\x00').is_fifo(), False) def test_is_socket_false(self): P = self.cls(BASE) @@ -1926,6 +1943,8 @@ def test_is_socket_false(self): self.assertFalse((P / 'dirA').is_socket()) self.assertFalse((P / 'non-existing').is_socket()) self.assertFalse((P / 'fileA' / 'bah').is_socket()) + self.assertIs((P / 'fileA\udfff').is_socket(), False) + self.assertIs((P / 'fileA\x00').is_socket(), False) @unittest.skipUnless(hasattr(socket, "AF_UNIX"), "Unix sockets required") def test_is_socket_true(self): @@ -1941,6 +1960,8 @@ def test_is_socket_true(self): self.assertTrue(P.is_socket()) self.assertFalse(P.is_fifo()) self.assertFalse(P.is_file()) + self.assertIs(self.cls(BASE, 'mysock\udfff').is_socket(), False) + self.assertIs(self.cls(BASE, 'mysock\x00').is_socket(), False) def test_is_block_device_false(self): P = self.cls(BASE) @@ -1948,6 +1969,8 @@ def test_is_block_device_false(self): self.assertFalse((P / 'dirA').is_block_device()) self.assertFalse((P / 'non-existing').is_block_device()) self.assertFalse((P / 'fileA' / 'bah').is_block_device()) + self.assertIs((P / 'fileA\udfff').is_block_device(), False) + self.assertIs((P / 'fileA\x00').is_block_device(), False) def test_is_char_device_false(self): P = self.cls(BASE) @@ -1955,6 +1978,8 @@ def test_is_char_device_false(self): self.assertFalse((P / 'dirA').is_char_device()) self.assertFalse((P / 'non-existing').is_char_device()) self.assertFalse((P / 'fileA' / 'bah').is_char_device()) + self.assertIs((P / 'fileA\udfff').is_char_device(), False) + self.assertIs((P / 'fileA\x00').is_char_device(), False) def test_is_char_device_true(self): # Under Unix, /dev/null should generally be a char device @@ -1964,6 +1989,8 @@ def test_is_char_device_true(self): self.assertTrue(P.is_char_device()) self.assertFalse(P.is_block_device()) self.assertFalse(P.is_file()) + self.assertIs(self.cls('/dev/null\udfff').is_char_device(), False) + self.assertIs(self.cls('/dev/null\x00').is_char_device(), False) def test_pickling_common(self): p = self.cls(BASE, 'fileA') diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py index 9476ede53193..ae59ef5927be 100644 --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -153,9 +153,11 @@ def test_dirname(self): def test_islink(self): self.assertIs(posixpath.islink(support.TESTFN + "1"), False) self.assertIs(posixpath.lexists(support.TESTFN + "2"), False) + with open(support.TESTFN + "1", "wb") as f: f.write(b"foo") self.assertIs(posixpath.islink(support.TESTFN + "1"), False) + if support.can_symlink(): os.symlink(support.TESTFN + "1", support.TESTFN + "2") self.assertIs(posixpath.islink(support.TESTFN + "2"), True) @@ -164,6 +166,11 @@ def test_islink(self): self.assertIs(posixpath.exists(support.TESTFN + "2"), False) self.assertIs(posixpath.lexists(support.TESTFN + "2"), True) + self.assertIs(posixpath.islink(support.TESTFN + "\udfff"), False) + self.assertIs(posixpath.islink(os.fsencode(support.TESTFN) + b"\xff"), False) + self.assertIs(posixpath.islink(support.TESTFN + "\x00"), False) + self.assertIs(posixpath.islink(os.fsencode(support.TESTFN) + b"\x00"), False) + def test_ismount(self): self.assertIs(posixpath.ismount("/"), True) self.assertIs(posixpath.ismount(b"/"), True) @@ -177,6 +184,11 @@ def test_ismount_non_existent(self): finally: safe_rmdir(ABSTFN) + self.assertIs(posixpath.ismount('/\udfff'), False) + self.assertIs(posixpath.ismount(b'/\xff'), False) + self.assertIs(posixpath.ismount('/\x00'), False) + self.assertIs(posixpath.ismount(b'/\x00'), False) + @unittest.skipUnless(support.can_symlink(), "Test requires symlink support") def test_ismount_symlinks(self): diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py index e3c9deebf08c..742be1ec03d4 100644 --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -159,13 +159,11 @@ def test_addpackage_import_bad_pth_file(self): # Issue 5258 pth_dir, pth_fn = self.make_pth("abc\x00def\n") with captured_stderr() as err_out: - site.addpackage(pth_dir, pth_fn, set()) - self.assertRegex(err_out.getvalue(), "line 1") - self.assertRegex(err_out.getvalue(), - re.escape(os.path.join(pth_dir, pth_fn))) - # XXX: ditto previous XXX comment. - self.assertRegex(err_out.getvalue(), 'Traceback') - self.assertRegex(err_out.getvalue(), 'ValueError') + self.assertFalse(site.addpackage(pth_dir, pth_fn, set())) + self.assertEqual(err_out.getvalue(), "") + for path in sys.path: + if isinstance(path, str): + self.assertNotIn("abc\x00def", path) def test_addsitedir(self): # Same tests for test_addpackage since addsitedir() essentially just diff --git a/Misc/NEWS.d/next/Library/2018-06-14-17-53-30.bpo-33721.8i9_9A.rst b/Misc/NEWS.d/next/Library/2018-06-14-17-53-30.bpo-33721.8i9_9A.rst new file mode 100644 index 000000000000..987bf4696d56 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-06-14-17-53-30.bpo-33721.8i9_9A.rst @@ -0,0 +1,12 @@ +:mod:`os.path` functions that return a boolean result like +:func:`~os.path.exists`, :func:`~os.path.lexists`, :func:`~os.path.isdir`, +:func:`~os.path.isfile`, :func:`~os.path.islink`, and :func:`~os.path.ismount`, +and :mod:`pathlib.Path` methods that return a boolean result like +:meth:`~pathlib.Path.exists()`, :meth:`~pathlib.Path.is_dir()`, +:meth:`~pathlib.Path.is_file()`, :meth:`~pathlib.Path.is_mount()`, +:meth:`~pathlib.Path.is_symlink()`, :meth:`~pathlib.Path.is_block_device()`, +:meth:`~pathlib.Path.is_char_device()`, :meth:`~pathlib.Path.is_fifo()`, +:meth:`~pathlib.Path.is_socket()` now return ``False`` instead of raising +:exc:`ValueError` or its subclasses :exc:`UnicodeEncodeError` and +:exc:`UnicodeDecodeError` for paths that contain characters or bytes +unrepresentable at the OS level. diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h index 2c46d4bf172e..704c824e9e32 100644 --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -1005,27 +1005,6 @@ PyDoc_STRVAR(os__isdir__doc__, #define OS__ISDIR_METHODDEF \ {"_isdir", (PyCFunction)os__isdir, METH_O, os__isdir__doc__}, -static PyObject * -os__isdir_impl(PyObject *module, path_t *path); - -static PyObject * -os__isdir(PyObject *module, PyObject *arg) -{ - PyObject *return_value = NULL; - path_t path = PATH_T_INITIALIZE("_isdir", "path", 0, 0); - - if (!PyArg_Parse(arg, "O&:_isdir", path_converter, &path)) { - goto exit; - } - return_value = os__isdir_impl(module, &path); - -exit: - /* Cleanup for path */ - path_cleanup(&path); - - return return_value; -} - #endif /* defined(MS_WINDOWS) */ #if defined(MS_WINDOWS) @@ -6778,4 +6757,4 @@ os_getrandom(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject #ifndef OS_GETRANDOM_METHODDEF #define OS_GETRANDOM_METHODDEF #endif /* !defined(OS_GETRANDOM_METHODDEF) */ -/*[clinic end generated code: output=0f23518dd4482e66 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=40cac0135f846202 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 53d2ce22439e..400ed979821d 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -3821,22 +3821,32 @@ os__getfinalpathname_impl(PyObject *module, path_t *path) /*[clinic input] os._isdir - path: path_t + path as arg: object / Return true if the pathname refers to an existing directory. [clinic start generated code]*/ static PyObject * -os__isdir_impl(PyObject *module, path_t *path) -/*[clinic end generated code: output=75f56f32720836cb input=5e0800149c0ad95f]*/ +os__isdir(PyObject *module, PyObject *arg) +/*[clinic end generated code: output=404f334d85d4bf25 input=36cb6785874d479e]*/ { DWORD attributes; + path_t path = PATH_T_INITIALIZE("_isdir", "path", 0, 0); + + if (!path_converter(arg, &path)) { + if (PyErr_ExceptionMatches(PyExc_ValueError)) { + PyErr_Clear(); + Py_RETURN_FALSE; + } + return NULL; + } Py_BEGIN_ALLOW_THREADS - attributes = GetFileAttributesW(path->wide); + attributes = GetFileAttributesW(path.wide); Py_END_ALLOW_THREADS + path_cleanup(&path); if (attributes == INVALID_FILE_ATTRIBUTES) Py_RETURN_FALSE; From solipsis at pitrou.net Tue Sep 18 05:11:13 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 18 Sep 2018 09:11:13 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=4 Message-ID: <20180918091113.1.79717DCA7B9256CB@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogYk9Ow6', '--timeout', '7200'] From webhook-mailer at python.org Tue Sep 18 08:39:03 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 18 Sep 2018 12:39:03 -0000 Subject: [Python-checkins] bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146) Message-ID: https://github.com/python/cpython/commit/cb5778f00ce48631c7140f33ba242496aaf7102b commit: cb5778f00ce48631c7140f33ba242496aaf7102b branch: master author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-18T05:38:58-07:00 summary: bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146) The C accelerated _elementtree module now initializes hash randomization salt from _Py_HashSecret instead of libexpat's default CPRNG. Signed-off-by: Christian Heimes https://bugs.python.org/issue34623 files: A Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst M Include/pyexpat.h M Modules/_elementtree.c M Modules/pyexpat.c diff --git a/Include/pyexpat.h b/Include/pyexpat.h index 44259bf6d716..07020b5dc964 100644 --- a/Include/pyexpat.h +++ b/Include/pyexpat.h @@ -3,7 +3,7 @@ /* note: you must import expat.h before importing this module! */ -#define PyExpat_CAPI_MAGIC "pyexpat.expat_CAPI 1.0" +#define PyExpat_CAPI_MAGIC "pyexpat.expat_CAPI 1.1" #define PyExpat_CAPSULE_NAME "pyexpat.expat_CAPI" struct PyExpat_CAPI @@ -48,6 +48,8 @@ struct PyExpat_CAPI enum XML_Status (*SetEncoding)(XML_Parser parser, const XML_Char *encoding); int (*DefaultUnknownEncodingHandler)( void *encodingHandlerData, const XML_Char *name, XML_Encoding *info); + /* might be none for expat < 2.1.0 */ + int (*SetHashSalt)(XML_Parser parser, unsigned long hash_salt); /* always add new stuff to the end! */ }; diff --git a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst new file mode 100644 index 000000000000..31ad92ef8582 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst @@ -0,0 +1,2 @@ +The C accelerated _elementtree module now initializes hash randomization +salt from _Py_HashSecret instead of libexpat's default CSPRNG. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 02781d5d892b..bba687388797 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3272,6 +3272,11 @@ _elementtree_XMLParser___init___impl(XMLParserObject *self, PyObject *target, PyErr_NoMemory(); return -1; } + /* expat < 2.1.0 has no XML_SetHashSalt() */ + if (EXPAT(SetHashSalt) != NULL) { + EXPAT(SetHashSalt)(self->parser, + (unsigned long)_Py_HashSecret.expat.hashsalt); + } if (target) { Py_INCREF(target); diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index c8a01d4e088e..c52079e518f2 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1877,6 +1877,11 @@ MODULE_INITFUNC(void) capi.SetStartDoctypeDeclHandler = XML_SetStartDoctypeDeclHandler; capi.SetEncoding = XML_SetEncoding; capi.DefaultUnknownEncodingHandler = PyUnknownEncodingHandler; +#if XML_COMBINED_VERSION >= 20100 + capi.SetHashSalt = XML_SetHashSalt; +#else + capi.SetHashSalt = NULL; +#endif /* export using capsule */ capi_object = PyCapsule_New(&capi, PyExpat_CAPSULE_NAME, NULL); From webhook-mailer at python.org Tue Sep 18 09:11:13 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 18 Sep 2018 13:11:13 -0000 Subject: [Python-checkins] bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146) Message-ID: https://github.com/python/cpython/commit/470a435f3b42c9be5fdb7f7b04f3df5663ba7305 commit: 470a435f3b42c9be5fdb7f7b04f3df5663ba7305 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-18T06:11:09-07:00 summary: bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146) The C accelerated _elementtree module now initializes hash randomization salt from _Py_HashSecret instead of libexpat's default CPRNG. Signed-off-by: Christian Heimes https://bugs.python.org/issue34623 (cherry picked from commit cb5778f00ce48631c7140f33ba242496aaf7102b) Co-authored-by: Christian Heimes files: A Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst M Include/pyexpat.h M Modules/_elementtree.c M Modules/pyexpat.c diff --git a/Include/pyexpat.h b/Include/pyexpat.h index 44259bf6d716..07020b5dc964 100644 --- a/Include/pyexpat.h +++ b/Include/pyexpat.h @@ -3,7 +3,7 @@ /* note: you must import expat.h before importing this module! */ -#define PyExpat_CAPI_MAGIC "pyexpat.expat_CAPI 1.0" +#define PyExpat_CAPI_MAGIC "pyexpat.expat_CAPI 1.1" #define PyExpat_CAPSULE_NAME "pyexpat.expat_CAPI" struct PyExpat_CAPI @@ -48,6 +48,8 @@ struct PyExpat_CAPI enum XML_Status (*SetEncoding)(XML_Parser parser, const XML_Char *encoding); int (*DefaultUnknownEncodingHandler)( void *encodingHandlerData, const XML_Char *name, XML_Encoding *info); + /* might be none for expat < 2.1.0 */ + int (*SetHashSalt)(XML_Parser parser, unsigned long hash_salt); /* always add new stuff to the end! */ }; diff --git a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst new file mode 100644 index 000000000000..31ad92ef8582 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst @@ -0,0 +1,2 @@ +The C accelerated _elementtree module now initializes hash randomization +salt from _Py_HashSecret instead of libexpat's default CSPRNG. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 1dfdb3ce34f3..4b86f96a70d3 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3305,6 +3305,11 @@ _elementtree_XMLParser___init___impl(XMLParserObject *self, PyObject *html, PyErr_NoMemory(); return -1; } + /* expat < 2.1.0 has no XML_SetHashSalt() */ + if (EXPAT(SetHashSalt) != NULL) { + EXPAT(SetHashSalt)(self->parser, + (unsigned long)_Py_HashSecret.expat.hashsalt); + } if (target) { Py_INCREF(target); diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index c8a01d4e088e..c52079e518f2 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1877,6 +1877,11 @@ MODULE_INITFUNC(void) capi.SetStartDoctypeDeclHandler = XML_SetStartDoctypeDeclHandler; capi.SetEncoding = XML_SetEncoding; capi.DefaultUnknownEncodingHandler = PyUnknownEncodingHandler; +#if XML_COMBINED_VERSION >= 20100 + capi.SetHashSalt = XML_SetHashSalt; +#else + capi.SetHashSalt = NULL; +#endif /* export using capsule */ capi_object = PyCapsule_New(&capi, PyExpat_CAPSULE_NAME, NULL); From webhook-mailer at python.org Tue Sep 18 09:13:12 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 18 Sep 2018 13:13:12 -0000 Subject: [Python-checkins] [2.7] bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146) (GH-9394) Message-ID: https://github.com/python/cpython/commit/18b20bad75b4ff0486940fba4ec680e96e70f3a2 commit: 18b20bad75b4ff0486940fba4ec680e96e70f3a2 branch: 2.7 author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-18T06:13:09-07:00 summary: [2.7] bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146) (GH-9394) The C accelerated _elementtree module now initializes hash randomization salt from _Py_HashSecret instead of libexpat's default CPRNG. Signed-off-by: Christian Heimes https://bugs.python.org/issue34623. (cherry picked from commit cb5778f00ce48631c7140f33ba242496aaf7102b) Co-authored-by: Christian Heimes https://bugs.python.org/issue34623 files: A Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst M Include/pyexpat.h M Modules/_elementtree.c M Modules/pyexpat.c diff --git a/Include/pyexpat.h b/Include/pyexpat.h index 5340ef5fa386..3fc5fa54da63 100644 --- a/Include/pyexpat.h +++ b/Include/pyexpat.h @@ -3,7 +3,7 @@ /* note: you must import expat.h before importing this module! */ -#define PyExpat_CAPI_MAGIC "pyexpat.expat_CAPI 1.0" +#define PyExpat_CAPI_MAGIC "pyexpat.expat_CAPI 1.1" #define PyExpat_CAPSULE_NAME "pyexpat.expat_CAPI" struct PyExpat_CAPI @@ -43,6 +43,8 @@ struct PyExpat_CAPI XML_Parser parser, XML_UnknownEncodingHandler handler, void *encodingHandlerData); void (*SetUserData)(XML_Parser parser, void *userData); + /* might be none for expat < 2.1.0 */ + int (*SetHashSalt)(XML_Parser parser, unsigned long hash_salt); /* always add new stuff to the end! */ }; diff --git a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst new file mode 100644 index 000000000000..31ad92ef8582 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst @@ -0,0 +1,2 @@ +The C accelerated _elementtree module now initializes hash randomization +salt from _Py_HashSecret instead of libexpat's default CSPRNG. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index f7f992dd3a95..b38e0ab329c7 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -2574,6 +2574,11 @@ xmlparser(PyObject* self_, PyObject* args, PyObject* kw) PyErr_NoMemory(); return NULL; } + /* expat < 2.1.0 has no XML_SetHashSalt() */ + if (EXPAT(SetHashSalt) != NULL) { + EXPAT(SetHashSalt)(self->parser, + (unsigned long)_Py_HashSecret.prefix); + } ALLOC(sizeof(XMLParserObject), "create expatparser"); diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index 2b4d31293c64..1f8c0d70a559 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -2042,6 +2042,11 @@ MODULE_INITFUNC(void) capi.SetProcessingInstructionHandler = XML_SetProcessingInstructionHandler; capi.SetUnknownEncodingHandler = XML_SetUnknownEncodingHandler; capi.SetUserData = XML_SetUserData; +#if XML_COMBINED_VERSION >= 20100 + capi.SetHashSalt = XML_SetHashSalt; +#else + capi.SetHashSalt = NULL; +#endif /* export using capsule */ capi_object = PyCapsule_New(&capi, PyExpat_CAPSULE_NAME, NULL); From webhook-mailer at python.org Tue Sep 18 09:14:18 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 18 Sep 2018 13:14:18 -0000 Subject: [Python-checkins] bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146) Message-ID: https://github.com/python/cpython/commit/f7666e828cc3d5873136473ea36ba2013d624fa1 commit: f7666e828cc3d5873136473ea36ba2013d624fa1 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-18T06:14:13-07:00 summary: bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146) The C accelerated _elementtree module now initializes hash randomization salt from _Py_HashSecret instead of libexpat's default CPRNG. Signed-off-by: Christian Heimes https://bugs.python.org/issue34623 (cherry picked from commit cb5778f00ce48631c7140f33ba242496aaf7102b) Co-authored-by: Christian Heimes files: A Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst M Include/pyexpat.h M Modules/_elementtree.c M Modules/pyexpat.c diff --git a/Include/pyexpat.h b/Include/pyexpat.h index 44259bf6d716..07020b5dc964 100644 --- a/Include/pyexpat.h +++ b/Include/pyexpat.h @@ -3,7 +3,7 @@ /* note: you must import expat.h before importing this module! */ -#define PyExpat_CAPI_MAGIC "pyexpat.expat_CAPI 1.0" +#define PyExpat_CAPI_MAGIC "pyexpat.expat_CAPI 1.1" #define PyExpat_CAPSULE_NAME "pyexpat.expat_CAPI" struct PyExpat_CAPI @@ -48,6 +48,8 @@ struct PyExpat_CAPI enum XML_Status (*SetEncoding)(XML_Parser parser, const XML_Char *encoding); int (*DefaultUnknownEncodingHandler)( void *encodingHandlerData, const XML_Char *name, XML_Encoding *info); + /* might be none for expat < 2.1.0 */ + int (*SetHashSalt)(XML_Parser parser, unsigned long hash_salt); /* always add new stuff to the end! */ }; diff --git a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst new file mode 100644 index 000000000000..31ad92ef8582 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst @@ -0,0 +1,2 @@ +The C accelerated _elementtree module now initializes hash randomization +salt from _Py_HashSecret instead of libexpat's default CSPRNG. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 707ab2912b35..53f05f937ffb 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3261,6 +3261,11 @@ _elementtree_XMLParser___init___impl(XMLParserObject *self, PyObject *html, PyErr_NoMemory(); return -1; } + /* expat < 2.1.0 has no XML_SetHashSalt() */ + if (EXPAT(SetHashSalt) != NULL) { + EXPAT(SetHashSalt)(self->parser, + (unsigned long)_Py_HashSecret.expat.hashsalt); + } if (target) { Py_INCREF(target); diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index 47c3e86c209d..aa21d93c115a 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1887,6 +1887,11 @@ MODULE_INITFUNC(void) capi.SetStartDoctypeDeclHandler = XML_SetStartDoctypeDeclHandler; capi.SetEncoding = XML_SetEncoding; capi.DefaultUnknownEncodingHandler = PyUnknownEncodingHandler; +#if XML_COMBINED_VERSION >= 20100 + capi.SetHashSalt = XML_SetHashSalt; +#else + capi.SetHashSalt = NULL; +#endif /* export using capsule */ capi_object = PyCapsule_New(&capi, PyExpat_CAPSULE_NAME, NULL); From webhook-mailer at python.org Tue Sep 18 12:10:32 2018 From: webhook-mailer at python.org (Steve Dower) Date: Tue, 18 Sep 2018 16:10:32 -0000 Subject: [Python-checkins] bpo-34582: Adds JUnit XML output for regression tests (GH-9210) Message-ID: https://github.com/python/cpython/commit/d0f49d2f5085ca68e3dc8725f1fb1c9674bfb5ed commit: d0f49d2f5085ca68e3dc8725f1fb1c9674bfb5ed branch: master author: Steve Dower committer: GitHub date: 2018-09-18T09:10:26-07:00 summary: bpo-34582: Adds JUnit XML output for regression tests (GH-9210) files: A Lib/test/support/testresult.py A Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst M .vsts/linux-pr.yml M .vsts/macos-pr.yml M .vsts/windows-pr.yml M Lib/test/eintrdata/eintr_tester.py M Lib/test/libregrtest/cmdline.py M Lib/test/libregrtest/main.py M Lib/test/libregrtest/runtest.py M Lib/test/libregrtest/runtest_mp.py M Lib/test/support/__init__.py M Lib/test/test_argparse.py diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml index 6e4ac7c65c4d..d11a4f06e4e1 100644 --- a/.vsts/linux-pr.yml +++ b/.vsts/linux-pr.yml @@ -70,6 +70,15 @@ steps: displayName: 'Run patchcheck.py' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(system.pullRequest.targetBranch)-linux' + platform: linux + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml index c56e66b5090b..69b619e47577 100644 --- a/.vsts/macos-pr.yml +++ b/.vsts/macos-pr.yml @@ -50,6 +50,15 @@ steps: displayName: 'Display build info' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(system.pullRequest.targetBranch)-macOS' + platform: macOS + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/windows-pr.yml b/.vsts/windows-pr.yml index 3dd5609a32e5..7134120d6414 100644 --- a/.vsts/windows-pr.yml +++ b/.vsts/windows-pr.yml @@ -54,8 +54,17 @@ steps: displayName: 'Display build info' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" displayName: 'Tests' env: PREFIX: $(Py_OutDir)\$(outDirSuffix) condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' + mergeTestResults: true + testRunTitle: '$(System.PullRequest.TargetBranch)-$(outDirSuffix)' + platform: $(outDirSuffix) + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py index bc308fe107b3..1caeafe25d9b 100644 --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -52,7 +52,8 @@ def setUpClass(cls): # Issue #25277: Use faulthandler to try to debug a hang on FreeBSD if hasattr(faulthandler, 'dump_traceback_later'): - faulthandler.dump_traceback_later(10 * 60, exit=True) + faulthandler.dump_traceback_later(10 * 60, exit=True, + file=sys.__stderr__) @classmethod def stop_alarm(cls): diff --git a/Lib/test/libregrtest/cmdline.py b/Lib/test/libregrtest/cmdline.py index bd126b33c9bd..538ff05489ea 100644 --- a/Lib/test/libregrtest/cmdline.py +++ b/Lib/test/libregrtest/cmdline.py @@ -268,6 +268,10 @@ def _create_parser(): help='if a test file alters the environment, mark ' 'the test as failed') + group.add_argument('--junit-xml', dest='xmlpath', metavar='FILENAME', + help='writes JUnit-style XML results to the specified ' + 'file') + return parser diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index a176db59c0fb..1fd41320d111 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -100,8 +100,11 @@ def __init__(self): self.next_single_test = None self.next_single_filename = None + # used by --junit-xml + self.testsuite_xml = None + def accumulate_result(self, test, result): - ok, test_time = result + ok, test_time, xml_data = result if ok not in (CHILD_ERROR, INTERRUPTED): self.test_times.append((test_time, test)) if ok == PASSED: @@ -118,6 +121,15 @@ def accumulate_result(self, test, result): elif ok != INTERRUPTED: raise ValueError("invalid test result: %r" % ok) + if xml_data: + import xml.etree.ElementTree as ET + for e in xml_data: + try: + self.testsuite_xml.append(ET.fromstring(e)) + except ET.ParseError: + print(xml_data, file=sys.__stderr__) + raise + def display_progress(self, test_index, test): if self.ns.quiet: return @@ -164,6 +176,9 @@ def parse_args(self, kwargs): file=sys.stderr) ns.findleaks = False + if ns.xmlpath: + support.junit_xml_list = self.testsuite_xml = [] + # Strip .py extensions. removepy(ns.args) @@ -384,7 +399,7 @@ def run_tests_sequential(self): result = runtest(self.ns, test) except KeyboardInterrupt: self.interrupted = True - self.accumulate_result(test, (INTERRUPTED, None)) + self.accumulate_result(test, (INTERRUPTED, None, None)) break else: self.accumulate_result(test, result) @@ -508,6 +523,31 @@ def finalize(self): if self.ns.runleaks: os.system("leaks %d" % os.getpid()) + def save_xml_result(self): + if not self.ns.xmlpath and not self.testsuite_xml: + return + + import xml.etree.ElementTree as ET + root = ET.Element("testsuites") + + # Manually count the totals for the overall summary + totals = {'tests': 0, 'errors': 0, 'failures': 0} + for suite in self.testsuite_xml: + root.append(suite) + for k in totals: + try: + totals[k] += int(suite.get(k, 0)) + except ValueError: + pass + + for k, v in totals.items(): + root.set(k, str(v)) + + xmlpath = os.path.join(support.SAVEDCWD, self.ns.xmlpath) + with open(xmlpath, 'wb') as f: + for s in ET.tostringlist(root): + f.write(s) + def main(self, tests=None, **kwargs): global TEMPDIR @@ -570,6 +610,9 @@ def _main(self, tests, kwargs): self.rerun_failed_tests() self.finalize() + + self.save_xml_result() + if self.bad: sys.exit(2) if self.interrupted: diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py index 3e1afd41997a..4f41080d37b9 100644 --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -85,8 +85,8 @@ def runtest(ns, test): ns -- regrtest namespace of options test -- the name of the test - Returns the tuple (result, test_time), where result is one of the - constants: + Returns the tuple (result, test_time, xml_data), where result is one + of the constants: INTERRUPTED KeyboardInterrupt when run under -j RESOURCE_DENIED test skipped because resource denied @@ -94,6 +94,9 @@ def runtest(ns, test): ENV_CHANGED test failed because it changed the execution environment FAILED test failed PASSED test passed + + If ns.xmlpath is not None, xml_data is a list containing each + generated testsuite element. """ output_on_failure = ns.verbose3 @@ -106,22 +109,13 @@ def runtest(ns, test): # reset the environment_altered flag to detect if a test altered # the environment support.environment_altered = False + support.junit_xml_list = xml_list = [] if ns.xmlpath else None if ns.failfast: support.failfast = True if output_on_failure: support.verbose = True - # Reuse the same instance to all calls to runtest(). Some - # tests keep a reference to sys.stdout or sys.stderr - # (eg. test_argparse). - if runtest.stringio is None: - stream = io.StringIO() - runtest.stringio = stream - else: - stream = runtest.stringio - stream.seek(0) - stream.truncate() - + stream = io.StringIO() orig_stdout = sys.stdout orig_stderr = sys.stderr try: @@ -138,12 +132,18 @@ def runtest(ns, test): else: support.verbose = ns.verbose # Tell tests to be moderately quiet result = runtest_inner(ns, test, display_failure=not ns.verbose) - return result + + if xml_list: + import xml.etree.ElementTree as ET + xml_data = [ET.tostring(x).decode('us-ascii') for x in xml_list] + else: + xml_data = None + return result + (xml_data,) finally: if use_timeout: faulthandler.cancel_dump_traceback_later() cleanup_test_droppings(test, ns.verbose) -runtest.stringio = None + support.junit_xml_list = None def post_test_cleanup(): diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py index 1f07cfbd8cbe..6190574afdf8 100644 --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -67,7 +67,7 @@ def run_tests_worker(worker_args): try: result = runtest(ns, testname) except KeyboardInterrupt: - result = INTERRUPTED, '' + result = INTERRUPTED, '', None except BaseException as e: traceback.print_exc() result = CHILD_ERROR, str(e) @@ -122,7 +122,7 @@ def _runtest(self): self.current_test = None if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) + result = (CHILD_ERROR, "Exit code %s" % retcode, None) self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) return False @@ -133,6 +133,7 @@ def _runtest(self): return True result = json.loads(result) + assert len(result) == 3, f"Invalid result tuple: {result!r}" self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) return False @@ -195,7 +196,7 @@ def get_running(workers): regrtest.accumulate_result(test, result) # Display progress - ok, test_time = result + ok, test_time, xml_data = result text = format_test_result(test, ok) if (ok not in (CHILD_ERROR, INTERRUPTED) and test_time >= PROGRESS_MIN_TIME diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index de997b253a69..19701cf388fe 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -6,6 +6,7 @@ import asyncio.events import collections.abc import contextlib +import datetime import errno import faulthandler import fnmatch @@ -13,6 +14,7 @@ import gc import importlib import importlib.util +import io import logging.handlers import nntplib import os @@ -34,6 +36,8 @@ import urllib.error import warnings +from .testresult import get_test_runner + try: import multiprocessing.process except ImportError: @@ -295,6 +299,7 @@ def get_attribute(obj, name): max_memuse = 0 # Disable bigmem tests (they will still be run with # small sizes, to make sure they work.) real_max_memuse = 0 +junit_xml_list = None # list of testsuite XML elements failfast = False # _original_stdout is meant to hold stdout at the time regrtest began. @@ -1891,13 +1896,16 @@ def _filter_suite(suite, pred): def _run_suite(suite): """Run tests from a unittest.TestSuite-derived class.""" - if verbose: - runner = unittest.TextTestRunner(sys.stdout, verbosity=2, - failfast=failfast) - else: - runner = BasicTestRunner() + runner = get_test_runner(sys.stdout, verbosity=verbose) + + # TODO: Remove this before merging (here for easy comparison with old impl) + #runner = unittest.TextTestRunner(sys.stdout, verbosity=2, failfast=failfast) result = runner.run(suite) + + if junit_xml_list is not None: + junit_xml_list.append(result.get_xml_element()) + if not result.wasSuccessful(): if len(result.errors) == 1 and not result.failures: err = result.errors[0][1] diff --git a/Lib/test/support/testresult.py b/Lib/test/support/testresult.py new file mode 100644 index 000000000000..8988d3d15278 --- /dev/null +++ b/Lib/test/support/testresult.py @@ -0,0 +1,201 @@ +'''Test runner and result class for the regression test suite. + +''' + +import functools +import io +import sys +import time +import traceback +import unittest + +import xml.etree.ElementTree as ET + +from datetime import datetime + +class RegressionTestResult(unittest.TextTestResult): + separator1 = '=' * 70 + '\n' + separator2 = '-' * 70 + '\n' + + def __init__(self, stream, descriptions, verbosity): + super().__init__(stream=stream, descriptions=descriptions, verbosity=0) + self.buffer = True + self.__suite = ET.Element('testsuite') + self.__suite.set('start', datetime.utcnow().isoformat(' ')) + + self.__e = None + self.__start_time = None + self.__results = [] + self.__verbose = bool(verbosity) + + @classmethod + def __getId(cls, test): + try: + test_id = test.id + except AttributeError: + return str(test) + try: + return test_id() + except TypeError: + return str(test_id) + return repr(test) + + def startTest(self, test): + super().startTest(test) + self.__e = e = ET.SubElement(self.__suite, 'testcase') + self.__start_time = time.perf_counter() + if self.__verbose: + self.stream.write(f'{self.getDescription(test)} ... ') + self.stream.flush() + + def _add_result(self, test, capture=False, **args): + e = self.__e + self.__e = None + if e is None: + return + e.set('name', args.pop('name', self.__getId(test))) + e.set('status', args.pop('status', 'run')) + e.set('result', args.pop('result', 'completed')) + if self.__start_time: + e.set('time', f'{time.perf_counter() - self.__start_time:0.6f}') + + if capture: + stdout = self._stdout_buffer.getvalue().rstrip() + ET.SubElement(e, 'system-out').text = stdout + stderr = self._stderr_buffer.getvalue().rstrip() + ET.SubElement(e, 'system-err').text = stderr + + for k, v in args.items(): + if not k or not v: + continue + e2 = ET.SubElement(e, k) + if hasattr(v, 'items'): + for k2, v2 in v.items(): + if k2: + e2.set(k2, str(v2)) + else: + e2.text = str(v2) + else: + e2.text = str(v) + + def __write(self, c, word): + if self.__verbose: + self.stream.write(f'{word}\n') + + @classmethod + def __makeErrorDict(cls, err_type, err_value, err_tb): + if isinstance(err_type, type): + if err_type.__module__ == 'builtins': + typename = err_type.__name__ + else: + typename = f'{err_type.__module__}.{err_type.__name__}' + else: + typename = repr(err_type) + + msg = traceback.format_exception(err_type, err_value, None) + tb = traceback.format_exception(err_type, err_value, err_tb) + + return { + 'type': typename, + 'message': ''.join(msg), + '': ''.join(tb), + } + + def addError(self, test, err): + self._add_result(test, True, error=self.__makeErrorDict(*err)) + super().addError(test, err) + self.__write('E', 'ERROR') + + def addExpectedFailure(self, test, err): + self._add_result(test, True, output=self.__makeErrorDict(*err)) + super().addExpectedFailure(test, err) + self.__write('x', 'expected failure') + + def addFailure(self, test, err): + self._add_result(test, True, failure=self.__makeErrorDict(*err)) + super().addFailure(test, err) + self.__write('F', 'FAIL') + + def addSkip(self, test, reason): + self._add_result(test, skipped=reason) + super().addSkip(test, reason) + self.__write('S', f'skipped {reason!r}') + + def addSuccess(self, test): + self._add_result(test) + super().addSuccess(test) + self.__write('.', 'ok') + + def addUnexpectedSuccess(self, test): + self._add_result(test, outcome='UNEXPECTED_SUCCESS') + super().addUnexpectedSuccess(test) + self.__write('u', 'unexpected success') + + def printErrors(self): + if self.__verbose: + self.stream.write('\n') + self.printErrorList('ERROR', self.errors) + self.printErrorList('FAIL', self.failures) + + def printErrorList(self, flavor, errors): + for test, err in errors: + self.stream.write(self.separator1) + self.stream.write(f'{flavor}: {self.getDescription(test)}\n') + self.stream.write(self.separator2) + self.stream.write('%s\n' % err) + + def get_xml_element(self): + e = self.__suite + e.set('tests', str(self.testsRun)) + e.set('errors', str(len(self.errors))) + e.set('failures', str(len(self.failures))) + return e + +class QuietRegressionTestRunner: + def __init__(self, stream): + self.result = RegressionTestResult(stream, None, 0) + + def run(self, test): + test(self.result) + return self.result + +def get_test_runner_class(verbosity): + if verbosity: + return functools.partial(unittest.TextTestRunner, + resultclass=RegressionTestResult, + buffer=True, + verbosity=verbosity) + return QuietRegressionTestRunner + +def get_test_runner(stream, verbosity): + return get_test_runner_class(verbosity)(stream) + +if __name__ == '__main__': + class TestTests(unittest.TestCase): + def test_pass(self): + pass + + def test_pass_slow(self): + time.sleep(1.0) + + def test_fail(self): + print('stdout', file=sys.stdout) + print('stderr', file=sys.stderr) + self.fail('failure message') + + def test_error(self): + print('stdout', file=sys.stdout) + print('stderr', file=sys.stderr) + raise RuntimeError('error message') + + suite = unittest.TestSuite() + suite.addTest(unittest.makeSuite(TestTests)) + stream = io.StringIO() + runner_cls = get_test_runner_class(sum(a == '-v' for a in sys.argv)) + runner = runner_cls(sys.stdout) + result = runner.run(suite) + print('Output:', stream.getvalue()) + print('XML: ', end='') + for s in ET.tostringlist(result.get_xml_element()): + print(s.decode(), end='') + print() diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py index f0802a509731..c0c7cb05940b 100644 --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -1459,6 +1459,16 @@ def test_r_1_replace(self): type = argparse.FileType('r', 1, errors='replace') self.assertEqual("FileType('r', 1, errors='replace')", repr(type)) +class StdStreamComparer: + def __init__(self, attr): + self.attr = attr + + def __eq__(self, other): + return other == getattr(sys, self.attr) + +eq_stdin = StdStreamComparer('stdin') +eq_stdout = StdStreamComparer('stdout') +eq_stderr = StdStreamComparer('stderr') class RFile(object): seen = {} @@ -1497,7 +1507,7 @@ def setUp(self): ('foo', NS(x=None, spam=RFile('foo'))), ('-x foo bar', NS(x=RFile('foo'), spam=RFile('bar'))), ('bar -x foo', NS(x=RFile('foo'), spam=RFile('bar'))), - ('-x - -', NS(x=sys.stdin, spam=sys.stdin)), + ('-x - -', NS(x=eq_stdin, spam=eq_stdin)), ('readonly', NS(x=None, spam=RFile('readonly'))), ] @@ -1537,7 +1547,7 @@ def setUp(self): ('foo', NS(x=None, spam=RFile('foo'))), ('-x foo bar', NS(x=RFile('foo'), spam=RFile('bar'))), ('bar -x foo', NS(x=RFile('foo'), spam=RFile('bar'))), - ('-x - -', NS(x=sys.stdin, spam=sys.stdin)), + ('-x - -', NS(x=eq_stdin, spam=eq_stdin)), ] @@ -1576,7 +1586,7 @@ def setUp(self): ('foo', NS(x=None, spam=WFile('foo'))), ('-x foo bar', NS(x=WFile('foo'), spam=WFile('bar'))), ('bar -x foo', NS(x=WFile('foo'), spam=WFile('bar'))), - ('-x - -', NS(x=sys.stdout, spam=sys.stdout)), + ('-x - -', NS(x=eq_stdout, spam=eq_stdout)), ] @@ -1591,7 +1601,7 @@ class TestFileTypeWB(TempDirMixin, ParserTestCase): ('foo', NS(x=None, spam=WFile('foo'))), ('-x foo bar', NS(x=WFile('foo'), spam=WFile('bar'))), ('bar -x foo', NS(x=WFile('foo'), spam=WFile('bar'))), - ('-x - -', NS(x=sys.stdout, spam=sys.stdout)), + ('-x - -', NS(x=eq_stdout, spam=eq_stdout)), ] diff --git a/Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst b/Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst new file mode 100644 index 000000000000..582c15f27c8c --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst @@ -0,0 +1 @@ +Add JUnit XML output for regression tests and update Azure DevOps builds. From webhook-mailer at python.org Tue Sep 18 13:00:09 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 18 Sep 2018 17:00:09 -0000 Subject: [Python-checkins] bpo-34472: Add data descriptor signature to zipfile (GH-8871) Message-ID: https://github.com/python/cpython/commit/4ba3b50bfe6d50cd82d208023ea23e203ab50589 commit: 4ba3b50bfe6d50cd82d208023ea23e203ab50589 branch: master author: Silas Sewell committer: Serhiy Storchaka date: 2018-09-18T20:00:05+03:00 summary: bpo-34472: Add data descriptor signature to zipfile (GH-8871) This makes streamed zips compatible with MacOS Archive Utility and other applications. files: A Misc/NEWS.d/next/Library/2018-08-23-09-25-08.bpo-34472.cGyYrO.rst M Lib/zipfile.py M Misc/ACKS diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 89df90b25209..4a6b40ee441c 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -159,6 +159,8 @@ class LargeZipFile(Exception): _CD64_DIRECTORY_SIZE = 8 _CD64_OFFSET_START_CENTDIR = 9 +_DD_SIGNATURE = 0x08074b50 + _EXTRA_FIELD_STRUCT = struct.Struct(' https://github.com/python/cpython/commit/1a89cb5c479d8e4f467d7f96e1781c7275cefa88 commit: 1a89cb5c479d8e4f467d7f96e1781c7275cefa88 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Steve Dower date: 2018-09-18T11:48:22-07:00 summary: bpo-34582: Adds JUnit XML output for regression tests (GH-9210) (cherry picked from commit d0f49d2f5085ca68e3dc8725f1fb1c9674bfb5ed) Co-authored-by: Steve Dower files: A Lib/test/support/testresult.py A Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst M .vsts/linux-pr.yml M .vsts/macos-pr.yml M .vsts/windows-pr.yml M Lib/test/eintrdata/eintr_tester.py M Lib/test/libregrtest/cmdline.py M Lib/test/libregrtest/main.py M Lib/test/libregrtest/runtest.py M Lib/test/libregrtest/runtest_mp.py M Lib/test/support/__init__.py M Lib/test/test_argparse.py diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml index 6e4ac7c65c4d..d11a4f06e4e1 100644 --- a/.vsts/linux-pr.yml +++ b/.vsts/linux-pr.yml @@ -70,6 +70,15 @@ steps: displayName: 'Run patchcheck.py' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(system.pullRequest.targetBranch)-linux' + platform: linux + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml index c56e66b5090b..69b619e47577 100644 --- a/.vsts/macos-pr.yml +++ b/.vsts/macos-pr.yml @@ -50,6 +50,15 @@ steps: displayName: 'Display build info' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(system.pullRequest.targetBranch)-macOS' + platform: macOS + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/windows-pr.yml b/.vsts/windows-pr.yml index 3dd5609a32e5..7134120d6414 100644 --- a/.vsts/windows-pr.yml +++ b/.vsts/windows-pr.yml @@ -54,8 +54,17 @@ steps: displayName: 'Display build info' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" displayName: 'Tests' env: PREFIX: $(Py_OutDir)\$(outDirSuffix) condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' + mergeTestResults: true + testRunTitle: '$(System.PullRequest.TargetBranch)-$(outDirSuffix)' + platform: $(outDirSuffix) + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py index bc308fe107b3..1caeafe25d9b 100644 --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -52,7 +52,8 @@ def setUpClass(cls): # Issue #25277: Use faulthandler to try to debug a hang on FreeBSD if hasattr(faulthandler, 'dump_traceback_later'): - faulthandler.dump_traceback_later(10 * 60, exit=True) + faulthandler.dump_traceback_later(10 * 60, exit=True, + file=sys.__stderr__) @classmethod def stop_alarm(cls): diff --git a/Lib/test/libregrtest/cmdline.py b/Lib/test/libregrtest/cmdline.py index dab17c3edf32..2af839a182db 100644 --- a/Lib/test/libregrtest/cmdline.py +++ b/Lib/test/libregrtest/cmdline.py @@ -268,6 +268,10 @@ def _create_parser(): help='if a test file alters the environment, mark ' 'the test as failed') + group.add_argument('--junit-xml', dest='xmlpath', metavar='FILENAME', + help='writes JUnit-style XML results to the specified ' + 'file') + return parser diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index e262a7a172b9..b491a08c2424 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -100,8 +100,11 @@ def __init__(self): self.next_single_test = None self.next_single_filename = None + # used by --junit-xml + self.testsuite_xml = None + def accumulate_result(self, test, result): - ok, test_time = result + ok, test_time, xml_data = result if ok not in (CHILD_ERROR, INTERRUPTED): self.test_times.append((test_time, test)) if ok == PASSED: @@ -118,6 +121,15 @@ def accumulate_result(self, test, result): elif ok != INTERRUPTED: raise ValueError("invalid test result: %r" % ok) + if xml_data: + import xml.etree.ElementTree as ET + for e in xml_data: + try: + self.testsuite_xml.append(ET.fromstring(e)) + except ET.ParseError: + print(xml_data, file=sys.__stderr__) + raise + def display_progress(self, test_index, test): if self.ns.quiet: return @@ -164,6 +176,9 @@ def parse_args(self, kwargs): file=sys.stderr) ns.findleaks = False + if ns.xmlpath: + support.junit_xml_list = self.testsuite_xml = [] + # Strip .py extensions. removepy(ns.args) @@ -384,7 +399,7 @@ def run_tests_sequential(self): result = runtest(self.ns, test) except KeyboardInterrupt: self.interrupted = True - self.accumulate_result(test, (INTERRUPTED, None)) + self.accumulate_result(test, (INTERRUPTED, None, None)) break else: self.accumulate_result(test, result) @@ -508,6 +523,31 @@ def finalize(self): if self.ns.runleaks: os.system("leaks %d" % os.getpid()) + def save_xml_result(self): + if not self.ns.xmlpath and not self.testsuite_xml: + return + + import xml.etree.ElementTree as ET + root = ET.Element("testsuites") + + # Manually count the totals for the overall summary + totals = {'tests': 0, 'errors': 0, 'failures': 0} + for suite in self.testsuite_xml: + root.append(suite) + for k in totals: + try: + totals[k] += int(suite.get(k, 0)) + except ValueError: + pass + + for k, v in totals.items(): + root.set(k, str(v)) + + xmlpath = os.path.join(support.SAVEDCWD, self.ns.xmlpath) + with open(xmlpath, 'wb') as f: + for s in ET.tostringlist(root): + f.write(s) + def main(self, tests=None, **kwargs): global TEMPDIR @@ -570,6 +610,9 @@ def _main(self, tests, kwargs): self.rerun_failed_tests() self.finalize() + + self.save_xml_result() + if self.bad: sys.exit(2) if self.interrupted: diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py index 3e1afd41997a..4f41080d37b9 100644 --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -85,8 +85,8 @@ def runtest(ns, test): ns -- regrtest namespace of options test -- the name of the test - Returns the tuple (result, test_time), where result is one of the - constants: + Returns the tuple (result, test_time, xml_data), where result is one + of the constants: INTERRUPTED KeyboardInterrupt when run under -j RESOURCE_DENIED test skipped because resource denied @@ -94,6 +94,9 @@ def runtest(ns, test): ENV_CHANGED test failed because it changed the execution environment FAILED test failed PASSED test passed + + If ns.xmlpath is not None, xml_data is a list containing each + generated testsuite element. """ output_on_failure = ns.verbose3 @@ -106,22 +109,13 @@ def runtest(ns, test): # reset the environment_altered flag to detect if a test altered # the environment support.environment_altered = False + support.junit_xml_list = xml_list = [] if ns.xmlpath else None if ns.failfast: support.failfast = True if output_on_failure: support.verbose = True - # Reuse the same instance to all calls to runtest(). Some - # tests keep a reference to sys.stdout or sys.stderr - # (eg. test_argparse). - if runtest.stringio is None: - stream = io.StringIO() - runtest.stringio = stream - else: - stream = runtest.stringio - stream.seek(0) - stream.truncate() - + stream = io.StringIO() orig_stdout = sys.stdout orig_stderr = sys.stderr try: @@ -138,12 +132,18 @@ def runtest(ns, test): else: support.verbose = ns.verbose # Tell tests to be moderately quiet result = runtest_inner(ns, test, display_failure=not ns.verbose) - return result + + if xml_list: + import xml.etree.ElementTree as ET + xml_data = [ET.tostring(x).decode('us-ascii') for x in xml_list] + else: + xml_data = None + return result + (xml_data,) finally: if use_timeout: faulthandler.cancel_dump_traceback_later() cleanup_test_droppings(test, ns.verbose) -runtest.stringio = None + support.junit_xml_list = None def post_test_cleanup(): diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py index 907451cf6311..779c429bff52 100644 --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -67,7 +67,7 @@ def run_tests_slave(slaveargs): try: result = runtest(ns, testname) except KeyboardInterrupt: - result = INTERRUPTED, '' + result = INTERRUPTED, '', None except BaseException as e: traceback.print_exc() result = CHILD_ERROR, str(e) @@ -122,7 +122,7 @@ def _runtest(self): self.current_test = None if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) + result = (CHILD_ERROR, "Exit code %s" % retcode, None) self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) return False @@ -133,6 +133,7 @@ def _runtest(self): return True result = json.loads(result) + assert len(result) == 3, f"Invalid result tuple: {result!r}" self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) return False @@ -195,7 +196,7 @@ def get_running(workers): regrtest.accumulate_result(test, result) # Display progress - ok, test_time = result + ok, test_time, xml_data = result text = format_test_result(test, ok) if (ok not in (CHILD_ERROR, INTERRUPTED) and test_time >= PROGRESS_MIN_TIME diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index ddcd2cc38759..2ad563469ae3 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -5,6 +5,7 @@ import collections.abc import contextlib +import datetime import errno import faulthandler import fnmatch @@ -12,6 +13,7 @@ import gc import importlib import importlib.util +import io import logging.handlers import nntplib import os @@ -33,6 +35,8 @@ import urllib.error import warnings +from .testresult import get_test_runner + try: import multiprocessing.process except ImportError: @@ -277,6 +281,7 @@ def get_attribute(obj, name): max_memuse = 0 # Disable bigmem tests (they will still be run with # small sizes, to make sure they work.) real_max_memuse = 0 +junit_xml_list = None # list of testsuite XML elements failfast = False # _original_stdout is meant to hold stdout at the time regrtest began. @@ -1873,13 +1878,16 @@ def _filter_suite(suite, pred): def _run_suite(suite): """Run tests from a unittest.TestSuite-derived class.""" - if verbose: - runner = unittest.TextTestRunner(sys.stdout, verbosity=2, - failfast=failfast) - else: - runner = BasicTestRunner() + runner = get_test_runner(sys.stdout, verbosity=verbose) + + # TODO: Remove this before merging (here for easy comparison with old impl) + #runner = unittest.TextTestRunner(sys.stdout, verbosity=2, failfast=failfast) result = runner.run(suite) + + if junit_xml_list is not None: + junit_xml_list.append(result.get_xml_element()) + if not result.wasSuccessful(): if len(result.errors) == 1 and not result.failures: err = result.errors[0][1] diff --git a/Lib/test/support/testresult.py b/Lib/test/support/testresult.py new file mode 100644 index 000000000000..8988d3d15278 --- /dev/null +++ b/Lib/test/support/testresult.py @@ -0,0 +1,201 @@ +'''Test runner and result class for the regression test suite. + +''' + +import functools +import io +import sys +import time +import traceback +import unittest + +import xml.etree.ElementTree as ET + +from datetime import datetime + +class RegressionTestResult(unittest.TextTestResult): + separator1 = '=' * 70 + '\n' + separator2 = '-' * 70 + '\n' + + def __init__(self, stream, descriptions, verbosity): + super().__init__(stream=stream, descriptions=descriptions, verbosity=0) + self.buffer = True + self.__suite = ET.Element('testsuite') + self.__suite.set('start', datetime.utcnow().isoformat(' ')) + + self.__e = None + self.__start_time = None + self.__results = [] + self.__verbose = bool(verbosity) + + @classmethod + def __getId(cls, test): + try: + test_id = test.id + except AttributeError: + return str(test) + try: + return test_id() + except TypeError: + return str(test_id) + return repr(test) + + def startTest(self, test): + super().startTest(test) + self.__e = e = ET.SubElement(self.__suite, 'testcase') + self.__start_time = time.perf_counter() + if self.__verbose: + self.stream.write(f'{self.getDescription(test)} ... ') + self.stream.flush() + + def _add_result(self, test, capture=False, **args): + e = self.__e + self.__e = None + if e is None: + return + e.set('name', args.pop('name', self.__getId(test))) + e.set('status', args.pop('status', 'run')) + e.set('result', args.pop('result', 'completed')) + if self.__start_time: + e.set('time', f'{time.perf_counter() - self.__start_time:0.6f}') + + if capture: + stdout = self._stdout_buffer.getvalue().rstrip() + ET.SubElement(e, 'system-out').text = stdout + stderr = self._stderr_buffer.getvalue().rstrip() + ET.SubElement(e, 'system-err').text = stderr + + for k, v in args.items(): + if not k or not v: + continue + e2 = ET.SubElement(e, k) + if hasattr(v, 'items'): + for k2, v2 in v.items(): + if k2: + e2.set(k2, str(v2)) + else: + e2.text = str(v2) + else: + e2.text = str(v) + + def __write(self, c, word): + if self.__verbose: + self.stream.write(f'{word}\n') + + @classmethod + def __makeErrorDict(cls, err_type, err_value, err_tb): + if isinstance(err_type, type): + if err_type.__module__ == 'builtins': + typename = err_type.__name__ + else: + typename = f'{err_type.__module__}.{err_type.__name__}' + else: + typename = repr(err_type) + + msg = traceback.format_exception(err_type, err_value, None) + tb = traceback.format_exception(err_type, err_value, err_tb) + + return { + 'type': typename, + 'message': ''.join(msg), + '': ''.join(tb), + } + + def addError(self, test, err): + self._add_result(test, True, error=self.__makeErrorDict(*err)) + super().addError(test, err) + self.__write('E', 'ERROR') + + def addExpectedFailure(self, test, err): + self._add_result(test, True, output=self.__makeErrorDict(*err)) + super().addExpectedFailure(test, err) + self.__write('x', 'expected failure') + + def addFailure(self, test, err): + self._add_result(test, True, failure=self.__makeErrorDict(*err)) + super().addFailure(test, err) + self.__write('F', 'FAIL') + + def addSkip(self, test, reason): + self._add_result(test, skipped=reason) + super().addSkip(test, reason) + self.__write('S', f'skipped {reason!r}') + + def addSuccess(self, test): + self._add_result(test) + super().addSuccess(test) + self.__write('.', 'ok') + + def addUnexpectedSuccess(self, test): + self._add_result(test, outcome='UNEXPECTED_SUCCESS') + super().addUnexpectedSuccess(test) + self.__write('u', 'unexpected success') + + def printErrors(self): + if self.__verbose: + self.stream.write('\n') + self.printErrorList('ERROR', self.errors) + self.printErrorList('FAIL', self.failures) + + def printErrorList(self, flavor, errors): + for test, err in errors: + self.stream.write(self.separator1) + self.stream.write(f'{flavor}: {self.getDescription(test)}\n') + self.stream.write(self.separator2) + self.stream.write('%s\n' % err) + + def get_xml_element(self): + e = self.__suite + e.set('tests', str(self.testsRun)) + e.set('errors', str(len(self.errors))) + e.set('failures', str(len(self.failures))) + return e + +class QuietRegressionTestRunner: + def __init__(self, stream): + self.result = RegressionTestResult(stream, None, 0) + + def run(self, test): + test(self.result) + return self.result + +def get_test_runner_class(verbosity): + if verbosity: + return functools.partial(unittest.TextTestRunner, + resultclass=RegressionTestResult, + buffer=True, + verbosity=verbosity) + return QuietRegressionTestRunner + +def get_test_runner(stream, verbosity): + return get_test_runner_class(verbosity)(stream) + +if __name__ == '__main__': + class TestTests(unittest.TestCase): + def test_pass(self): + pass + + def test_pass_slow(self): + time.sleep(1.0) + + def test_fail(self): + print('stdout', file=sys.stdout) + print('stderr', file=sys.stderr) + self.fail('failure message') + + def test_error(self): + print('stdout', file=sys.stdout) + print('stderr', file=sys.stderr) + raise RuntimeError('error message') + + suite = unittest.TestSuite() + suite.addTest(unittest.makeSuite(TestTests)) + stream = io.StringIO() + runner_cls = get_test_runner_class(sum(a == '-v' for a in sys.argv)) + runner = runner_cls(sys.stdout) + result = runner.run(suite) + print('Output:', stream.getvalue()) + print('XML: ', end='') + for s in ET.tostringlist(result.get_xml_element()): + print(s.decode(), end='') + print() diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py index 85449c729902..51f0effaf2ff 100644 --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -1459,6 +1459,16 @@ def test_r_1_replace(self): type = argparse.FileType('r', 1, errors='replace') self.assertEqual("FileType('r', 1, errors='replace')", repr(type)) +class StdStreamComparer: + def __init__(self, attr): + self.attr = attr + + def __eq__(self, other): + return other == getattr(sys, self.attr) + +eq_stdin = StdStreamComparer('stdin') +eq_stdout = StdStreamComparer('stdout') +eq_stderr = StdStreamComparer('stderr') class RFile(object): seen = {} @@ -1497,7 +1507,7 @@ def setUp(self): ('foo', NS(x=None, spam=RFile('foo'))), ('-x foo bar', NS(x=RFile('foo'), spam=RFile('bar'))), ('bar -x foo', NS(x=RFile('foo'), spam=RFile('bar'))), - ('-x - -', NS(x=sys.stdin, spam=sys.stdin)), + ('-x - -', NS(x=eq_stdin, spam=eq_stdin)), ('readonly', NS(x=None, spam=RFile('readonly'))), ] @@ -1537,7 +1547,7 @@ def setUp(self): ('foo', NS(x=None, spam=RFile('foo'))), ('-x foo bar', NS(x=RFile('foo'), spam=RFile('bar'))), ('bar -x foo', NS(x=RFile('foo'), spam=RFile('bar'))), - ('-x - -', NS(x=sys.stdin, spam=sys.stdin)), + ('-x - -', NS(x=eq_stdin, spam=eq_stdin)), ] @@ -1576,7 +1586,7 @@ def setUp(self): ('foo', NS(x=None, spam=WFile('foo'))), ('-x foo bar', NS(x=WFile('foo'), spam=WFile('bar'))), ('bar -x foo', NS(x=WFile('foo'), spam=WFile('bar'))), - ('-x - -', NS(x=sys.stdout, spam=sys.stdout)), + ('-x - -', NS(x=eq_stdout, spam=eq_stdout)), ] @@ -1591,7 +1601,7 @@ class TestFileTypeWB(TempDirMixin, ParserTestCase): ('foo', NS(x=None, spam=WFile('foo'))), ('-x foo bar', NS(x=WFile('foo'), spam=WFile('bar'))), ('bar -x foo', NS(x=WFile('foo'), spam=WFile('bar'))), - ('-x - -', NS(x=sys.stdout, spam=sys.stdout)), + ('-x - -', NS(x=eq_stdout, spam=eq_stdout)), ] diff --git a/Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst b/Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst new file mode 100644 index 000000000000..582c15f27c8c --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst @@ -0,0 +1 @@ +Add JUnit XML output for regression tests and update Azure DevOps builds. From webhook-mailer at python.org Tue Sep 18 14:58:15 2018 From: webhook-mailer at python.org (Steve Dower) Date: Tue, 18 Sep 2018 18:58:15 -0000 Subject: [Python-checkins] bpo-34582: Adds JUnit XML output for regression tests (GH-9210) Message-ID: https://github.com/python/cpython/commit/c9276495fc497898544a38b244b38ff61797f32d commit: c9276495fc497898544a38b244b38ff61797f32d branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Steve Dower date: 2018-09-18T11:58:08-07:00 summary: bpo-34582: Adds JUnit XML output for regression tests (GH-9210) (cherry picked from commit d0f49d2f5085ca68e3dc8725f1fb1c9674bfb5ed) Co-authored-by: Steve Dower files: A Lib/test/support/testresult.py A Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst M .vsts/linux-pr.yml M .vsts/macos-pr.yml M .vsts/windows-pr.yml M Lib/test/eintrdata/eintr_tester.py M Lib/test/libregrtest/cmdline.py M Lib/test/libregrtest/main.py M Lib/test/libregrtest/runtest.py M Lib/test/libregrtest/runtest_mp.py M Lib/test/support/__init__.py M Lib/test/test_argparse.py diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml index 945ac097e5e8..cf810052f8e9 100644 --- a/.vsts/linux-pr.yml +++ b/.vsts/linux-pr.yml @@ -56,6 +56,15 @@ steps: displayName: 'Run patchcheck.py' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(system.pullRequest.targetBranch)-linux' + platform: linux + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml index c56e66b5090b..69b619e47577 100644 --- a/.vsts/macos-pr.yml +++ b/.vsts/macos-pr.yml @@ -50,6 +50,15 @@ steps: displayName: 'Display build info' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(system.pullRequest.targetBranch)-macOS' + platform: macOS + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/windows-pr.yml b/.vsts/windows-pr.yml index 3dd5609a32e5..7134120d6414 100644 --- a/.vsts/windows-pr.yml +++ b/.vsts/windows-pr.yml @@ -54,8 +54,17 @@ steps: displayName: 'Display build info' condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" displayName: 'Tests' env: PREFIX: $(Py_OutDir)\$(outDirSuffix) condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' + mergeTestResults: true + testRunTitle: '$(System.PullRequest.TargetBranch)-$(outDirSuffix)' + platform: $(outDirSuffix) + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py index bc308fe107b3..1caeafe25d9b 100644 --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -52,7 +52,8 @@ def setUpClass(cls): # Issue #25277: Use faulthandler to try to debug a hang on FreeBSD if hasattr(faulthandler, 'dump_traceback_later'): - faulthandler.dump_traceback_later(10 * 60, exit=True) + faulthandler.dump_traceback_later(10 * 60, exit=True, + file=sys.__stderr__) @classmethod def stop_alarm(cls): diff --git a/Lib/test/libregrtest/cmdline.py b/Lib/test/libregrtest/cmdline.py index dab17c3edf32..2af839a182db 100644 --- a/Lib/test/libregrtest/cmdline.py +++ b/Lib/test/libregrtest/cmdline.py @@ -268,6 +268,10 @@ def _create_parser(): help='if a test file alters the environment, mark ' 'the test as failed') + group.add_argument('--junit-xml', dest='xmlpath', metavar='FILENAME', + help='writes JUnit-style XML results to the specified ' + 'file') + return parser diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index e262a7a172b9..b491a08c2424 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -100,8 +100,11 @@ def __init__(self): self.next_single_test = None self.next_single_filename = None + # used by --junit-xml + self.testsuite_xml = None + def accumulate_result(self, test, result): - ok, test_time = result + ok, test_time, xml_data = result if ok not in (CHILD_ERROR, INTERRUPTED): self.test_times.append((test_time, test)) if ok == PASSED: @@ -118,6 +121,15 @@ def accumulate_result(self, test, result): elif ok != INTERRUPTED: raise ValueError("invalid test result: %r" % ok) + if xml_data: + import xml.etree.ElementTree as ET + for e in xml_data: + try: + self.testsuite_xml.append(ET.fromstring(e)) + except ET.ParseError: + print(xml_data, file=sys.__stderr__) + raise + def display_progress(self, test_index, test): if self.ns.quiet: return @@ -164,6 +176,9 @@ def parse_args(self, kwargs): file=sys.stderr) ns.findleaks = False + if ns.xmlpath: + support.junit_xml_list = self.testsuite_xml = [] + # Strip .py extensions. removepy(ns.args) @@ -384,7 +399,7 @@ def run_tests_sequential(self): result = runtest(self.ns, test) except KeyboardInterrupt: self.interrupted = True - self.accumulate_result(test, (INTERRUPTED, None)) + self.accumulate_result(test, (INTERRUPTED, None, None)) break else: self.accumulate_result(test, result) @@ -508,6 +523,31 @@ def finalize(self): if self.ns.runleaks: os.system("leaks %d" % os.getpid()) + def save_xml_result(self): + if not self.ns.xmlpath and not self.testsuite_xml: + return + + import xml.etree.ElementTree as ET + root = ET.Element("testsuites") + + # Manually count the totals for the overall summary + totals = {'tests': 0, 'errors': 0, 'failures': 0} + for suite in self.testsuite_xml: + root.append(suite) + for k in totals: + try: + totals[k] += int(suite.get(k, 0)) + except ValueError: + pass + + for k, v in totals.items(): + root.set(k, str(v)) + + xmlpath = os.path.join(support.SAVEDCWD, self.ns.xmlpath) + with open(xmlpath, 'wb') as f: + for s in ET.tostringlist(root): + f.write(s) + def main(self, tests=None, **kwargs): global TEMPDIR @@ -570,6 +610,9 @@ def _main(self, tests, kwargs): self.rerun_failed_tests() self.finalize() + + self.save_xml_result() + if self.bad: sys.exit(2) if self.interrupted: diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py index 3e1afd41997a..4f41080d37b9 100644 --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -85,8 +85,8 @@ def runtest(ns, test): ns -- regrtest namespace of options test -- the name of the test - Returns the tuple (result, test_time), where result is one of the - constants: + Returns the tuple (result, test_time, xml_data), where result is one + of the constants: INTERRUPTED KeyboardInterrupt when run under -j RESOURCE_DENIED test skipped because resource denied @@ -94,6 +94,9 @@ def runtest(ns, test): ENV_CHANGED test failed because it changed the execution environment FAILED test failed PASSED test passed + + If ns.xmlpath is not None, xml_data is a list containing each + generated testsuite element. """ output_on_failure = ns.verbose3 @@ -106,22 +109,13 @@ def runtest(ns, test): # reset the environment_altered flag to detect if a test altered # the environment support.environment_altered = False + support.junit_xml_list = xml_list = [] if ns.xmlpath else None if ns.failfast: support.failfast = True if output_on_failure: support.verbose = True - # Reuse the same instance to all calls to runtest(). Some - # tests keep a reference to sys.stdout or sys.stderr - # (eg. test_argparse). - if runtest.stringio is None: - stream = io.StringIO() - runtest.stringio = stream - else: - stream = runtest.stringio - stream.seek(0) - stream.truncate() - + stream = io.StringIO() orig_stdout = sys.stdout orig_stderr = sys.stderr try: @@ -138,12 +132,18 @@ def runtest(ns, test): else: support.verbose = ns.verbose # Tell tests to be moderately quiet result = runtest_inner(ns, test, display_failure=not ns.verbose) - return result + + if xml_list: + import xml.etree.ElementTree as ET + xml_data = [ET.tostring(x).decode('us-ascii') for x in xml_list] + else: + xml_data = None + return result + (xml_data,) finally: if use_timeout: faulthandler.cancel_dump_traceback_later() cleanup_test_droppings(test, ns.verbose) -runtest.stringio = None + support.junit_xml_list = None def post_test_cleanup(): diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py index fa9178a863e1..e5e808a6aa53 100644 --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -71,7 +71,7 @@ def run_tests_slave(slaveargs): try: result = runtest(ns, testname) except KeyboardInterrupt: - result = INTERRUPTED, '' + result = INTERRUPTED, '', None except BaseException as e: traceback.print_exc() result = CHILD_ERROR, str(e) @@ -126,7 +126,7 @@ def _runtest(self): self.current_test = None if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) + result = (CHILD_ERROR, "Exit code %s" % retcode, None) self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) return False @@ -137,6 +137,7 @@ def _runtest(self): return True result = json.loads(result) + assert len(result) == 3, f"Invalid result tuple: {result!r}" self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) return False @@ -199,7 +200,7 @@ def get_running(workers): regrtest.accumulate_result(test, result) # Display progress - ok, test_time = result + ok, test_time, xml_data = result text = format_test_result(test, ok) if (ok not in (CHILD_ERROR, INTERRUPTED) and test_time >= PROGRESS_MIN_TIME diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 20c1edc53c21..9b3ebbadfe44 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -5,6 +5,7 @@ import collections.abc import contextlib +import datetime import errno import faulthandler import fnmatch @@ -12,6 +13,7 @@ import gc import importlib import importlib.util +import io import logging.handlers import nntplib import os @@ -31,6 +33,8 @@ import urllib.error import warnings +from .testresult import get_test_runner + try: import _thread, threading except ImportError: @@ -280,6 +284,7 @@ def get_attribute(obj, name): max_memuse = 0 # Disable bigmem tests (they will still be run with # small sizes, to make sure they work.) real_max_memuse = 0 +junit_xml_list = None # list of testsuite XML elements failfast = False # _original_stdout is meant to hold stdout at the time regrtest began. @@ -1899,13 +1904,16 @@ def _filter_suite(suite, pred): def _run_suite(suite): """Run tests from a unittest.TestSuite-derived class.""" - if verbose: - runner = unittest.TextTestRunner(sys.stdout, verbosity=2, - failfast=failfast) - else: - runner = BasicTestRunner() + runner = get_test_runner(sys.stdout, verbosity=verbose) + + # TODO: Remove this before merging (here for easy comparison with old impl) + #runner = unittest.TextTestRunner(sys.stdout, verbosity=2, failfast=failfast) result = runner.run(suite) + + if junit_xml_list is not None: + junit_xml_list.append(result.get_xml_element()) + if not result.wasSuccessful(): if len(result.errors) == 1 and not result.failures: err = result.errors[0][1] diff --git a/Lib/test/support/testresult.py b/Lib/test/support/testresult.py new file mode 100644 index 000000000000..8988d3d15278 --- /dev/null +++ b/Lib/test/support/testresult.py @@ -0,0 +1,201 @@ +'''Test runner and result class for the regression test suite. + +''' + +import functools +import io +import sys +import time +import traceback +import unittest + +import xml.etree.ElementTree as ET + +from datetime import datetime + +class RegressionTestResult(unittest.TextTestResult): + separator1 = '=' * 70 + '\n' + separator2 = '-' * 70 + '\n' + + def __init__(self, stream, descriptions, verbosity): + super().__init__(stream=stream, descriptions=descriptions, verbosity=0) + self.buffer = True + self.__suite = ET.Element('testsuite') + self.__suite.set('start', datetime.utcnow().isoformat(' ')) + + self.__e = None + self.__start_time = None + self.__results = [] + self.__verbose = bool(verbosity) + + @classmethod + def __getId(cls, test): + try: + test_id = test.id + except AttributeError: + return str(test) + try: + return test_id() + except TypeError: + return str(test_id) + return repr(test) + + def startTest(self, test): + super().startTest(test) + self.__e = e = ET.SubElement(self.__suite, 'testcase') + self.__start_time = time.perf_counter() + if self.__verbose: + self.stream.write(f'{self.getDescription(test)} ... ') + self.stream.flush() + + def _add_result(self, test, capture=False, **args): + e = self.__e + self.__e = None + if e is None: + return + e.set('name', args.pop('name', self.__getId(test))) + e.set('status', args.pop('status', 'run')) + e.set('result', args.pop('result', 'completed')) + if self.__start_time: + e.set('time', f'{time.perf_counter() - self.__start_time:0.6f}') + + if capture: + stdout = self._stdout_buffer.getvalue().rstrip() + ET.SubElement(e, 'system-out').text = stdout + stderr = self._stderr_buffer.getvalue().rstrip() + ET.SubElement(e, 'system-err').text = stderr + + for k, v in args.items(): + if not k or not v: + continue + e2 = ET.SubElement(e, k) + if hasattr(v, 'items'): + for k2, v2 in v.items(): + if k2: + e2.set(k2, str(v2)) + else: + e2.text = str(v2) + else: + e2.text = str(v) + + def __write(self, c, word): + if self.__verbose: + self.stream.write(f'{word}\n') + + @classmethod + def __makeErrorDict(cls, err_type, err_value, err_tb): + if isinstance(err_type, type): + if err_type.__module__ == 'builtins': + typename = err_type.__name__ + else: + typename = f'{err_type.__module__}.{err_type.__name__}' + else: + typename = repr(err_type) + + msg = traceback.format_exception(err_type, err_value, None) + tb = traceback.format_exception(err_type, err_value, err_tb) + + return { + 'type': typename, + 'message': ''.join(msg), + '': ''.join(tb), + } + + def addError(self, test, err): + self._add_result(test, True, error=self.__makeErrorDict(*err)) + super().addError(test, err) + self.__write('E', 'ERROR') + + def addExpectedFailure(self, test, err): + self._add_result(test, True, output=self.__makeErrorDict(*err)) + super().addExpectedFailure(test, err) + self.__write('x', 'expected failure') + + def addFailure(self, test, err): + self._add_result(test, True, failure=self.__makeErrorDict(*err)) + super().addFailure(test, err) + self.__write('F', 'FAIL') + + def addSkip(self, test, reason): + self._add_result(test, skipped=reason) + super().addSkip(test, reason) + self.__write('S', f'skipped {reason!r}') + + def addSuccess(self, test): + self._add_result(test) + super().addSuccess(test) + self.__write('.', 'ok') + + def addUnexpectedSuccess(self, test): + self._add_result(test, outcome='UNEXPECTED_SUCCESS') + super().addUnexpectedSuccess(test) + self.__write('u', 'unexpected success') + + def printErrors(self): + if self.__verbose: + self.stream.write('\n') + self.printErrorList('ERROR', self.errors) + self.printErrorList('FAIL', self.failures) + + def printErrorList(self, flavor, errors): + for test, err in errors: + self.stream.write(self.separator1) + self.stream.write(f'{flavor}: {self.getDescription(test)}\n') + self.stream.write(self.separator2) + self.stream.write('%s\n' % err) + + def get_xml_element(self): + e = self.__suite + e.set('tests', str(self.testsRun)) + e.set('errors', str(len(self.errors))) + e.set('failures', str(len(self.failures))) + return e + +class QuietRegressionTestRunner: + def __init__(self, stream): + self.result = RegressionTestResult(stream, None, 0) + + def run(self, test): + test(self.result) + return self.result + +def get_test_runner_class(verbosity): + if verbosity: + return functools.partial(unittest.TextTestRunner, + resultclass=RegressionTestResult, + buffer=True, + verbosity=verbosity) + return QuietRegressionTestRunner + +def get_test_runner(stream, verbosity): + return get_test_runner_class(verbosity)(stream) + +if __name__ == '__main__': + class TestTests(unittest.TestCase): + def test_pass(self): + pass + + def test_pass_slow(self): + time.sleep(1.0) + + def test_fail(self): + print('stdout', file=sys.stdout) + print('stderr', file=sys.stderr) + self.fail('failure message') + + def test_error(self): + print('stdout', file=sys.stdout) + print('stderr', file=sys.stderr) + raise RuntimeError('error message') + + suite = unittest.TestSuite() + suite.addTest(unittest.makeSuite(TestTests)) + stream = io.StringIO() + runner_cls = get_test_runner_class(sum(a == '-v' for a in sys.argv)) + runner = runner_cls(sys.stdout) + result = runner.run(suite) + print('Output:', stream.getvalue()) + print('XML: ', end='') + for s in ET.tostringlist(result.get_xml_element()): + print(s.decode(), end='') + print() diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py index 41a7f4145815..1ba899677435 100644 --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -1459,6 +1459,16 @@ def test_r_1_replace(self): type = argparse.FileType('r', 1, errors='replace') self.assertEqual("FileType('r', 1, errors='replace')", repr(type)) +class StdStreamComparer: + def __init__(self, attr): + self.attr = attr + + def __eq__(self, other): + return other == getattr(sys, self.attr) + +eq_stdin = StdStreamComparer('stdin') +eq_stdout = StdStreamComparer('stdout') +eq_stderr = StdStreamComparer('stderr') class RFile(object): seen = {} @@ -1497,7 +1507,7 @@ def setUp(self): ('foo', NS(x=None, spam=RFile('foo'))), ('-x foo bar', NS(x=RFile('foo'), spam=RFile('bar'))), ('bar -x foo', NS(x=RFile('foo'), spam=RFile('bar'))), - ('-x - -', NS(x=sys.stdin, spam=sys.stdin)), + ('-x - -', NS(x=eq_stdin, spam=eq_stdin)), ('readonly', NS(x=None, spam=RFile('readonly'))), ] @@ -1537,7 +1547,7 @@ def setUp(self): ('foo', NS(x=None, spam=RFile('foo'))), ('-x foo bar', NS(x=RFile('foo'), spam=RFile('bar'))), ('bar -x foo', NS(x=RFile('foo'), spam=RFile('bar'))), - ('-x - -', NS(x=sys.stdin, spam=sys.stdin)), + ('-x - -', NS(x=eq_stdin, spam=eq_stdin)), ] @@ -1576,7 +1586,7 @@ def setUp(self): ('foo', NS(x=None, spam=WFile('foo'))), ('-x foo bar', NS(x=WFile('foo'), spam=WFile('bar'))), ('bar -x foo', NS(x=WFile('foo'), spam=WFile('bar'))), - ('-x - -', NS(x=sys.stdout, spam=sys.stdout)), + ('-x - -', NS(x=eq_stdout, spam=eq_stdout)), ] @@ -1591,7 +1601,7 @@ class TestFileTypeWB(TempDirMixin, ParserTestCase): ('foo', NS(x=None, spam=WFile('foo'))), ('-x foo bar', NS(x=WFile('foo'), spam=WFile('bar'))), ('bar -x foo', NS(x=WFile('foo'), spam=WFile('bar'))), - ('-x - -', NS(x=sys.stdout, spam=sys.stdout)), + ('-x - -', NS(x=eq_stdout, spam=eq_stdout)), ] diff --git a/Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst b/Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst new file mode 100644 index 000000000000..582c15f27c8c --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-09-14-09-53-21.bpo-34582.j3omgk.rst @@ -0,0 +1 @@ +Add JUnit XML output for regression tests and update Azure DevOps builds. From webhook-mailer at python.org Tue Sep 18 15:22:36 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 18 Sep 2018 19:22:36 -0000 Subject: [Python-checkins] bpo-25711: Rewrite zipimport in pure Python. (GH-6809) Message-ID: https://github.com/python/cpython/commit/79d1c2e6c9d1bc1cf41ec3041801ca1a2b9a995b commit: 79d1c2e6c9d1bc1cf41ec3041801ca1a2b9a995b branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-18T22:22:29+03:00 summary: bpo-25711: Rewrite zipimport in pure Python. (GH-6809) files: A Lib/zipimport.py A Misc/NEWS.d/next/Core and Builtins/2018-05-14-18-54-03.bpo-25711.9xfq-v.rst A Python/importlib_zipimport.h D Modules/clinic/zipimport.c.h D Modules/zipimport.c M Lib/ctypes/test/test_values.py M Lib/importlib/__init__.py M Lib/importlib/_bootstrap_external.py M Lib/test/test_bdb.py M Lib/test/test_importlib/source/test_file_loader.py M Lib/test/test_importlib/test_abc.py M Lib/test/test_zipimport.py M Makefile.pre.in M Modules/Setup M PC/config.c M PCbuild/_freeze_importlib.vcxproj M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters M Programs/_freeze_importlib.c M Python/frozen.c M Python/importlib.h M Python/importlib_external.h M Python/pylifecycle.c diff --git a/Lib/ctypes/test/test_values.py b/Lib/ctypes/test/test_values.py index e71b48020f9d..b38b63f870a6 100644 --- a/Lib/ctypes/test/test_values.py +++ b/Lib/ctypes/test/test_values.py @@ -64,6 +64,7 @@ class struct_frozen(Structure): bootstrap_expected = [ b'_frozen_importlib', b'_frozen_importlib_external', + b'zipimport', ] for entry in ft: # This is dangerous. We *can* iterate over a pointer, but diff --git a/Lib/importlib/__init__.py b/Lib/importlib/__init__.py index cb37d246a707..0c73c505f98d 100644 --- a/Lib/importlib/__init__.py +++ b/Lib/importlib/__init__.py @@ -48,8 +48,8 @@ sys.modules['importlib._bootstrap_external'] = _bootstrap_external # To simplify imports in test code -_w_long = _bootstrap_external._w_long -_r_long = _bootstrap_external._r_long +_pack_uint32 = _bootstrap_external._pack_uint32 +_unpack_uint32 = _bootstrap_external._unpack_uint32 # Fully bootstrapped at this point, import whatever you like, circular # dependencies and startup overhead minimisation permitting :) diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py index 7f8bc2167c1e..6ef6bf8ab6a4 100644 --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -43,14 +43,20 @@ def _relax_case(): return _relax_case -def _w_long(x): +def _pack_uint32(x): """Convert a 32-bit integer to little-endian.""" return (int(x) & 0xFFFFFFFF).to_bytes(4, 'little') -def _r_long(int_bytes): +def _unpack_uint32(data): """Convert 4 bytes in little-endian to an integer.""" - return int.from_bytes(int_bytes, 'little') + assert len(data) == 4 + return int.from_bytes(data, 'little') + +def _unpack_uint16(data): + """Convert 2 bytes in little-endian to an integer.""" + assert len(data) == 2 + return int.from_bytes(data, 'little') def _path_join(*path_parts): @@ -503,7 +509,7 @@ def _classify_pyc(data, name, exc_details): message = f'reached EOF while reading pyc header of {name!r}' _bootstrap._verbose_message('{}', message) raise EOFError(message) - flags = _r_long(data[4:8]) + flags = _unpack_uint32(data[4:8]) # Only the first two flags are defined. if flags & ~0b11: message = f'invalid flags {flags!r} in {name!r}' @@ -530,12 +536,12 @@ def _validate_timestamp_pyc(data, source_mtime, source_size, name, An ImportError is raised if the bytecode is stale. """ - if _r_long(data[8:12]) != (source_mtime & 0xFFFFFFFF): + if _unpack_uint32(data[8:12]) != (source_mtime & 0xFFFFFFFF): message = f'bytecode is stale for {name!r}' _bootstrap._verbose_message('{}', message) raise ImportError(message, **exc_details) if (source_size is not None and - _r_long(data[12:16]) != (source_size & 0xFFFFFFFF)): + _unpack_uint32(data[12:16]) != (source_size & 0xFFFFFFFF)): raise ImportError(f'bytecode is stale for {name!r}', **exc_details) @@ -579,9 +585,9 @@ def _compile_bytecode(data, name=None, bytecode_path=None, source_path=None): def _code_to_timestamp_pyc(code, mtime=0, source_size=0): "Produce the data for a timestamp-based pyc." data = bytearray(MAGIC_NUMBER) - data.extend(_w_long(0)) - data.extend(_w_long(mtime)) - data.extend(_w_long(source_size)) + data.extend(_pack_uint32(0)) + data.extend(_pack_uint32(mtime)) + data.extend(_pack_uint32(source_size)) data.extend(marshal.dumps(code)) return data @@ -590,7 +596,7 @@ def _code_to_hash_pyc(code, source_hash, checked=True): "Produce the data for a hash-based pyc." data = bytearray(MAGIC_NUMBER) flags = 0b1 | checked << 1 - data.extend(_w_long(flags)) + data.extend(_pack_uint32(flags)) assert len(source_hash) == 8 data.extend(source_hash) data.extend(marshal.dumps(code)) diff --git a/Lib/test/test_bdb.py b/Lib/test/test_bdb.py index 616c3a864984..8a9c4dd4376c 100644 --- a/Lib/test/test_bdb.py +++ b/Lib/test/test_bdb.py @@ -726,7 +726,7 @@ def main(): ('line', 2, 'tfunc_import'), ('step', ), ('line', 3, 'tfunc_import'), ('quit', ), ] - skip = ('importlib*', TEST_MODULE) + skip = ('importlib*', 'zipimport', TEST_MODULE) with TracerRun(self, skip=skip) as tracer: tracer.runcall(tfunc_import) diff --git a/Lib/test/test_importlib/source/test_file_loader.py b/Lib/test/test_importlib/source/test_file_loader.py index cc80f26357ed..c70c5eb34ae3 100644 --- a/Lib/test/test_importlib/source/test_file_loader.py +++ b/Lib/test/test_importlib/source/test_file_loader.py @@ -629,7 +629,7 @@ def test_old_timestamp(self): bytecode_file.write(zeros) self.import_(mapping['_temp'], '_temp') source_mtime = os.path.getmtime(mapping['_temp']) - source_timestamp = self.importlib._w_long(source_mtime) + source_timestamp = self.importlib._pack_uint32(source_mtime) with open(bytecode_path, 'rb') as bytecode_file: bytecode_file.seek(8) self.assertEqual(bytecode_file.read(4), source_timestamp) diff --git a/Lib/test/test_importlib/test_abc.py b/Lib/test/test_importlib/test_abc.py index f1e1db356232..05608bbb8b9d 100644 --- a/Lib/test/test_importlib/test_abc.py +++ b/Lib/test/test_importlib/test_abc.py @@ -712,9 +712,9 @@ def __init__(self, path, magic=None): if magic is None: magic = self.util.MAGIC_NUMBER data = bytearray(magic) - data.extend(self.init._w_long(0)) - data.extend(self.init._w_long(self.source_mtime)) - data.extend(self.init._w_long(self.source_size)) + data.extend(self.init._pack_uint32(0)) + data.extend(self.init._pack_uint32(self.source_mtime)) + data.extend(self.init._pack_uint32(self.source_size)) code_object = compile(self.source, self.path, 'exec', dont_inherit=True) data.extend(marshal.dumps(code_object)) @@ -876,9 +876,9 @@ def verify_code(self, code_object, *, bytecode_written=False): if bytecode_written: self.assertIn(self.cached, self.loader.written) data = bytearray(self.util.MAGIC_NUMBER) - data.extend(self.init._w_long(0)) - data.extend(self.init._w_long(self.loader.source_mtime)) - data.extend(self.init._w_long(self.loader.source_size)) + data.extend(self.init._pack_uint32(0)) + data.extend(self.init._pack_uint32(self.loader.source_mtime)) + data.extend(self.init._pack_uint32(self.loader.source_size)) data.extend(marshal.dumps(code_object)) self.assertEqual(self.loader.written[self.cached], bytes(data)) diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py index 901bebdd71db..cad73ea3f146 100644 --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -551,7 +551,12 @@ def replace(self, old, new): z.writestr(name, data) z.close() zi = zipimport.zipimporter(TEMP_ZIP) - self.assertEqual(data, zi.get_data(FunnyStr(name))) + try: + data2 = zi.get_data(FunnyStr(name)) + except AttributeError: + pass + else: + self.assertEqual(data2, data) finally: z.close() os.remove(TEMP_ZIP) @@ -677,9 +682,9 @@ def testBytesPath(self): zipimport.zipimporter(filename) zipimport.zipimporter(os.fsencode(filename)) - with self.assertWarns(DeprecationWarning): + with self.assertRaises(TypeError): zipimport.zipimporter(bytearray(os.fsencode(filename))) - with self.assertWarns(DeprecationWarning): + with self.assertRaises(TypeError): zipimport.zipimporter(memoryview(os.fsencode(filename))) @support.cpython_only @@ -687,14 +692,14 @@ def testUninitializedZipimporter(self): # The interpreter shouldn't crash in case of calling methods of an # uninitialized zipimport.zipimporter object. zi = zipimport.zipimporter.__new__(zipimport.zipimporter) - self.assertRaises(ValueError, zi.find_module, 'foo') - self.assertRaises(ValueError, zi.find_loader, 'foo') - self.assertRaises(ValueError, zi.load_module, 'foo') - self.assertRaises(ValueError, zi.get_filename, 'foo') - self.assertRaises(ValueError, zi.is_package, 'foo') - self.assertRaises(ValueError, zi.get_data, 'foo') - self.assertRaises(ValueError, zi.get_code, 'foo') - self.assertRaises(ValueError, zi.get_source, 'foo') + self.assertRaises((ValueError, AttributeError), zi.find_module, 'foo') + self.assertRaises((ValueError, AttributeError), zi.find_loader, 'foo') + self.assertRaises((ValueError, AttributeError), zi.load_module, 'foo') + self.assertRaises((ValueError, AttributeError), zi.get_filename, 'foo') + self.assertRaises((ValueError, AttributeError), zi.is_package, 'foo') + self.assertRaises((ValueError, AttributeError), zi.get_data, 'foo') + self.assertRaises((ValueError, AttributeError), zi.get_code, 'foo') + self.assertRaises((ValueError, AttributeError), zi.get_source, 'foo') @support.requires_zlib @@ -712,7 +717,7 @@ def bad_decompress(*args): zip_file.writestr('bar.py', b'print("hello world")', ZIP_DEFLATED) zi = zipimport.zipimporter(TEMP_ZIP) with support.swap_attr(zlib, 'decompress', bad_decompress): - self.assertRaises(TypeError, zi.get_source, 'bar') + self.assertRaises((TypeError, AttributeError), zi.get_source, 'bar') class BadFileZipImportTestCase(unittest.TestCase): diff --git a/Lib/zipimport.py b/Lib/zipimport.py new file mode 100644 index 000000000000..059f124512dc --- /dev/null +++ b/Lib/zipimport.py @@ -0,0 +1,650 @@ +"""zipimport provides support for importing Python modules from Zip archives. + +This module exports three objects: +- zipimporter: a class; its constructor takes a path to a Zip archive. +- ZipImportError: exception raised by zipimporter objects. It's a + subclass of ImportError, so it can be caught as ImportError, too. +- _zip_directory_cache: a dict, mapping archive paths to zip directory + info dicts, as used in zipimporter._files. + +It is usually not needed to use the zipimport module explicitly; it is +used by the builtin import mechanism for sys.path items that are paths +to Zip archives. +""" + +#from importlib import _bootstrap_external +#from importlib import _bootstrap # for _verbose_message +import _frozen_importlib_external as _bootstrap_external +from _frozen_importlib_external import _unpack_uint16, _unpack_uint32 +import _frozen_importlib as _bootstrap # for _verbose_message +import _imp # for check_hash_based_pycs +import _io # for open +import marshal # for loads +import sys # for modules +import time # for mktime + +__all__ = ['ZipImportError', 'zipimporter'] + + +path_sep = _bootstrap_external.path_sep +alt_path_sep = _bootstrap_external.path_separators[1:] + + +class ZipImportError(ImportError): + pass + +# _read_directory() cache +_zip_directory_cache = {} + +_module_type = type(sys) + + +class zipimporter: + """zipimporter(archivepath) -> zipimporter object + + Create a new zipimporter instance. 'archivepath' must be a path to + a zipfile, or to a specific path inside a zipfile. For example, it can be + '/tmp/myimport.zip', or '/tmp/myimport.zip/mydirectory', if mydirectory is a + valid directory inside the archive. + + 'ZipImportError is raised if 'archivepath' doesn't point to a valid Zip + archive. + + The 'archive' attribute of zipimporter objects contains the name of the + zipfile targeted. + """ + + # Split the "subdirectory" from the Zip archive path, lookup a matching + # entry in sys.path_importer_cache, fetch the file directory from there + # if found, or else read it from the archive. + def __init__(self, path): + if not isinstance(path, str): + import os + path = os.fsdecode(path) + if not path: + raise ZipImportError('archive path is empty', path=path) + if alt_path_sep: + path = path.replace(alt_path_sep, path_sep) + + prefix = [] + while True: + try: + st = _bootstrap_external._path_stat(path) + except (OSError, ValueError): + # On Windows a ValueError is raised for too long paths. + # Back up one path element. + dirname, basename = _bootstrap_external._path_split(path) + if dirname == path: + raise ZipImportError('not a Zip file', path=path) + path = dirname + prefix.append(basename) + else: + # it exists + if (st.st_mode & 0o170000) != 0o100000: # stat.S_ISREG + # it's a not file + raise ZipImportError('not a Zip file', path=path) + break + + try: + files = _zip_directory_cache[path] + except KeyError: + files = _read_directory(path) + _zip_directory_cache[path] = files + self._files = files + self.archive = path + # a prefix directory following the ZIP file path. + self.prefix = _bootstrap_external._path_join(*prefix[::-1]) + if self.prefix: + self.prefix += path_sep + + + # Check whether we can satisfy the import of the module named by + # 'fullname', or whether it could be a portion of a namespace + # package. Return self if we can load it, a string containing the + # full path if it's a possible namespace portion, None if we + # can't load it. + def find_loader(self, fullname, path=None): + """find_loader(fullname, path=None) -> self, str or None. + + Search for a module specified by 'fullname'. 'fullname' must be the + fully qualified (dotted) module name. It returns the zipimporter + instance itself if the module was found, a string containing the + full path name if it's possibly a portion of a namespace package, + or None otherwise. The optional 'path' argument is ignored -- it's + there for compatibility with the importer protocol. + """ + mi = _get_module_info(self, fullname) + if mi is not None: + # This is a module or package. + return self, [] + + # Not a module or regular package. See if this is a directory, and + # therefore possibly a portion of a namespace package. + + # We're only interested in the last path component of fullname + # earlier components are recorded in self.prefix. + modpath = _get_module_path(self, fullname) + if _is_dir(self, modpath): + # This is possibly a portion of a namespace + # package. Return the string representing its path, + # without a trailing separator. + return None, [f'{self.archive}{path_sep}{modpath}'] + + return None, [] + + + # Check whether we can satisfy the import of the module named by + # 'fullname'. Return self if we can, None if we can't. + def find_module(self, fullname, path=None): + """find_module(fullname, path=None) -> self or None. + + Search for a module specified by 'fullname'. 'fullname' must be the + fully qualified (dotted) module name. It returns the zipimporter + instance itself if the module was found, or None if it wasn't. + The optional 'path' argument is ignored -- it's there for compatibility + with the importer protocol. + """ + return self.find_loader(fullname, path)[0] + + + def get_code(self, fullname): + """get_code(fullname) -> code object. + + Return the code object for the specified module. Raise ZipImportError + if the module couldn't be found. + """ + code, ispackage, modpath = _get_module_code(self, fullname) + return code + + + def get_data(self, pathname): + """get_data(pathname) -> string with file data. + + Return the data associated with 'pathname'. Raise OSError if + the file wasn't found. + """ + if alt_path_sep: + pathname = pathname.replace(alt_path_sep, path_sep) + + key = pathname + if pathname.startswith(self.archive + path_sep): + key = pathname[len(self.archive + path_sep):] + + try: + toc_entry = self._files[key] + except KeyError: + raise OSError(0, '', key) + return _get_data(self.archive, toc_entry) + + + # Return a string matching __file__ for the named module + def get_filename(self, fullname): + """get_filename(fullname) -> filename string. + + Return the filename for the specified module. + """ + # Deciding the filename requires working out where the code + # would come from if the module was actually loaded + code, ispackage, modpath = _get_module_code(self, fullname) + return modpath + + + def get_source(self, fullname): + """get_source(fullname) -> source string. + + Return the source code for the specified module. Raise ZipImportError + if the module couldn't be found, return None if the archive does + contain the module, but has no source for it. + """ + mi = _get_module_info(self, fullname) + if mi is None: + raise ZipImportError(f"can't find module {fullname!r}", name=fullname) + + path = _get_module_path(self, fullname) + if mi: + fullpath = _bootstrap_external._path_join(path, '__init__.py') + else: + fullpath = f'{path}.py' + + try: + toc_entry = self._files[fullpath] + except KeyError: + # we have the module, but no source + return None + return _get_data(self.archive, toc_entry).decode() + + + # Return a bool signifying whether the module is a package or not. + def is_package(self, fullname): + """is_package(fullname) -> bool. + + Return True if the module specified by fullname is a package. + Raise ZipImportError if the module couldn't be found. + """ + mi = _get_module_info(self, fullname) + if mi is None: + raise ZipImportError(f"can't find module {fullname!r}", name=fullname) + return mi + + + # Load and return the module named by 'fullname'. + def load_module(self, fullname): + """load_module(fullname) -> module. + + Load the module specified by 'fullname'. 'fullname' must be the + fully qualified (dotted) module name. It returns the imported + module, or raises ZipImportError if it wasn't found. + """ + code, ispackage, modpath = _get_module_code(self, fullname) + mod = sys.modules.get(fullname) + if mod is None or not isinstance(mod, _module_type): + mod = _module_type(fullname) + sys.modules[fullname] = mod + mod.__loader__ = self + + try: + if ispackage: + # add __path__ to the module *before* the code gets + # executed + path = _get_module_path(self, fullname) + fullpath = _bootstrap_external._path_join(self.archive, path) + mod.__path__ = [fullpath] + + if not hasattr(mod, '__builtins__'): + mod.__builtins__ = __builtins__ + _bootstrap_external._fix_up_module(mod.__dict__, fullname, modpath) + exec(code, mod.__dict__) + except: + del sys.modules[fullname] + raise + + try: + mod = sys.modules[fullname] + except KeyError: + raise ImportError(f'Loaded module {fullname!r} not found in sys.modules') + _bootstrap._verbose_message('import {} # loaded from Zip {}', fullname, modpath) + return mod + + + def get_resource_reader(self, fullname): + """Return the ResourceReader for a package in a zip file. + + If 'fullname' is a package within the zip file, return the + 'ResourceReader' object for the package. Otherwise return None. + """ + from importlib import resources + return resources._zipimport_get_resource_reader(self, fullname) + + + def __repr__(self): + return f'' + + +# _zip_searchorder defines how we search for a module in the Zip +# archive: we first search for a package __init__, then for +# non-package .pyc, and .py entries. The .pyc entries +# are swapped by initzipimport() if we run in optimized mode. Also, +# '/' is replaced by path_sep there. +_zip_searchorder = ( + (path_sep + '__init__.pyc', True, True), + (path_sep + '__init__.py', False, True), + ('.pyc', True, False), + ('.py', False, False), +) + +# Given a module name, return the potential file path in the +# archive (without extension). +def _get_module_path(self, fullname): + return self.prefix + fullname.rpartition('.')[2] + +# Does this path represent a directory? +def _is_dir(self, path): + # See if this is a "directory". If so, it's eligible to be part + # of a namespace package. We test by seeing if the name, with an + # appended path separator, exists. + dirpath = path + path_sep + # If dirpath is present in self._files, we have a directory. + return dirpath in self._files + +# Return some information about a module. +def _get_module_info(self, fullname): + path = _get_module_path(self, fullname) + for suffix, isbytecode, ispackage in _zip_searchorder: + fullpath = path + suffix + if fullpath in self._files: + return ispackage + return None + + +# implementation + +# _read_directory(archive) -> files dict (new reference) +# +# Given a path to a Zip archive, build a dict, mapping file names +# (local to the archive, using SEP as a separator) to toc entries. +# +# A toc_entry is a tuple: +# +# (__file__, # value to use for __file__, available for all files, +# # encoded to the filesystem encoding +# compress, # compression kind; 0 for uncompressed +# data_size, # size of compressed data on disk +# file_size, # size of decompressed data +# file_offset, # offset of file header from start of archive +# time, # mod time of file (in dos format) +# date, # mod data of file (in dos format) +# crc, # crc checksum of the data +# ) +# +# Directories can be recognized by the trailing path_sep in the name, +# data_size and file_offset are 0. +def _read_directory(archive): + try: + fp = _io.open(archive, 'rb') + except OSError: + raise ZipImportError(f"can't open Zip file: {archive!r}", path=archive) + + with fp: + try: + fp.seek(-22, 2) + header_position = fp.tell() + buffer = fp.read(22) + except OSError: + raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) + if len(buffer) != 22: + raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) + if buffer[:4] != b'PK\x05\x06': + # Bad: End of Central Dir signature + raise ZipImportError(f'not a Zip file: {archive!r}', path=archive) + + header_size = _unpack_uint32(buffer[12:16]) + header_offset = _unpack_uint32(buffer[16:20]) + if header_position < header_size: + raise ZipImportError(f'bad central directory size: {archive!r}', path=archive) + if header_position < header_offset: + raise ZipImportError(f'bad central directory offset: {archive!r}', path=archive) + header_position -= header_size + arc_offset = header_position - header_offset + if arc_offset < 0: + raise ZipImportError(f'bad central directory size or offset: {archive!r}', path=archive) + + files = {} + # Start of Central Directory + count = 0 + try: + fp.seek(header_position) + except OSError: + raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) + while True: + buffer = fp.read(46) + if len(buffer) < 4: + raise EOFError('EOF read where not expected') + # Start of file header + if buffer[:4] != b'PK\x01\x02': + break # Bad: Central Dir File Header + if len(buffer) != 46: + raise EOFError('EOF read where not expected') + flags = _unpack_uint16(buffer[8:10]) + compress = _unpack_uint16(buffer[10:12]) + time = _unpack_uint16(buffer[12:14]) + date = _unpack_uint16(buffer[14:16]) + crc = _unpack_uint32(buffer[16:20]) + data_size = _unpack_uint32(buffer[20:24]) + file_size = _unpack_uint32(buffer[24:28]) + name_size = _unpack_uint16(buffer[28:30]) + extra_size = _unpack_uint16(buffer[30:32]) + comment_size = _unpack_uint16(buffer[32:34]) + file_offset = _unpack_uint32(buffer[42:46]) + header_size = name_size + extra_size + comment_size + if file_offset > header_offset: + raise ZipImportError(f'bad local header offset: {archive!r}', path=archive) + file_offset += arc_offset + + try: + name = fp.read(name_size) + except OSError: + raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) + if len(name) != name_size: + raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) + # On Windows, calling fseek to skip over the fields we don't use is + # slower than reading the data because fseek flushes stdio's + # internal buffers. See issue #8745. + try: + if len(fp.read(header_size - name_size)) != header_size - name_size: + raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) + except OSError: + raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) + + if flags & 0x800: + # UTF-8 file names extension + name = name.decode() + else: + # Historical ZIP filename encoding + try: + name = name.decode('ascii') + except UnicodeDecodeError: + name = name.decode('latin1').translate(cp437_table) + + name = name.replace('/', path_sep) + path = _bootstrap_external._path_join(archive, name) + t = (path, compress, data_size, file_size, file_offset, time, date, crc) + files[name] = t + count += 1 + _bootstrap._verbose_message('zipimport: found {} names in {!r}', count, archive) + return files + +# During bootstrap, we may need to load the encodings +# package from a ZIP file. But the cp437 encoding is implemented +# in Python in the encodings package. +# +# Break out of this dependency by using the translation table for +# the cp437 encoding. +cp437_table = ( + # ASCII part, 8 rows x 16 chars + '\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' + '\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f' + ' !"#$%&\'()*+,-./' + '0123456789:;<=>?' + '@ABCDEFGHIJKLMNO' + 'PQRSTUVWXYZ[\\]^_' + '`abcdefghijklmno' + 'pqrstuvwxyz{|}~\x7f' + # non-ASCII part, 16 rows x 8 chars + '\xc7\xfc\xe9\xe2\xe4\xe0\xe5\xe7' + '\xea\xeb\xe8\xef\xee\xec\xc4\xc5' + '\xc9\xe6\xc6\xf4\xf6\xf2\xfb\xf9' + '\xff\xd6\xdc\xa2\xa3\xa5\u20a7\u0192' + '\xe1\xed\xf3\xfa\xf1\xd1\xaa\xba' + '\xbf\u2310\xac\xbd\xbc\xa1\xab\xbb' + '\u2591\u2592\u2593\u2502\u2524\u2561\u2562\u2556' + '\u2555\u2563\u2551\u2557\u255d\u255c\u255b\u2510' + '\u2514\u2534\u252c\u251c\u2500\u253c\u255e\u255f' + '\u255a\u2554\u2569\u2566\u2560\u2550\u256c\u2567' + '\u2568\u2564\u2565\u2559\u2558\u2552\u2553\u256b' + '\u256a\u2518\u250c\u2588\u2584\u258c\u2590\u2580' + '\u03b1\xdf\u0393\u03c0\u03a3\u03c3\xb5\u03c4' + '\u03a6\u0398\u03a9\u03b4\u221e\u03c6\u03b5\u2229' + '\u2261\xb1\u2265\u2264\u2320\u2321\xf7\u2248' + '\xb0\u2219\xb7\u221a\u207f\xb2\u25a0\xa0' +) + +_importing_zlib = False + +# Return the zlib.decompress function object, or NULL if zlib couldn't +# be imported. The function is cached when found, so subsequent calls +# don't import zlib again. +def _get_decompress_func(): + global _importing_zlib + if _importing_zlib: + # Someone has a zlib.py[co] in their Zip file + # let's avoid a stack overflow. + _bootstrap._verbose_message('zipimport: zlib UNAVAILABLE') + raise ZipImportError("can't decompress data; zlib not available") + + _importing_zlib = True + try: + from zlib import decompress + except Exception: + _bootstrap._verbose_message('zipimport: zlib UNAVAILABLE') + raise ZipImportError("can't decompress data; zlib not available") + finally: + _importing_zlib = False + + _bootstrap._verbose_message('zipimport: zlib available') + return decompress + +# Given a path to a Zip file and a toc_entry, return the (uncompressed) data. +def _get_data(archive, toc_entry): + datapath, compress, data_size, file_size, file_offset, time, date, crc = toc_entry + if data_size < 0: + raise ZipImportError('negative data size') + + with _io.open(archive, 'rb') as fp: + # Check to make sure the local file header is correct + try: + fp.seek(file_offset) + except OSError: + raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) + buffer = fp.read(30) + if len(buffer) != 30: + raise EOFError('EOF read where not expected') + + if buffer[:4] != b'PK\x03\x04': + # Bad: Local File Header + raise ZipImportError(f'bad local file header: {archive!r}', path=archive) + + name_size = _unpack_uint16(buffer[26:28]) + extra_size = _unpack_uint16(buffer[28:30]) + header_size = 30 + name_size + extra_size + file_offset += header_size # Start of file data + try: + fp.seek(file_offset) + except OSError: + raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) + raw_data = fp.read(data_size) + if len(raw_data) != data_size: + raise OSError("zipimport: can't read data") + + if compress == 0: + # data is not compressed + return raw_data + + # Decompress with zlib + try: + decompress = _get_decompress_func() + except Exception: + raise ZipImportError("can't decompress data; zlib not available") + return decompress(raw_data, -15) + + +# Lenient date/time comparison function. The precision of the mtime +# in the archive is lower than the mtime stored in a .pyc: we +# must allow a difference of at most one second. +def _eq_mtime(t1, t2): + # dostime only stores even seconds, so be lenient + return abs(t1 - t2) <= 1 + +# Given the contents of a .py[co] file, unmarshal the data +# and return the code object. Return None if it the magic word doesn't +# match (we do this instead of raising an exception as we fall back +# to .py if available and we don't want to mask other errors). +def _unmarshal_code(pathname, data, mtime): + if len(data) < 16: + raise ZipImportError('bad pyc data') + + if data[:4] != _bootstrap_external.MAGIC_NUMBER: + _bootstrap._verbose_message('{!r} has bad magic', pathname) + return None # signal caller to try alternative + + flags = _unpack_uint32(data[4:8]) + if flags != 0: + # Hash-based pyc. We currently refuse to handle checked hash-based + # pycs. We could validate hash-based pycs against the source, but it + # seems likely that most people putting hash-based pycs in a zipfile + # will use unchecked ones. + if (_imp.check_hash_based_pycs != 'never' and + (flags != 0x1 or _imp.check_hash_based_pycs == 'always')): + return None + elif mtime != 0 and not _eq_mtime(_unpack_uint32(data[8:12]), mtime): + _bootstrap._verbose_message('{!r} has bad mtime', pathname) + return None # signal caller to try alternative + + # XXX the pyc's size field is ignored; timestamp collisions are probably + # unimportant with zip files. + code = marshal.loads(data[16:]) + if not isinstance(code, _code_type): + raise TypeError(f'compiled module {pathname!r} is not a code object') + return code + +_code_type = type(_unmarshal_code.__code__) + + +# Replace any occurrences of '\r\n?' in the input string with '\n'. +# This converts DOS and Mac line endings to Unix line endings. +def _normalize_line_endings(source): + source = source.replace(b'\r\n', b'\n') + source = source.replace(b'\r', b'\n') + return source + +# Given a string buffer containing Python source code, compile it +# and return a code object. +def _compile_source(pathname, source): + source = _normalize_line_endings(source) + return compile(source, pathname, 'exec', dont_inherit=True) + +# Convert the date/time values found in the Zip archive to a value +# that's compatible with the time stamp stored in .pyc files. +def _parse_dostime(d, t): + return time.mktime(( + (d >> 9) + 1980, # bits 9..15: year + (d >> 5) & 0xF, # bits 5..8: month + d & 0x1F, # bits 0..4: day + t >> 11, # bits 11..15: hours + (t >> 5) & 0x3F, # bits 8..10: minutes + (t & 0x1F) * 2, # bits 0..7: seconds / 2 + -1, -1, -1)) + +# Given a path to a .pyc file in the archive, return the +# modification time of the matching .py file, or 0 if no source +# is available. +def _get_mtime_of_source(self, path): + try: + # strip 'c' or 'o' from *.py[co] + assert path[-1:] in ('c', 'o') + path = path[:-1] + toc_entry = self._files[path] + # fetch the time stamp of the .py file for comparison + # with an embedded pyc time stamp + time = toc_entry[5] + date = toc_entry[6] + return _parse_dostime(date, time) + except (KeyError, IndexError, TypeError): + return 0 + +# Get the code object associated with the module specified by +# 'fullname'. +def _get_module_code(self, fullname): + path = _get_module_path(self, fullname) + for suffix, isbytecode, ispackage in _zip_searchorder: + fullpath = path + suffix + _bootstrap._verbose_message('trying {}{}{}', self.archive, path_sep, fullpath, verbosity=2) + try: + toc_entry = self._files[fullpath] + except KeyError: + pass + else: + modpath = toc_entry[0] + data = _get_data(self.archive, toc_entry) + if isbytecode: + mtime = _get_mtime_of_source(self, fullpath) + code = _unmarshal_code(modpath, data, mtime) + else: + code = _compile_source(modpath, data) + if code is None: + # bad magic number or non-matching mtime + # in byte code, try next + continue + modpath = toc_entry[0] + return code, ispackage, modpath + else: + raise ZipImportError(f"can't find module {fullname!r}", name=fullname) diff --git a/Makefile.pre.in b/Makefile.pre.in index d60d48e5f9bb..d2e7377e98d5 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -713,16 +713,22 @@ Programs/_freeze_importlib: Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FRO regen-importlib: Programs/_freeze_importlib # Regenerate Python/importlib_external.h # from Lib/importlib/_bootstrap_external.py using _freeze_importlib - ./Programs/_freeze_importlib \ + ./Programs/_freeze_importlib importlib._bootstrap_external \ $(srcdir)/Lib/importlib/_bootstrap_external.py \ $(srcdir)/Python/importlib_external.h.new $(UPDATE_FILE) $(srcdir)/Python/importlib_external.h $(srcdir)/Python/importlib_external.h.new # Regenerate Python/importlib.h from Lib/importlib/_bootstrap.py # using _freeze_importlib - ./Programs/_freeze_importlib \ + ./Programs/_freeze_importlib importlib._bootstrap \ $(srcdir)/Lib/importlib/_bootstrap.py \ $(srcdir)/Python/importlib.h.new $(UPDATE_FILE) $(srcdir)/Python/importlib.h $(srcdir)/Python/importlib.h.new + # Regenerate Python/importlib_zipimport.h from Lib/zipimport.py + # using _freeze_importlib + ./Programs/_freeze_importlib zipimport \ + $(srcdir)/Lib/zipimport.py \ + $(srcdir)/Python/importlib_zipimport.h.new + $(UPDATE_FILE) $(srcdir)/Python/importlib_zipimport.h $(srcdir)/Python/importlib_zipimport.h.new ############################################################################ @@ -897,7 +903,8 @@ regen-opcode-targets: Python/ceval.o: $(srcdir)/Python/opcode_targets.h $(srcdir)/Python/ceval_gil.h -Python/frozen.o: $(srcdir)/Python/importlib.h $(srcdir)/Python/importlib_external.h +Python/frozen.o: $(srcdir)/Python/importlib.h $(srcdir)/Python/importlib_external.h \ + $(srcdir)/Python/importlib_zipimport.h # Generate DTrace probe macros, then rename them (PYTHON_ -> PyDTrace_) to # follow our naming conventions. dtrace(1) uses the output filename to generate diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-05-14-18-54-03.bpo-25711.9xfq-v.rst b/Misc/NEWS.d/next/Core and Builtins/2018-05-14-18-54-03.bpo-25711.9xfq-v.rst new file mode 100644 index 000000000000..07f9e722baf5 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-05-14-18-54-03.bpo-25711.9xfq-v.rst @@ -0,0 +1 @@ +The :mod:`zipimport` module has been rewritten in pure Python. diff --git a/Modules/Setup b/Modules/Setup index 9787ea2d2c48..a0622cc8c647 100644 --- a/Modules/Setup +++ b/Modules/Setup @@ -125,10 +125,6 @@ _locale _localemodule.c # -lintl # Standard I/O baseline _io -DPy_BUILD_CORE -I$(srcdir)/Modules/_io _io/_iomodule.c _io/iobase.c _io/fileio.c _io/bytesio.c _io/bufferedio.c _io/textio.c _io/stringio.c -# The zipimport module is always imported at startup. Having it as a -# builtin module avoids some bootstrapping problems and reduces overhead. -zipimport -DPy_BUILD_CORE zipimport.c - # faulthandler module faulthandler faulthandler.c diff --git a/Modules/clinic/zipimport.c.h b/Modules/clinic/zipimport.c.h deleted file mode 100644 index 565b06530360..000000000000 --- a/Modules/clinic/zipimport.c.h +++ /dev/null @@ -1,325 +0,0 @@ -/*[clinic input] -preserve -[clinic start generated code]*/ - -PyDoc_STRVAR(zipimport_zipimporter___init____doc__, -"zipimporter(archivepath, /)\n" -"--\n" -"\n" -"Create a new zipimporter instance.\n" -"\n" -" archivepath\n" -" A path-like object to a zipfile, or to a specific path inside\n" -" a zipfile.\n" -"\n" -"\'archivepath\' must be a path-like object to a zipfile, or to a specific path\n" -"inside a zipfile. For example, it can be \'/tmp/myimport.zip\', or\n" -"\'/tmp/myimport.zip/mydirectory\', if mydirectory is a valid directory inside\n" -"the archive.\n" -"\n" -"\'ZipImportError\' is raised if \'archivepath\' doesn\'t point to a valid Zip\n" -"archive.\n" -"\n" -"The \'archive\' attribute of the zipimporter object contains the name of the\n" -"zipfile targeted."); - -static int -zipimport_zipimporter___init___impl(ZipImporter *self, PyObject *path); - -static int -zipimport_zipimporter___init__(PyObject *self, PyObject *args, PyObject *kwargs) -{ - int return_value = -1; - PyObject *path; - - if ((Py_TYPE(self) == &ZipImporter_Type) && - !_PyArg_NoKeywords("zipimporter", kwargs)) { - goto exit; - } - if (!PyArg_ParseTuple(args, "O&:zipimporter", - PyUnicode_FSDecoder, &path)) { - goto exit; - } - return_value = zipimport_zipimporter___init___impl((ZipImporter *)self, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_find_module__doc__, -"find_module($self, fullname, path=None, /)\n" -"--\n" -"\n" -"Search for a module specified by \'fullname\'.\n" -"\n" -"\'fullname\' must be the fully qualified (dotted) module name. It returns the\n" -"zipimporter instance itself if the module was found, or None if it wasn\'t.\n" -"The optional \'path\' argument is ignored -- it\'s there for compatibility\n" -"with the importer protocol."); - -#define ZIPIMPORT_ZIPIMPORTER_FIND_MODULE_METHODDEF \ - {"find_module", (PyCFunction)zipimport_zipimporter_find_module, METH_FASTCALL, zipimport_zipimporter_find_module__doc__}, - -static PyObject * -zipimport_zipimporter_find_module_impl(ZipImporter *self, PyObject *fullname, - PyObject *path); - -static PyObject * -zipimport_zipimporter_find_module(ZipImporter *self, PyObject *const *args, Py_ssize_t nargs) -{ - PyObject *return_value = NULL; - PyObject *fullname; - PyObject *path = Py_None; - - if (!_PyArg_ParseStack(args, nargs, "U|O:find_module", - &fullname, &path)) { - goto exit; - } - return_value = zipimport_zipimporter_find_module_impl(self, fullname, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_find_loader__doc__, -"find_loader($self, fullname, path=None, /)\n" -"--\n" -"\n" -"Search for a module specified by \'fullname\'.\n" -"\n" -"\'fullname\' must be the fully qualified (dotted) module name. It returns the\n" -"zipimporter instance itself if the module was found, a string containing the\n" -"full path name if it\'s possibly a portion of a namespace package,\n" -"or None otherwise. The optional \'path\' argument is ignored -- it\'s\n" -"there for compatibility with the importer protocol."); - -#define ZIPIMPORT_ZIPIMPORTER_FIND_LOADER_METHODDEF \ - {"find_loader", (PyCFunction)zipimport_zipimporter_find_loader, METH_FASTCALL, zipimport_zipimporter_find_loader__doc__}, - -static PyObject * -zipimport_zipimporter_find_loader_impl(ZipImporter *self, PyObject *fullname, - PyObject *path); - -static PyObject * -zipimport_zipimporter_find_loader(ZipImporter *self, PyObject *const *args, Py_ssize_t nargs) -{ - PyObject *return_value = NULL; - PyObject *fullname; - PyObject *path = Py_None; - - if (!_PyArg_ParseStack(args, nargs, "U|O:find_loader", - &fullname, &path)) { - goto exit; - } - return_value = zipimport_zipimporter_find_loader_impl(self, fullname, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_load_module__doc__, -"load_module($self, fullname, /)\n" -"--\n" -"\n" -"Load the module specified by \'fullname\'.\n" -"\n" -"\'fullname\' must be the fully qualified (dotted) module name. It returns the\n" -"imported module, or raises ZipImportError if it wasn\'t found."); - -#define ZIPIMPORT_ZIPIMPORTER_LOAD_MODULE_METHODDEF \ - {"load_module", (PyCFunction)zipimport_zipimporter_load_module, METH_O, zipimport_zipimporter_load_module__doc__}, - -static PyObject * -zipimport_zipimporter_load_module_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_load_module(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:load_module", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_load_module_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_filename__doc__, -"get_filename($self, fullname, /)\n" -"--\n" -"\n" -"Return the filename for the specified module."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_FILENAME_METHODDEF \ - {"get_filename", (PyCFunction)zipimport_zipimporter_get_filename, METH_O, zipimport_zipimporter_get_filename__doc__}, - -static PyObject * -zipimport_zipimporter_get_filename_impl(ZipImporter *self, - PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_filename(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_filename", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_filename_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_is_package__doc__, -"is_package($self, fullname, /)\n" -"--\n" -"\n" -"Return True if the module specified by fullname is a package.\n" -"\n" -"Raise ZipImportError if the module couldn\'t be found."); - -#define ZIPIMPORT_ZIPIMPORTER_IS_PACKAGE_METHODDEF \ - {"is_package", (PyCFunction)zipimport_zipimporter_is_package, METH_O, zipimport_zipimporter_is_package__doc__}, - -static PyObject * -zipimport_zipimporter_is_package_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_is_package(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:is_package", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_is_package_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_data__doc__, -"get_data($self, pathname, /)\n" -"--\n" -"\n" -"Return the data associated with \'pathname\'.\n" -"\n" -"Raise OSError if the file was not found."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_DATA_METHODDEF \ - {"get_data", (PyCFunction)zipimport_zipimporter_get_data, METH_O, zipimport_zipimporter_get_data__doc__}, - -static PyObject * -zipimport_zipimporter_get_data_impl(ZipImporter *self, PyObject *path); - -static PyObject * -zipimport_zipimporter_get_data(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *path; - - if (!PyArg_Parse(arg, "U:get_data", &path)) { - goto exit; - } - return_value = zipimport_zipimporter_get_data_impl(self, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_code__doc__, -"get_code($self, fullname, /)\n" -"--\n" -"\n" -"Return the code object for the specified module.\n" -"\n" -"Raise ZipImportError if the module couldn\'t be found."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_CODE_METHODDEF \ - {"get_code", (PyCFunction)zipimport_zipimporter_get_code, METH_O, zipimport_zipimporter_get_code__doc__}, - -static PyObject * -zipimport_zipimporter_get_code_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_code(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_code", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_code_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_source__doc__, -"get_source($self, fullname, /)\n" -"--\n" -"\n" -"Return the source code for the specified module.\n" -"\n" -"Raise ZipImportError if the module couldn\'t be found, return None if the\n" -"archive does contain the module, but has no source for it."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_SOURCE_METHODDEF \ - {"get_source", (PyCFunction)zipimport_zipimporter_get_source, METH_O, zipimport_zipimporter_get_source__doc__}, - -static PyObject * -zipimport_zipimporter_get_source_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_source(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_source", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_source_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_resource_reader__doc__, -"get_resource_reader($self, fullname, /)\n" -"--\n" -"\n" -"Return the ResourceReader for a package in a zip file.\n" -"\n" -"If \'fullname\' is a package within the zip file, return the \'ResourceReader\'\n" -"object for the package. Otherwise return None."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_RESOURCE_READER_METHODDEF \ - {"get_resource_reader", (PyCFunction)zipimport_zipimporter_get_resource_reader, METH_O, zipimport_zipimporter_get_resource_reader__doc__}, - -static PyObject * -zipimport_zipimporter_get_resource_reader_impl(ZipImporter *self, - PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_resource_reader(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_resource_reader", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_resource_reader_impl(self, fullname); - -exit: - return return_value; -} -/*[clinic end generated code: output=0b57adfe21373512 input=a9049054013a1b77]*/ diff --git a/Modules/zipimport.c b/Modules/zipimport.c deleted file mode 100644 index 4df4ba9f8b82..000000000000 --- a/Modules/zipimport.c +++ /dev/null @@ -1,1664 +0,0 @@ -#include "Python.h" -#include "internal/pystate.h" -#include "structmember.h" -#include "osdefs.h" -#include "marshal.h" -#include - - -#define IS_SOURCE 0x0 -#define IS_BYTECODE 0x1 -#define IS_PACKAGE 0x2 - -struct st_zip_searchorder { - char suffix[14]; - int type; -}; - -#ifdef ALTSEP -_Py_IDENTIFIER(replace); -#endif - -/* zip_searchorder defines how we search for a module in the Zip - archive: we first search for a package __init__, then for - non-package .pyc, and .py entries. The .pyc entries - are swapped by initzipimport() if we run in optimized mode. Also, - '/' is replaced by SEP there. */ -static struct st_zip_searchorder zip_searchorder[] = { - {"/__init__.pyc", IS_PACKAGE | IS_BYTECODE}, - {"/__init__.py", IS_PACKAGE | IS_SOURCE}, - {".pyc", IS_BYTECODE}, - {".py", IS_SOURCE}, - {"", 0} -}; - -/* zipimporter object definition and support */ - -typedef struct _zipimporter ZipImporter; - -struct _zipimporter { - PyObject_HEAD - PyObject *archive; /* pathname of the Zip archive, - decoded from the filesystem encoding */ - PyObject *prefix; /* file prefix: "a/sub/directory/", - encoded to the filesystem encoding */ - PyObject *files; /* dict with file info {path: toc_entry} */ -}; - -static PyObject *ZipImportError; -/* read_directory() cache */ -static PyObject *zip_directory_cache = NULL; - -/* forward decls */ -static PyObject *read_directory(PyObject *archive); -static PyObject *get_data(PyObject *archive, PyObject *toc_entry); -static PyObject *get_module_code(ZipImporter *self, PyObject *fullname, - int *p_ispackage, PyObject **p_modpath); - -static PyTypeObject ZipImporter_Type; - -#define ZipImporter_Check(op) PyObject_TypeCheck(op, &ZipImporter_Type) - -/*[clinic input] -module zipimport -class zipimport.zipimporter "ZipImporter *" "&ZipImporter_Type" -[clinic start generated code]*/ -/*[clinic end generated code: output=da39a3ee5e6b4b0d input=9db8b61557d911e7]*/ -#include "clinic/zipimport.c.h" - - -/* zipimporter.__init__ - Split the "subdirectory" from the Zip archive path, lookup a matching - entry in sys.path_importer_cache, fetch the file directory from there - if found, or else read it from the archive. */ - -/*[clinic input] -zipimport.zipimporter.__init__ - - archivepath as path: object(converter="PyUnicode_FSDecoder") - A path-like object to a zipfile, or to a specific path inside - a zipfile. - / - -Create a new zipimporter instance. - -'archivepath' must be a path-like object to a zipfile, or to a specific path -inside a zipfile. For example, it can be '/tmp/myimport.zip', or -'/tmp/myimport.zip/mydirectory', if mydirectory is a valid directory inside -the archive. - -'ZipImportError' is raised if 'archivepath' doesn't point to a valid Zip -archive. - -The 'archive' attribute of the zipimporter object contains the name of the -zipfile targeted. - -[clinic start generated code]*/ - -static int -zipimport_zipimporter___init___impl(ZipImporter *self, PyObject *path) -/*[clinic end generated code: output=141558fefdb46dc8 input=92b9ebeed1f6a704]*/ -{ - PyObject *files, *tmp; - PyObject *filename = NULL; - Py_ssize_t len, flen; - - if (PyUnicode_READY(path) == -1) - return -1; - - len = PyUnicode_GET_LENGTH(path); - if (len == 0) { - PyErr_SetString(ZipImportError, "archive path is empty"); - goto error; - } - -#ifdef ALTSEP - tmp = _PyObject_CallMethodId(path, &PyId_replace, "CC", ALTSEP, SEP); - if (!tmp) - goto error; - Py_DECREF(path); - path = tmp; -#endif - - filename = path; - Py_INCREF(filename); - flen = len; - for (;;) { - struct stat statbuf; - int rv; - - rv = _Py_stat(filename, &statbuf); - if (rv == -2) - goto error; - if (rv == 0) { - /* it exists */ - if (!S_ISREG(statbuf.st_mode)) - /* it's a not file */ - Py_CLEAR(filename); - break; - } - Py_CLEAR(filename); - /* back up one path element */ - flen = PyUnicode_FindChar(path, SEP, 0, flen, -1); - if (flen == -1) - break; - filename = PyUnicode_Substring(path, 0, flen); - if (filename == NULL) - goto error; - } - if (filename == NULL) { - PyErr_SetString(ZipImportError, "not a Zip file"); - goto error; - } - - if (PyUnicode_READY(filename) < 0) - goto error; - - files = PyDict_GetItem(zip_directory_cache, filename); - if (files == NULL) { - files = read_directory(filename); - if (files == NULL) - goto error; - if (PyDict_SetItem(zip_directory_cache, filename, files) != 0) - goto error; - } - else - Py_INCREF(files); - Py_XSETREF(self->files, files); - - /* Transfer reference */ - Py_XSETREF(self->archive, filename); - filename = NULL; - - /* Check if there is a prefix directory following the filename. */ - if (flen != len) { - tmp = PyUnicode_Substring(path, flen+1, - PyUnicode_GET_LENGTH(path)); - if (tmp == NULL) - goto error; - Py_XSETREF(self->prefix, tmp); - if (PyUnicode_READ_CHAR(path, len-1) != SEP) { - /* add trailing SEP */ - tmp = PyUnicode_FromFormat("%U%c", self->prefix, SEP); - if (tmp == NULL) - goto error; - Py_SETREF(self->prefix, tmp); - } - } - else { - Py_XSETREF(self->prefix, PyUnicode_New(0, 0)); - } - Py_DECREF(path); - return 0; - -error: - Py_DECREF(path); - Py_XDECREF(filename); - return -1; -} - -/* GC support. */ -static int -zipimporter_traverse(PyObject *obj, visitproc visit, void *arg) -{ - ZipImporter *self = (ZipImporter *)obj; - Py_VISIT(self->files); - return 0; -} - -static void -zipimporter_dealloc(ZipImporter *self) -{ - PyObject_GC_UnTrack(self); - Py_XDECREF(self->archive); - Py_XDECREF(self->prefix); - Py_XDECREF(self->files); - Py_TYPE(self)->tp_free((PyObject *)self); -} - -static PyObject * -zipimporter_repr(ZipImporter *self) -{ - if (self->archive == NULL) - return PyUnicode_FromString(""); - else if (self->prefix != NULL && PyUnicode_GET_LENGTH(self->prefix) != 0) - return PyUnicode_FromFormat("", - self->archive, SEP, self->prefix); - else - return PyUnicode_FromFormat("", - self->archive); -} - -/* return fullname.split(".")[-1] */ -static PyObject * -get_subname(PyObject *fullname) -{ - Py_ssize_t len, dot; - if (PyUnicode_READY(fullname) < 0) - return NULL; - len = PyUnicode_GET_LENGTH(fullname); - dot = PyUnicode_FindChar(fullname, '.', 0, len, -1); - if (dot == -1) { - Py_INCREF(fullname); - return fullname; - } else - return PyUnicode_Substring(fullname, dot+1, len); -} - -/* Given a (sub)modulename, write the potential file path in the - archive (without extension) to the path buffer. Return the - length of the resulting string. - - return self.prefix + name.replace('.', os.sep) */ -static PyObject* -make_filename(PyObject *prefix, PyObject *name) -{ - PyObject *pathobj; - Py_UCS4 *p, *buf; - Py_ssize_t len; - - len = PyUnicode_GET_LENGTH(prefix) + PyUnicode_GET_LENGTH(name) + 1; - p = buf = PyMem_New(Py_UCS4, len); - if (buf == NULL) { - PyErr_NoMemory(); - return NULL; - } - - if (!PyUnicode_AsUCS4(prefix, p, len, 0)) { - PyMem_Free(buf); - return NULL; - } - p += PyUnicode_GET_LENGTH(prefix); - len -= PyUnicode_GET_LENGTH(prefix); - if (!PyUnicode_AsUCS4(name, p, len, 1)) { - PyMem_Free(buf); - return NULL; - } - for (; *p; p++) { - if (*p == '.') - *p = SEP; - } - pathobj = PyUnicode_FromKindAndData(PyUnicode_4BYTE_KIND, - buf, p-buf); - PyMem_Free(buf); - return pathobj; -} - -enum zi_module_info { - MI_ERROR, - MI_NOT_FOUND, - MI_MODULE, - MI_PACKAGE -}; - -/* Does this path represent a directory? - on error, return < 0 - if not a dir, return 0 - if a dir, return 1 -*/ -static int -check_is_directory(ZipImporter *self, PyObject* prefix, PyObject *path) -{ - PyObject *dirpath; - int res; - - /* See if this is a "directory". If so, it's eligible to be part - of a namespace package. We test by seeing if the name, with an - appended path separator, exists. */ - dirpath = PyUnicode_FromFormat("%U%U%c", prefix, path, SEP); - if (dirpath == NULL) - return -1; - /* If dirpath is present in self->files, we have a directory. */ - res = PyDict_Contains(self->files, dirpath); - Py_DECREF(dirpath); - return res; -} - -/* Return some information about a module. */ -static enum zi_module_info -get_module_info(ZipImporter *self, PyObject *fullname) -{ - PyObject *subname; - PyObject *path, *fullpath, *item; - struct st_zip_searchorder *zso; - - if (self->prefix == NULL) { - PyErr_SetString(PyExc_ValueError, - "zipimporter.__init__() wasn't called"); - return MI_ERROR; - } - - subname = get_subname(fullname); - if (subname == NULL) - return MI_ERROR; - - path = make_filename(self->prefix, subname); - Py_DECREF(subname); - if (path == NULL) - return MI_ERROR; - - for (zso = zip_searchorder; *zso->suffix; zso++) { - fullpath = PyUnicode_FromFormat("%U%s", path, zso->suffix); - if (fullpath == NULL) { - Py_DECREF(path); - return MI_ERROR; - } - item = PyDict_GetItem(self->files, fullpath); - Py_DECREF(fullpath); - if (item != NULL) { - Py_DECREF(path); - if (zso->type & IS_PACKAGE) - return MI_PACKAGE; - else - return MI_MODULE; - } - } - Py_DECREF(path); - return MI_NOT_FOUND; -} - -typedef enum { - FL_ERROR = -1, /* error */ - FL_NOT_FOUND, /* no loader or namespace portions found */ - FL_MODULE_FOUND, /* module/package found */ - FL_NS_FOUND /* namespace portion found: */ - /* *namespace_portion will point to the name */ -} find_loader_result; - -/* The guts of "find_loader" and "find_module". -*/ -static find_loader_result -find_loader(ZipImporter *self, PyObject *fullname, PyObject **namespace_portion) -{ - enum zi_module_info mi; - - *namespace_portion = NULL; - - mi = get_module_info(self, fullname); - if (mi == MI_ERROR) - return FL_ERROR; - if (mi == MI_NOT_FOUND) { - /* Not a module or regular package. See if this is a directory, and - therefore possibly a portion of a namespace package. */ - find_loader_result result = FL_NOT_FOUND; - PyObject *subname; - int is_dir; - - /* We're only interested in the last path component of fullname; - earlier components are recorded in self->prefix. */ - subname = get_subname(fullname); - if (subname == NULL) { - return FL_ERROR; - } - - is_dir = check_is_directory(self, self->prefix, subname); - if (is_dir < 0) - result = FL_ERROR; - else if (is_dir) { - /* This is possibly a portion of a namespace - package. Return the string representing its path, - without a trailing separator. */ - *namespace_portion = PyUnicode_FromFormat("%U%c%U%U", - self->archive, SEP, - self->prefix, subname); - if (*namespace_portion == NULL) - result = FL_ERROR; - else - result = FL_NS_FOUND; - } - Py_DECREF(subname); - return result; - } - /* This is a module or package. */ - return FL_MODULE_FOUND; -} - -/*[clinic input] -zipimport.zipimporter.find_module - - fullname: unicode - path: object = None - / - -Search for a module specified by 'fullname'. - -'fullname' must be the fully qualified (dotted) module name. It returns the -zipimporter instance itself if the module was found, or None if it wasn't. -The optional 'path' argument is ignored -- it's there for compatibility -with the importer protocol. - -[clinic start generated code]*/ - -static PyObject * -zipimport_zipimporter_find_module_impl(ZipImporter *self, PyObject *fullname, - PyObject *path) -/*[clinic end generated code: output=506087f609466dc7 input=e3528520e075063f]*/ -{ - PyObject *namespace_portion = NULL; - PyObject *result = NULL; - - switch (find_loader(self, fullname, &namespace_portion)) { - case FL_ERROR: - return NULL; - case FL_NS_FOUND: - /* A namespace portion is not allowed via find_module, so return None. */ - Py_DECREF(namespace_portion); - /* FALL THROUGH */ - case FL_NOT_FOUND: - result = Py_None; - break; - case FL_MODULE_FOUND: - result = (PyObject *)self; - break; - default: - PyErr_BadInternalCall(); - return NULL; - } - Py_INCREF(result); - return result; -} - - -/*[clinic input] -zipimport.zipimporter.find_loader - - fullname: unicode - path: object = None - / - -Search for a module specified by 'fullname'. - -'fullname' must be the fully qualified (dotted) module name. It returns the -zipimporter instance itself if the module was found, a string containing the -full path name if it's possibly a portion of a namespace package, -or None otherwise. The optional 'path' argument is ignored -- it's -there for compatibility with the importer protocol. - -[clinic start generated code]*/ - -static PyObject * -zipimport_zipimporter_find_loader_impl(ZipImporter *self, PyObject *fullname, - PyObject *path) -/*[clinic end generated code: output=601599a43bc0f49a input=dc73f275b0d5be23]*/ -{ - PyObject *result = NULL; - PyObject *namespace_portion = NULL; - - switch (find_loader(self, fullname, &namespace_portion)) { - case FL_ERROR: - return NULL; - case FL_NOT_FOUND: /* Not found, return (None, []) */ - result = Py_BuildValue("O[]", Py_None); - break; - case FL_MODULE_FOUND: /* Return (self, []) */ - result = Py_BuildValue("O[]", self); - break; - case FL_NS_FOUND: /* Return (None, [namespace_portion]) */ - result = Py_BuildValue("O[O]", Py_None, namespace_portion); - Py_DECREF(namespace_portion); - return result; - default: - PyErr_BadInternalCall(); - return NULL; - } - return result; -} - -/*[clinic input] -zipimport.zipimporter.load_module - - fullname: unicode - / - -Load the module specified by 'fullname'. - -'fullname' must be the fully qualified (dotted) module name. It returns the -imported module, or raises ZipImportError if it wasn't found. - -[clinic start generated code]*/ - -static PyObject * -zipimport_zipimporter_load_module_impl(ZipImporter *self, PyObject *fullname) -/*[clinic end generated code: output=7303cebf88d47953 input=c236e2e8621f04ef]*/ -{ - PyObject *code = NULL, *mod, *dict; - PyObject *modpath = NULL; - int ispackage; - - if (PyUnicode_READY(fullname) == -1) - return NULL; - - code = get_module_code(self, fullname, &ispackage, &modpath); - if (code == NULL) - goto error; - - mod = PyImport_AddModuleObject(fullname); - if (mod == NULL) - goto error; - dict = PyModule_GetDict(mod); - - /* mod.__loader__ = self */ - if (PyDict_SetItemString(dict, "__loader__", (PyObject *)self) != 0) - goto error; - - if (ispackage) { - /* add __path__ to the module *before* the code gets - executed */ - PyObject *pkgpath, *fullpath, *subname; - int err; - - subname = get_subname(fullname); - if (subname == NULL) - goto error; - - fullpath = PyUnicode_FromFormat("%U%c%U%U", - self->archive, SEP, - self->prefix, subname); - Py_DECREF(subname); - if (fullpath == NULL) - goto error; - - pkgpath = Py_BuildValue("[N]", fullpath); - if (pkgpath == NULL) - goto error; - err = PyDict_SetItemString(dict, "__path__", pkgpath); - Py_DECREF(pkgpath); - if (err != 0) - goto error; - } - mod = PyImport_ExecCodeModuleObject(fullname, code, modpath, NULL); - Py_CLEAR(code); - if (mod == NULL) - goto error; - - if (Py_VerboseFlag) - PySys_FormatStderr("import %U # loaded from Zip %U\n", - fullname, modpath); - Py_DECREF(modpath); - return mod; -error: - Py_XDECREF(code); - Py_XDECREF(modpath); - return NULL; -} - -/*[clinic input] -zipimport.zipimporter.get_filename - - fullname: unicode - / - -Return the filename for the specified module. -[clinic start generated code]*/ - -static PyObject * -zipimport_zipimporter_get_filename_impl(ZipImporter *self, - PyObject *fullname) -/*[clinic end generated code: output=c5b92b58bea86506 input=28d2eb57e4f25c8a]*/ -{ - PyObject *code, *modpath; - int ispackage; - - /* Deciding the filename requires working out where the code - would come from if the module was actually loaded */ - code = get_module_code(self, fullname, &ispackage, &modpath); - if (code == NULL) - return NULL; - Py_DECREF(code); /* Only need the path info */ - - return modpath; -} - -/*[clinic input] -zipimport.zipimporter.is_package - - fullname: unicode - / - -Return True if the module specified by fullname is a package. - -Raise ZipImportError if the module couldn't be found. - -[clinic start generated code]*/ - -static PyObject * -zipimport_zipimporter_is_package_impl(ZipImporter *self, PyObject *fullname) -/*[clinic end generated code: output=c32958c2a5216ae6 input=a7ba752f64345062]*/ -{ - enum zi_module_info mi; - - mi = get_module_info(self, fullname); - if (mi == MI_ERROR) - return NULL; - if (mi == MI_NOT_FOUND) { - PyErr_Format(ZipImportError, "can't find module %R", fullname); - return NULL; - } - return PyBool_FromLong(mi == MI_PACKAGE); -} - - -/*[clinic input] -zipimport.zipimporter.get_data - - pathname as path: unicode - / - -Return the data associated with 'pathname'. - -Raise OSError if the file was not found. - -[clinic start generated code]*/ - -static PyObject * -zipimport_zipimporter_get_data_impl(ZipImporter *self, PyObject *path) -/*[clinic end generated code: output=65dc506aaa268436 input=fa6428b74843c4ae]*/ -{ - PyObject *key; - PyObject *toc_entry; - Py_ssize_t path_start, path_len, len; - - if (self->archive == NULL) { - PyErr_SetString(PyExc_ValueError, - "zipimporter.__init__() wasn't called"); - return NULL; - } - -#ifdef ALTSEP - path = _PyObject_CallMethodId((PyObject *)&PyUnicode_Type, &PyId_replace, - "OCC", path, ALTSEP, SEP); - if (!path) - return NULL; -#else - Py_INCREF(path); -#endif - if (PyUnicode_READY(path) == -1) - goto error; - - path_len = PyUnicode_GET_LENGTH(path); - - len = PyUnicode_GET_LENGTH(self->archive); - path_start = 0; - if (PyUnicode_Tailmatch(path, self->archive, 0, len, -1) - && PyUnicode_READ_CHAR(path, len) == SEP) { - path_start = len + 1; - } - - key = PyUnicode_Substring(path, path_start, path_len); - if (key == NULL) - goto error; - toc_entry = PyDict_GetItem(self->files, key); - if (toc_entry == NULL) { - PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, key); - Py_DECREF(key); - goto error; - } - Py_DECREF(key); - Py_DECREF(path); - return get_data(self->archive, toc_entry); - error: - Py_DECREF(path); - return NULL; -} - -/*[clinic input] -zipimport.zipimporter.get_code - - fullname: unicode - / - -Return the code object for the specified module. - -Raise ZipImportError if the module couldn't be found. - -[clinic start generated code]*/ - -static PyObject * -zipimport_zipimporter_get_code_impl(ZipImporter *self, PyObject *fullname) -/*[clinic end generated code: output=b923c37fa99cbac4 input=2761412bc37f3549]*/ -{ - return get_module_code(self, fullname, NULL, NULL); -} - -/*[clinic input] -zipimport.zipimporter.get_source - - fullname: unicode - / - -Return the source code for the specified module. - -Raise ZipImportError if the module couldn't be found, return None if the -archive does contain the module, but has no source for it. - -[clinic start generated code]*/ - -static PyObject * -zipimport_zipimporter_get_source_impl(ZipImporter *self, PyObject *fullname) -/*[clinic end generated code: output=bc059301b0c33729 input=4e4b186f2e690716]*/ -{ - PyObject *toc_entry; - PyObject *subname, *path, *fullpath; - enum zi_module_info mi; - - mi = get_module_info(self, fullname); - if (mi == MI_ERROR) - return NULL; - if (mi == MI_NOT_FOUND) { - PyErr_Format(ZipImportError, "can't find module %R", fullname); - return NULL; - } - - subname = get_subname(fullname); - if (subname == NULL) - return NULL; - - path = make_filename(self->prefix, subname); - Py_DECREF(subname); - if (path == NULL) - return NULL; - - if (mi == MI_PACKAGE) - fullpath = PyUnicode_FromFormat("%U%c__init__.py", path, SEP); - else - fullpath = PyUnicode_FromFormat("%U.py", path); - Py_DECREF(path); - if (fullpath == NULL) - return NULL; - - toc_entry = PyDict_GetItem(self->files, fullpath); - Py_DECREF(fullpath); - if (toc_entry != NULL) { - PyObject *res, *bytes; - bytes = get_data(self->archive, toc_entry); - if (bytes == NULL) - return NULL; - res = PyUnicode_FromStringAndSize(PyBytes_AS_STRING(bytes), - PyBytes_GET_SIZE(bytes)); - Py_DECREF(bytes); - return res; - } - - /* we have the module, but no source */ - Py_RETURN_NONE; -} - -/*[clinic input] -zipimport.zipimporter.get_resource_reader - - fullname: unicode - / - -Return the ResourceReader for a package in a zip file. - -If 'fullname' is a package within the zip file, return the 'ResourceReader' -object for the package. Otherwise return None. - -[clinic start generated code]*/ - -static PyObject * -zipimport_zipimporter_get_resource_reader_impl(ZipImporter *self, - PyObject *fullname) -/*[clinic end generated code: output=5e367d431f830726 input=bfab94d736e99151]*/ -{ - PyObject *module = PyImport_ImportModule("importlib.resources"); - if (module == NULL) { - return NULL; - } - PyObject *retval = PyObject_CallMethod( - module, "_zipimport_get_resource_reader", - "OO", (PyObject *)self, fullname); - Py_DECREF(module); - return retval; -} - - -static PyMethodDef zipimporter_methods[] = { - ZIPIMPORT_ZIPIMPORTER_FIND_MODULE_METHODDEF - ZIPIMPORT_ZIPIMPORTER_FIND_LOADER_METHODDEF - ZIPIMPORT_ZIPIMPORTER_LOAD_MODULE_METHODDEF - ZIPIMPORT_ZIPIMPORTER_GET_FILENAME_METHODDEF - ZIPIMPORT_ZIPIMPORTER_IS_PACKAGE_METHODDEF - ZIPIMPORT_ZIPIMPORTER_GET_DATA_METHODDEF - ZIPIMPORT_ZIPIMPORTER_GET_CODE_METHODDEF - ZIPIMPORT_ZIPIMPORTER_GET_SOURCE_METHODDEF - ZIPIMPORT_ZIPIMPORTER_GET_RESOURCE_READER_METHODDEF - {NULL, NULL} /* sentinel */ -}; - -static PyMemberDef zipimporter_members[] = { - {"archive", T_OBJECT, offsetof(ZipImporter, archive), READONLY}, - {"prefix", T_OBJECT, offsetof(ZipImporter, prefix), READONLY}, - {"_files", T_OBJECT, offsetof(ZipImporter, files), READONLY}, - {NULL} -}; - -#define DEFERRED_ADDRESS(ADDR) 0 - -static PyTypeObject ZipImporter_Type = { - PyVarObject_HEAD_INIT(DEFERRED_ADDRESS(&PyType_Type), 0) - "zipimport.zipimporter", - sizeof(ZipImporter), - 0, /* tp_itemsize */ - (destructor)zipimporter_dealloc, /* tp_dealloc */ - 0, /* tp_print */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_reserved */ - (reprfunc)zipimporter_repr, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | - Py_TPFLAGS_HAVE_GC, /* tp_flags */ - zipimport_zipimporter___init____doc__, /* tp_doc */ - zipimporter_traverse, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - zipimporter_methods, /* tp_methods */ - zipimporter_members, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - (initproc)zipimport_zipimporter___init__, /* tp_init */ - PyType_GenericAlloc, /* tp_alloc */ - PyType_GenericNew, /* tp_new */ - PyObject_GC_Del, /* tp_free */ -}; - - -/* implementation */ - -/* Given a buffer, return the unsigned int that is represented by the first - 4 bytes, encoded as little endian. This partially reimplements - marshal.c:r_long() */ -static unsigned int -get_uint32(const unsigned char *buf) -{ - unsigned int x; - x = buf[0]; - x |= (unsigned int)buf[1] << 8; - x |= (unsigned int)buf[2] << 16; - x |= (unsigned int)buf[3] << 24; - return x; -} - -/* Given a buffer, return the unsigned int that is represented by the first - 2 bytes, encoded as little endian. This partially reimplements - marshal.c:r_short() */ -static unsigned short -get_uint16(const unsigned char *buf) -{ - unsigned short x; - x = buf[0]; - x |= (unsigned short)buf[1] << 8; - return x; -} - -static void -set_file_error(PyObject *archive, int eof) -{ - if (eof) { - PyErr_SetString(PyExc_EOFError, "EOF read where not expected"); - } - else { - PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, archive); - } -} - -/* - read_directory(archive) -> files dict (new reference) - - Given a path to a Zip archive, build a dict, mapping file names - (local to the archive, using SEP as a separator) to toc entries. - - A toc_entry is a tuple: - - (__file__, # value to use for __file__, available for all files, - # encoded to the filesystem encoding - compress, # compression kind; 0 for uncompressed - data_size, # size of compressed data on disk - file_size, # size of decompressed data - file_offset, # offset of file header from start of archive - time, # mod time of file (in dos format) - date, # mod data of file (in dos format) - crc, # crc checksum of the data - ) - - Directories can be recognized by the trailing SEP in the name, - data_size and file_offset are 0. -*/ -static PyObject * -read_directory(PyObject *archive) -{ - PyObject *files = NULL; - FILE *fp; - unsigned short flags, compress, time, date, name_size; - unsigned int crc, data_size, file_size, header_size, header_offset; - unsigned long file_offset, header_position; - unsigned long arc_offset; /* Absolute offset to start of the zip-archive. */ - unsigned int count, i; - unsigned char buffer[46]; - char name[MAXPATHLEN + 5]; - PyObject *nameobj = NULL; - PyObject *path; - const char *charset; - int bootstrap; - const char *errmsg = NULL; - - fp = _Py_fopen_obj(archive, "rb"); - if (fp == NULL) { - if (PyErr_ExceptionMatches(PyExc_OSError)) { - _PyErr_FormatFromCause(ZipImportError, - "can't open Zip file: %R", archive); - } - return NULL; - } - - if (fseek(fp, -22, SEEK_END) == -1) { - goto file_error; - } - header_position = (unsigned long)ftell(fp); - if (header_position == (unsigned long)-1) { - goto file_error; - } - assert(header_position <= (unsigned long)LONG_MAX); - if (fread(buffer, 1, 22, fp) != 22) { - goto file_error; - } - if (get_uint32(buffer) != 0x06054B50u) { - /* Bad: End of Central Dir signature */ - errmsg = "not a Zip file"; - goto invalid_header; - } - - header_size = get_uint32(buffer + 12); - header_offset = get_uint32(buffer + 16); - if (header_position < header_size) { - errmsg = "bad central directory size"; - goto invalid_header; - } - if (header_position < header_offset) { - errmsg = "bad central directory offset"; - goto invalid_header; - } - if (header_position - header_size < header_offset) { - errmsg = "bad central directory size or offset"; - goto invalid_header; - } - header_position -= header_size; - arc_offset = header_position - header_offset; - - files = PyDict_New(); - if (files == NULL) { - goto error; - } - /* Start of Central Directory */ - count = 0; - if (fseek(fp, (long)header_position, 0) == -1) { - goto file_error; - } - for (;;) { - PyObject *t; - size_t n; - int err; - - n = fread(buffer, 1, 46, fp); - if (n < 4) { - goto eof_error; - } - /* Start of file header */ - if (get_uint32(buffer) != 0x02014B50u) { - break; /* Bad: Central Dir File Header */ - } - if (n != 46) { - goto eof_error; - } - flags = get_uint16(buffer + 8); - compress = get_uint16(buffer + 10); - time = get_uint16(buffer + 12); - date = get_uint16(buffer + 14); - crc = get_uint32(buffer + 16); - data_size = get_uint32(buffer + 20); - file_size = get_uint32(buffer + 24); - name_size = get_uint16(buffer + 28); - header_size = (unsigned int)name_size + - get_uint16(buffer + 30) /* extra field */ + - get_uint16(buffer + 32) /* comment */; - - file_offset = get_uint32(buffer + 42); - if (file_offset > header_offset) { - errmsg = "bad local header offset"; - goto invalid_header; - } - file_offset += arc_offset; - - if (name_size > MAXPATHLEN) { - name_size = MAXPATHLEN; - } - if (fread(name, 1, name_size, fp) != name_size) { - goto file_error; - } - name[name_size] = '\0'; /* Add terminating null byte */ -#if SEP != '/' - for (i = 0; i < name_size; i++) { - if (name[i] == '/') { - name[i] = SEP; - } - } -#endif - /* Skip the rest of the header. - * On Windows, calling fseek to skip over the fields we don't use is - * slower than reading the data because fseek flushes stdio's - * internal buffers. See issue #8745. */ - assert(header_size <= 3*0xFFFFu); - for (i = name_size; i < header_size; i++) { - if (getc(fp) == EOF) { - goto file_error; - } - } - - bootstrap = 0; - if (flags & 0x0800) { - charset = "utf-8"; - } - else if (!_PyInterpreterState_Get()->codecs_initialized) { - /* During bootstrap, we may need to load the encodings - package from a ZIP file. But the cp437 encoding is implemented - in Python in the encodings package. - - Break out of this dependency by assuming that the path to - the encodings module is ASCII-only. */ - charset = "ascii"; - bootstrap = 1; - } - else { - charset = "cp437"; - } - nameobj = PyUnicode_Decode(name, name_size, charset, NULL); - if (nameobj == NULL) { - if (bootstrap) { - PyErr_Format(PyExc_NotImplementedError, - "bootstrap issue: python%i%i.zip contains non-ASCII " - "filenames without the unicode flag", - PY_MAJOR_VERSION, PY_MINOR_VERSION); - } - goto error; - } - if (PyUnicode_READY(nameobj) == -1) { - goto error; - } - path = PyUnicode_FromFormat("%U%c%U", archive, SEP, nameobj); - if (path == NULL) { - goto error; - } - t = Py_BuildValue("NHIIkHHI", path, compress, data_size, - file_size, file_offset, time, date, crc); - if (t == NULL) { - goto error; - } - err = PyDict_SetItem(files, nameobj, t); - Py_CLEAR(nameobj); - Py_DECREF(t); - if (err != 0) { - goto error; - } - count++; - } - fclose(fp); - if (Py_VerboseFlag) { - PySys_FormatStderr("# zipimport: found %u names in %R\n", - count, archive); - } - return files; - -eof_error: - set_file_error(archive, !ferror(fp)); - goto error; - -file_error: - PyErr_Format(ZipImportError, "can't read Zip file: %R", archive); - goto error; - -invalid_header: - assert(errmsg != NULL); - PyErr_Format(ZipImportError, "%s: %R", errmsg, archive); - goto error; - -error: - fclose(fp); - Py_XDECREF(files); - Py_XDECREF(nameobj); - return NULL; -} - -/* Return the zlib.decompress function object, or NULL if zlib couldn't - be imported. The function is cached when found, so subsequent calls - don't import zlib again. */ -static PyObject * -get_decompress_func(void) -{ - static int importing_zlib = 0; - PyObject *zlib; - PyObject *decompress; - _Py_IDENTIFIER(decompress); - - if (importing_zlib != 0) - /* Someone has a zlib.pyc in their Zip file; - let's avoid a stack overflow. */ - return NULL; - importing_zlib = 1; - zlib = PyImport_ImportModuleNoBlock("zlib"); - importing_zlib = 0; - if (zlib != NULL) { - decompress = _PyObject_GetAttrId(zlib, - &PyId_decompress); - Py_DECREF(zlib); - } - else { - PyErr_Clear(); - decompress = NULL; - } - if (Py_VerboseFlag) - PySys_WriteStderr("# zipimport: zlib %s\n", - zlib != NULL ? "available": "UNAVAILABLE"); - return decompress; -} - -/* Given a path to a Zip file and a toc_entry, return the (uncompressed) - data as a new reference. */ -static PyObject * -get_data(PyObject *archive, PyObject *toc_entry) -{ - PyObject *raw_data = NULL, *data, *decompress; - char *buf; - FILE *fp; - PyObject *datapath; - unsigned short compress, time, date; - unsigned int crc; - Py_ssize_t data_size, file_size, bytes_size; - long file_offset, header_size; - unsigned char buffer[30]; - const char *errmsg = NULL; - - if (!PyArg_ParseTuple(toc_entry, "OHnnlHHI", &datapath, &compress, - &data_size, &file_size, &file_offset, &time, - &date, &crc)) { - return NULL; - } - if (data_size < 0) { - PyErr_Format(ZipImportError, "negative data size"); - return NULL; - } - - fp = _Py_fopen_obj(archive, "rb"); - if (!fp) { - return NULL; - } - /* Check to make sure the local file header is correct */ - if (fseek(fp, file_offset, 0) == -1) { - goto file_error; - } - if (fread(buffer, 1, 30, fp) != 30) { - goto eof_error; - } - if (get_uint32(buffer) != 0x04034B50u) { - /* Bad: Local File Header */ - errmsg = "bad local file header"; - goto invalid_header; - } - - header_size = (unsigned int)30 + - get_uint16(buffer + 26) /* file name */ + - get_uint16(buffer + 28) /* extra field */; - if (file_offset > LONG_MAX - header_size) { - errmsg = "bad local file header size"; - goto invalid_header; - } - file_offset += header_size; /* Start of file data */ - - if (data_size > LONG_MAX - 1) { - fclose(fp); - PyErr_NoMemory(); - return NULL; - } - bytes_size = compress == 0 ? data_size : data_size + 1; - if (bytes_size == 0) { - bytes_size++; - } - raw_data = PyBytes_FromStringAndSize((char *)NULL, bytes_size); - if (raw_data == NULL) { - goto error; - } - buf = PyBytes_AsString(raw_data); - - if (fseek(fp, file_offset, 0) == -1) { - goto file_error; - } - if (fread(buf, 1, data_size, fp) != (size_t)data_size) { - PyErr_SetString(PyExc_OSError, - "zipimport: can't read data"); - goto error; - } - - fclose(fp); - fp = NULL; - - if (compress != 0) { - buf[data_size] = 'Z'; /* saw this in zipfile.py */ - data_size++; - } - buf[data_size] = '\0'; - - if (compress == 0) { /* data is not compressed */ - data = PyBytes_FromStringAndSize(buf, data_size); - Py_DECREF(raw_data); - return data; - } - - /* Decompress with zlib */ - decompress = get_decompress_func(); - if (decompress == NULL) { - PyErr_SetString(ZipImportError, - "can't decompress data; " - "zlib not available"); - goto error; - } - data = PyObject_CallFunction(decompress, "Oi", raw_data, -15); - Py_DECREF(decompress); - Py_DECREF(raw_data); - if (data != NULL && !PyBytes_Check(data)) { - PyErr_Format(PyExc_TypeError, - "zlib.decompress() must return a bytes object, not " - "%.200s", - Py_TYPE(data)->tp_name); - Py_DECREF(data); - return NULL; - } - return data; - -eof_error: - set_file_error(archive, !ferror(fp)); - goto error; - -file_error: - PyErr_Format(ZipImportError, "can't read Zip file: %R", archive); - goto error; - -invalid_header: - assert(errmsg != NULL); - PyErr_Format(ZipImportError, "%s: %R", errmsg, archive); - goto error; - -error: - if (fp != NULL) { - fclose(fp); - } - Py_XDECREF(raw_data); - return NULL; -} - -/* Lenient date/time comparison function. The precision of the mtime - in the archive is lower than the mtime stored in a .pyc: we - must allow a difference of at most one second. */ -static int -eq_mtime(time_t t1, time_t t2) -{ - time_t d = t1 - t2; - if (d < 0) - d = -d; - /* dostime only stores even seconds, so be lenient */ - return d <= 1; -} - -/* Given the contents of a .pyc file in a buffer, unmarshal the data - and return the code object. Return None if it the magic word doesn't - match (we do this instead of raising an exception as we fall back - to .py if available and we don't want to mask other errors). - Returns a new reference. */ -static PyObject * -unmarshal_code(PyObject *pathname, PyObject *data, time_t mtime) -{ - PyObject *code; - unsigned char *buf = (unsigned char *)PyBytes_AsString(data); - Py_ssize_t size = PyBytes_Size(data); - - if (size < 16) { - PyErr_SetString(ZipImportError, - "bad pyc data"); - return NULL; - } - - if (get_uint32(buf) != (unsigned int)PyImport_GetMagicNumber()) { - if (Py_VerboseFlag) { - PySys_FormatStderr("# %R has bad magic\n", - pathname); - } - Py_RETURN_NONE; /* signal caller to try alternative */ - } - - uint32_t flags = get_uint32(buf + 4); - if (flags != 0) { - _PyCoreConfig *config = &_PyInterpreterState_Get()->core_config; - // Hash-based pyc. We currently refuse to handle checked hash-based - // pycs. We could validate hash-based pycs against the source, but it - // seems likely that most people putting hash-based pycs in a zipfile - // will use unchecked ones. - if (strcmp(config->_check_hash_pycs_mode, "never") && - (flags != 0x1 || !strcmp(config->_check_hash_pycs_mode, "always"))) - Py_RETURN_NONE; - } else if ((mtime != 0 && !eq_mtime(get_uint32(buf + 8), mtime))) { - if (Py_VerboseFlag) { - PySys_FormatStderr("# %R has bad mtime\n", - pathname); - } - Py_RETURN_NONE; /* signal caller to try alternative */ - } - - /* XXX the pyc's size field is ignored; timestamp collisions are probably - unimportant with zip files. */ - code = PyMarshal_ReadObjectFromString((char *)buf + 16, size - 16); - if (code == NULL) { - return NULL; - } - if (!PyCode_Check(code)) { - Py_DECREF(code); - PyErr_Format(PyExc_TypeError, - "compiled module %R is not a code object", - pathname); - return NULL; - } - return code; -} - -/* Replace any occurrences of "\r\n?" in the input string with "\n". - This converts DOS and Mac line endings to Unix line endings. - Also append a trailing "\n" to be compatible with - PyParser_SimpleParseFile(). Returns a new reference. */ -static PyObject * -normalize_line_endings(PyObject *source) -{ - char *buf, *q, *p; - PyObject *fixed_source; - int len = 0; - - p = PyBytes_AsString(source); - if (p == NULL) { - return PyBytes_FromStringAndSize("\n\0", 2); - } - - /* one char extra for trailing \n and one for terminating \0 */ - buf = (char *)PyMem_Malloc(PyBytes_Size(source) + 2); - if (buf == NULL) { - PyErr_SetString(PyExc_MemoryError, - "zipimport: no memory to allocate " - "source buffer"); - return NULL; - } - /* replace "\r\n?" by "\n" */ - for (q = buf; *p != '\0'; p++) { - if (*p == '\r') { - *q++ = '\n'; - if (*(p + 1) == '\n') - p++; - } - else - *q++ = *p; - len++; - } - *q++ = '\n'; /* add trailing \n */ - *q = '\0'; - fixed_source = PyBytes_FromStringAndSize(buf, len + 2); - PyMem_Free(buf); - return fixed_source; -} - -/* Given a string buffer containing Python source code, compile it - and return a code object as a new reference. */ -static PyObject * -compile_source(PyObject *pathname, PyObject *source) -{ - PyObject *code, *fixed_source; - - fixed_source = normalize_line_endings(source); - if (fixed_source == NULL) { - return NULL; - } - - code = Py_CompileStringObject(PyBytes_AsString(fixed_source), - pathname, Py_file_input, NULL, -1); - - Py_DECREF(fixed_source); - return code; -} - -/* Convert the date/time values found in the Zip archive to a value - that's compatible with the time stamp stored in .pyc files. */ -static time_t -parse_dostime(int dostime, int dosdate) -{ - struct tm stm; - - memset((void *) &stm, '\0', sizeof(stm)); - - stm.tm_sec = (dostime & 0x1f) * 2; - stm.tm_min = (dostime >> 5) & 0x3f; - stm.tm_hour = (dostime >> 11) & 0x1f; - stm.tm_mday = dosdate & 0x1f; - stm.tm_mon = ((dosdate >> 5) & 0x0f) - 1; - stm.tm_year = ((dosdate >> 9) & 0x7f) + 80; - stm.tm_isdst = -1; /* wday/yday is ignored */ - - return mktime(&stm); -} - -/* Given a path to a .pyc file in the archive, return the - modification time of the matching .py file, or 0 if no source - is available. */ -static time_t -get_mtime_of_source(ZipImporter *self, PyObject *path) -{ - PyObject *toc_entry, *stripped; - time_t mtime; - - /* strip 'c' from *.pyc */ - if (PyUnicode_READY(path) == -1) - return (time_t)-1; - stripped = PyUnicode_FromKindAndData(PyUnicode_KIND(path), - PyUnicode_DATA(path), - PyUnicode_GET_LENGTH(path) - 1); - if (stripped == NULL) - return (time_t)-1; - - toc_entry = PyDict_GetItem(self->files, stripped); - Py_DECREF(stripped); - if (toc_entry != NULL && PyTuple_Check(toc_entry) && - PyTuple_Size(toc_entry) == 8) { - /* fetch the time stamp of the .py file for comparison - with an embedded pyc time stamp */ - int time, date; - time = PyLong_AsLong(PyTuple_GetItem(toc_entry, 5)); - date = PyLong_AsLong(PyTuple_GetItem(toc_entry, 6)); - mtime = parse_dostime(time, date); - } else - mtime = 0; - return mtime; -} - -/* Return the code object for the module named by 'fullname' from the - Zip archive as a new reference. */ -static PyObject * -get_code_from_data(ZipImporter *self, int ispackage, int isbytecode, - time_t mtime, PyObject *toc_entry) -{ - PyObject *data, *modpath, *code; - - data = get_data(self->archive, toc_entry); - if (data == NULL) - return NULL; - - modpath = PyTuple_GetItem(toc_entry, 0); - if (isbytecode) - code = unmarshal_code(modpath, data, mtime); - else - code = compile_source(modpath, data); - Py_DECREF(data); - return code; -} - -/* Get the code object associated with the module specified by - 'fullname'. */ -static PyObject * -get_module_code(ZipImporter *self, PyObject *fullname, - int *p_ispackage, PyObject **p_modpath) -{ - PyObject *code = NULL, *toc_entry, *subname; - PyObject *path, *fullpath = NULL; - struct st_zip_searchorder *zso; - - if (self->prefix == NULL) { - PyErr_SetString(PyExc_ValueError, - "zipimporter.__init__() wasn't called"); - return NULL; - } - - subname = get_subname(fullname); - if (subname == NULL) - return NULL; - - path = make_filename(self->prefix, subname); - Py_DECREF(subname); - if (path == NULL) - return NULL; - - for (zso = zip_searchorder; *zso->suffix; zso++) { - code = NULL; - - fullpath = PyUnicode_FromFormat("%U%s", path, zso->suffix); - if (fullpath == NULL) - goto exit; - - if (Py_VerboseFlag > 1) - PySys_FormatStderr("# trying %U%c%U\n", - self->archive, (int)SEP, fullpath); - toc_entry = PyDict_GetItem(self->files, fullpath); - if (toc_entry != NULL) { - time_t mtime = 0; - int ispackage = zso->type & IS_PACKAGE; - int isbytecode = zso->type & IS_BYTECODE; - - if (isbytecode) { - mtime = get_mtime_of_source(self, fullpath); - if (mtime == (time_t)-1 && PyErr_Occurred()) { - goto exit; - } - } - Py_CLEAR(fullpath); - if (p_ispackage != NULL) - *p_ispackage = ispackage; - code = get_code_from_data(self, ispackage, - isbytecode, mtime, - toc_entry); - if (code == Py_None) { - /* bad magic number or non-matching mtime - in byte code, try next */ - Py_DECREF(code); - continue; - } - if (code != NULL && p_modpath != NULL) { - *p_modpath = PyTuple_GetItem(toc_entry, 0); - Py_INCREF(*p_modpath); - } - goto exit; - } - else - Py_CLEAR(fullpath); - } - PyErr_Format(ZipImportError, "can't find module %R", fullname); -exit: - Py_DECREF(path); - Py_XDECREF(fullpath); - return code; -} - - -/* Module init */ - -PyDoc_STRVAR(zipimport_doc, -"zipimport provides support for importing Python modules from Zip archives.\n\ -\n\ -This module exports three objects:\n\ -- zipimporter: a class; its constructor takes a path to a Zip archive.\n\ -- ZipImportError: exception raised by zipimporter objects. It's a\n\ - subclass of ImportError, so it can be caught as ImportError, too.\n\ -- _zip_directory_cache: a dict, mapping archive paths to zip directory\n\ - info dicts, as used in zipimporter._files.\n\ -\n\ -It is usually not needed to use the zipimport module explicitly; it is\n\ -used by the builtin import mechanism for sys.path items that are paths\n\ -to Zip archives."); - -static struct PyModuleDef zipimportmodule = { - PyModuleDef_HEAD_INIT, - "zipimport", - zipimport_doc, - -1, - NULL, - NULL, - NULL, - NULL, - NULL -}; - -PyMODINIT_FUNC -PyInit_zipimport(void) -{ - PyObject *mod; - - if (PyType_Ready(&ZipImporter_Type) < 0) - return NULL; - - /* Correct directory separator */ - zip_searchorder[0].suffix[0] = SEP; - zip_searchorder[1].suffix[0] = SEP; - - mod = PyModule_Create(&zipimportmodule); - if (mod == NULL) - return NULL; - - ZipImportError = PyErr_NewException("zipimport.ZipImportError", - PyExc_ImportError, NULL); - if (ZipImportError == NULL) - return NULL; - - Py_INCREF(ZipImportError); - if (PyModule_AddObject(mod, "ZipImportError", - ZipImportError) < 0) - return NULL; - - Py_INCREF(&ZipImporter_Type); - if (PyModule_AddObject(mod, "zipimporter", - (PyObject *)&ZipImporter_Type) < 0) - return NULL; - - zip_directory_cache = PyDict_New(); - if (zip_directory_cache == NULL) - return NULL; - Py_INCREF(zip_directory_cache); - if (PyModule_AddObject(mod, "_zip_directory_cache", - zip_directory_cache) < 0) - return NULL; - return mod; -} diff --git a/PC/config.c b/PC/config.c index 568a0fa9a467..43347dddeba3 100644 --- a/PC/config.c +++ b/PC/config.c @@ -35,7 +35,6 @@ extern PyObject* PyInit__weakref(void); /* XXX: These two should really be extracted to standalone extensions. */ extern PyObject* PyInit_xxsubtype(void); extern PyObject* PyInit__xxsubinterpreters(void); -extern PyObject* PyInit_zipimport(void); extern PyObject* PyInit__random(void); extern PyObject* PyInit_itertools(void); extern PyObject* PyInit__collections(void); @@ -131,7 +130,6 @@ struct _inittab _PyImport_Inittab[] = { {"xxsubtype", PyInit_xxsubtype}, {"_xxsubinterpreters", PyInit__xxsubinterpreters}, - {"zipimport", PyInit_zipimport}, #ifdef _Py_HAVE_ZLIB {"zlib", PyInit_zlib}, #endif diff --git a/PCbuild/_freeze_importlib.vcxproj b/PCbuild/_freeze_importlib.vcxproj index c73266301e8b..34e754658d72 100644 --- a/PCbuild/_freeze_importlib.vcxproj +++ b/PCbuild/_freeze_importlib.vcxproj @@ -77,19 +77,26 @@ + importlib._bootstrap $(IntDir)importlib.g.h $(PySourcePath)Python\importlib.h + importlib._bootstrap_external $(IntDir)importlib_external.g.h $(PySourcePath)Python\importlib_external.h + + zipimport + $(IntDir)importlib_zipimport.g.h + $(PySourcePath)Python\importlib_zipimport.h + - + <_OldContent Condition="Exists($(OutTargetPath))"> @@ -114,6 +121,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 42455f5c7da6..4a05d590be5c 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -282,7 +282,6 @@ - diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 582114f6f475..25b1011340a2 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -617,9 +617,6 @@ Modules - - Modules - Modules diff --git a/Programs/_freeze_importlib.c b/Programs/_freeze_importlib.c index 8830d131d6f4..3f43757ecb1f 100644 --- a/Programs/_freeze_importlib.c +++ b/Programs/_freeze_importlib.c @@ -27,28 +27,30 @@ static const struct _frozen _PyImport_FrozenModules[] = { const struct _frozen *PyImport_FrozenModules; #endif -const char header[] = "/* Auto-generated by Programs/_freeze_importlib.c */"; +static const char header[] = + "/* Auto-generated by Programs/_freeze_importlib.c */"; int main(int argc, char *argv[]) { - char *inpath, *outpath, *code_name; + const char *name, *inpath, *outpath; + char buf[100]; FILE *infile = NULL, *outfile = NULL; struct _Py_stat_struct status; - size_t text_size, data_size, n; + size_t text_size, data_size, i, n; char *text = NULL; unsigned char *data; PyObject *code = NULL, *marshalled = NULL; - int is_bootstrap = 1; PyImport_FrozenModules = _PyImport_FrozenModules; - if (argc != 3) { - fprintf(stderr, "need to specify input and output paths\n"); + if (argc != 4) { + fprintf(stderr, "need to specify the name, input and output paths\n"); return 2; } - inpath = argv[1]; - outpath = argv[2]; + name = argv[1]; + inpath = argv[2]; + outpath = argv[3]; infile = fopen(inpath, "rb"); if (infile == NULL) { fprintf(stderr, "cannot open '%s' for reading\n", inpath); @@ -90,14 +92,8 @@ main(int argc, char *argv[]) _Py_FatalInitError(err); } - if (strstr(inpath, "_external") != NULL) { - is_bootstrap = 0; - } - - code_name = is_bootstrap ? - "" : - ""; - code = Py_CompileStringExFlags(text, code_name, Py_file_input, NULL, 0); + sprintf(buf, "", name); + code = Py_CompileStringExFlags(text, buf, Py_file_input, NULL, 0); if (code == NULL) goto error; free(text); @@ -120,11 +116,13 @@ main(int argc, char *argv[]) goto error; } fprintf(outfile, "%s\n", header); - if (is_bootstrap) - fprintf(outfile, "const unsigned char _Py_M__importlib[] = {\n"); - else - fprintf(outfile, - "const unsigned char _Py_M__importlib_external[] = {\n"); + for (i = n = 0; name[i] != '\0'; i++) { + if (name[i] != '.') { + buf[n++] = name[i]; + } + } + buf[n] = '\0'; + fprintf(outfile, "const unsigned char _Py_M__%s[] = {\n", buf); for (n = 0; n < data_size; n += 16) { size_t i, end = Py_MIN(n + 16, data_size); fprintf(outfile, " "); diff --git a/Python/frozen.c b/Python/frozen.c index 7e04f3ca0250..f1901d24deeb 100644 --- a/Python/frozen.c +++ b/Python/frozen.c @@ -4,6 +4,7 @@ #include "Python.h" #include "importlib.h" #include "importlib_external.h" +#include "importlib_zipimport.h" /* In order to test the support for frozen modules, by default we define a single frozen module, __hello__. Loading it will print @@ -29,9 +30,12 @@ static unsigned char M___hello__[] = { static const struct _frozen _PyImport_FrozenModules[] = { /* importlib */ - {"_frozen_importlib", _Py_M__importlib, (int)sizeof(_Py_M__importlib)}, - {"_frozen_importlib_external", _Py_M__importlib_external, - (int)sizeof(_Py_M__importlib_external)}, + {"_frozen_importlib", _Py_M__importlib_bootstrap, + (int)sizeof(_Py_M__importlib_bootstrap)}, + {"_frozen_importlib_external", _Py_M__importlib_bootstrap_external, + (int)sizeof(_Py_M__importlib_bootstrap_external)}, + {"zipimport", _Py_M__zipimport, + (int)sizeof(_Py_M__zipimport)}, /* Test module */ {"__hello__", M___hello__, SIZE}, /* Test package (negative size indicates package-ness) */ diff --git a/Python/importlib.h b/Python/importlib.h index 1d2576005e9e..82e340676b66 100644 --- a/Python/importlib.h +++ b/Python/importlib.h @@ -1,5 +1,5 @@ /* Auto-generated by Programs/_freeze_importlib.c */ -const unsigned char _Py_M__importlib[] = { +const unsigned char _Py_M__importlib_bootstrap[] = { 99,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0, 0,64,0,0,0,115,208,1,0,0,100,0,90,0,100,1, 97,1,100,2,100,3,132,0,90,2,100,4,100,5,132,0, diff --git a/Python/importlib_external.h b/Python/importlib_external.h index 9a813c963e03..a597ca3d935c 100644 --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -1,850 +1,865 @@ /* Auto-generated by Programs/_freeze_importlib.c */ -const unsigned char _Py_M__importlib_external[] = { +const unsigned char _Py_M__importlib_bootstrap_external[] = { 99,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0, - 0,64,0,0,0,115,44,2,0,0,100,0,90,0,100,1, + 0,64,0,0,0,115,52,2,0,0,100,0,90,0,100,1, 90,1,100,2,90,2,101,2,101,1,23,0,90,3,100,3, 100,4,132,0,90,4,100,5,100,6,132,0,90,5,100,7, 100,8,132,0,90,6,100,9,100,10,132,0,90,7,100,11, 100,12,132,0,90,8,100,13,100,14,132,0,90,9,100,15, 100,16,132,0,90,10,100,17,100,18,132,0,90,11,100,19, 100,20,132,0,90,12,100,21,100,22,132,0,90,13,100,23, - 102,1,100,24,100,25,132,1,90,14,101,15,101,14,106,16, - 131,1,90,17,100,26,160,18,100,27,100,28,161,2,100,29, - 23,0,90,19,101,20,160,21,101,19,100,28,161,2,90,22, - 100,30,90,23,100,31,90,24,100,32,103,1,90,25,100,33, - 103,1,90,26,101,26,4,0,90,27,90,28,100,34,102,1, - 100,34,100,35,156,1,100,36,100,37,132,3,90,29,100,38, - 100,39,132,0,90,30,100,40,100,41,132,0,90,31,100,42, + 100,24,132,0,90,14,100,25,102,1,100,26,100,27,132,1, + 90,15,101,16,101,15,106,17,131,1,90,18,100,28,160,19, + 100,29,100,30,161,2,100,31,23,0,90,20,101,21,160,22, + 101,20,100,30,161,2,90,23,100,32,90,24,100,33,90,25, + 100,34,103,1,90,26,100,35,103,1,90,27,101,27,4,0, + 90,28,90,29,100,36,102,1,100,36,100,37,156,1,100,38, + 100,39,132,3,90,30,100,40,100,41,132,0,90,31,100,42, 100,43,132,0,90,32,100,44,100,45,132,0,90,33,100,46, 100,47,132,0,90,34,100,48,100,49,132,0,90,35,100,50, 100,51,132,0,90,36,100,52,100,53,132,0,90,37,100,54, - 100,55,132,0,90,38,100,34,100,34,100,34,102,3,100,56, - 100,57,132,1,90,39,100,58,100,58,102,2,100,59,100,60, - 132,1,90,40,100,61,102,1,100,62,100,63,132,1,90,41, - 100,64,100,65,132,0,90,42,101,43,131,0,90,44,100,34, - 102,1,100,34,101,44,100,66,156,2,100,67,100,68,132,3, - 90,45,71,0,100,69,100,70,132,0,100,70,131,2,90,46, - 71,0,100,71,100,72,132,0,100,72,131,2,90,47,71,0, - 100,73,100,74,132,0,100,74,101,47,131,3,90,48,71,0, - 100,75,100,76,132,0,100,76,131,2,90,49,71,0,100,77, - 100,78,132,0,100,78,101,49,101,48,131,4,90,50,71,0, - 100,79,100,80,132,0,100,80,101,49,101,47,131,4,90,51, - 103,0,90,52,71,0,100,81,100,82,132,0,100,82,101,49, - 101,47,131,4,90,53,71,0,100,83,100,84,132,0,100,84, - 131,2,90,54,71,0,100,85,100,86,132,0,100,86,131,2, - 90,55,71,0,100,87,100,88,132,0,100,88,131,2,90,56, - 71,0,100,89,100,90,132,0,100,90,131,2,90,57,100,34, - 102,1,100,91,100,92,132,1,90,58,100,93,100,94,132,0, + 100,55,132,0,90,38,100,56,100,57,132,0,90,39,100,36, + 100,36,100,36,102,3,100,58,100,59,132,1,90,40,100,60, + 100,60,102,2,100,61,100,62,132,1,90,41,100,63,102,1, + 100,64,100,65,132,1,90,42,100,66,100,67,132,0,90,43, + 101,44,131,0,90,45,100,36,102,1,100,36,101,45,100,68, + 156,2,100,69,100,70,132,3,90,46,71,0,100,71,100,72, + 132,0,100,72,131,2,90,47,71,0,100,73,100,74,132,0, + 100,74,131,2,90,48,71,0,100,75,100,76,132,0,100,76, + 101,48,131,3,90,49,71,0,100,77,100,78,132,0,100,78, + 131,2,90,50,71,0,100,79,100,80,132,0,100,80,101,50, + 101,49,131,4,90,51,71,0,100,81,100,82,132,0,100,82, + 101,50,101,48,131,4,90,52,103,0,90,53,71,0,100,83, + 100,84,132,0,100,84,101,50,101,48,131,4,90,54,71,0, + 100,85,100,86,132,0,100,86,131,2,90,55,71,0,100,87, + 100,88,132,0,100,88,131,2,90,56,71,0,100,89,100,90, + 132,0,100,90,131,2,90,57,71,0,100,91,100,92,132,0, + 100,92,131,2,90,58,100,36,102,1,100,93,100,94,132,1, 90,59,100,95,100,96,132,0,90,60,100,97,100,98,132,0, - 90,61,100,34,83,0,41,99,97,94,1,0,0,67,111,114, - 101,32,105,109,112,108,101,109,101,110,116,97,116,105,111,110, - 32,111,102,32,112,97,116,104,45,98,97,115,101,100,32,105, - 109,112,111,114,116,46,10,10,84,104,105,115,32,109,111,100, - 117,108,101,32,105,115,32,78,79,84,32,109,101,97,110,116, - 32,116,111,32,98,101,32,100,105,114,101,99,116,108,121,32, - 105,109,112,111,114,116,101,100,33,32,73,116,32,104,97,115, - 32,98,101,101,110,32,100,101,115,105,103,110,101,100,32,115, - 117,99,104,10,116,104,97,116,32,105,116,32,99,97,110,32, - 98,101,32,98,111,111,116,115,116,114,97,112,112,101,100,32, - 105,110,116,111,32,80,121,116,104,111,110,32,97,115,32,116, - 104,101,32,105,109,112,108,101,109,101,110,116,97,116,105,111, - 110,32,111,102,32,105,109,112,111,114,116,46,32,65,115,10, - 115,117,99,104,32,105,116,32,114,101,113,117,105,114,101,115, - 32,116,104,101,32,105,110,106,101,99,116,105,111,110,32,111, - 102,32,115,112,101,99,105,102,105,99,32,109,111,100,117,108, - 101,115,32,97,110,100,32,97,116,116,114,105,98,117,116,101, - 115,32,105,110,32,111,114,100,101,114,32,116,111,10,119,111, - 114,107,46,32,79,110,101,32,115,104,111,117,108,100,32,117, - 115,101,32,105,109,112,111,114,116,108,105,98,32,97,115,32, - 116,104,101,32,112,117,98,108,105,99,45,102,97,99,105,110, - 103,32,118,101,114,115,105,111,110,32,111,102,32,116,104,105, - 115,32,109,111,100,117,108,101,46,10,10,41,1,218,3,119, - 105,110,41,2,90,6,99,121,103,119,105,110,90,6,100,97, - 114,119,105,110,99,0,0,0,0,0,0,0,0,1,0,0, - 0,3,0,0,0,3,0,0,0,115,60,0,0,0,116,0, - 106,1,160,2,116,3,161,1,114,48,116,0,106,1,160,2, - 116,4,161,1,114,30,100,1,137,0,110,4,100,2,137,0, - 135,0,102,1,100,3,100,4,132,8,125,0,110,8,100,5, - 100,4,132,0,125,0,124,0,83,0,41,6,78,90,12,80, - 89,84,72,79,78,67,65,83,69,79,75,115,12,0,0,0, - 80,89,84,72,79,78,67,65,83,69,79,75,99,0,0,0, - 0,0,0,0,0,0,0,0,0,2,0,0,0,19,0,0, - 0,115,10,0,0,0,136,0,116,0,106,1,107,6,83,0, - 41,1,122,53,84,114,117,101,32,105,102,32,102,105,108,101, - 110,97,109,101,115,32,109,117,115,116,32,98,101,32,99,104, - 101,99,107,101,100,32,99,97,115,101,45,105,110,115,101,110, - 115,105,116,105,118,101,108,121,46,41,2,218,3,95,111,115, - 90,7,101,110,118,105,114,111,110,169,0,41,1,218,3,107, - 101,121,114,2,0,0,0,250,38,60,102,114,111,122,101,110, - 32,105,109,112,111,114,116,108,105,98,46,95,98,111,111,116, - 115,116,114,97,112,95,101,120,116,101,114,110,97,108,62,218, - 11,95,114,101,108,97,120,95,99,97,115,101,36,0,0,0, - 115,2,0,0,0,0,2,122,37,95,109,97,107,101,95,114, - 101,108,97,120,95,99,97,115,101,46,60,108,111,99,97,108, - 115,62,46,95,114,101,108,97,120,95,99,97,115,101,99,0, - 0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,83, - 0,0,0,115,4,0,0,0,100,1,83,0,41,2,122,53, - 84,114,117,101,32,105,102,32,102,105,108,101,110,97,109,101, - 115,32,109,117,115,116,32,98,101,32,99,104,101,99,107,101, - 100,32,99,97,115,101,45,105,110,115,101,110,115,105,116,105, - 118,101,108,121,46,70,114,2,0,0,0,114,2,0,0,0, - 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,114, - 5,0,0,0,40,0,0,0,115,2,0,0,0,0,2,41, - 5,218,3,115,121,115,218,8,112,108,97,116,102,111,114,109, - 218,10,115,116,97,114,116,115,119,105,116,104,218,27,95,67, - 65,83,69,95,73,78,83,69,78,83,73,84,73,86,69,95, - 80,76,65,84,70,79,82,77,83,218,35,95,67,65,83,69, - 95,73,78,83,69,78,83,73,84,73,86,69,95,80,76,65, - 84,70,79,82,77,83,95,83,84,82,95,75,69,89,41,1, - 114,5,0,0,0,114,2,0,0,0,41,1,114,3,0,0, - 0,114,4,0,0,0,218,16,95,109,97,107,101,95,114,101, - 108,97,120,95,99,97,115,101,29,0,0,0,115,14,0,0, - 0,0,1,12,1,12,1,6,2,4,2,14,4,8,3,114, - 11,0,0,0,99,1,0,0,0,0,0,0,0,1,0,0, - 0,4,0,0,0,67,0,0,0,115,20,0,0,0,116,0, - 124,0,131,1,100,1,64,0,160,1,100,2,100,3,161,2, - 83,0,41,4,122,42,67,111,110,118,101,114,116,32,97,32, - 51,50,45,98,105,116,32,105,110,116,101,103,101,114,32,116, - 111,32,108,105,116,116,108,101,45,101,110,100,105,97,110,46, - 108,3,0,0,0,255,127,255,127,3,0,233,4,0,0,0, - 218,6,108,105,116,116,108,101,41,2,218,3,105,110,116,218, - 8,116,111,95,98,121,116,101,115,41,1,218,1,120,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,218,7,95, - 119,95,108,111,110,103,46,0,0,0,115,2,0,0,0,0, - 2,114,17,0,0,0,99,1,0,0,0,0,0,0,0,1, - 0,0,0,4,0,0,0,67,0,0,0,115,12,0,0,0, - 116,0,160,1,124,0,100,1,161,2,83,0,41,2,122,47, - 67,111,110,118,101,114,116,32,52,32,98,121,116,101,115,32, - 105,110,32,108,105,116,116,108,101,45,101,110,100,105,97,110, - 32,116,111,32,97,110,32,105,110,116,101,103,101,114,46,114, - 13,0,0,0,41,2,114,14,0,0,0,218,10,102,114,111, - 109,95,98,121,116,101,115,41,1,90,9,105,110,116,95,98, - 121,116,101,115,114,2,0,0,0,114,2,0,0,0,114,4, - 0,0,0,218,7,95,114,95,108,111,110,103,51,0,0,0, - 115,2,0,0,0,0,2,114,19,0,0,0,99,0,0,0, - 0,0,0,0,0,1,0,0,0,4,0,0,0,71,0,0, - 0,115,20,0,0,0,116,0,160,1,100,1,100,2,132,0, - 124,0,68,0,131,1,161,1,83,0,41,3,122,31,82,101, - 112,108,97,99,101,109,101,110,116,32,102,111,114,32,111,115, - 46,112,97,116,104,46,106,111,105,110,40,41,46,99,1,0, - 0,0,0,0,0,0,2,0,0,0,5,0,0,0,83,0, - 0,0,115,26,0,0,0,103,0,124,0,93,18,125,1,124, - 1,114,22,124,1,160,0,116,1,161,1,145,2,113,4,83, - 0,114,2,0,0,0,41,2,218,6,114,115,116,114,105,112, - 218,15,112,97,116,104,95,115,101,112,97,114,97,116,111,114, - 115,41,2,218,2,46,48,218,4,112,97,114,116,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,218,10,60,108, - 105,115,116,99,111,109,112,62,58,0,0,0,115,6,0,0, - 0,6,1,2,0,4,255,122,30,95,112,97,116,104,95,106, - 111,105,110,46,60,108,111,99,97,108,115,62,46,60,108,105, - 115,116,99,111,109,112,62,41,2,218,8,112,97,116,104,95, - 115,101,112,218,4,106,111,105,110,41,1,218,10,112,97,116, - 104,95,112,97,114,116,115,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,218,10,95,112,97,116,104,95,106,111, - 105,110,56,0,0,0,115,6,0,0,0,0,2,10,1,2, - 255,114,28,0,0,0,99,1,0,0,0,0,0,0,0,5, - 0,0,0,5,0,0,0,67,0,0,0,115,96,0,0,0, - 116,0,116,1,131,1,100,1,107,2,114,36,124,0,160,2, - 116,3,161,1,92,3,125,1,125,2,125,3,124,1,124,3, - 102,2,83,0,116,4,124,0,131,1,68,0,93,42,125,4, - 124,4,116,1,107,6,114,44,124,0,106,5,124,4,100,1, - 100,2,141,2,92,2,125,1,125,3,124,1,124,3,102,2, - 2,0,1,0,83,0,113,44,100,3,124,0,102,2,83,0, - 41,4,122,32,82,101,112,108,97,99,101,109,101,110,116,32, - 102,111,114,32,111,115,46,112,97,116,104,46,115,112,108,105, - 116,40,41,46,233,1,0,0,0,41,1,90,8,109,97,120, - 115,112,108,105,116,218,0,41,6,218,3,108,101,110,114,21, - 0,0,0,218,10,114,112,97,114,116,105,116,105,111,110,114, - 25,0,0,0,218,8,114,101,118,101,114,115,101,100,218,6, - 114,115,112,108,105,116,41,5,218,4,112,97,116,104,90,5, - 102,114,111,110,116,218,1,95,218,4,116,97,105,108,114,16, - 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,218,11,95,112,97,116,104,95,115,112,108,105,116,62, - 0,0,0,115,16,0,0,0,0,2,12,1,16,1,8,1, - 12,1,8,1,18,1,14,1,114,38,0,0,0,99,1,0, - 0,0,0,0,0,0,1,0,0,0,3,0,0,0,67,0, - 0,0,115,10,0,0,0,116,0,160,1,124,0,161,1,83, - 0,41,1,122,126,83,116,97,116,32,116,104,101,32,112,97, - 116,104,46,10,10,32,32,32,32,77,97,100,101,32,97,32, - 115,101,112,97,114,97,116,101,32,102,117,110,99,116,105,111, - 110,32,116,111,32,109,97,107,101,32,105,116,32,101,97,115, - 105,101,114,32,116,111,32,111,118,101,114,114,105,100,101,32, - 105,110,32,101,120,112,101,114,105,109,101,110,116,115,10,32, - 32,32,32,40,101,46,103,46,32,99,97,99,104,101,32,115, - 116,97,116,32,114,101,115,117,108,116,115,41,46,10,10,32, - 32,32,32,41,2,114,1,0,0,0,90,4,115,116,97,116, - 41,1,114,35,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,218,10,95,112,97,116,104,95,115,116, - 97,116,74,0,0,0,115,2,0,0,0,0,7,114,39,0, - 0,0,99,2,0,0,0,0,0,0,0,3,0,0,0,8, - 0,0,0,67,0,0,0,115,50,0,0,0,122,12,116,0, - 124,0,131,1,125,2,87,0,110,22,4,0,116,1,107,10, - 114,34,1,0,1,0,1,0,89,0,100,1,83,0,88,0, - 124,2,106,2,100,2,64,0,124,1,107,2,83,0,41,3, - 122,49,84,101,115,116,32,119,104,101,116,104,101,114,32,116, - 104,101,32,112,97,116,104,32,105,115,32,116,104,101,32,115, - 112,101,99,105,102,105,101,100,32,109,111,100,101,32,116,121, - 112,101,46,70,105,0,240,0,0,41,3,114,39,0,0,0, - 218,7,79,83,69,114,114,111,114,218,7,115,116,95,109,111, - 100,101,41,3,114,35,0,0,0,218,4,109,111,100,101,90, - 9,115,116,97,116,95,105,110,102,111,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,218,18,95,112,97,116,104, - 95,105,115,95,109,111,100,101,95,116,121,112,101,84,0,0, - 0,115,10,0,0,0,0,2,2,1,12,1,14,1,8,1, - 114,43,0,0,0,99,1,0,0,0,0,0,0,0,1,0, - 0,0,3,0,0,0,67,0,0,0,115,10,0,0,0,116, - 0,124,0,100,1,131,2,83,0,41,2,122,31,82,101,112, - 108,97,99,101,109,101,110,116,32,102,111,114,32,111,115,46, - 112,97,116,104,46,105,115,102,105,108,101,46,105,0,128,0, - 0,41,1,114,43,0,0,0,41,1,114,35,0,0,0,114, - 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,12, - 95,112,97,116,104,95,105,115,102,105,108,101,93,0,0,0, - 115,2,0,0,0,0,2,114,44,0,0,0,99,1,0,0, - 0,0,0,0,0,1,0,0,0,3,0,0,0,67,0,0, - 0,115,22,0,0,0,124,0,115,12,116,0,160,1,161,0, - 125,0,116,2,124,0,100,1,131,2,83,0,41,2,122,30, + 90,61,100,99,100,100,132,0,90,62,100,36,83,0,41,101, + 97,94,1,0,0,67,111,114,101,32,105,109,112,108,101,109, + 101,110,116,97,116,105,111,110,32,111,102,32,112,97,116,104, + 45,98,97,115,101,100,32,105,109,112,111,114,116,46,10,10, + 84,104,105,115,32,109,111,100,117,108,101,32,105,115,32,78, + 79,84,32,109,101,97,110,116,32,116,111,32,98,101,32,100, + 105,114,101,99,116,108,121,32,105,109,112,111,114,116,101,100, + 33,32,73,116,32,104,97,115,32,98,101,101,110,32,100,101, + 115,105,103,110,101,100,32,115,117,99,104,10,116,104,97,116, + 32,105,116,32,99,97,110,32,98,101,32,98,111,111,116,115, + 116,114,97,112,112,101,100,32,105,110,116,111,32,80,121,116, + 104,111,110,32,97,115,32,116,104,101,32,105,109,112,108,101, + 109,101,110,116,97,116,105,111,110,32,111,102,32,105,109,112, + 111,114,116,46,32,65,115,10,115,117,99,104,32,105,116,32, + 114,101,113,117,105,114,101,115,32,116,104,101,32,105,110,106, + 101,99,116,105,111,110,32,111,102,32,115,112,101,99,105,102, + 105,99,32,109,111,100,117,108,101,115,32,97,110,100,32,97, + 116,116,114,105,98,117,116,101,115,32,105,110,32,111,114,100, + 101,114,32,116,111,10,119,111,114,107,46,32,79,110,101,32, + 115,104,111,117,108,100,32,117,115,101,32,105,109,112,111,114, + 116,108,105,98,32,97,115,32,116,104,101,32,112,117,98,108, + 105,99,45,102,97,99,105,110,103,32,118,101,114,115,105,111, + 110,32,111,102,32,116,104,105,115,32,109,111,100,117,108,101, + 46,10,10,41,1,218,3,119,105,110,41,2,90,6,99,121, + 103,119,105,110,90,6,100,97,114,119,105,110,99,0,0,0, + 0,0,0,0,0,1,0,0,0,3,0,0,0,3,0,0, + 0,115,60,0,0,0,116,0,106,1,160,2,116,3,161,1, + 114,48,116,0,106,1,160,2,116,4,161,1,114,30,100,1, + 137,0,110,4,100,2,137,0,135,0,102,1,100,3,100,4, + 132,8,125,0,110,8,100,5,100,4,132,0,125,0,124,0, + 83,0,41,6,78,90,12,80,89,84,72,79,78,67,65,83, + 69,79,75,115,12,0,0,0,80,89,84,72,79,78,67,65, + 83,69,79,75,99,0,0,0,0,0,0,0,0,0,0,0, + 0,2,0,0,0,19,0,0,0,115,10,0,0,0,136,0, + 116,0,106,1,107,6,83,0,41,1,122,53,84,114,117,101, + 32,105,102,32,102,105,108,101,110,97,109,101,115,32,109,117, + 115,116,32,98,101,32,99,104,101,99,107,101,100,32,99,97, + 115,101,45,105,110,115,101,110,115,105,116,105,118,101,108,121, + 46,41,2,218,3,95,111,115,90,7,101,110,118,105,114,111, + 110,169,0,41,1,218,3,107,101,121,114,2,0,0,0,250, + 38,60,102,114,111,122,101,110,32,105,109,112,111,114,116,108, + 105,98,46,95,98,111,111,116,115,116,114,97,112,95,101,120, + 116,101,114,110,97,108,62,218,11,95,114,101,108,97,120,95, + 99,97,115,101,36,0,0,0,115,2,0,0,0,0,2,122, + 37,95,109,97,107,101,95,114,101,108,97,120,95,99,97,115, + 101,46,60,108,111,99,97,108,115,62,46,95,114,101,108,97, + 120,95,99,97,115,101,99,0,0,0,0,0,0,0,0,0, + 0,0,0,1,0,0,0,83,0,0,0,115,4,0,0,0, + 100,1,83,0,41,2,122,53,84,114,117,101,32,105,102,32, + 102,105,108,101,110,97,109,101,115,32,109,117,115,116,32,98, + 101,32,99,104,101,99,107,101,100,32,99,97,115,101,45,105, + 110,115,101,110,115,105,116,105,118,101,108,121,46,70,114,2, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,2,0, + 0,0,114,4,0,0,0,114,5,0,0,0,40,0,0,0, + 115,2,0,0,0,0,2,41,5,218,3,115,121,115,218,8, + 112,108,97,116,102,111,114,109,218,10,115,116,97,114,116,115, + 119,105,116,104,218,27,95,67,65,83,69,95,73,78,83,69, + 78,83,73,84,73,86,69,95,80,76,65,84,70,79,82,77, + 83,218,35,95,67,65,83,69,95,73,78,83,69,78,83,73, + 84,73,86,69,95,80,76,65,84,70,79,82,77,83,95,83, + 84,82,95,75,69,89,41,1,114,5,0,0,0,114,2,0, + 0,0,41,1,114,3,0,0,0,114,4,0,0,0,218,16, + 95,109,97,107,101,95,114,101,108,97,120,95,99,97,115,101, + 29,0,0,0,115,14,0,0,0,0,1,12,1,12,1,6, + 2,4,2,14,4,8,3,114,11,0,0,0,99,1,0,0, + 0,0,0,0,0,1,0,0,0,4,0,0,0,67,0,0, + 0,115,20,0,0,0,116,0,124,0,131,1,100,1,64,0, + 160,1,100,2,100,3,161,2,83,0,41,4,122,42,67,111, + 110,118,101,114,116,32,97,32,51,50,45,98,105,116,32,105, + 110,116,101,103,101,114,32,116,111,32,108,105,116,116,108,101, + 45,101,110,100,105,97,110,46,108,3,0,0,0,255,127,255, + 127,3,0,233,4,0,0,0,218,6,108,105,116,116,108,101, + 41,2,218,3,105,110,116,218,8,116,111,95,98,121,116,101, + 115,41,1,218,1,120,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,218,12,95,112,97,99,107,95,117,105,110, + 116,51,50,46,0,0,0,115,2,0,0,0,0,2,114,17, + 0,0,0,99,1,0,0,0,0,0,0,0,1,0,0,0, + 4,0,0,0,67,0,0,0,115,28,0,0,0,116,0,124, + 0,131,1,100,1,107,2,115,16,116,1,130,1,116,2,160, + 3,124,0,100,2,161,2,83,0,41,3,122,47,67,111,110, + 118,101,114,116,32,52,32,98,121,116,101,115,32,105,110,32, + 108,105,116,116,108,101,45,101,110,100,105,97,110,32,116,111, + 32,97,110,32,105,110,116,101,103,101,114,46,114,12,0,0, + 0,114,13,0,0,0,41,4,218,3,108,101,110,218,14,65, + 115,115,101,114,116,105,111,110,69,114,114,111,114,114,14,0, + 0,0,218,10,102,114,111,109,95,98,121,116,101,115,41,1, + 218,4,100,97,116,97,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,218,14,95,117,110,112,97,99,107,95,117, + 105,110,116,51,50,51,0,0,0,115,4,0,0,0,0,2, + 16,1,114,22,0,0,0,99,1,0,0,0,0,0,0,0, + 1,0,0,0,4,0,0,0,67,0,0,0,115,28,0,0, + 0,116,0,124,0,131,1,100,1,107,2,115,16,116,1,130, + 1,116,2,160,3,124,0,100,2,161,2,83,0,41,3,122, + 47,67,111,110,118,101,114,116,32,50,32,98,121,116,101,115, + 32,105,110,32,108,105,116,116,108,101,45,101,110,100,105,97, + 110,32,116,111,32,97,110,32,105,110,116,101,103,101,114,46, + 233,2,0,0,0,114,13,0,0,0,41,4,114,18,0,0, + 0,114,19,0,0,0,114,14,0,0,0,114,20,0,0,0, + 41,1,114,21,0,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,14,95,117,110,112,97,99,107,95, + 117,105,110,116,49,54,56,0,0,0,115,4,0,0,0,0, + 2,16,1,114,24,0,0,0,99,0,0,0,0,0,0,0, + 0,1,0,0,0,4,0,0,0,71,0,0,0,115,20,0, + 0,0,116,0,160,1,100,1,100,2,132,0,124,0,68,0, + 131,1,161,1,83,0,41,3,122,31,82,101,112,108,97,99, + 101,109,101,110,116,32,102,111,114,32,111,115,46,112,97,116, + 104,46,106,111,105,110,40,41,46,99,1,0,0,0,0,0, + 0,0,2,0,0,0,5,0,0,0,83,0,0,0,115,26, + 0,0,0,103,0,124,0,93,18,125,1,124,1,114,22,124, + 1,160,0,116,1,161,1,145,2,113,4,83,0,114,2,0, + 0,0,41,2,218,6,114,115,116,114,105,112,218,15,112,97, + 116,104,95,115,101,112,97,114,97,116,111,114,115,41,2,218, + 2,46,48,218,4,112,97,114,116,114,2,0,0,0,114,2, + 0,0,0,114,4,0,0,0,218,10,60,108,105,115,116,99, + 111,109,112,62,64,0,0,0,115,6,0,0,0,6,1,2, + 0,4,255,122,30,95,112,97,116,104,95,106,111,105,110,46, + 60,108,111,99,97,108,115,62,46,60,108,105,115,116,99,111, + 109,112,62,41,2,218,8,112,97,116,104,95,115,101,112,218, + 4,106,111,105,110,41,1,218,10,112,97,116,104,95,112,97, + 114,116,115,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,218,10,95,112,97,116,104,95,106,111,105,110,62,0, + 0,0,115,6,0,0,0,0,2,10,1,2,255,114,33,0, + 0,0,99,1,0,0,0,0,0,0,0,5,0,0,0,5, + 0,0,0,67,0,0,0,115,96,0,0,0,116,0,116,1, + 131,1,100,1,107,2,114,36,124,0,160,2,116,3,161,1, + 92,3,125,1,125,2,125,3,124,1,124,3,102,2,83,0, + 116,4,124,0,131,1,68,0,93,42,125,4,124,4,116,1, + 107,6,114,44,124,0,106,5,124,4,100,1,100,2,141,2, + 92,2,125,1,125,3,124,1,124,3,102,2,2,0,1,0, + 83,0,113,44,100,3,124,0,102,2,83,0,41,4,122,32, 82,101,112,108,97,99,101,109,101,110,116,32,102,111,114,32, - 111,115,46,112,97,116,104,46,105,115,100,105,114,46,105,0, - 64,0,0,41,3,114,1,0,0,0,218,6,103,101,116,99, - 119,100,114,43,0,0,0,41,1,114,35,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,218,11,95, - 112,97,116,104,95,105,115,100,105,114,98,0,0,0,115,6, - 0,0,0,0,2,4,1,8,1,114,46,0,0,0,99,1, - 0,0,0,0,0,0,0,1,0,0,0,3,0,0,0,67, - 0,0,0,115,26,0,0,0,124,0,160,0,116,1,161,1, - 112,24,124,0,100,1,100,2,133,2,25,0,116,2,107,6, - 83,0,41,3,122,142,82,101,112,108,97,99,101,109,101,110, - 116,32,102,111,114,32,111,115,46,112,97,116,104,46,105,115, - 97,98,115,46,10,10,32,32,32,32,67,111,110,115,105,100, - 101,114,115,32,97,32,87,105,110,100,111,119,115,32,100,114, - 105,118,101,45,114,101,108,97,116,105,118,101,32,112,97,116, - 104,32,40,110,111,32,100,114,105,118,101,44,32,98,117,116, - 32,115,116,97,114,116,115,32,119,105,116,104,32,115,108,97, - 115,104,41,32,116,111,10,32,32,32,32,115,116,105,108,108, - 32,98,101,32,34,97,98,115,111,108,117,116,101,34,46,10, - 32,32,32,32,114,29,0,0,0,233,3,0,0,0,41,3, - 114,8,0,0,0,114,21,0,0,0,218,20,95,112,97,116, - 104,115,101,112,115,95,119,105,116,104,95,99,111,108,111,110, - 41,1,114,35,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,218,11,95,112,97,116,104,95,105,115, - 97,98,115,105,0,0,0,115,2,0,0,0,0,6,114,49, - 0,0,0,105,182,1,0,0,99,3,0,0,0,0,0,0, - 0,6,0,0,0,11,0,0,0,67,0,0,0,115,162,0, - 0,0,100,1,160,0,124,0,116,1,124,0,131,1,161,2, - 125,3,116,2,160,3,124,3,116,2,106,4,116,2,106,5, - 66,0,116,2,106,6,66,0,124,2,100,2,64,0,161,3, - 125,4,122,50,116,7,160,8,124,4,100,3,161,2,143,16, - 125,5,124,5,160,9,124,1,161,1,1,0,87,0,53,0, - 81,0,82,0,88,0,116,2,160,10,124,3,124,0,161,2, - 1,0,87,0,110,58,4,0,116,11,107,10,114,156,1,0, - 1,0,1,0,122,14,116,2,160,12,124,3,161,1,1,0, - 87,0,110,20,4,0,116,11,107,10,114,148,1,0,1,0, - 1,0,89,0,110,2,88,0,130,0,89,0,110,2,88,0, - 100,4,83,0,41,5,122,162,66,101,115,116,45,101,102,102, - 111,114,116,32,102,117,110,99,116,105,111,110,32,116,111,32, - 119,114,105,116,101,32,100,97,116,97,32,116,111,32,97,32, - 112,97,116,104,32,97,116,111,109,105,99,97,108,108,121,46, - 10,32,32,32,32,66,101,32,112,114,101,112,97,114,101,100, - 32,116,111,32,104,97,110,100,108,101,32,97,32,70,105,108, - 101,69,120,105,115,116,115,69,114,114,111,114,32,105,102,32, - 99,111,110,99,117,114,114,101,110,116,32,119,114,105,116,105, - 110,103,32,111,102,32,116,104,101,10,32,32,32,32,116,101, - 109,112,111,114,97,114,121,32,102,105,108,101,32,105,115,32, - 97,116,116,101,109,112,116,101,100,46,122,5,123,125,46,123, - 125,105,182,1,0,0,90,2,119,98,78,41,13,218,6,102, - 111,114,109,97,116,218,2,105,100,114,1,0,0,0,90,4, - 111,112,101,110,90,6,79,95,69,88,67,76,90,7,79,95, - 67,82,69,65,84,90,8,79,95,87,82,79,78,76,89,218, - 3,95,105,111,218,6,70,105,108,101,73,79,218,5,119,114, - 105,116,101,218,7,114,101,112,108,97,99,101,114,40,0,0, - 0,90,6,117,110,108,105,110,107,41,6,114,35,0,0,0, - 218,4,100,97,116,97,114,42,0,0,0,90,8,112,97,116, - 104,95,116,109,112,90,2,102,100,218,4,102,105,108,101,114, - 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,13, - 95,119,114,105,116,101,95,97,116,111,109,105,99,114,0,0, - 0,115,30,0,0,0,0,5,16,1,6,1,16,0,6,255, - 4,2,2,3,14,1,20,1,16,1,14,1,2,1,14,1, - 14,1,6,1,114,58,0,0,0,105,73,13,0,0,233,2, - 0,0,0,114,13,0,0,0,115,2,0,0,0,13,10,90, - 11,95,95,112,121,99,97,99,104,101,95,95,122,4,111,112, - 116,45,122,3,46,112,121,122,4,46,112,121,99,78,41,1, - 218,12,111,112,116,105,109,105,122,97,116,105,111,110,99,2, - 0,0,0,1,0,0,0,12,0,0,0,5,0,0,0,67, - 0,0,0,115,88,1,0,0,124,1,100,1,107,9,114,52, - 116,0,160,1,100,2,116,2,161,2,1,0,124,2,100,1, - 107,9,114,40,100,3,125,3,116,3,124,3,131,1,130,1, - 124,1,114,48,100,4,110,2,100,5,125,2,116,4,160,5, - 124,0,161,1,125,0,116,6,124,0,131,1,92,2,125,4, - 125,5,124,5,160,7,100,6,161,1,92,3,125,6,125,7, - 125,8,116,8,106,9,106,10,125,9,124,9,100,1,107,8, - 114,114,116,11,100,7,131,1,130,1,100,4,160,12,124,6, - 114,126,124,6,110,2,124,8,124,7,124,9,103,3,161,1, - 125,10,124,2,100,1,107,8,114,172,116,8,106,13,106,14, - 100,8,107,2,114,164,100,4,125,2,110,8,116,8,106,13, - 106,14,125,2,116,15,124,2,131,1,125,2,124,2,100,4, - 107,3,114,224,124,2,160,16,161,0,115,210,116,17,100,9, - 160,18,124,2,161,1,131,1,130,1,100,10,160,18,124,10, - 116,19,124,2,161,3,125,10,124,10,116,20,100,8,25,0, - 23,0,125,11,116,8,106,21,100,1,107,9,144,1,114,76, - 116,22,124,4,131,1,144,1,115,16,116,23,116,4,160,24, - 161,0,124,4,131,2,125,4,124,4,100,5,25,0,100,11, - 107,2,144,1,114,56,124,4,100,8,25,0,116,25,107,7, - 144,1,114,56,124,4,100,12,100,1,133,2,25,0,125,4, - 116,23,116,8,106,21,124,4,160,26,116,25,161,1,124,11, - 131,3,83,0,116,23,124,4,116,27,124,11,131,3,83,0, - 41,13,97,254,2,0,0,71,105,118,101,110,32,116,104,101, - 32,112,97,116,104,32,116,111,32,97,32,46,112,121,32,102, - 105,108,101,44,32,114,101,116,117,114,110,32,116,104,101,32, - 112,97,116,104,32,116,111,32,105,116,115,32,46,112,121,99, - 32,102,105,108,101,46,10,10,32,32,32,32,84,104,101,32, - 46,112,121,32,102,105,108,101,32,100,111,101,115,32,110,111, - 116,32,110,101,101,100,32,116,111,32,101,120,105,115,116,59, - 32,116,104,105,115,32,115,105,109,112,108,121,32,114,101,116, - 117,114,110,115,32,116,104,101,32,112,97,116,104,32,116,111, - 32,116,104,101,10,32,32,32,32,46,112,121,99,32,102,105, - 108,101,32,99,97,108,99,117,108,97,116,101,100,32,97,115, - 32,105,102,32,116,104,101,32,46,112,121,32,102,105,108,101, - 32,119,101,114,101,32,105,109,112,111,114,116,101,100,46,10, - 10,32,32,32,32,84,104,101,32,39,111,112,116,105,109,105, - 122,97,116,105,111,110,39,32,112,97,114,97,109,101,116,101, - 114,32,99,111,110,116,114,111,108,115,32,116,104,101,32,112, - 114,101,115,117,109,101,100,32,111,112,116,105,109,105,122,97, - 116,105,111,110,32,108,101,118,101,108,32,111,102,10,32,32, - 32,32,116,104,101,32,98,121,116,101,99,111,100,101,32,102, - 105,108,101,46,32,73,102,32,39,111,112,116,105,109,105,122, - 97,116,105,111,110,39,32,105,115,32,110,111,116,32,78,111, - 110,101,44,32,116,104,101,32,115,116,114,105,110,103,32,114, - 101,112,114,101,115,101,110,116,97,116,105,111,110,10,32,32, - 32,32,111,102,32,116,104,101,32,97,114,103,117,109,101,110, - 116,32,105,115,32,116,97,107,101,110,32,97,110,100,32,118, - 101,114,105,102,105,101,100,32,116,111,32,98,101,32,97,108, - 112,104,97,110,117,109,101,114,105,99,32,40,101,108,115,101, - 32,86,97,108,117,101,69,114,114,111,114,10,32,32,32,32, - 105,115,32,114,97,105,115,101,100,41,46,10,10,32,32,32, - 32,84,104,101,32,100,101,98,117,103,95,111,118,101,114,114, - 105,100,101,32,112,97,114,97,109,101,116,101,114,32,105,115, - 32,100,101,112,114,101,99,97,116,101,100,46,32,73,102,32, - 100,101,98,117,103,95,111,118,101,114,114,105,100,101,32,105, - 115,32,110,111,116,32,78,111,110,101,44,10,32,32,32,32, - 97,32,84,114,117,101,32,118,97,108,117,101,32,105,115,32, - 116,104,101,32,115,97,109,101,32,97,115,32,115,101,116,116, - 105,110,103,32,39,111,112,116,105,109,105,122,97,116,105,111, - 110,39,32,116,111,32,116,104,101,32,101,109,112,116,121,32, - 115,116,114,105,110,103,10,32,32,32,32,119,104,105,108,101, - 32,97,32,70,97,108,115,101,32,118,97,108,117,101,32,105, - 115,32,101,113,117,105,118,97,108,101,110,116,32,116,111,32, - 115,101,116,116,105,110,103,32,39,111,112,116,105,109,105,122, - 97,116,105,111,110,39,32,116,111,32,39,49,39,46,10,10, - 32,32,32,32,73,102,32,115,121,115,46,105,109,112,108,101, - 109,101,110,116,97,116,105,111,110,46,99,97,99,104,101,95, - 116,97,103,32,105,115,32,78,111,110,101,32,116,104,101,110, - 32,78,111,116,73,109,112,108,101,109,101,110,116,101,100,69, - 114,114,111,114,32,105,115,32,114,97,105,115,101,100,46,10, - 10,32,32,32,32,78,122,70,116,104,101,32,100,101,98,117, - 103,95,111,118,101,114,114,105,100,101,32,112,97,114,97,109, - 101,116,101,114,32,105,115,32,100,101,112,114,101,99,97,116, - 101,100,59,32,117,115,101,32,39,111,112,116,105,109,105,122, - 97,116,105,111,110,39,32,105,110,115,116,101,97,100,122,50, - 100,101,98,117,103,95,111,118,101,114,114,105,100,101,32,111, - 114,32,111,112,116,105,109,105,122,97,116,105,111,110,32,109, - 117,115,116,32,98,101,32,115,101,116,32,116,111,32,78,111, - 110,101,114,30,0,0,0,114,29,0,0,0,218,1,46,122, - 36,115,121,115,46,105,109,112,108,101,109,101,110,116,97,116, - 105,111,110,46,99,97,99,104,101,95,116,97,103,32,105,115, - 32,78,111,110,101,233,0,0,0,0,122,24,123,33,114,125, - 32,105,115,32,110,111,116,32,97,108,112,104,97,110,117,109, - 101,114,105,99,122,7,123,125,46,123,125,123,125,250,1,58, - 114,59,0,0,0,41,28,218,9,95,119,97,114,110,105,110, - 103,115,218,4,119,97,114,110,218,18,68,101,112,114,101,99, - 97,116,105,111,110,87,97,114,110,105,110,103,218,9,84,121, - 112,101,69,114,114,111,114,114,1,0,0,0,218,6,102,115, - 112,97,116,104,114,38,0,0,0,114,32,0,0,0,114,6, - 0,0,0,218,14,105,109,112,108,101,109,101,110,116,97,116, - 105,111,110,218,9,99,97,99,104,101,95,116,97,103,218,19, - 78,111,116,73,109,112,108,101,109,101,110,116,101,100,69,114, - 114,111,114,114,26,0,0,0,218,5,102,108,97,103,115,218, - 8,111,112,116,105,109,105,122,101,218,3,115,116,114,218,7, - 105,115,97,108,110,117,109,218,10,86,97,108,117,101,69,114, - 114,111,114,114,50,0,0,0,218,4,95,79,80,84,218,17, - 66,89,84,69,67,79,68,69,95,83,85,70,70,73,88,69, - 83,218,14,112,121,99,97,99,104,101,95,112,114,101,102,105, - 120,114,49,0,0,0,114,28,0,0,0,114,45,0,0,0, - 114,21,0,0,0,218,6,108,115,116,114,105,112,218,8,95, - 80,89,67,65,67,72,69,41,12,114,35,0,0,0,90,14, - 100,101,98,117,103,95,111,118,101,114,114,105,100,101,114,60, - 0,0,0,218,7,109,101,115,115,97,103,101,218,4,104,101, - 97,100,114,37,0,0,0,90,4,98,97,115,101,218,3,115, - 101,112,218,4,114,101,115,116,90,3,116,97,103,90,15,97, - 108,109,111,115,116,95,102,105,108,101,110,97,109,101,218,8, - 102,105,108,101,110,97,109,101,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,218,17,99,97,99,104,101,95,102, - 114,111,109,95,115,111,117,114,99,101,26,1,0,0,115,72, - 0,0,0,0,18,8,1,6,1,2,255,4,2,8,1,4, - 1,8,1,12,1,10,1,12,1,16,1,8,1,8,1,8, - 1,24,1,8,1,12,1,6,2,8,1,8,1,8,1,8, - 1,14,1,14,1,12,1,12,9,10,1,14,5,28,1,12, - 4,2,1,4,1,8,1,2,253,4,5,114,87,0,0,0, - 99,1,0,0,0,0,0,0,0,10,0,0,0,5,0,0, - 0,67,0,0,0,115,46,1,0,0,116,0,106,1,106,2, - 100,1,107,8,114,20,116,3,100,2,131,1,130,1,116,4, - 160,5,124,0,161,1,125,0,116,6,124,0,131,1,92,2, - 125,1,125,2,100,3,125,3,116,0,106,7,100,1,107,9, - 114,102,116,0,106,7,160,8,116,9,161,1,125,4,124,1, - 160,10,124,4,116,11,23,0,161,1,114,102,124,1,116,12, - 124,4,131,1,100,1,133,2,25,0,125,1,100,4,125,3, - 124,3,115,144,116,6,124,1,131,1,92,2,125,1,125,5, - 124,5,116,13,107,3,114,144,116,14,116,13,155,0,100,5, - 124,0,155,2,157,3,131,1,130,1,124,2,160,15,100,6, - 161,1,125,6,124,6,100,7,107,7,114,178,116,14,100,8, - 124,2,155,2,157,2,131,1,130,1,110,92,124,6,100,9, - 107,2,144,1,114,14,124,2,160,16,100,6,100,10,161,2, - 100,11,25,0,125,7,124,7,160,10,116,17,161,1,115,228, - 116,14,100,12,116,17,155,2,157,2,131,1,130,1,124,7, - 116,12,116,17,131,1,100,1,133,2,25,0,125,8,124,8, - 160,18,161,0,144,1,115,14,116,14,100,13,124,7,155,2, - 100,14,157,3,131,1,130,1,124,2,160,19,100,6,161,1, - 100,15,25,0,125,9,116,20,124,1,124,9,116,21,100,15, - 25,0,23,0,131,2,83,0,41,16,97,110,1,0,0,71, - 105,118,101,110,32,116,104,101,32,112,97,116,104,32,116,111, - 32,97,32,46,112,121,99,46,32,102,105,108,101,44,32,114, - 101,116,117,114,110,32,116,104,101,32,112,97,116,104,32,116, - 111,32,105,116,115,32,46,112,121,32,102,105,108,101,46,10, - 10,32,32,32,32,84,104,101,32,46,112,121,99,32,102,105, - 108,101,32,100,111,101,115,32,110,111,116,32,110,101,101,100, - 32,116,111,32,101,120,105,115,116,59,32,116,104,105,115,32, - 115,105,109,112,108,121,32,114,101,116,117,114,110,115,32,116, - 104,101,32,112,97,116,104,32,116,111,10,32,32,32,32,116, - 104,101,32,46,112,121,32,102,105,108,101,32,99,97,108,99, - 117,108,97,116,101,100,32,116,111,32,99,111,114,114,101,115, - 112,111,110,100,32,116,111,32,116,104,101,32,46,112,121,99, - 32,102,105,108,101,46,32,32,73,102,32,112,97,116,104,32, - 100,111,101,115,10,32,32,32,32,110,111,116,32,99,111,110, - 102,111,114,109,32,116,111,32,80,69,80,32,51,49,52,55, - 47,52,56,56,32,102,111,114,109,97,116,44,32,86,97,108, - 117,101,69,114,114,111,114,32,119,105,108,108,32,98,101,32, - 114,97,105,115,101,100,46,32,73,102,10,32,32,32,32,115, - 121,115,46,105,109,112,108,101,109,101,110,116,97,116,105,111, - 110,46,99,97,99,104,101,95,116,97,103,32,105,115,32,78, - 111,110,101,32,116,104,101,110,32,78,111,116,73,109,112,108, - 101,109,101,110,116,101,100,69,114,114,111,114,32,105,115,32, - 114,97,105,115,101,100,46,10,10,32,32,32,32,78,122,36, - 115,121,115,46,105,109,112,108,101,109,101,110,116,97,116,105, - 111,110,46,99,97,99,104,101,95,116,97,103,32,105,115,32, - 78,111,110,101,70,84,122,31,32,110,111,116,32,98,111,116, - 116,111,109,45,108,101,118,101,108,32,100,105,114,101,99,116, - 111,114,121,32,105,110,32,114,61,0,0,0,62,2,0,0, - 0,114,59,0,0,0,114,47,0,0,0,122,29,101,120,112, - 101,99,116,101,100,32,111,110,108,121,32,50,32,111,114,32, - 51,32,100,111,116,115,32,105,110,32,114,47,0,0,0,114, - 59,0,0,0,233,254,255,255,255,122,53,111,112,116,105,109, - 105,122,97,116,105,111,110,32,112,111,114,116,105,111,110,32, - 111,102,32,102,105,108,101,110,97,109,101,32,100,111,101,115, - 32,110,111,116,32,115,116,97,114,116,32,119,105,116,104,32, - 122,19,111,112,116,105,109,105,122,97,116,105,111,110,32,108, - 101,118,101,108,32,122,29,32,105,115,32,110,111,116,32,97, - 110,32,97,108,112,104,97,110,117,109,101,114,105,99,32,118, - 97,108,117,101,114,62,0,0,0,41,22,114,6,0,0,0, - 114,69,0,0,0,114,70,0,0,0,114,71,0,0,0,114, - 1,0,0,0,114,68,0,0,0,114,38,0,0,0,114,79, - 0,0,0,114,20,0,0,0,114,21,0,0,0,114,8,0, - 0,0,114,25,0,0,0,114,31,0,0,0,114,81,0,0, - 0,114,76,0,0,0,218,5,99,111,117,110,116,114,34,0, - 0,0,114,77,0,0,0,114,75,0,0,0,218,9,112,97, - 114,116,105,116,105,111,110,114,28,0,0,0,218,15,83,79, - 85,82,67,69,95,83,85,70,70,73,88,69,83,41,10,114, - 35,0,0,0,114,83,0,0,0,90,16,112,121,99,97,99, - 104,101,95,102,105,108,101,110,97,109,101,90,23,102,111,117, - 110,100,95,105,110,95,112,121,99,97,99,104,101,95,112,114, - 101,102,105,120,90,13,115,116,114,105,112,112,101,100,95,112, - 97,116,104,90,7,112,121,99,97,99,104,101,90,9,100,111, - 116,95,99,111,117,110,116,114,60,0,0,0,90,9,111,112, - 116,95,108,101,118,101,108,90,13,98,97,115,101,95,102,105, - 108,101,110,97,109,101,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,218,17,115,111,117,114,99,101,95,102,114, - 111,109,95,99,97,99,104,101,97,1,0,0,115,52,0,0, - 0,0,9,12,1,8,1,10,1,12,1,4,1,10,1,12, - 1,14,1,16,1,4,1,4,1,12,1,8,1,18,2,10, - 1,8,1,16,1,10,1,16,1,10,1,14,2,16,1,10, - 1,16,2,14,1,114,92,0,0,0,99,1,0,0,0,0, - 0,0,0,5,0,0,0,9,0,0,0,67,0,0,0,115, - 126,0,0,0,116,0,124,0,131,1,100,1,107,2,114,16, - 100,2,83,0,124,0,160,1,100,3,161,1,92,3,125,1, - 125,2,125,3,124,1,114,56,124,3,160,2,161,0,100,4, - 100,5,133,2,25,0,100,6,107,3,114,60,124,0,83,0, - 122,12,116,3,124,0,131,1,125,4,87,0,110,36,4,0, - 116,4,116,5,102,2,107,10,114,108,1,0,1,0,1,0, - 124,0,100,2,100,5,133,2,25,0,125,4,89,0,110,2, - 88,0,116,6,124,4,131,1,114,122,124,4,83,0,124,0, - 83,0,41,7,122,188,67,111,110,118,101,114,116,32,97,32, - 98,121,116,101,99,111,100,101,32,102,105,108,101,32,112,97, - 116,104,32,116,111,32,97,32,115,111,117,114,99,101,32,112, - 97,116,104,32,40,105,102,32,112,111,115,115,105,98,108,101, - 41,46,10,10,32,32,32,32,84,104,105,115,32,102,117,110, - 99,116,105,111,110,32,101,120,105,115,116,115,32,112,117,114, - 101,108,121,32,102,111,114,32,98,97,99,107,119,97,114,100, - 115,45,99,111,109,112,97,116,105,98,105,108,105,116,121,32, - 102,111,114,10,32,32,32,32,80,121,73,109,112,111,114,116, - 95,69,120,101,99,67,111,100,101,77,111,100,117,108,101,87, - 105,116,104,70,105,108,101,110,97,109,101,115,40,41,32,105, - 110,32,116,104,101,32,67,32,65,80,73,46,10,10,32,32, - 32,32,114,62,0,0,0,78,114,61,0,0,0,233,253,255, - 255,255,233,255,255,255,255,90,2,112,121,41,7,114,31,0, - 0,0,114,32,0,0,0,218,5,108,111,119,101,114,114,92, - 0,0,0,114,71,0,0,0,114,76,0,0,0,114,44,0, - 0,0,41,5,218,13,98,121,116,101,99,111,100,101,95,112, - 97,116,104,114,85,0,0,0,114,36,0,0,0,90,9,101, - 120,116,101,110,115,105,111,110,218,11,115,111,117,114,99,101, - 95,112,97,116,104,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,218,15,95,103,101,116,95,115,111,117,114,99, - 101,102,105,108,101,137,1,0,0,115,20,0,0,0,0,7, - 12,1,4,1,16,1,24,1,4,1,2,1,12,1,18,1, - 18,1,114,98,0,0,0,99,1,0,0,0,0,0,0,0, - 1,0,0,0,8,0,0,0,67,0,0,0,115,74,0,0, - 0,124,0,160,0,116,1,116,2,131,1,161,1,114,48,122, - 10,116,3,124,0,131,1,87,0,83,0,4,0,116,4,107, - 10,114,44,1,0,1,0,1,0,89,0,113,70,88,0,110, - 22,124,0,160,0,116,1,116,5,131,1,161,1,114,66,124, - 0,83,0,100,0,83,0,100,0,83,0,41,1,78,41,6, - 218,8,101,110,100,115,119,105,116,104,218,5,116,117,112,108, - 101,114,91,0,0,0,114,87,0,0,0,114,71,0,0,0, - 114,78,0,0,0,41,1,114,86,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,218,11,95,103,101, - 116,95,99,97,99,104,101,100,156,1,0,0,115,16,0,0, - 0,0,1,14,1,2,1,10,1,14,1,8,1,14,1,4, - 2,114,101,0,0,0,99,1,0,0,0,0,0,0,0,2, - 0,0,0,8,0,0,0,67,0,0,0,115,52,0,0,0, - 122,14,116,0,124,0,131,1,106,1,125,1,87,0,110,24, - 4,0,116,2,107,10,114,38,1,0,1,0,1,0,100,1, - 125,1,89,0,110,2,88,0,124,1,100,2,79,0,125,1, - 124,1,83,0,41,3,122,51,67,97,108,99,117,108,97,116, - 101,32,116,104,101,32,109,111,100,101,32,112,101,114,109,105, - 115,115,105,111,110,115,32,102,111,114,32,97,32,98,121,116, - 101,99,111,100,101,32,102,105,108,101,46,105,182,1,0,0, - 233,128,0,0,0,41,3,114,39,0,0,0,114,41,0,0, - 0,114,40,0,0,0,41,2,114,35,0,0,0,114,42,0, - 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, - 0,218,10,95,99,97,108,99,95,109,111,100,101,168,1,0, - 0,115,12,0,0,0,0,2,2,1,14,1,14,1,10,3, - 8,1,114,103,0,0,0,99,1,0,0,0,0,0,0,0, - 3,0,0,0,8,0,0,0,3,0,0,0,115,68,0,0, - 0,100,6,135,0,102,1,100,2,100,3,132,9,125,1,122, - 10,116,0,106,1,125,2,87,0,110,28,4,0,116,2,107, - 10,114,52,1,0,1,0,1,0,100,4,100,5,132,0,125, - 2,89,0,110,2,88,0,124,2,124,1,136,0,131,2,1, - 0,124,1,83,0,41,7,122,252,68,101,99,111,114,97,116, - 111,114,32,116,111,32,118,101,114,105,102,121,32,116,104,97, - 116,32,116,104,101,32,109,111,100,117,108,101,32,98,101,105, - 110,103,32,114,101,113,117,101,115,116,101,100,32,109,97,116, - 99,104,101,115,32,116,104,101,32,111,110,101,32,116,104,101, - 10,32,32,32,32,108,111,97,100,101,114,32,99,97,110,32, - 104,97,110,100,108,101,46,10,10,32,32,32,32,84,104,101, - 32,102,105,114,115,116,32,97,114,103,117,109,101,110,116,32, - 40,115,101,108,102,41,32,109,117,115,116,32,100,101,102,105, - 110,101,32,95,110,97,109,101,32,119,104,105,99,104,32,116, - 104,101,32,115,101,99,111,110,100,32,97,114,103,117,109,101, - 110,116,32,105,115,10,32,32,32,32,99,111,109,112,97,114, - 101,100,32,97,103,97,105,110,115,116,46,32,73,102,32,116, - 104,101,32,99,111,109,112,97,114,105,115,111,110,32,102,97, - 105,108,115,32,116,104,101,110,32,73,109,112,111,114,116,69, - 114,114,111,114,32,105,115,32,114,97,105,115,101,100,46,10, - 10,32,32,32,32,78,99,2,0,0,0,0,0,0,0,4, - 0,0,0,4,0,0,0,31,0,0,0,115,66,0,0,0, - 124,1,100,0,107,8,114,16,124,0,106,0,125,1,110,32, - 124,0,106,0,124,1,107,3,114,48,116,1,100,1,124,0, - 106,0,124,1,102,2,22,0,124,1,100,2,141,2,130,1, - 136,0,124,0,124,1,102,2,124,2,158,2,124,3,142,1, - 83,0,41,3,78,122,30,108,111,97,100,101,114,32,102,111, - 114,32,37,115,32,99,97,110,110,111,116,32,104,97,110,100, - 108,101,32,37,115,41,1,218,4,110,97,109,101,41,2,114, - 104,0,0,0,218,11,73,109,112,111,114,116,69,114,114,111, - 114,41,4,218,4,115,101,108,102,114,104,0,0,0,218,4, - 97,114,103,115,90,6,107,119,97,114,103,115,41,1,218,6, - 109,101,116,104,111,100,114,2,0,0,0,114,4,0,0,0, - 218,19,95,99,104,101,99,107,95,110,97,109,101,95,119,114, - 97,112,112,101,114,188,1,0,0,115,18,0,0,0,0,1, - 8,1,8,1,10,1,4,1,8,255,2,1,2,255,6,2, - 122,40,95,99,104,101,99,107,95,110,97,109,101,46,60,108, - 111,99,97,108,115,62,46,95,99,104,101,99,107,95,110,97, - 109,101,95,119,114,97,112,112,101,114,99,2,0,0,0,0, - 0,0,0,3,0,0,0,7,0,0,0,83,0,0,0,115, - 56,0,0,0,100,1,68,0,93,32,125,2,116,0,124,1, - 124,2,131,2,114,4,116,1,124,0,124,2,116,2,124,1, - 124,2,131,2,131,3,1,0,113,4,124,0,106,3,160,4, - 124,1,106,3,161,1,1,0,100,0,83,0,41,2,78,41, - 4,218,10,95,95,109,111,100,117,108,101,95,95,218,8,95, - 95,110,97,109,101,95,95,218,12,95,95,113,117,97,108,110, - 97,109,101,95,95,218,7,95,95,100,111,99,95,95,41,5, - 218,7,104,97,115,97,116,116,114,218,7,115,101,116,97,116, - 116,114,218,7,103,101,116,97,116,116,114,218,8,95,95,100, - 105,99,116,95,95,218,6,117,112,100,97,116,101,41,3,90, - 3,110,101,119,90,3,111,108,100,114,55,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,218,5,95, - 119,114,97,112,199,1,0,0,115,8,0,0,0,0,1,8, - 1,10,1,20,1,122,26,95,99,104,101,99,107,95,110,97, - 109,101,46,60,108,111,99,97,108,115,62,46,95,119,114,97, - 112,41,1,78,41,3,218,10,95,98,111,111,116,115,116,114, - 97,112,114,119,0,0,0,218,9,78,97,109,101,69,114,114, - 111,114,41,3,114,108,0,0,0,114,109,0,0,0,114,119, - 0,0,0,114,2,0,0,0,41,1,114,108,0,0,0,114, - 4,0,0,0,218,11,95,99,104,101,99,107,95,110,97,109, - 101,180,1,0,0,115,14,0,0,0,0,8,14,7,2,1, - 10,1,14,2,14,5,10,1,114,122,0,0,0,99,2,0, - 0,0,0,0,0,0,5,0,0,0,6,0,0,0,67,0, - 0,0,115,60,0,0,0,124,0,160,0,124,1,161,1,92, - 2,125,2,125,3,124,2,100,1,107,8,114,56,116,1,124, - 3,131,1,114,56,100,2,125,4,116,2,160,3,124,4,160, - 4,124,3,100,3,25,0,161,1,116,5,161,2,1,0,124, - 2,83,0,41,4,122,155,84,114,121,32,116,111,32,102,105, - 110,100,32,97,32,108,111,97,100,101,114,32,102,111,114,32, - 116,104,101,32,115,112,101,99,105,102,105,101,100,32,109,111, - 100,117,108,101,32,98,121,32,100,101,108,101,103,97,116,105, - 110,103,32,116,111,10,32,32,32,32,115,101,108,102,46,102, - 105,110,100,95,108,111,97,100,101,114,40,41,46,10,10,32, - 32,32,32,84,104,105,115,32,109,101,116,104,111,100,32,105, - 115,32,100,101,112,114,101,99,97,116,101,100,32,105,110,32, - 102,97,118,111,114,32,111,102,32,102,105,110,100,101,114,46, - 102,105,110,100,95,115,112,101,99,40,41,46,10,10,32,32, - 32,32,78,122,44,78,111,116,32,105,109,112,111,114,116,105, - 110,103,32,100,105,114,101,99,116,111,114,121,32,123,125,58, - 32,109,105,115,115,105,110,103,32,95,95,105,110,105,116,95, - 95,114,62,0,0,0,41,6,218,11,102,105,110,100,95,108, - 111,97,100,101,114,114,31,0,0,0,114,64,0,0,0,114, - 65,0,0,0,114,50,0,0,0,218,13,73,109,112,111,114, - 116,87,97,114,110,105,110,103,41,5,114,106,0,0,0,218, - 8,102,117,108,108,110,97,109,101,218,6,108,111,97,100,101, - 114,218,8,112,111,114,116,105,111,110,115,218,3,109,115,103, + 111,115,46,112,97,116,104,46,115,112,108,105,116,40,41,46, + 233,1,0,0,0,41,1,90,8,109,97,120,115,112,108,105, + 116,218,0,41,6,114,18,0,0,0,114,26,0,0,0,218, + 10,114,112,97,114,116,105,116,105,111,110,114,30,0,0,0, + 218,8,114,101,118,101,114,115,101,100,218,6,114,115,112,108, + 105,116,41,5,218,4,112,97,116,104,90,5,102,114,111,110, + 116,218,1,95,218,4,116,97,105,108,114,16,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,11, + 95,112,97,116,104,95,115,112,108,105,116,68,0,0,0,115, + 16,0,0,0,0,2,12,1,16,1,8,1,12,1,8,1, + 18,1,14,1,114,42,0,0,0,99,1,0,0,0,0,0, + 0,0,1,0,0,0,3,0,0,0,67,0,0,0,115,10, + 0,0,0,116,0,160,1,124,0,161,1,83,0,41,1,122, + 126,83,116,97,116,32,116,104,101,32,112,97,116,104,46,10, + 10,32,32,32,32,77,97,100,101,32,97,32,115,101,112,97, + 114,97,116,101,32,102,117,110,99,116,105,111,110,32,116,111, + 32,109,97,107,101,32,105,116,32,101,97,115,105,101,114,32, + 116,111,32,111,118,101,114,114,105,100,101,32,105,110,32,101, + 120,112,101,114,105,109,101,110,116,115,10,32,32,32,32,40, + 101,46,103,46,32,99,97,99,104,101,32,115,116,97,116,32, + 114,101,115,117,108,116,115,41,46,10,10,32,32,32,32,41, + 2,114,1,0,0,0,90,4,115,116,97,116,41,1,114,39, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,218,10,95,112,97,116,104,95,115,116,97,116,80,0, + 0,0,115,2,0,0,0,0,7,114,43,0,0,0,99,2, + 0,0,0,0,0,0,0,3,0,0,0,8,0,0,0,67, + 0,0,0,115,50,0,0,0,122,12,116,0,124,0,131,1, + 125,2,87,0,110,22,4,0,116,1,107,10,114,34,1,0, + 1,0,1,0,89,0,100,1,83,0,88,0,124,2,106,2, + 100,2,64,0,124,1,107,2,83,0,41,3,122,49,84,101, + 115,116,32,119,104,101,116,104,101,114,32,116,104,101,32,112, + 97,116,104,32,105,115,32,116,104,101,32,115,112,101,99,105, + 102,105,101,100,32,109,111,100,101,32,116,121,112,101,46,70, + 105,0,240,0,0,41,3,114,43,0,0,0,218,7,79,83, + 69,114,114,111,114,218,7,115,116,95,109,111,100,101,41,3, + 114,39,0,0,0,218,4,109,111,100,101,90,9,115,116,97, + 116,95,105,110,102,111,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,218,18,95,112,97,116,104,95,105,115,95, + 109,111,100,101,95,116,121,112,101,90,0,0,0,115,10,0, + 0,0,0,2,2,1,12,1,14,1,8,1,114,47,0,0, + 0,99,1,0,0,0,0,0,0,0,1,0,0,0,3,0, + 0,0,67,0,0,0,115,10,0,0,0,116,0,124,0,100, + 1,131,2,83,0,41,2,122,31,82,101,112,108,97,99,101, + 109,101,110,116,32,102,111,114,32,111,115,46,112,97,116,104, + 46,105,115,102,105,108,101,46,105,0,128,0,0,41,1,114, + 47,0,0,0,41,1,114,39,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,218,12,95,112,97,116, + 104,95,105,115,102,105,108,101,99,0,0,0,115,2,0,0, + 0,0,2,114,48,0,0,0,99,1,0,0,0,0,0,0, + 0,1,0,0,0,3,0,0,0,67,0,0,0,115,22,0, + 0,0,124,0,115,12,116,0,160,1,161,0,125,0,116,2, + 124,0,100,1,131,2,83,0,41,2,122,30,82,101,112,108, + 97,99,101,109,101,110,116,32,102,111,114,32,111,115,46,112, + 97,116,104,46,105,115,100,105,114,46,105,0,64,0,0,41, + 3,114,1,0,0,0,218,6,103,101,116,99,119,100,114,47, + 0,0,0,41,1,114,39,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,218,11,95,112,97,116,104, + 95,105,115,100,105,114,104,0,0,0,115,6,0,0,0,0, + 2,4,1,8,1,114,50,0,0,0,99,1,0,0,0,0, + 0,0,0,1,0,0,0,3,0,0,0,67,0,0,0,115, + 26,0,0,0,124,0,160,0,116,1,161,1,112,24,124,0, + 100,1,100,2,133,2,25,0,116,2,107,6,83,0,41,3, + 122,142,82,101,112,108,97,99,101,109,101,110,116,32,102,111, + 114,32,111,115,46,112,97,116,104,46,105,115,97,98,115,46, + 10,10,32,32,32,32,67,111,110,115,105,100,101,114,115,32, + 97,32,87,105,110,100,111,119,115,32,100,114,105,118,101,45, + 114,101,108,97,116,105,118,101,32,112,97,116,104,32,40,110, + 111,32,100,114,105,118,101,44,32,98,117,116,32,115,116,97, + 114,116,115,32,119,105,116,104,32,115,108,97,115,104,41,32, + 116,111,10,32,32,32,32,115,116,105,108,108,32,98,101,32, + 34,97,98,115,111,108,117,116,101,34,46,10,32,32,32,32, + 114,34,0,0,0,233,3,0,0,0,41,3,114,8,0,0, + 0,114,26,0,0,0,218,20,95,112,97,116,104,115,101,112, + 115,95,119,105,116,104,95,99,111,108,111,110,41,1,114,39, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,218,11,95,112,97,116,104,95,105,115,97,98,115,111, + 0,0,0,115,2,0,0,0,0,6,114,53,0,0,0,105, + 182,1,0,0,99,3,0,0,0,0,0,0,0,6,0,0, + 0,11,0,0,0,67,0,0,0,115,162,0,0,0,100,1, + 160,0,124,0,116,1,124,0,131,1,161,2,125,3,116,2, + 160,3,124,3,116,2,106,4,116,2,106,5,66,0,116,2, + 106,6,66,0,124,2,100,2,64,0,161,3,125,4,122,50, + 116,7,160,8,124,4,100,3,161,2,143,16,125,5,124,5, + 160,9,124,1,161,1,1,0,87,0,53,0,81,0,82,0, + 88,0,116,2,160,10,124,3,124,0,161,2,1,0,87,0, + 110,58,4,0,116,11,107,10,114,156,1,0,1,0,1,0, + 122,14,116,2,160,12,124,3,161,1,1,0,87,0,110,20, + 4,0,116,11,107,10,114,148,1,0,1,0,1,0,89,0, + 110,2,88,0,130,0,89,0,110,2,88,0,100,4,83,0, + 41,5,122,162,66,101,115,116,45,101,102,102,111,114,116,32, + 102,117,110,99,116,105,111,110,32,116,111,32,119,114,105,116, + 101,32,100,97,116,97,32,116,111,32,97,32,112,97,116,104, + 32,97,116,111,109,105,99,97,108,108,121,46,10,32,32,32, + 32,66,101,32,112,114,101,112,97,114,101,100,32,116,111,32, + 104,97,110,100,108,101,32,97,32,70,105,108,101,69,120,105, + 115,116,115,69,114,114,111,114,32,105,102,32,99,111,110,99, + 117,114,114,101,110,116,32,119,114,105,116,105,110,103,32,111, + 102,32,116,104,101,10,32,32,32,32,116,101,109,112,111,114, + 97,114,121,32,102,105,108,101,32,105,115,32,97,116,116,101, + 109,112,116,101,100,46,122,5,123,125,46,123,125,105,182,1, + 0,0,90,2,119,98,78,41,13,218,6,102,111,114,109,97, + 116,218,2,105,100,114,1,0,0,0,90,4,111,112,101,110, + 90,6,79,95,69,88,67,76,90,7,79,95,67,82,69,65, + 84,90,8,79,95,87,82,79,78,76,89,218,3,95,105,111, + 218,6,70,105,108,101,73,79,218,5,119,114,105,116,101,218, + 7,114,101,112,108,97,99,101,114,44,0,0,0,90,6,117, + 110,108,105,110,107,41,6,114,39,0,0,0,114,21,0,0, + 0,114,46,0,0,0,90,8,112,97,116,104,95,116,109,112, + 90,2,102,100,218,4,102,105,108,101,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,218,13,95,119,114,105,116, + 101,95,97,116,111,109,105,99,120,0,0,0,115,30,0,0, + 0,0,5,16,1,6,1,16,0,6,255,4,2,2,3,14, + 1,20,1,16,1,14,1,2,1,14,1,14,1,6,1,114, + 61,0,0,0,105,73,13,0,0,114,23,0,0,0,114,13, + 0,0,0,115,2,0,0,0,13,10,90,11,95,95,112,121, + 99,97,99,104,101,95,95,122,4,111,112,116,45,122,3,46, + 112,121,122,4,46,112,121,99,78,41,1,218,12,111,112,116, + 105,109,105,122,97,116,105,111,110,99,2,0,0,0,1,0, + 0,0,12,0,0,0,5,0,0,0,67,0,0,0,115,88, + 1,0,0,124,1,100,1,107,9,114,52,116,0,160,1,100, + 2,116,2,161,2,1,0,124,2,100,1,107,9,114,40,100, + 3,125,3,116,3,124,3,131,1,130,1,124,1,114,48,100, + 4,110,2,100,5,125,2,116,4,160,5,124,0,161,1,125, + 0,116,6,124,0,131,1,92,2,125,4,125,5,124,5,160, + 7,100,6,161,1,92,3,125,6,125,7,125,8,116,8,106, + 9,106,10,125,9,124,9,100,1,107,8,114,114,116,11,100, + 7,131,1,130,1,100,4,160,12,124,6,114,126,124,6,110, + 2,124,8,124,7,124,9,103,3,161,1,125,10,124,2,100, + 1,107,8,114,172,116,8,106,13,106,14,100,8,107,2,114, + 164,100,4,125,2,110,8,116,8,106,13,106,14,125,2,116, + 15,124,2,131,1,125,2,124,2,100,4,107,3,114,224,124, + 2,160,16,161,0,115,210,116,17,100,9,160,18,124,2,161, + 1,131,1,130,1,100,10,160,18,124,10,116,19,124,2,161, + 3,125,10,124,10,116,20,100,8,25,0,23,0,125,11,116, + 8,106,21,100,1,107,9,144,1,114,76,116,22,124,4,131, + 1,144,1,115,16,116,23,116,4,160,24,161,0,124,4,131, + 2,125,4,124,4,100,5,25,0,100,11,107,2,144,1,114, + 56,124,4,100,8,25,0,116,25,107,7,144,1,114,56,124, + 4,100,12,100,1,133,2,25,0,125,4,116,23,116,8,106, + 21,124,4,160,26,116,25,161,1,124,11,131,3,83,0,116, + 23,124,4,116,27,124,11,131,3,83,0,41,13,97,254,2, + 0,0,71,105,118,101,110,32,116,104,101,32,112,97,116,104, + 32,116,111,32,97,32,46,112,121,32,102,105,108,101,44,32, + 114,101,116,117,114,110,32,116,104,101,32,112,97,116,104,32, + 116,111,32,105,116,115,32,46,112,121,99,32,102,105,108,101, + 46,10,10,32,32,32,32,84,104,101,32,46,112,121,32,102, + 105,108,101,32,100,111,101,115,32,110,111,116,32,110,101,101, + 100,32,116,111,32,101,120,105,115,116,59,32,116,104,105,115, + 32,115,105,109,112,108,121,32,114,101,116,117,114,110,115,32, + 116,104,101,32,112,97,116,104,32,116,111,32,116,104,101,10, + 32,32,32,32,46,112,121,99,32,102,105,108,101,32,99,97, + 108,99,117,108,97,116,101,100,32,97,115,32,105,102,32,116, + 104,101,32,46,112,121,32,102,105,108,101,32,119,101,114,101, + 32,105,109,112,111,114,116,101,100,46,10,10,32,32,32,32, + 84,104,101,32,39,111,112,116,105,109,105,122,97,116,105,111, + 110,39,32,112,97,114,97,109,101,116,101,114,32,99,111,110, + 116,114,111,108,115,32,116,104,101,32,112,114,101,115,117,109, + 101,100,32,111,112,116,105,109,105,122,97,116,105,111,110,32, + 108,101,118,101,108,32,111,102,10,32,32,32,32,116,104,101, + 32,98,121,116,101,99,111,100,101,32,102,105,108,101,46,32, + 73,102,32,39,111,112,116,105,109,105,122,97,116,105,111,110, + 39,32,105,115,32,110,111,116,32,78,111,110,101,44,32,116, + 104,101,32,115,116,114,105,110,103,32,114,101,112,114,101,115, + 101,110,116,97,116,105,111,110,10,32,32,32,32,111,102,32, + 116,104,101,32,97,114,103,117,109,101,110,116,32,105,115,32, + 116,97,107,101,110,32,97,110,100,32,118,101,114,105,102,105, + 101,100,32,116,111,32,98,101,32,97,108,112,104,97,110,117, + 109,101,114,105,99,32,40,101,108,115,101,32,86,97,108,117, + 101,69,114,114,111,114,10,32,32,32,32,105,115,32,114,97, + 105,115,101,100,41,46,10,10,32,32,32,32,84,104,101,32, + 100,101,98,117,103,95,111,118,101,114,114,105,100,101,32,112, + 97,114,97,109,101,116,101,114,32,105,115,32,100,101,112,114, + 101,99,97,116,101,100,46,32,73,102,32,100,101,98,117,103, + 95,111,118,101,114,114,105,100,101,32,105,115,32,110,111,116, + 32,78,111,110,101,44,10,32,32,32,32,97,32,84,114,117, + 101,32,118,97,108,117,101,32,105,115,32,116,104,101,32,115, + 97,109,101,32,97,115,32,115,101,116,116,105,110,103,32,39, + 111,112,116,105,109,105,122,97,116,105,111,110,39,32,116,111, + 32,116,104,101,32,101,109,112,116,121,32,115,116,114,105,110, + 103,10,32,32,32,32,119,104,105,108,101,32,97,32,70,97, + 108,115,101,32,118,97,108,117,101,32,105,115,32,101,113,117, + 105,118,97,108,101,110,116,32,116,111,32,115,101,116,116,105, + 110,103,32,39,111,112,116,105,109,105,122,97,116,105,111,110, + 39,32,116,111,32,39,49,39,46,10,10,32,32,32,32,73, + 102,32,115,121,115,46,105,109,112,108,101,109,101,110,116,97, + 116,105,111,110,46,99,97,99,104,101,95,116,97,103,32,105, + 115,32,78,111,110,101,32,116,104,101,110,32,78,111,116,73, + 109,112,108,101,109,101,110,116,101,100,69,114,114,111,114,32, + 105,115,32,114,97,105,115,101,100,46,10,10,32,32,32,32, + 78,122,70,116,104,101,32,100,101,98,117,103,95,111,118,101, + 114,114,105,100,101,32,112,97,114,97,109,101,116,101,114,32, + 105,115,32,100,101,112,114,101,99,97,116,101,100,59,32,117, + 115,101,32,39,111,112,116,105,109,105,122,97,116,105,111,110, + 39,32,105,110,115,116,101,97,100,122,50,100,101,98,117,103, + 95,111,118,101,114,114,105,100,101,32,111,114,32,111,112,116, + 105,109,105,122,97,116,105,111,110,32,109,117,115,116,32,98, + 101,32,115,101,116,32,116,111,32,78,111,110,101,114,35,0, + 0,0,114,34,0,0,0,218,1,46,122,36,115,121,115,46, + 105,109,112,108,101,109,101,110,116,97,116,105,111,110,46,99, + 97,99,104,101,95,116,97,103,32,105,115,32,78,111,110,101, + 233,0,0,0,0,122,24,123,33,114,125,32,105,115,32,110, + 111,116,32,97,108,112,104,97,110,117,109,101,114,105,99,122, + 7,123,125,46,123,125,123,125,250,1,58,114,23,0,0,0, + 41,28,218,9,95,119,97,114,110,105,110,103,115,218,4,119, + 97,114,110,218,18,68,101,112,114,101,99,97,116,105,111,110, + 87,97,114,110,105,110,103,218,9,84,121,112,101,69,114,114, + 111,114,114,1,0,0,0,218,6,102,115,112,97,116,104,114, + 42,0,0,0,114,36,0,0,0,114,6,0,0,0,218,14, + 105,109,112,108,101,109,101,110,116,97,116,105,111,110,218,9, + 99,97,99,104,101,95,116,97,103,218,19,78,111,116,73,109, + 112,108,101,109,101,110,116,101,100,69,114,114,111,114,114,31, + 0,0,0,218,5,102,108,97,103,115,218,8,111,112,116,105, + 109,105,122,101,218,3,115,116,114,218,7,105,115,97,108,110, + 117,109,218,10,86,97,108,117,101,69,114,114,111,114,114,54, + 0,0,0,218,4,95,79,80,84,218,17,66,89,84,69,67, + 79,68,69,95,83,85,70,70,73,88,69,83,218,14,112,121, + 99,97,99,104,101,95,112,114,101,102,105,120,114,53,0,0, + 0,114,33,0,0,0,114,49,0,0,0,114,26,0,0,0, + 218,6,108,115,116,114,105,112,218,8,95,80,89,67,65,67, + 72,69,41,12,114,39,0,0,0,90,14,100,101,98,117,103, + 95,111,118,101,114,114,105,100,101,114,62,0,0,0,218,7, + 109,101,115,115,97,103,101,218,4,104,101,97,100,114,41,0, + 0,0,90,4,98,97,115,101,218,3,115,101,112,218,4,114, + 101,115,116,90,3,116,97,103,90,15,97,108,109,111,115,116, + 95,102,105,108,101,110,97,109,101,218,8,102,105,108,101,110, + 97,109,101,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,218,17,99,97,99,104,101,95,102,114,111,109,95,115, + 111,117,114,99,101,32,1,0,0,115,72,0,0,0,0,18, + 8,1,6,1,2,255,4,2,8,1,4,1,8,1,12,1, + 10,1,12,1,16,1,8,1,8,1,8,1,24,1,8,1, + 12,1,6,2,8,1,8,1,8,1,8,1,14,1,14,1, + 12,1,12,9,10,1,14,5,28,1,12,4,2,1,4,1, + 8,1,2,253,4,5,114,89,0,0,0,99,1,0,0,0, + 0,0,0,0,10,0,0,0,5,0,0,0,67,0,0,0, + 115,46,1,0,0,116,0,106,1,106,2,100,1,107,8,114, + 20,116,3,100,2,131,1,130,1,116,4,160,5,124,0,161, + 1,125,0,116,6,124,0,131,1,92,2,125,1,125,2,100, + 3,125,3,116,0,106,7,100,1,107,9,114,102,116,0,106, + 7,160,8,116,9,161,1,125,4,124,1,160,10,124,4,116, + 11,23,0,161,1,114,102,124,1,116,12,124,4,131,1,100, + 1,133,2,25,0,125,1,100,4,125,3,124,3,115,144,116, + 6,124,1,131,1,92,2,125,1,125,5,124,5,116,13,107, + 3,114,144,116,14,116,13,155,0,100,5,124,0,155,2,157, + 3,131,1,130,1,124,2,160,15,100,6,161,1,125,6,124, + 6,100,7,107,7,114,178,116,14,100,8,124,2,155,2,157, + 2,131,1,130,1,110,92,124,6,100,9,107,2,144,1,114, + 14,124,2,160,16,100,6,100,10,161,2,100,11,25,0,125, + 7,124,7,160,10,116,17,161,1,115,228,116,14,100,12,116, + 17,155,2,157,2,131,1,130,1,124,7,116,12,116,17,131, + 1,100,1,133,2,25,0,125,8,124,8,160,18,161,0,144, + 1,115,14,116,14,100,13,124,7,155,2,100,14,157,3,131, + 1,130,1,124,2,160,19,100,6,161,1,100,15,25,0,125, + 9,116,20,124,1,124,9,116,21,100,15,25,0,23,0,131, + 2,83,0,41,16,97,110,1,0,0,71,105,118,101,110,32, + 116,104,101,32,112,97,116,104,32,116,111,32,97,32,46,112, + 121,99,46,32,102,105,108,101,44,32,114,101,116,117,114,110, + 32,116,104,101,32,112,97,116,104,32,116,111,32,105,116,115, + 32,46,112,121,32,102,105,108,101,46,10,10,32,32,32,32, + 84,104,101,32,46,112,121,99,32,102,105,108,101,32,100,111, + 101,115,32,110,111,116,32,110,101,101,100,32,116,111,32,101, + 120,105,115,116,59,32,116,104,105,115,32,115,105,109,112,108, + 121,32,114,101,116,117,114,110,115,32,116,104,101,32,112,97, + 116,104,32,116,111,10,32,32,32,32,116,104,101,32,46,112, + 121,32,102,105,108,101,32,99,97,108,99,117,108,97,116,101, + 100,32,116,111,32,99,111,114,114,101,115,112,111,110,100,32, + 116,111,32,116,104,101,32,46,112,121,99,32,102,105,108,101, + 46,32,32,73,102,32,112,97,116,104,32,100,111,101,115,10, + 32,32,32,32,110,111,116,32,99,111,110,102,111,114,109,32, + 116,111,32,80,69,80,32,51,49,52,55,47,52,56,56,32, + 102,111,114,109,97,116,44,32,86,97,108,117,101,69,114,114, + 111,114,32,119,105,108,108,32,98,101,32,114,97,105,115,101, + 100,46,32,73,102,10,32,32,32,32,115,121,115,46,105,109, + 112,108,101,109,101,110,116,97,116,105,111,110,46,99,97,99, + 104,101,95,116,97,103,32,105,115,32,78,111,110,101,32,116, + 104,101,110,32,78,111,116,73,109,112,108,101,109,101,110,116, + 101,100,69,114,114,111,114,32,105,115,32,114,97,105,115,101, + 100,46,10,10,32,32,32,32,78,122,36,115,121,115,46,105, + 109,112,108,101,109,101,110,116,97,116,105,111,110,46,99,97, + 99,104,101,95,116,97,103,32,105,115,32,78,111,110,101,70, + 84,122,31,32,110,111,116,32,98,111,116,116,111,109,45,108, + 101,118,101,108,32,100,105,114,101,99,116,111,114,121,32,105, + 110,32,114,63,0,0,0,62,2,0,0,0,114,23,0,0, + 0,114,51,0,0,0,122,29,101,120,112,101,99,116,101,100, + 32,111,110,108,121,32,50,32,111,114,32,51,32,100,111,116, + 115,32,105,110,32,114,51,0,0,0,114,23,0,0,0,233, + 254,255,255,255,122,53,111,112,116,105,109,105,122,97,116,105, + 111,110,32,112,111,114,116,105,111,110,32,111,102,32,102,105, + 108,101,110,97,109,101,32,100,111,101,115,32,110,111,116,32, + 115,116,97,114,116,32,119,105,116,104,32,122,19,111,112,116, + 105,109,105,122,97,116,105,111,110,32,108,101,118,101,108,32, + 122,29,32,105,115,32,110,111,116,32,97,110,32,97,108,112, + 104,97,110,117,109,101,114,105,99,32,118,97,108,117,101,114, + 64,0,0,0,41,22,114,6,0,0,0,114,71,0,0,0, + 114,72,0,0,0,114,73,0,0,0,114,1,0,0,0,114, + 70,0,0,0,114,42,0,0,0,114,81,0,0,0,114,25, + 0,0,0,114,26,0,0,0,114,8,0,0,0,114,30,0, + 0,0,114,18,0,0,0,114,83,0,0,0,114,78,0,0, + 0,218,5,99,111,117,110,116,114,38,0,0,0,114,79,0, + 0,0,114,77,0,0,0,218,9,112,97,114,116,105,116,105, + 111,110,114,33,0,0,0,218,15,83,79,85,82,67,69,95, + 83,85,70,70,73,88,69,83,41,10,114,39,0,0,0,114, + 85,0,0,0,90,16,112,121,99,97,99,104,101,95,102,105, + 108,101,110,97,109,101,90,23,102,111,117,110,100,95,105,110, + 95,112,121,99,97,99,104,101,95,112,114,101,102,105,120,90, + 13,115,116,114,105,112,112,101,100,95,112,97,116,104,90,7, + 112,121,99,97,99,104,101,90,9,100,111,116,95,99,111,117, + 110,116,114,62,0,0,0,90,9,111,112,116,95,108,101,118, + 101,108,90,13,98,97,115,101,95,102,105,108,101,110,97,109, + 101,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, + 218,17,115,111,117,114,99,101,95,102,114,111,109,95,99,97, + 99,104,101,103,1,0,0,115,52,0,0,0,0,9,12,1, + 8,1,10,1,12,1,4,1,10,1,12,1,14,1,16,1, + 4,1,4,1,12,1,8,1,18,2,10,1,8,1,16,1, + 10,1,16,1,10,1,14,2,16,1,10,1,16,2,14,1, + 114,94,0,0,0,99,1,0,0,0,0,0,0,0,5,0, + 0,0,9,0,0,0,67,0,0,0,115,126,0,0,0,116, + 0,124,0,131,1,100,1,107,2,114,16,100,2,83,0,124, + 0,160,1,100,3,161,1,92,3,125,1,125,2,125,3,124, + 1,114,56,124,3,160,2,161,0,100,4,100,5,133,2,25, + 0,100,6,107,3,114,60,124,0,83,0,122,12,116,3,124, + 0,131,1,125,4,87,0,110,36,4,0,116,4,116,5,102, + 2,107,10,114,108,1,0,1,0,1,0,124,0,100,2,100, + 5,133,2,25,0,125,4,89,0,110,2,88,0,116,6,124, + 4,131,1,114,122,124,4,83,0,124,0,83,0,41,7,122, + 188,67,111,110,118,101,114,116,32,97,32,98,121,116,101,99, + 111,100,101,32,102,105,108,101,32,112,97,116,104,32,116,111, + 32,97,32,115,111,117,114,99,101,32,112,97,116,104,32,40, + 105,102,32,112,111,115,115,105,98,108,101,41,46,10,10,32, + 32,32,32,84,104,105,115,32,102,117,110,99,116,105,111,110, + 32,101,120,105,115,116,115,32,112,117,114,101,108,121,32,102, + 111,114,32,98,97,99,107,119,97,114,100,115,45,99,111,109, + 112,97,116,105,98,105,108,105,116,121,32,102,111,114,10,32, + 32,32,32,80,121,73,109,112,111,114,116,95,69,120,101,99, + 67,111,100,101,77,111,100,117,108,101,87,105,116,104,70,105, + 108,101,110,97,109,101,115,40,41,32,105,110,32,116,104,101, + 32,67,32,65,80,73,46,10,10,32,32,32,32,114,64,0, + 0,0,78,114,63,0,0,0,233,253,255,255,255,233,255,255, + 255,255,90,2,112,121,41,7,114,18,0,0,0,114,36,0, + 0,0,218,5,108,111,119,101,114,114,94,0,0,0,114,73, + 0,0,0,114,78,0,0,0,114,48,0,0,0,41,5,218, + 13,98,121,116,101,99,111,100,101,95,112,97,116,104,114,87, + 0,0,0,114,40,0,0,0,90,9,101,120,116,101,110,115, + 105,111,110,218,11,115,111,117,114,99,101,95,112,97,116,104, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 17,95,102,105,110,100,95,109,111,100,117,108,101,95,115,104, - 105,109,208,1,0,0,115,10,0,0,0,0,10,14,1,16, - 1,4,1,22,1,114,129,0,0,0,99,3,0,0,0,0, - 0,0,0,6,0,0,0,4,0,0,0,67,0,0,0,115, - 158,0,0,0,124,0,100,1,100,2,133,2,25,0,125,3, - 124,3,116,0,107,3,114,60,100,3,124,1,155,2,100,4, - 124,3,155,2,157,4,125,4,116,1,160,2,100,5,124,4, - 161,2,1,0,116,3,124,4,102,1,124,2,142,1,130,1, - 116,4,124,0,131,1,100,6,107,0,114,102,100,7,124,1, - 155,2,157,2,125,4,116,1,160,2,100,5,124,4,161,2, - 1,0,116,5,124,4,131,1,130,1,116,6,124,0,100,2, - 100,8,133,2,25,0,131,1,125,5,124,5,100,9,64,0, - 114,154,100,10,124,5,155,2,100,11,124,1,155,2,157,4, - 125,4,116,3,124,4,102,1,124,2,142,1,130,1,124,5, - 83,0,41,12,97,84,2,0,0,80,101,114,102,111,114,109, - 32,98,97,115,105,99,32,118,97,108,105,100,105,116,121,32, - 99,104,101,99,107,105,110,103,32,111,102,32,97,32,112,121, - 99,32,104,101,97,100,101,114,32,97,110,100,32,114,101,116, - 117,114,110,32,116,104,101,32,102,108,97,103,115,32,102,105, - 101,108,100,44,10,32,32,32,32,119,104,105,99,104,32,100, - 101,116,101,114,109,105,110,101,115,32,104,111,119,32,116,104, - 101,32,112,121,99,32,115,104,111,117,108,100,32,98,101,32, - 102,117,114,116,104,101,114,32,118,97,108,105,100,97,116,101, - 100,32,97,103,97,105,110,115,116,32,116,104,101,32,115,111, - 117,114,99,101,46,10,10,32,32,32,32,42,100,97,116,97, - 42,32,105,115,32,116,104,101,32,99,111,110,116,101,110,116, - 115,32,111,102,32,116,104,101,32,112,121,99,32,102,105,108, - 101,46,32,40,79,110,108,121,32,116,104,101,32,102,105,114, - 115,116,32,49,54,32,98,121,116,101,115,32,97,114,101,10, - 32,32,32,32,114,101,113,117,105,114,101,100,44,32,116,104, - 111,117,103,104,46,41,10,10,32,32,32,32,42,110,97,109, - 101,42,32,105,115,32,116,104,101,32,110,97,109,101,32,111, - 102,32,116,104,101,32,109,111,100,117,108,101,32,98,101,105, - 110,103,32,105,109,112,111,114,116,101,100,46,32,73,116,32, - 105,115,32,117,115,101,100,32,102,111,114,32,108,111,103,103, - 105,110,103,46,10,10,32,32,32,32,42,101,120,99,95,100, - 101,116,97,105,108,115,42,32,105,115,32,97,32,100,105,99, - 116,105,111,110,97,114,121,32,112,97,115,115,101,100,32,116, - 111,32,73,109,112,111,114,116,69,114,114,111,114,32,105,102, - 32,105,116,32,114,97,105,115,101,100,32,102,111,114,10,32, - 32,32,32,105,109,112,114,111,118,101,100,32,100,101,98,117, - 103,103,105,110,103,46,10,10,32,32,32,32,73,109,112,111, - 114,116,69,114,114,111,114,32,105,115,32,114,97,105,115,101, - 100,32,119,104,101,110,32,116,104,101,32,109,97,103,105,99, - 32,110,117,109,98,101,114,32,105,115,32,105,110,99,111,114, - 114,101,99,116,32,111,114,32,119,104,101,110,32,116,104,101, - 32,102,108,97,103,115,10,32,32,32,32,102,105,101,108,100, - 32,105,115,32,105,110,118,97,108,105,100,46,32,69,79,70, - 69,114,114,111,114,32,105,115,32,114,97,105,115,101,100,32, - 119,104,101,110,32,116,104,101,32,100,97,116,97,32,105,115, - 32,102,111,117,110,100,32,116,111,32,98,101,32,116,114,117, - 110,99,97,116,101,100,46,10,10,32,32,32,32,78,114,12, - 0,0,0,122,20,98,97,100,32,109,97,103,105,99,32,110, - 117,109,98,101,114,32,105,110,32,122,2,58,32,122,2,123, - 125,233,16,0,0,0,122,40,114,101,97,99,104,101,100,32, - 69,79,70,32,119,104,105,108,101,32,114,101,97,100,105,110, - 103,32,112,121,99,32,104,101,97,100,101,114,32,111,102,32, - 233,8,0,0,0,233,252,255,255,255,122,14,105,110,118,97, - 108,105,100,32,102,108,97,103,115,32,122,4,32,105,110,32, - 41,7,218,12,77,65,71,73,67,95,78,85,77,66,69,82, - 114,120,0,0,0,218,16,95,118,101,114,98,111,115,101,95, - 109,101,115,115,97,103,101,114,105,0,0,0,114,31,0,0, - 0,218,8,69,79,70,69,114,114,111,114,114,19,0,0,0, - 41,6,114,56,0,0,0,114,104,0,0,0,218,11,101,120, - 99,95,100,101,116,97,105,108,115,90,5,109,97,103,105,99, - 114,82,0,0,0,114,72,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,218,13,95,99,108,97,115, - 115,105,102,121,95,112,121,99,225,1,0,0,115,28,0,0, - 0,0,16,12,1,8,1,16,1,12,1,12,1,12,1,10, - 1,12,1,8,1,16,2,8,1,16,1,12,1,114,137,0, - 0,0,99,5,0,0,0,0,0,0,0,6,0,0,0,4, - 0,0,0,67,0,0,0,115,112,0,0,0,116,0,124,0, - 100,1,100,2,133,2,25,0,131,1,124,1,100,3,64,0, - 107,3,114,58,100,4,124,3,155,2,157,2,125,5,116,1, - 160,2,100,5,124,5,161,2,1,0,116,3,124,5,102,1, - 124,4,142,1,130,1,124,2,100,6,107,9,114,108,116,0, - 124,0,100,2,100,7,133,2,25,0,131,1,124,2,100,3, - 64,0,107,3,114,108,116,3,100,4,124,3,155,2,157,2, - 102,1,124,4,142,1,130,1,100,6,83,0,41,8,97,7, - 2,0,0,86,97,108,105,100,97,116,101,32,97,32,112,121, - 99,32,97,103,97,105,110,115,116,32,116,104,101,32,115,111, - 117,114,99,101,32,108,97,115,116,45,109,111,100,105,102,105, - 101,100,32,116,105,109,101,46,10,10,32,32,32,32,42,100, - 97,116,97,42,32,105,115,32,116,104,101,32,99,111,110,116, - 101,110,116,115,32,111,102,32,116,104,101,32,112,121,99,32, - 102,105,108,101,46,32,40,79,110,108,121,32,116,104,101,32, - 102,105,114,115,116,32,49,54,32,98,121,116,101,115,32,97, - 114,101,10,32,32,32,32,114,101,113,117,105,114,101,100,46, - 41,10,10,32,32,32,32,42,115,111,117,114,99,101,95,109, - 116,105,109,101,42,32,105,115,32,116,104,101,32,108,97,115, - 116,32,109,111,100,105,102,105,101,100,32,116,105,109,101,115, - 116,97,109,112,32,111,102,32,116,104,101,32,115,111,117,114, - 99,101,32,102,105,108,101,46,10,10,32,32,32,32,42,115, - 111,117,114,99,101,95,115,105,122,101,42,32,105,115,32,78, - 111,110,101,32,111,114,32,116,104,101,32,115,105,122,101,32, + 15,95,103,101,116,95,115,111,117,114,99,101,102,105,108,101, + 143,1,0,0,115,20,0,0,0,0,7,12,1,4,1,16, + 1,24,1,4,1,2,1,12,1,18,1,18,1,114,100,0, + 0,0,99,1,0,0,0,0,0,0,0,1,0,0,0,8, + 0,0,0,67,0,0,0,115,74,0,0,0,124,0,160,0, + 116,1,116,2,131,1,161,1,114,48,122,10,116,3,124,0, + 131,1,87,0,83,0,4,0,116,4,107,10,114,44,1,0, + 1,0,1,0,89,0,113,70,88,0,110,22,124,0,160,0, + 116,1,116,5,131,1,161,1,114,66,124,0,83,0,100,0, + 83,0,100,0,83,0,41,1,78,41,6,218,8,101,110,100, + 115,119,105,116,104,218,5,116,117,112,108,101,114,93,0,0, + 0,114,89,0,0,0,114,73,0,0,0,114,80,0,0,0, + 41,1,114,88,0,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,218,11,95,103,101,116,95,99,97,99, + 104,101,100,162,1,0,0,115,16,0,0,0,0,1,14,1, + 2,1,10,1,14,1,8,1,14,1,4,2,114,103,0,0, + 0,99,1,0,0,0,0,0,0,0,2,0,0,0,8,0, + 0,0,67,0,0,0,115,52,0,0,0,122,14,116,0,124, + 0,131,1,106,1,125,1,87,0,110,24,4,0,116,2,107, + 10,114,38,1,0,1,0,1,0,100,1,125,1,89,0,110, + 2,88,0,124,1,100,2,79,0,125,1,124,1,83,0,41, + 3,122,51,67,97,108,99,117,108,97,116,101,32,116,104,101, + 32,109,111,100,101,32,112,101,114,109,105,115,115,105,111,110, + 115,32,102,111,114,32,97,32,98,121,116,101,99,111,100,101, + 32,102,105,108,101,46,105,182,1,0,0,233,128,0,0,0, + 41,3,114,43,0,0,0,114,45,0,0,0,114,44,0,0, + 0,41,2,114,39,0,0,0,114,46,0,0,0,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,218,10,95,99, + 97,108,99,95,109,111,100,101,174,1,0,0,115,12,0,0, + 0,0,2,2,1,14,1,14,1,10,3,8,1,114,105,0, + 0,0,99,1,0,0,0,0,0,0,0,3,0,0,0,8, + 0,0,0,3,0,0,0,115,68,0,0,0,100,6,135,0, + 102,1,100,2,100,3,132,9,125,1,122,10,116,0,106,1, + 125,2,87,0,110,28,4,0,116,2,107,10,114,52,1,0, + 1,0,1,0,100,4,100,5,132,0,125,2,89,0,110,2, + 88,0,124,2,124,1,136,0,131,2,1,0,124,1,83,0, + 41,7,122,252,68,101,99,111,114,97,116,111,114,32,116,111, + 32,118,101,114,105,102,121,32,116,104,97,116,32,116,104,101, + 32,109,111,100,117,108,101,32,98,101,105,110,103,32,114,101, + 113,117,101,115,116,101,100,32,109,97,116,99,104,101,115,32, + 116,104,101,32,111,110,101,32,116,104,101,10,32,32,32,32, + 108,111,97,100,101,114,32,99,97,110,32,104,97,110,100,108, + 101,46,10,10,32,32,32,32,84,104,101,32,102,105,114,115, + 116,32,97,114,103,117,109,101,110,116,32,40,115,101,108,102, + 41,32,109,117,115,116,32,100,101,102,105,110,101,32,95,110, + 97,109,101,32,119,104,105,99,104,32,116,104,101,32,115,101, + 99,111,110,100,32,97,114,103,117,109,101,110,116,32,105,115, + 10,32,32,32,32,99,111,109,112,97,114,101,100,32,97,103, + 97,105,110,115,116,46,32,73,102,32,116,104,101,32,99,111, + 109,112,97,114,105,115,111,110,32,102,97,105,108,115,32,116, + 104,101,110,32,73,109,112,111,114,116,69,114,114,111,114,32, + 105,115,32,114,97,105,115,101,100,46,10,10,32,32,32,32, + 78,99,2,0,0,0,0,0,0,0,4,0,0,0,4,0, + 0,0,31,0,0,0,115,66,0,0,0,124,1,100,0,107, + 8,114,16,124,0,106,0,125,1,110,32,124,0,106,0,124, + 1,107,3,114,48,116,1,100,1,124,0,106,0,124,1,102, + 2,22,0,124,1,100,2,141,2,130,1,136,0,124,0,124, + 1,102,2,124,2,158,2,124,3,142,1,83,0,41,3,78, + 122,30,108,111,97,100,101,114,32,102,111,114,32,37,115,32, + 99,97,110,110,111,116,32,104,97,110,100,108,101,32,37,115, + 41,1,218,4,110,97,109,101,41,2,114,106,0,0,0,218, + 11,73,109,112,111,114,116,69,114,114,111,114,41,4,218,4, + 115,101,108,102,114,106,0,0,0,218,4,97,114,103,115,90, + 6,107,119,97,114,103,115,41,1,218,6,109,101,116,104,111, + 100,114,2,0,0,0,114,4,0,0,0,218,19,95,99,104, + 101,99,107,95,110,97,109,101,95,119,114,97,112,112,101,114, + 194,1,0,0,115,18,0,0,0,0,1,8,1,8,1,10, + 1,4,1,8,255,2,1,2,255,6,2,122,40,95,99,104, + 101,99,107,95,110,97,109,101,46,60,108,111,99,97,108,115, + 62,46,95,99,104,101,99,107,95,110,97,109,101,95,119,114, + 97,112,112,101,114,99,2,0,0,0,0,0,0,0,3,0, + 0,0,7,0,0,0,83,0,0,0,115,56,0,0,0,100, + 1,68,0,93,32,125,2,116,0,124,1,124,2,131,2,114, + 4,116,1,124,0,124,2,116,2,124,1,124,2,131,2,131, + 3,1,0,113,4,124,0,106,3,160,4,124,1,106,3,161, + 1,1,0,100,0,83,0,41,2,78,41,4,218,10,95,95, + 109,111,100,117,108,101,95,95,218,8,95,95,110,97,109,101, + 95,95,218,12,95,95,113,117,97,108,110,97,109,101,95,95, + 218,7,95,95,100,111,99,95,95,41,5,218,7,104,97,115, + 97,116,116,114,218,7,115,101,116,97,116,116,114,218,7,103, + 101,116,97,116,116,114,218,8,95,95,100,105,99,116,95,95, + 218,6,117,112,100,97,116,101,41,3,90,3,110,101,119,90, + 3,111,108,100,114,59,0,0,0,114,2,0,0,0,114,2, + 0,0,0,114,4,0,0,0,218,5,95,119,114,97,112,205, + 1,0,0,115,8,0,0,0,0,1,8,1,10,1,20,1, + 122,26,95,99,104,101,99,107,95,110,97,109,101,46,60,108, + 111,99,97,108,115,62,46,95,119,114,97,112,41,1,78,41, + 3,218,10,95,98,111,111,116,115,116,114,97,112,114,121,0, + 0,0,218,9,78,97,109,101,69,114,114,111,114,41,3,114, + 110,0,0,0,114,111,0,0,0,114,121,0,0,0,114,2, + 0,0,0,41,1,114,110,0,0,0,114,4,0,0,0,218, + 11,95,99,104,101,99,107,95,110,97,109,101,186,1,0,0, + 115,14,0,0,0,0,8,14,7,2,1,10,1,14,2,14, + 5,10,1,114,124,0,0,0,99,2,0,0,0,0,0,0, + 0,5,0,0,0,6,0,0,0,67,0,0,0,115,60,0, + 0,0,124,0,160,0,124,1,161,1,92,2,125,2,125,3, + 124,2,100,1,107,8,114,56,116,1,124,3,131,1,114,56, + 100,2,125,4,116,2,160,3,124,4,160,4,124,3,100,3, + 25,0,161,1,116,5,161,2,1,0,124,2,83,0,41,4, + 122,155,84,114,121,32,116,111,32,102,105,110,100,32,97,32, + 108,111,97,100,101,114,32,102,111,114,32,116,104,101,32,115, + 112,101,99,105,102,105,101,100,32,109,111,100,117,108,101,32, + 98,121,32,100,101,108,101,103,97,116,105,110,103,32,116,111, + 10,32,32,32,32,115,101,108,102,46,102,105,110,100,95,108, + 111,97,100,101,114,40,41,46,10,10,32,32,32,32,84,104, + 105,115,32,109,101,116,104,111,100,32,105,115,32,100,101,112, + 114,101,99,97,116,101,100,32,105,110,32,102,97,118,111,114, + 32,111,102,32,102,105,110,100,101,114,46,102,105,110,100,95, + 115,112,101,99,40,41,46,10,10,32,32,32,32,78,122,44, + 78,111,116,32,105,109,112,111,114,116,105,110,103,32,100,105, + 114,101,99,116,111,114,121,32,123,125,58,32,109,105,115,115, + 105,110,103,32,95,95,105,110,105,116,95,95,114,64,0,0, + 0,41,6,218,11,102,105,110,100,95,108,111,97,100,101,114, + 114,18,0,0,0,114,66,0,0,0,114,67,0,0,0,114, + 54,0,0,0,218,13,73,109,112,111,114,116,87,97,114,110, + 105,110,103,41,5,114,108,0,0,0,218,8,102,117,108,108, + 110,97,109,101,218,6,108,111,97,100,101,114,218,8,112,111, + 114,116,105,111,110,115,218,3,109,115,103,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,218,17,95,102,105,110, + 100,95,109,111,100,117,108,101,95,115,104,105,109,214,1,0, + 0,115,10,0,0,0,0,10,14,1,16,1,4,1,22,1, + 114,131,0,0,0,99,3,0,0,0,0,0,0,0,6,0, + 0,0,4,0,0,0,67,0,0,0,115,158,0,0,0,124, + 0,100,1,100,2,133,2,25,0,125,3,124,3,116,0,107, + 3,114,60,100,3,124,1,155,2,100,4,124,3,155,2,157, + 4,125,4,116,1,160,2,100,5,124,4,161,2,1,0,116, + 3,124,4,102,1,124,2,142,1,130,1,116,4,124,0,131, + 1,100,6,107,0,114,102,100,7,124,1,155,2,157,2,125, + 4,116,1,160,2,100,5,124,4,161,2,1,0,116,5,124, + 4,131,1,130,1,116,6,124,0,100,2,100,8,133,2,25, + 0,131,1,125,5,124,5,100,9,64,0,114,154,100,10,124, + 5,155,2,100,11,124,1,155,2,157,4,125,4,116,3,124, + 4,102,1,124,2,142,1,130,1,124,5,83,0,41,12,97, + 84,2,0,0,80,101,114,102,111,114,109,32,98,97,115,105, + 99,32,118,97,108,105,100,105,116,121,32,99,104,101,99,107, + 105,110,103,32,111,102,32,97,32,112,121,99,32,104,101,97, + 100,101,114,32,97,110,100,32,114,101,116,117,114,110,32,116, + 104,101,32,102,108,97,103,115,32,102,105,101,108,100,44,10, + 32,32,32,32,119,104,105,99,104,32,100,101,116,101,114,109, + 105,110,101,115,32,104,111,119,32,116,104,101,32,112,121,99, + 32,115,104,111,117,108,100,32,98,101,32,102,117,114,116,104, + 101,114,32,118,97,108,105,100,97,116,101,100,32,97,103,97, + 105,110,115,116,32,116,104,101,32,115,111,117,114,99,101,46, + 10,10,32,32,32,32,42,100,97,116,97,42,32,105,115,32, + 116,104,101,32,99,111,110,116,101,110,116,115,32,111,102,32, + 116,104,101,32,112,121,99,32,102,105,108,101,46,32,40,79, + 110,108,121,32,116,104,101,32,102,105,114,115,116,32,49,54, + 32,98,121,116,101,115,32,97,114,101,10,32,32,32,32,114, + 101,113,117,105,114,101,100,44,32,116,104,111,117,103,104,46, + 41,10,10,32,32,32,32,42,110,97,109,101,42,32,105,115, + 32,116,104,101,32,110,97,109,101,32,111,102,32,116,104,101, + 32,109,111,100,117,108,101,32,98,101,105,110,103,32,105,109, + 112,111,114,116,101,100,46,32,73,116,32,105,115,32,117,115, + 101,100,32,102,111,114,32,108,111,103,103,105,110,103,46,10, + 10,32,32,32,32,42,101,120,99,95,100,101,116,97,105,108, + 115,42,32,105,115,32,97,32,100,105,99,116,105,111,110,97, + 114,121,32,112,97,115,115,101,100,32,116,111,32,73,109,112, + 111,114,116,69,114,114,111,114,32,105,102,32,105,116,32,114, + 97,105,115,101,100,32,102,111,114,10,32,32,32,32,105,109, + 112,114,111,118,101,100,32,100,101,98,117,103,103,105,110,103, + 46,10,10,32,32,32,32,73,109,112,111,114,116,69,114,114, + 111,114,32,105,115,32,114,97,105,115,101,100,32,119,104,101, + 110,32,116,104,101,32,109,97,103,105,99,32,110,117,109,98, + 101,114,32,105,115,32,105,110,99,111,114,114,101,99,116,32, + 111,114,32,119,104,101,110,32,116,104,101,32,102,108,97,103, + 115,10,32,32,32,32,102,105,101,108,100,32,105,115,32,105, + 110,118,97,108,105,100,46,32,69,79,70,69,114,114,111,114, + 32,105,115,32,114,97,105,115,101,100,32,119,104,101,110,32, + 116,104,101,32,100,97,116,97,32,105,115,32,102,111,117,110, + 100,32,116,111,32,98,101,32,116,114,117,110,99,97,116,101, + 100,46,10,10,32,32,32,32,78,114,12,0,0,0,122,20, + 98,97,100,32,109,97,103,105,99,32,110,117,109,98,101,114, + 32,105,110,32,122,2,58,32,122,2,123,125,233,16,0,0, + 0,122,40,114,101,97,99,104,101,100,32,69,79,70,32,119, + 104,105,108,101,32,114,101,97,100,105,110,103,32,112,121,99, + 32,104,101,97,100,101,114,32,111,102,32,233,8,0,0,0, + 233,252,255,255,255,122,14,105,110,118,97,108,105,100,32,102, + 108,97,103,115,32,122,4,32,105,110,32,41,7,218,12,77, + 65,71,73,67,95,78,85,77,66,69,82,114,122,0,0,0, + 218,16,95,118,101,114,98,111,115,101,95,109,101,115,115,97, + 103,101,114,107,0,0,0,114,18,0,0,0,218,8,69,79, + 70,69,114,114,111,114,114,22,0,0,0,41,6,114,21,0, + 0,0,114,106,0,0,0,218,11,101,120,99,95,100,101,116, + 97,105,108,115,90,5,109,97,103,105,99,114,84,0,0,0, + 114,74,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 4,0,0,0,218,13,95,99,108,97,115,115,105,102,121,95, + 112,121,99,231,1,0,0,115,28,0,0,0,0,16,12,1, + 8,1,16,1,12,1,12,1,12,1,10,1,12,1,8,1, + 16,2,8,1,16,1,12,1,114,139,0,0,0,99,5,0, + 0,0,0,0,0,0,6,0,0,0,4,0,0,0,67,0, + 0,0,115,112,0,0,0,116,0,124,0,100,1,100,2,133, + 2,25,0,131,1,124,1,100,3,64,0,107,3,114,58,100, + 4,124,3,155,2,157,2,125,5,116,1,160,2,100,5,124, + 5,161,2,1,0,116,3,124,5,102,1,124,4,142,1,130, + 1,124,2,100,6,107,9,114,108,116,0,124,0,100,2,100, + 7,133,2,25,0,131,1,124,2,100,3,64,0,107,3,114, + 108,116,3,100,4,124,3,155,2,157,2,102,1,124,4,142, + 1,130,1,100,6,83,0,41,8,97,7,2,0,0,86,97, + 108,105,100,97,116,101,32,97,32,112,121,99,32,97,103,97, + 105,110,115,116,32,116,104,101,32,115,111,117,114,99,101,32, + 108,97,115,116,45,109,111,100,105,102,105,101,100,32,116,105, + 109,101,46,10,10,32,32,32,32,42,100,97,116,97,42,32, + 105,115,32,116,104,101,32,99,111,110,116,101,110,116,115,32, + 111,102,32,116,104,101,32,112,121,99,32,102,105,108,101,46, + 32,40,79,110,108,121,32,116,104,101,32,102,105,114,115,116, + 32,49,54,32,98,121,116,101,115,32,97,114,101,10,32,32, + 32,32,114,101,113,117,105,114,101,100,46,41,10,10,32,32, + 32,32,42,115,111,117,114,99,101,95,109,116,105,109,101,42, + 32,105,115,32,116,104,101,32,108,97,115,116,32,109,111,100, + 105,102,105,101,100,32,116,105,109,101,115,116,97,109,112,32, 111,102,32,116,104,101,32,115,111,117,114,99,101,32,102,105, - 108,101,32,105,110,32,98,121,116,101,115,46,10,10,32,32, - 32,32,42,110,97,109,101,42,32,105,115,32,116,104,101,32, - 110,97,109,101,32,111,102,32,116,104,101,32,109,111,100,117, - 108,101,32,98,101,105,110,103,32,105,109,112,111,114,116,101, - 100,46,32,73,116,32,105,115,32,117,115,101,100,32,102,111, - 114,32,108,111,103,103,105,110,103,46,10,10,32,32,32,32, - 42,101,120,99,95,100,101,116,97,105,108,115,42,32,105,115, - 32,97,32,100,105,99,116,105,111,110,97,114,121,32,112,97, - 115,115,101,100,32,116,111,32,73,109,112,111,114,116,69,114, - 114,111,114,32,105,102,32,105,116,32,114,97,105,115,101,100, - 32,102,111,114,10,32,32,32,32,105,109,112,114,111,118,101, - 100,32,100,101,98,117,103,103,105,110,103,46,10,10,32,32, - 32,32,65,110,32,73,109,112,111,114,116,69,114,114,111,114, - 32,105,115,32,114,97,105,115,101,100,32,105,102,32,116,104, - 101,32,98,121,116,101,99,111,100,101,32,105,115,32,115,116, - 97,108,101,46,10,10,32,32,32,32,114,131,0,0,0,233, - 12,0,0,0,108,3,0,0,0,255,127,255,127,3,0,122, - 22,98,121,116,101,99,111,100,101,32,105,115,32,115,116,97, - 108,101,32,102,111,114,32,122,2,123,125,78,114,130,0,0, - 0,41,4,114,19,0,0,0,114,120,0,0,0,114,134,0, - 0,0,114,105,0,0,0,41,6,114,56,0,0,0,218,12, - 115,111,117,114,99,101,95,109,116,105,109,101,218,11,115,111, - 117,114,99,101,95,115,105,122,101,114,104,0,0,0,114,136, - 0,0,0,114,82,0,0,0,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,218,23,95,118,97,108,105,100,97, - 116,101,95,116,105,109,101,115,116,97,109,112,95,112,121,99, - 2,2,0,0,115,16,0,0,0,0,19,24,1,10,1,12, - 1,12,1,8,1,22,255,2,2,114,141,0,0,0,99,4, - 0,0,0,0,0,0,0,4,0,0,0,3,0,0,0,67, - 0,0,0,115,38,0,0,0,124,0,100,1,100,2,133,2, - 25,0,124,1,107,3,114,34,116,0,100,3,124,2,155,2, - 157,2,102,1,124,3,142,1,130,1,100,4,83,0,41,5, - 97,243,1,0,0,86,97,108,105,100,97,116,101,32,97,32, - 104,97,115,104,45,98,97,115,101,100,32,112,121,99,32,98, - 121,32,99,104,101,99,107,105,110,103,32,116,104,101,32,114, - 101,97,108,32,115,111,117,114,99,101,32,104,97,115,104,32, - 97,103,97,105,110,115,116,32,116,104,101,32,111,110,101,32, - 105,110,10,32,32,32,32,116,104,101,32,112,121,99,32,104, - 101,97,100,101,114,46,10,10,32,32,32,32,42,100,97,116, - 97,42,32,105,115,32,116,104,101,32,99,111,110,116,101,110, - 116,115,32,111,102,32,116,104,101,32,112,121,99,32,102,105, - 108,101,46,32,40,79,110,108,121,32,116,104,101,32,102,105, - 114,115,116,32,49,54,32,98,121,116,101,115,32,97,114,101, - 10,32,32,32,32,114,101,113,117,105,114,101,100,46,41,10, - 10,32,32,32,32,42,115,111,117,114,99,101,95,104,97,115, - 104,42,32,105,115,32,116,104,101,32,105,109,112,111,114,116, - 108,105,98,46,117,116,105,108,46,115,111,117,114,99,101,95, - 104,97,115,104,40,41,32,111,102,32,116,104,101,32,115,111, - 117,114,99,101,32,102,105,108,101,46,10,10,32,32,32,32, - 42,110,97,109,101,42,32,105,115,32,116,104,101,32,110,97, - 109,101,32,111,102,32,116,104,101,32,109,111,100,117,108,101, - 32,98,101,105,110,103,32,105,109,112,111,114,116,101,100,46, - 32,73,116,32,105,115,32,117,115,101,100,32,102,111,114,32, - 108,111,103,103,105,110,103,46,10,10,32,32,32,32,42,101, - 120,99,95,100,101,116,97,105,108,115,42,32,105,115,32,97, - 32,100,105,99,116,105,111,110,97,114,121,32,112,97,115,115, - 101,100,32,116,111,32,73,109,112,111,114,116,69,114,114,111, - 114,32,105,102,32,105,116,32,114,97,105,115,101,100,32,102, - 111,114,10,32,32,32,32,105,109,112,114,111,118,101,100,32, - 100,101,98,117,103,103,105,110,103,46,10,10,32,32,32,32, - 65,110,32,73,109,112,111,114,116,69,114,114,111,114,32,105, - 115,32,114,97,105,115,101,100,32,105,102,32,116,104,101,32, - 98,121,116,101,99,111,100,101,32,105,115,32,115,116,97,108, - 101,46,10,10,32,32,32,32,114,131,0,0,0,114,130,0, - 0,0,122,46,104,97,115,104,32,105,110,32,98,121,116,101, - 99,111,100,101,32,100,111,101,115,110,39,116,32,109,97,116, - 99,104,32,104,97,115,104,32,111,102,32,115,111,117,114,99, - 101,32,78,41,1,114,105,0,0,0,41,4,114,56,0,0, - 0,218,11,115,111,117,114,99,101,95,104,97,115,104,114,104, - 0,0,0,114,136,0,0,0,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,218,18,95,118,97,108,105,100,97, - 116,101,95,104,97,115,104,95,112,121,99,30,2,0,0,115, - 12,0,0,0,0,17,16,1,2,1,8,255,2,2,2,254, - 114,143,0,0,0,99,4,0,0,0,0,0,0,0,5,0, - 0,0,5,0,0,0,67,0,0,0,115,82,0,0,0,116, - 0,160,1,124,0,161,1,125,4,116,2,124,4,116,3,131, - 2,114,58,116,4,160,5,100,1,124,2,161,2,1,0,124, - 3,100,2,107,9,114,52,116,6,160,7,124,4,124,3,161, - 2,1,0,124,4,83,0,110,20,116,8,100,3,160,9,124, - 2,161,1,124,1,124,2,100,4,141,3,130,1,100,2,83, - 0,41,5,122,35,67,111,109,112,105,108,101,32,98,121,116, - 101,99,111,100,101,32,97,115,32,102,111,117,110,100,32,105, - 110,32,97,32,112,121,99,46,122,21,99,111,100,101,32,111, - 98,106,101,99,116,32,102,114,111,109,32,123,33,114,125,78, - 122,23,78,111,110,45,99,111,100,101,32,111,98,106,101,99, - 116,32,105,110,32,123,33,114,125,41,2,114,104,0,0,0, - 114,35,0,0,0,41,10,218,7,109,97,114,115,104,97,108, - 90,5,108,111,97,100,115,218,10,105,115,105,110,115,116,97, - 110,99,101,218,10,95,99,111,100,101,95,116,121,112,101,114, - 120,0,0,0,114,134,0,0,0,218,4,95,105,109,112,90, - 16,95,102,105,120,95,99,111,95,102,105,108,101,110,97,109, - 101,114,105,0,0,0,114,50,0,0,0,41,5,114,56,0, - 0,0,114,104,0,0,0,114,96,0,0,0,114,97,0,0, - 0,218,4,99,111,100,101,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,218,17,95,99,111,109,112,105,108,101, - 95,98,121,116,101,99,111,100,101,54,2,0,0,115,20,0, - 0,0,0,2,10,1,10,1,12,1,8,1,12,1,6,2, - 10,1,2,0,2,255,114,149,0,0,0,114,62,0,0,0, - 99,3,0,0,0,0,0,0,0,4,0,0,0,5,0,0, - 0,67,0,0,0,115,70,0,0,0,116,0,116,1,131,1, - 125,3,124,3,160,2,116,3,100,1,131,1,161,1,1,0, - 124,3,160,2,116,3,124,1,131,1,161,1,1,0,124,3, - 160,2,116,3,124,2,131,1,161,1,1,0,124,3,160,2, - 116,4,160,5,124,0,161,1,161,1,1,0,124,3,83,0, - 41,2,122,43,80,114,111,100,117,99,101,32,116,104,101,32, - 100,97,116,97,32,102,111,114,32,97,32,116,105,109,101,115, - 116,97,109,112,45,98,97,115,101,100,32,112,121,99,46,114, - 62,0,0,0,41,6,218,9,98,121,116,101,97,114,114,97, - 121,114,133,0,0,0,218,6,101,120,116,101,110,100,114,17, - 0,0,0,114,144,0,0,0,218,5,100,117,109,112,115,41, - 4,114,148,0,0,0,218,5,109,116,105,109,101,114,140,0, - 0,0,114,56,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,218,22,95,99,111,100,101,95,116,111, - 95,116,105,109,101,115,116,97,109,112,95,112,121,99,67,2, - 0,0,115,12,0,0,0,0,2,8,1,14,1,14,1,14, - 1,16,1,114,154,0,0,0,84,99,3,0,0,0,0,0, - 0,0,5,0,0,0,5,0,0,0,67,0,0,0,115,80, - 0,0,0,116,0,116,1,131,1,125,3,100,1,124,2,100, - 1,62,0,66,0,125,4,124,3,160,2,116,3,124,4,131, - 1,161,1,1,0,116,4,124,1,131,1,100,2,107,2,115, - 50,116,5,130,1,124,3,160,2,124,1,161,1,1,0,124, - 3,160,2,116,6,160,7,124,0,161,1,161,1,1,0,124, - 3,83,0,41,3,122,38,80,114,111,100,117,99,101,32,116, - 104,101,32,100,97,116,97,32,102,111,114,32,97,32,104,97, - 115,104,45,98,97,115,101,100,32,112,121,99,46,114,29,0, - 0,0,114,131,0,0,0,41,8,114,150,0,0,0,114,133, - 0,0,0,114,151,0,0,0,114,17,0,0,0,114,31,0, - 0,0,218,14,65,115,115,101,114,116,105,111,110,69,114,114, - 111,114,114,144,0,0,0,114,152,0,0,0,41,5,114,148, - 0,0,0,114,142,0,0,0,90,7,99,104,101,99,107,101, - 100,114,56,0,0,0,114,72,0,0,0,114,2,0,0,0, + 108,101,46,10,10,32,32,32,32,42,115,111,117,114,99,101, + 95,115,105,122,101,42,32,105,115,32,78,111,110,101,32,111, + 114,32,116,104,101,32,115,105,122,101,32,111,102,32,116,104, + 101,32,115,111,117,114,99,101,32,102,105,108,101,32,105,110, + 32,98,121,116,101,115,46,10,10,32,32,32,32,42,110,97, + 109,101,42,32,105,115,32,116,104,101,32,110,97,109,101,32, + 111,102,32,116,104,101,32,109,111,100,117,108,101,32,98,101, + 105,110,103,32,105,109,112,111,114,116,101,100,46,32,73,116, + 32,105,115,32,117,115,101,100,32,102,111,114,32,108,111,103, + 103,105,110,103,46,10,10,32,32,32,32,42,101,120,99,95, + 100,101,116,97,105,108,115,42,32,105,115,32,97,32,100,105, + 99,116,105,111,110,97,114,121,32,112,97,115,115,101,100,32, + 116,111,32,73,109,112,111,114,116,69,114,114,111,114,32,105, + 102,32,105,116,32,114,97,105,115,101,100,32,102,111,114,10, + 32,32,32,32,105,109,112,114,111,118,101,100,32,100,101,98, + 117,103,103,105,110,103,46,10,10,32,32,32,32,65,110,32, + 73,109,112,111,114,116,69,114,114,111,114,32,105,115,32,114, + 97,105,115,101,100,32,105,102,32,116,104,101,32,98,121,116, + 101,99,111,100,101,32,105,115,32,115,116,97,108,101,46,10, + 10,32,32,32,32,114,133,0,0,0,233,12,0,0,0,108, + 3,0,0,0,255,127,255,127,3,0,122,22,98,121,116,101, + 99,111,100,101,32,105,115,32,115,116,97,108,101,32,102,111, + 114,32,122,2,123,125,78,114,132,0,0,0,41,4,114,22, + 0,0,0,114,122,0,0,0,114,136,0,0,0,114,107,0, + 0,0,41,6,114,21,0,0,0,218,12,115,111,117,114,99, + 101,95,109,116,105,109,101,218,11,115,111,117,114,99,101,95, + 115,105,122,101,114,106,0,0,0,114,138,0,0,0,114,84, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,218,23,95,118,97,108,105,100,97,116,101,95,116,105, + 109,101,115,116,97,109,112,95,112,121,99,8,2,0,0,115, + 16,0,0,0,0,19,24,1,10,1,12,1,12,1,8,1, + 22,255,2,2,114,143,0,0,0,99,4,0,0,0,0,0, + 0,0,4,0,0,0,3,0,0,0,67,0,0,0,115,38, + 0,0,0,124,0,100,1,100,2,133,2,25,0,124,1,107, + 3,114,34,116,0,100,3,124,2,155,2,157,2,102,1,124, + 3,142,1,130,1,100,4,83,0,41,5,97,243,1,0,0, + 86,97,108,105,100,97,116,101,32,97,32,104,97,115,104,45, + 98,97,115,101,100,32,112,121,99,32,98,121,32,99,104,101, + 99,107,105,110,103,32,116,104,101,32,114,101,97,108,32,115, + 111,117,114,99,101,32,104,97,115,104,32,97,103,97,105,110, + 115,116,32,116,104,101,32,111,110,101,32,105,110,10,32,32, + 32,32,116,104,101,32,112,121,99,32,104,101,97,100,101,114, + 46,10,10,32,32,32,32,42,100,97,116,97,42,32,105,115, + 32,116,104,101,32,99,111,110,116,101,110,116,115,32,111,102, + 32,116,104,101,32,112,121,99,32,102,105,108,101,46,32,40, + 79,110,108,121,32,116,104,101,32,102,105,114,115,116,32,49, + 54,32,98,121,116,101,115,32,97,114,101,10,32,32,32,32, + 114,101,113,117,105,114,101,100,46,41,10,10,32,32,32,32, + 42,115,111,117,114,99,101,95,104,97,115,104,42,32,105,115, + 32,116,104,101,32,105,109,112,111,114,116,108,105,98,46,117, + 116,105,108,46,115,111,117,114,99,101,95,104,97,115,104,40, + 41,32,111,102,32,116,104,101,32,115,111,117,114,99,101,32, + 102,105,108,101,46,10,10,32,32,32,32,42,110,97,109,101, + 42,32,105,115,32,116,104,101,32,110,97,109,101,32,111,102, + 32,116,104,101,32,109,111,100,117,108,101,32,98,101,105,110, + 103,32,105,109,112,111,114,116,101,100,46,32,73,116,32,105, + 115,32,117,115,101,100,32,102,111,114,32,108,111,103,103,105, + 110,103,46,10,10,32,32,32,32,42,101,120,99,95,100,101, + 116,97,105,108,115,42,32,105,115,32,97,32,100,105,99,116, + 105,111,110,97,114,121,32,112,97,115,115,101,100,32,116,111, + 32,73,109,112,111,114,116,69,114,114,111,114,32,105,102,32, + 105,116,32,114,97,105,115,101,100,32,102,111,114,10,32,32, + 32,32,105,109,112,114,111,118,101,100,32,100,101,98,117,103, + 103,105,110,103,46,10,10,32,32,32,32,65,110,32,73,109, + 112,111,114,116,69,114,114,111,114,32,105,115,32,114,97,105, + 115,101,100,32,105,102,32,116,104,101,32,98,121,116,101,99, + 111,100,101,32,105,115,32,115,116,97,108,101,46,10,10,32, + 32,32,32,114,133,0,0,0,114,132,0,0,0,122,46,104, + 97,115,104,32,105,110,32,98,121,116,101,99,111,100,101,32, + 100,111,101,115,110,39,116,32,109,97,116,99,104,32,104,97, + 115,104,32,111,102,32,115,111,117,114,99,101,32,78,41,1, + 114,107,0,0,0,41,4,114,21,0,0,0,218,11,115,111, + 117,114,99,101,95,104,97,115,104,114,106,0,0,0,114,138, + 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, + 0,0,218,18,95,118,97,108,105,100,97,116,101,95,104,97, + 115,104,95,112,121,99,36,2,0,0,115,12,0,0,0,0, + 17,16,1,2,1,8,255,2,2,2,254,114,145,0,0,0, + 99,4,0,0,0,0,0,0,0,5,0,0,0,5,0,0, + 0,67,0,0,0,115,82,0,0,0,116,0,160,1,124,0, + 161,1,125,4,116,2,124,4,116,3,131,2,114,58,116,4, + 160,5,100,1,124,2,161,2,1,0,124,3,100,2,107,9, + 114,52,116,6,160,7,124,4,124,3,161,2,1,0,124,4, + 83,0,110,20,116,8,100,3,160,9,124,2,161,1,124,1, + 124,2,100,4,141,3,130,1,100,2,83,0,41,5,122,35, + 67,111,109,112,105,108,101,32,98,121,116,101,99,111,100,101, + 32,97,115,32,102,111,117,110,100,32,105,110,32,97,32,112, + 121,99,46,122,21,99,111,100,101,32,111,98,106,101,99,116, + 32,102,114,111,109,32,123,33,114,125,78,122,23,78,111,110, + 45,99,111,100,101,32,111,98,106,101,99,116,32,105,110,32, + 123,33,114,125,41,2,114,106,0,0,0,114,39,0,0,0, + 41,10,218,7,109,97,114,115,104,97,108,90,5,108,111,97, + 100,115,218,10,105,115,105,110,115,116,97,110,99,101,218,10, + 95,99,111,100,101,95,116,121,112,101,114,122,0,0,0,114, + 136,0,0,0,218,4,95,105,109,112,90,16,95,102,105,120, + 95,99,111,95,102,105,108,101,110,97,109,101,114,107,0,0, + 0,114,54,0,0,0,41,5,114,21,0,0,0,114,106,0, + 0,0,114,98,0,0,0,114,99,0,0,0,218,4,99,111, + 100,101,114,2,0,0,0,114,2,0,0,0,114,4,0,0, + 0,218,17,95,99,111,109,112,105,108,101,95,98,121,116,101, + 99,111,100,101,60,2,0,0,115,20,0,0,0,0,2,10, + 1,10,1,12,1,8,1,12,1,6,2,10,1,2,0,2, + 255,114,151,0,0,0,114,64,0,0,0,99,3,0,0,0, + 0,0,0,0,4,0,0,0,5,0,0,0,67,0,0,0, + 115,70,0,0,0,116,0,116,1,131,1,125,3,124,3,160, + 2,116,3,100,1,131,1,161,1,1,0,124,3,160,2,116, + 3,124,1,131,1,161,1,1,0,124,3,160,2,116,3,124, + 2,131,1,161,1,1,0,124,3,160,2,116,4,160,5,124, + 0,161,1,161,1,1,0,124,3,83,0,41,2,122,43,80, + 114,111,100,117,99,101,32,116,104,101,32,100,97,116,97,32, + 102,111,114,32,97,32,116,105,109,101,115,116,97,109,112,45, + 98,97,115,101,100,32,112,121,99,46,114,64,0,0,0,41, + 6,218,9,98,121,116,101,97,114,114,97,121,114,135,0,0, + 0,218,6,101,120,116,101,110,100,114,17,0,0,0,114,146, + 0,0,0,218,5,100,117,109,112,115,41,4,114,150,0,0, + 0,218,5,109,116,105,109,101,114,142,0,0,0,114,21,0, + 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, + 0,218,22,95,99,111,100,101,95,116,111,95,116,105,109,101, + 115,116,97,109,112,95,112,121,99,73,2,0,0,115,12,0, + 0,0,0,2,8,1,14,1,14,1,14,1,16,1,114,156, + 0,0,0,84,99,3,0,0,0,0,0,0,0,5,0,0, + 0,5,0,0,0,67,0,0,0,115,80,0,0,0,116,0, + 116,1,131,1,125,3,100,1,124,2,100,1,62,0,66,0, + 125,4,124,3,160,2,116,3,124,4,131,1,161,1,1,0, + 116,4,124,1,131,1,100,2,107,2,115,50,116,5,130,1, + 124,3,160,2,124,1,161,1,1,0,124,3,160,2,116,6, + 160,7,124,0,161,1,161,1,1,0,124,3,83,0,41,3, + 122,38,80,114,111,100,117,99,101,32,116,104,101,32,100,97, + 116,97,32,102,111,114,32,97,32,104,97,115,104,45,98,97, + 115,101,100,32,112,121,99,46,114,34,0,0,0,114,133,0, + 0,0,41,8,114,152,0,0,0,114,135,0,0,0,114,153, + 0,0,0,114,17,0,0,0,114,18,0,0,0,114,19,0, + 0,0,114,146,0,0,0,114,154,0,0,0,41,5,114,150, + 0,0,0,114,144,0,0,0,90,7,99,104,101,99,107,101, + 100,114,21,0,0,0,114,74,0,0,0,114,2,0,0,0, 114,2,0,0,0,114,4,0,0,0,218,17,95,99,111,100, - 101,95,116,111,95,104,97,115,104,95,112,121,99,77,2,0, + 101,95,116,111,95,104,97,115,104,95,112,121,99,83,2,0, 0,115,14,0,0,0,0,2,8,1,12,1,14,1,16,1, - 10,1,16,1,114,156,0,0,0,99,1,0,0,0,0,0, + 10,1,16,1,114,157,0,0,0,99,1,0,0,0,0,0, 0,0,5,0,0,0,6,0,0,0,67,0,0,0,115,62, 0,0,0,100,1,100,2,108,0,125,1,116,1,160,2,124, 0,161,1,106,3,125,2,124,1,160,4,124,2,161,1,125, @@ -857,21 +872,21 @@ const unsigned char _Py_M__importlib_external[] = { 46,10,10,32,32,32,32,85,110,105,118,101,114,115,97,108, 32,110,101,119,108,105,110,101,32,115,117,112,112,111,114,116, 32,105,115,32,117,115,101,100,32,105,110,32,116,104,101,32, - 100,101,99,111,100,105,110,103,46,10,32,32,32,32,114,62, + 100,101,99,111,100,105,110,103,46,10,32,32,32,32,114,64, 0,0,0,78,84,41,7,218,8,116,111,107,101,110,105,122, - 101,114,52,0,0,0,90,7,66,121,116,101,115,73,79,90, + 101,114,56,0,0,0,90,7,66,121,116,101,115,73,79,90, 8,114,101,97,100,108,105,110,101,90,15,100,101,116,101,99, 116,95,101,110,99,111,100,105,110,103,90,25,73,110,99,114, 101,109,101,110,116,97,108,78,101,119,108,105,110,101,68,101, 99,111,100,101,114,218,6,100,101,99,111,100,101,41,5,218, - 12,115,111,117,114,99,101,95,98,121,116,101,115,114,157,0, + 12,115,111,117,114,99,101,95,98,121,116,101,115,114,158,0, 0,0,90,21,115,111,117,114,99,101,95,98,121,116,101,115, 95,114,101,97,100,108,105,110,101,218,8,101,110,99,111,100, 105,110,103,90,15,110,101,119,108,105,110,101,95,100,101,99, 111,100,101,114,114,2,0,0,0,114,2,0,0,0,114,4, 0,0,0,218,13,100,101,99,111,100,101,95,115,111,117,114, - 99,101,88,2,0,0,115,10,0,0,0,0,5,8,1,12, - 1,10,1,12,1,114,161,0,0,0,41,2,114,126,0,0, + 99,101,94,2,0,0,115,10,0,0,0,0,5,8,1,12, + 1,10,1,12,1,114,162,0,0,0,41,2,114,128,0,0, 0,218,26,115,117,98,109,111,100,117,108,101,95,115,101,97, 114,99,104,95,108,111,99,97,116,105,111,110,115,99,2,0, 0,0,2,0,0,0,9,0,0,0,8,0,0,0,67,0, @@ -915,28 +930,28 @@ const unsigned char _Py_M__importlib_external[] = { 32,97,114,103,46,10,10,32,32,32,32,78,122,9,60,117, 110,107,110,111,119,110,62,218,12,103,101,116,95,102,105,108, 101,110,97,109,101,41,1,218,6,111,114,105,103,105,110,84, - 218,10,105,115,95,112,97,99,107,97,103,101,114,62,0,0, - 0,41,17,114,114,0,0,0,114,163,0,0,0,114,105,0, - 0,0,114,1,0,0,0,114,68,0,0,0,114,120,0,0, + 218,10,105,115,95,112,97,99,107,97,103,101,114,64,0,0, + 0,41,17,114,116,0,0,0,114,164,0,0,0,114,107,0, + 0,0,114,1,0,0,0,114,70,0,0,0,114,122,0,0, 0,218,10,77,111,100,117,108,101,83,112,101,99,90,13,95, 115,101,116,95,102,105,108,101,97,116,116,114,218,27,95,103, 101,116,95,115,117,112,112,111,114,116,101,100,95,102,105,108, - 101,95,108,111,97,100,101,114,115,114,99,0,0,0,114,100, - 0,0,0,114,126,0,0,0,218,9,95,80,79,80,85,76, - 65,84,69,114,165,0,0,0,114,162,0,0,0,114,38,0, - 0,0,218,6,97,112,112,101,110,100,41,9,114,104,0,0, - 0,90,8,108,111,99,97,116,105,111,110,114,126,0,0,0, - 114,162,0,0,0,218,4,115,112,101,99,218,12,108,111,97, + 101,95,108,111,97,100,101,114,115,114,101,0,0,0,114,102, + 0,0,0,114,128,0,0,0,218,9,95,80,79,80,85,76, + 65,84,69,114,166,0,0,0,114,163,0,0,0,114,42,0, + 0,0,218,6,97,112,112,101,110,100,41,9,114,106,0,0, + 0,90,8,108,111,99,97,116,105,111,110,114,128,0,0,0, + 114,163,0,0,0,218,4,115,112,101,99,218,12,108,111,97, 100,101,114,95,99,108,97,115,115,218,8,115,117,102,102,105, - 120,101,115,114,165,0,0,0,90,7,100,105,114,110,97,109, + 120,101,115,114,166,0,0,0,90,7,100,105,114,110,97,109, 101,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, 218,23,115,112,101,99,95,102,114,111,109,95,102,105,108,101, - 95,108,111,99,97,116,105,111,110,105,2,0,0,115,62,0, + 95,108,111,99,97,116,105,111,110,111,2,0,0,115,62,0, 0,0,0,12,8,4,4,1,10,2,2,1,14,1,14,1, 8,2,10,8,16,1,6,3,8,1,14,1,14,1,10,1, 6,1,6,2,4,3,8,2,10,1,2,1,14,1,14,1, 6,2,4,1,8,2,6,1,12,1,6,1,12,1,12,2, - 114,173,0,0,0,99,0,0,0,0,0,0,0,0,0,0, + 114,174,0,0,0,99,0,0,0,0,0,0,0,0,0,0, 0,0,4,0,0,0,64,0,0,0,115,86,0,0,0,101, 0,90,1,100,0,90,2,100,1,90,3,100,2,90,4,100, 3,90,5,100,4,90,6,101,7,100,5,100,6,132,0,131, @@ -964,11 +979,11 @@ const unsigned char _Py_M__importlib_external[] = { 4,124,1,161,2,6,0,89,0,83,0,88,0,100,0,83, 0,41,1,78,41,5,218,7,95,119,105,110,114,101,103,90, 7,79,112,101,110,75,101,121,90,17,72,75,69,89,95,67, - 85,82,82,69,78,84,95,85,83,69,82,114,40,0,0,0, + 85,82,82,69,78,84,95,85,83,69,82,114,44,0,0,0, 90,18,72,75,69,89,95,76,79,67,65,76,95,77,65,67, 72,73,78,69,41,2,218,3,99,108,115,114,3,0,0,0, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 14,95,111,112,101,110,95,114,101,103,105,115,116,114,121,185, + 14,95,111,112,101,110,95,114,101,103,105,115,116,114,121,191, 2,0,0,115,8,0,0,0,0,2,2,1,16,1,14,1, 122,36,87,105,110,100,111,119,115,82,101,103,105,115,116,114, 121,70,105,110,100,101,114,46,95,111,112,101,110,95,114,101, @@ -982,20 +997,20 @@ const unsigned char _Py_M__importlib_external[] = { 82,0,88,0,87,0,110,26,4,0,116,9,107,10,114,112, 1,0,1,0,1,0,89,0,100,0,83,0,89,0,110,2, 88,0,124,5,83,0,41,5,78,122,5,37,100,46,37,100, - 114,59,0,0,0,41,2,114,125,0,0,0,90,11,115,121, - 115,95,118,101,114,115,105,111,110,114,30,0,0,0,41,10, + 114,23,0,0,0,41,2,114,127,0,0,0,90,11,115,121, + 115,95,118,101,114,115,105,111,110,114,35,0,0,0,41,10, 218,11,68,69,66,85,71,95,66,85,73,76,68,218,18,82, 69,71,73,83,84,82,89,95,75,69,89,95,68,69,66,85, 71,218,12,82,69,71,73,83,84,82,89,95,75,69,89,114, - 50,0,0,0,114,6,0,0,0,218,12,118,101,114,115,105, - 111,110,95,105,110,102,111,114,177,0,0,0,114,175,0,0, - 0,90,10,81,117,101,114,121,86,97,108,117,101,114,40,0, - 0,0,41,6,114,176,0,0,0,114,125,0,0,0,90,12, + 54,0,0,0,114,6,0,0,0,218,12,118,101,114,115,105, + 111,110,95,105,110,102,111,114,178,0,0,0,114,176,0,0, + 0,90,10,81,117,101,114,121,86,97,108,117,101,114,44,0, + 0,0,41,6,114,177,0,0,0,114,127,0,0,0,90,12, 114,101,103,105,115,116,114,121,95,107,101,121,114,3,0,0, 0,90,4,104,107,101,121,218,8,102,105,108,101,112,97,116, 104,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, 218,16,95,115,101,97,114,99,104,95,114,101,103,105,115,116, - 114,121,192,2,0,0,115,24,0,0,0,0,2,6,1,8, + 114,121,198,2,0,0,115,24,0,0,0,0,2,6,1,8, 2,6,1,6,1,16,255,6,2,2,1,12,1,26,1,14, 1,12,1,122,38,87,105,110,100,111,119,115,82,101,103,105, 115,116,114,121,70,105,110,100,101,114,46,95,115,101,97,114, @@ -1009,15 +1024,15 @@ const unsigned char _Py_M__importlib_external[] = { 160,4,116,5,124,6,131,1,161,1,114,64,116,6,106,7, 124,1,124,5,124,1,124,4,131,2,124,4,100,1,141,3, 125,7,124,7,2,0,1,0,83,0,113,64,100,0,83,0, - 41,2,78,41,1,114,164,0,0,0,41,8,114,183,0,0, - 0,114,39,0,0,0,114,40,0,0,0,114,167,0,0,0, - 114,99,0,0,0,114,100,0,0,0,114,120,0,0,0,218, + 41,2,78,41,1,114,165,0,0,0,41,8,114,184,0,0, + 0,114,43,0,0,0,114,44,0,0,0,114,168,0,0,0, + 114,101,0,0,0,114,102,0,0,0,114,122,0,0,0,218, 16,115,112,101,99,95,102,114,111,109,95,108,111,97,100,101, - 114,41,8,114,176,0,0,0,114,125,0,0,0,114,35,0, - 0,0,218,6,116,97,114,103,101,116,114,182,0,0,0,114, - 126,0,0,0,114,172,0,0,0,114,170,0,0,0,114,2, + 114,41,8,114,177,0,0,0,114,127,0,0,0,114,39,0, + 0,0,218,6,116,97,114,103,101,116,114,183,0,0,0,114, + 128,0,0,0,114,173,0,0,0,114,171,0,0,0,114,2, 0,0,0,114,2,0,0,0,114,4,0,0,0,218,9,102, - 105,110,100,95,115,112,101,99,207,2,0,0,115,28,0,0, + 105,110,100,95,115,112,101,99,213,2,0,0,115,28,0,0, 0,0,2,10,1,8,1,4,1,2,1,12,1,14,1,8, 1,14,1,14,1,6,1,8,1,2,254,6,3,122,31,87, 105,110,100,111,119,115,82,101,103,105,115,116,114,121,70,105, @@ -1032,23 +1047,23 @@ const unsigned char _Py_M__importlib_external[] = { 101,116,104,111,100,32,105,115,32,100,101,112,114,101,99,97, 116,101,100,46,32,32,85,115,101,32,101,120,101,99,95,109, 111,100,117,108,101,40,41,32,105,110,115,116,101,97,100,46, - 10,10,32,32,32,32,32,32,32,32,78,41,2,114,186,0, - 0,0,114,126,0,0,0,41,4,114,176,0,0,0,114,125, - 0,0,0,114,35,0,0,0,114,170,0,0,0,114,2,0, + 10,10,32,32,32,32,32,32,32,32,78,41,2,114,187,0, + 0,0,114,128,0,0,0,41,4,114,177,0,0,0,114,127, + 0,0,0,114,39,0,0,0,114,171,0,0,0,114,2,0, 0,0,114,2,0,0,0,114,4,0,0,0,218,11,102,105, - 110,100,95,109,111,100,117,108,101,223,2,0,0,115,8,0, + 110,100,95,109,111,100,117,108,101,229,2,0,0,115,8,0, 0,0,0,7,12,1,8,1,6,2,122,33,87,105,110,100, 111,119,115,82,101,103,105,115,116,114,121,70,105,110,100,101, 114,46,102,105,110,100,95,109,111,100,117,108,101,41,12,114, - 111,0,0,0,114,110,0,0,0,114,112,0,0,0,114,113, - 0,0,0,114,180,0,0,0,114,179,0,0,0,114,178,0, + 113,0,0,0,114,112,0,0,0,114,114,0,0,0,114,115, + 0,0,0,114,181,0,0,0,114,180,0,0,0,114,179,0, 0,0,218,11,99,108,97,115,115,109,101,116,104,111,100,114, - 177,0,0,0,114,183,0,0,0,114,186,0,0,0,114,187, + 178,0,0,0,114,184,0,0,0,114,187,0,0,0,114,188, 0,0,0,114,2,0,0,0,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,114,174,0,0,0,173,2,0,0, + 0,0,114,4,0,0,0,114,175,0,0,0,179,2,0,0, 115,30,0,0,0,8,2,4,3,2,255,2,4,2,255,2, 3,4,2,12,7,12,15,2,1,2,0,2,255,12,16,2, - 1,2,255,114,174,0,0,0,99,0,0,0,0,0,0,0, + 1,2,255,114,175,0,0,0,99,0,0,0,0,0,0,0, 0,0,0,0,0,2,0,0,0,64,0,0,0,115,48,0, 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, @@ -1074,14 +1089,14 @@ const unsigned char _Py_M__importlib_external[] = { 101,100,32,98,121,32,103,101,116,95,102,105,108,101,110,97, 109,101,32,104,97,115,32,97,32,102,105,108,101,110,97,109, 101,32,111,102,32,39,95,95,105,110,105,116,95,95,46,112, - 121,39,46,114,29,0,0,0,114,61,0,0,0,114,62,0, - 0,0,114,59,0,0,0,218,8,95,95,105,110,105,116,95, - 95,41,4,114,38,0,0,0,114,163,0,0,0,114,34,0, - 0,0,114,32,0,0,0,41,5,114,106,0,0,0,114,125, - 0,0,0,114,86,0,0,0,90,13,102,105,108,101,110,97, + 121,39,46,114,34,0,0,0,114,63,0,0,0,114,64,0, + 0,0,114,23,0,0,0,218,8,95,95,105,110,105,116,95, + 95,41,4,114,42,0,0,0,114,164,0,0,0,114,38,0, + 0,0,114,36,0,0,0,41,5,114,108,0,0,0,114,127, + 0,0,0,114,88,0,0,0,90,13,102,105,108,101,110,97, 109,101,95,98,97,115,101,90,9,116,97,105,108,95,110,97, 109,101,114,2,0,0,0,114,2,0,0,0,114,4,0,0, - 0,114,165,0,0,0,242,2,0,0,115,8,0,0,0,0, + 0,114,166,0,0,0,248,2,0,0,115,8,0,0,0,0, 3,18,1,16,1,14,1,122,24,95,76,111,97,100,101,114, 66,97,115,105,99,115,46,105,115,95,112,97,99,107,97,103, 101,99,2,0,0,0,0,0,0,0,2,0,0,0,1,0, @@ -1089,9 +1104,9 @@ const unsigned char _Py_M__importlib_external[] = { 2,122,42,85,115,101,32,100,101,102,97,117,108,116,32,115, 101,109,97,110,116,105,99,115,32,102,111,114,32,109,111,100, 117,108,101,32,99,114,101,97,116,105,111,110,46,78,114,2, - 0,0,0,41,2,114,106,0,0,0,114,170,0,0,0,114, + 0,0,0,41,2,114,108,0,0,0,114,171,0,0,0,114, 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,13, - 99,114,101,97,116,101,95,109,111,100,117,108,101,250,2,0, + 99,114,101,97,116,101,95,109,111,100,117,108,101,0,3,0, 0,115,2,0,0,0,0,1,122,27,95,76,111,97,100,101, 114,66,97,115,105,99,115,46,99,114,101,97,116,101,95,109, 111,100,117,108,101,99,2,0,0,0,0,0,0,0,3,0, @@ -1105,13 +1120,13 @@ const unsigned char _Py_M__importlib_external[] = { 117,108,101,32,123,33,114,125,32,119,104,101,110,32,103,101, 116,95,99,111,100,101,40,41,32,114,101,116,117,114,110,115, 32,78,111,110,101,41,8,218,8,103,101,116,95,99,111,100, - 101,114,111,0,0,0,114,105,0,0,0,114,50,0,0,0, - 114,120,0,0,0,218,25,95,99,97,108,108,95,119,105,116, + 101,114,113,0,0,0,114,107,0,0,0,114,54,0,0,0, + 114,122,0,0,0,218,25,95,99,97,108,108,95,119,105,116, 104,95,102,114,97,109,101,115,95,114,101,109,111,118,101,100, - 218,4,101,120,101,99,114,117,0,0,0,41,3,114,106,0, - 0,0,218,6,109,111,100,117,108,101,114,148,0,0,0,114, + 218,4,101,120,101,99,114,119,0,0,0,41,3,114,108,0, + 0,0,218,6,109,111,100,117,108,101,114,150,0,0,0,114, 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,11, - 101,120,101,99,95,109,111,100,117,108,101,253,2,0,0,115, + 101,120,101,99,95,109,111,100,117,108,101,3,3,0,0,115, 12,0,0,0,0,2,12,1,8,1,6,1,4,255,6,2, 122,25,95,76,111,97,100,101,114,66,97,115,105,99,115,46, 101,120,101,99,95,109,111,100,117,108,101,99,2,0,0,0, @@ -1119,18 +1134,18 @@ const unsigned char _Py_M__importlib_external[] = { 115,12,0,0,0,116,0,160,1,124,0,124,1,161,2,83, 0,41,1,122,26,84,104,105,115,32,109,111,100,117,108,101, 32,105,115,32,100,101,112,114,101,99,97,116,101,100,46,41, - 2,114,120,0,0,0,218,17,95,108,111,97,100,95,109,111, - 100,117,108,101,95,115,104,105,109,41,2,114,106,0,0,0, - 114,125,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 2,114,122,0,0,0,218,17,95,108,111,97,100,95,109,111, + 100,117,108,101,95,115,104,105,109,41,2,114,108,0,0,0, + 114,127,0,0,0,114,2,0,0,0,114,2,0,0,0,114, 4,0,0,0,218,11,108,111,97,100,95,109,111,100,117,108, - 101,5,3,0,0,115,2,0,0,0,0,2,122,25,95,76, + 101,11,3,0,0,115,2,0,0,0,0,2,122,25,95,76, 111,97,100,101,114,66,97,115,105,99,115,46,108,111,97,100, - 95,109,111,100,117,108,101,78,41,8,114,111,0,0,0,114, - 110,0,0,0,114,112,0,0,0,114,113,0,0,0,114,165, - 0,0,0,114,191,0,0,0,114,196,0,0,0,114,198,0, + 95,109,111,100,117,108,101,78,41,8,114,113,0,0,0,114, + 112,0,0,0,114,114,0,0,0,114,115,0,0,0,114,166, + 0,0,0,114,192,0,0,0,114,197,0,0,0,114,199,0, 0,0,114,2,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,114,189,0,0,0,237,2,0,0,115, - 10,0,0,0,8,3,4,2,8,8,8,3,8,8,114,189, + 0,114,4,0,0,0,114,190,0,0,0,243,2,0,0,115, + 10,0,0,0,8,3,4,2,8,8,8,3,8,8,114,190, 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, 3,0,0,0,64,0,0,0,115,74,0,0,0,101,0,90, 1,100,0,90,2,100,1,100,2,132,0,90,3,100,3,100, @@ -1152,9 +1167,9 @@ const unsigned char _Py_M__importlib_external[] = { 69,114,114,111,114,32,119,104,101,110,32,116,104,101,32,112, 97,116,104,32,99,97,110,110,111,116,32,98,101,32,104,97, 110,100,108,101,100,46,10,32,32,32,32,32,32,32,32,78, - 41,1,114,40,0,0,0,41,2,114,106,0,0,0,114,35, + 41,1,114,44,0,0,0,41,2,114,108,0,0,0,114,39, 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,218,10,112,97,116,104,95,109,116,105,109,101,12,3, + 0,0,218,10,112,97,116,104,95,109,116,105,109,101,18,3, 0,0,115,2,0,0,0,0,6,122,23,83,111,117,114,99, 101,76,111,97,100,101,114,46,112,97,116,104,95,109,116,105, 109,101,99,2,0,0,0,0,0,0,0,2,0,0,0,4, @@ -1186,10 +1201,10 @@ const unsigned char _Py_M__importlib_external[] = { 32,32,32,82,97,105,115,101,115,32,79,83,69,114,114,111, 114,32,119,104,101,110,32,116,104,101,32,112,97,116,104,32, 99,97,110,110,111,116,32,98,101,32,104,97,110,100,108,101, - 100,46,10,32,32,32,32,32,32,32,32,114,153,0,0,0, - 41,1,114,200,0,0,0,41,2,114,106,0,0,0,114,35, + 100,46,10,32,32,32,32,32,32,32,32,114,155,0,0,0, + 41,1,114,201,0,0,0,41,2,114,108,0,0,0,114,39, 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,218,10,112,97,116,104,95,115,116,97,116,115,20,3, + 0,0,218,10,112,97,116,104,95,115,116,97,116,115,26,3, 0,0,115,2,0,0,0,0,11,122,23,83,111,117,114,99, 101,76,111,97,100,101,114,46,112,97,116,104,95,115,116,97, 116,115,99,4,0,0,0,0,0,0,0,4,0,0,0,4, @@ -1209,11 +1224,11 @@ const unsigned char _Py_M__importlib_external[] = { 111,114,100,101,114,32,116,111,32,99,111,114,114,101,99,116, 108,121,32,116,114,97,110,115,102,101,114,32,112,101,114,109, 105,115,115,105,111,110,115,10,32,32,32,32,32,32,32,32, - 41,1,218,8,115,101,116,95,100,97,116,97,41,4,114,106, - 0,0,0,114,97,0,0,0,90,10,99,97,99,104,101,95, - 112,97,116,104,114,56,0,0,0,114,2,0,0,0,114,2, + 41,1,218,8,115,101,116,95,100,97,116,97,41,4,114,108, + 0,0,0,114,99,0,0,0,90,10,99,97,99,104,101,95, + 112,97,116,104,114,21,0,0,0,114,2,0,0,0,114,2, 0,0,0,114,4,0,0,0,218,15,95,99,97,99,104,101, - 95,98,121,116,101,99,111,100,101,33,3,0,0,115,2,0, + 95,98,121,116,101,99,111,100,101,39,3,0,0,115,2,0, 0,0,0,8,122,28,83,111,117,114,99,101,76,111,97,100, 101,114,46,95,99,97,99,104,101,95,98,121,116,101,99,111, 100,101,99,3,0,0,0,0,0,0,0,3,0,0,0,1, @@ -1228,9 +1243,9 @@ const unsigned char _Py_M__importlib_external[] = { 111,114,32,116,104,101,32,119,114,105,116,105,110,103,32,111, 102,32,98,121,116,101,99,111,100,101,32,102,105,108,101,115, 46,10,32,32,32,32,32,32,32,32,78,114,2,0,0,0, - 41,3,114,106,0,0,0,114,35,0,0,0,114,56,0,0, + 41,3,114,108,0,0,0,114,39,0,0,0,114,21,0,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 114,202,0,0,0,43,3,0,0,115,2,0,0,0,0,4, + 114,203,0,0,0,49,3,0,0,115,2,0,0,0,0,4, 122,21,83,111,117,114,99,101,76,111,97,100,101,114,46,115, 101,116,95,100,97,116,97,99,2,0,0,0,0,0,0,0, 5,0,0,0,10,0,0,0,67,0,0,0,115,82,0,0, @@ -1245,16 +1260,16 @@ const unsigned char _Py_M__importlib_external[] = { 103,101,116,95,115,111,117,114,99,101,46,122,39,115,111,117, 114,99,101,32,110,111,116,32,97,118,97,105,108,97,98,108, 101,32,116,104,114,111,117,103,104,32,103,101,116,95,100,97, - 116,97,40,41,41,1,114,104,0,0,0,78,41,5,114,163, - 0,0,0,218,8,103,101,116,95,100,97,116,97,114,40,0, - 0,0,114,105,0,0,0,114,161,0,0,0,41,5,114,106, - 0,0,0,114,125,0,0,0,114,35,0,0,0,114,159,0, + 116,97,40,41,41,1,114,106,0,0,0,78,41,5,114,164, + 0,0,0,218,8,103,101,116,95,100,97,116,97,114,44,0, + 0,0,114,107,0,0,0,114,162,0,0,0,41,5,114,108, + 0,0,0,114,127,0,0,0,114,39,0,0,0,114,160,0, 0,0,218,3,101,120,99,114,2,0,0,0,114,2,0,0, 0,114,4,0,0,0,218,10,103,101,116,95,115,111,117,114, - 99,101,50,3,0,0,115,20,0,0,0,0,2,10,1,2, + 99,101,56,3,0,0,115,20,0,0,0,0,2,10,1,2, 1,14,1,16,1,4,1,2,255,4,1,2,255,20,2,122, 23,83,111,117,114,99,101,76,111,97,100,101,114,46,103,101, - 116,95,115,111,117,114,99,101,114,94,0,0,0,41,1,218, + 116,95,115,111,117,114,99,101,114,96,0,0,0,41,1,218, 9,95,111,112,116,105,109,105,122,101,99,3,0,0,0,1, 0,0,0,4,0,0,0,8,0,0,0,67,0,0,0,115, 22,0,0,0,116,0,106,1,116,2,124,1,124,2,100,1, @@ -1267,13 +1282,13 @@ const unsigned char _Py_M__importlib_external[] = { 110,121,32,111,98,106,101,99,116,32,116,121,112,101,32,116, 104,97,116,32,99,111,109,112,105,108,101,40,41,32,115,117, 112,112,111,114,116,115,46,10,32,32,32,32,32,32,32,32, - 114,194,0,0,0,84,41,2,218,12,100,111,110,116,95,105, - 110,104,101,114,105,116,114,73,0,0,0,41,3,114,120,0, - 0,0,114,193,0,0,0,218,7,99,111,109,112,105,108,101, - 41,4,114,106,0,0,0,114,56,0,0,0,114,35,0,0, - 0,114,207,0,0,0,114,2,0,0,0,114,2,0,0,0, + 114,195,0,0,0,84,41,2,218,12,100,111,110,116,95,105, + 110,104,101,114,105,116,114,75,0,0,0,41,3,114,122,0, + 0,0,114,194,0,0,0,218,7,99,111,109,112,105,108,101, + 41,4,114,108,0,0,0,114,21,0,0,0,114,39,0,0, + 0,114,208,0,0,0,114,2,0,0,0,114,2,0,0,0, 114,4,0,0,0,218,14,115,111,117,114,99,101,95,116,111, - 95,99,111,100,101,60,3,0,0,115,8,0,0,0,0,5, + 95,99,111,100,101,66,3,0,0,115,8,0,0,0,0,5, 12,1,2,0,2,255,122,27,83,111,117,114,99,101,76,111, 97,100,101,114,46,115,111,117,114,99,101,95,116,111,95,99, 111,100,101,99,2,0,0,0,0,0,0,0,15,0,0,0, @@ -1325,34 +1340,34 @@ const unsigned char _Py_M__importlib_external[] = { 32,115,101,116,95,100,97,116,97,32,109,117,115,116,32,97, 108,115,111,32,98,101,32,105,109,112,108,101,109,101,110,116, 101,100,46,10,10,32,32,32,32,32,32,32,32,78,70,84, - 114,153,0,0,0,41,2,114,104,0,0,0,114,35,0,0, - 0,114,130,0,0,0,114,29,0,0,0,114,62,0,0,0, - 114,59,0,0,0,90,5,110,101,118,101,114,90,6,97,108, + 114,155,0,0,0,41,2,114,106,0,0,0,114,39,0,0, + 0,114,132,0,0,0,114,34,0,0,0,114,64,0,0,0, + 114,23,0,0,0,90,5,110,101,118,101,114,90,6,97,108, 119,97,121,115,218,4,115,105,122,101,122,13,123,125,32,109, - 97,116,99,104,101,115,32,123,125,41,3,114,104,0,0,0, - 114,96,0,0,0,114,97,0,0,0,122,19,99,111,100,101, + 97,116,99,104,101,115,32,123,125,41,3,114,106,0,0,0, + 114,98,0,0,0,114,99,0,0,0,122,19,99,111,100,101, 32,111,98,106,101,99,116,32,102,114,111,109,32,123,125,122, - 10,119,114,111,116,101,32,123,33,114,125,41,27,114,163,0, - 0,0,114,87,0,0,0,114,71,0,0,0,114,201,0,0, - 0,114,40,0,0,0,114,14,0,0,0,114,204,0,0,0, - 114,137,0,0,0,218,10,109,101,109,111,114,121,118,105,101, - 119,114,147,0,0,0,90,21,99,104,101,99,107,95,104,97, - 115,104,95,98,97,115,101,100,95,112,121,99,115,114,142,0, + 10,119,114,111,116,101,32,123,33,114,125,41,27,114,164,0, + 0,0,114,89,0,0,0,114,73,0,0,0,114,202,0,0, + 0,114,44,0,0,0,114,14,0,0,0,114,205,0,0,0, + 114,139,0,0,0,218,10,109,101,109,111,114,121,118,105,101, + 119,114,149,0,0,0,90,21,99,104,101,99,107,95,104,97, + 115,104,95,98,97,115,101,100,95,112,121,99,115,114,144,0, 0,0,218,17,95,82,65,87,95,77,65,71,73,67,95,78, - 85,77,66,69,82,114,143,0,0,0,114,141,0,0,0,114, - 105,0,0,0,114,135,0,0,0,114,120,0,0,0,114,134, - 0,0,0,114,149,0,0,0,114,210,0,0,0,114,6,0, + 85,77,66,69,82,114,145,0,0,0,114,143,0,0,0,114, + 107,0,0,0,114,137,0,0,0,114,122,0,0,0,114,136, + 0,0,0,114,151,0,0,0,114,211,0,0,0,114,6,0, 0,0,218,19,100,111,110,116,95,119,114,105,116,101,95,98, - 121,116,101,99,111,100,101,114,156,0,0,0,114,154,0,0, - 0,114,31,0,0,0,114,203,0,0,0,41,15,114,106,0, - 0,0,114,125,0,0,0,114,97,0,0,0,114,139,0,0, - 0,114,159,0,0,0,114,142,0,0,0,90,10,104,97,115, + 121,116,101,99,111,100,101,114,157,0,0,0,114,156,0,0, + 0,114,18,0,0,0,114,204,0,0,0,41,15,114,108,0, + 0,0,114,127,0,0,0,114,99,0,0,0,114,141,0,0, + 0,114,160,0,0,0,114,144,0,0,0,90,10,104,97,115, 104,95,98,97,115,101,100,90,12,99,104,101,99,107,95,115, - 111,117,114,99,101,114,96,0,0,0,218,2,115,116,114,56, - 0,0,0,114,136,0,0,0,114,72,0,0,0,90,10,98, + 111,117,114,99,101,114,98,0,0,0,218,2,115,116,114,21, + 0,0,0,114,138,0,0,0,114,74,0,0,0,90,10,98, 121,116,101,115,95,100,97,116,97,90,11,99,111,100,101,95, 111,98,106,101,99,116,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,114,192,0,0,0,68,3,0,0,115,154, + 114,4,0,0,0,114,193,0,0,0,74,3,0,0,115,154, 0,0,0,0,7,10,1,4,1,4,1,4,1,4,1,4, 1,2,1,12,1,14,1,12,2,2,1,14,1,14,1,8, 2,12,1,2,1,14,1,14,1,6,3,2,1,2,254,6, @@ -1364,13 +1379,13 @@ const unsigned char _Py_M__importlib_external[] = { 1,6,255,4,2,6,1,10,1,10,1,14,2,6,1,6, 255,4,2,2,1,14,1,16,1,16,1,6,1,122,21,83, 111,117,114,99,101,76,111,97,100,101,114,46,103,101,116,95, - 99,111,100,101,78,41,10,114,111,0,0,0,114,110,0,0, - 0,114,112,0,0,0,114,200,0,0,0,114,201,0,0,0, - 114,203,0,0,0,114,202,0,0,0,114,206,0,0,0,114, - 210,0,0,0,114,192,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,114,199,0, - 0,0,10,3,0,0,115,14,0,0,0,8,2,8,8,8, - 13,8,10,8,7,8,10,14,8,114,199,0,0,0,99,0, + 99,111,100,101,78,41,10,114,113,0,0,0,114,112,0,0, + 0,114,114,0,0,0,114,201,0,0,0,114,202,0,0,0, + 114,204,0,0,0,114,203,0,0,0,114,207,0,0,0,114, + 211,0,0,0,114,193,0,0,0,114,2,0,0,0,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,200,0, + 0,0,16,3,0,0,115,14,0,0,0,8,2,8,8,8, + 13,8,10,8,7,8,10,14,8,114,200,0,0,0,99,0, 0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0, 0,0,0,115,124,0,0,0,101,0,90,1,100,0,90,2, 100,1,90,3,100,2,100,3,132,0,90,4,100,4,100,5, @@ -1395,27 +1410,27 @@ const unsigned char _Py_M__importlib_external[] = { 110,100,32,116,104,101,32,112,97,116,104,32,116,111,32,116, 104,101,32,102,105,108,101,32,102,111,117,110,100,32,98,121, 32,116,104,101,10,32,32,32,32,32,32,32,32,102,105,110, - 100,101,114,46,78,41,2,114,104,0,0,0,114,35,0,0, - 0,41,3,114,106,0,0,0,114,125,0,0,0,114,35,0, + 100,101,114,46,78,41,2,114,106,0,0,0,114,39,0,0, + 0,41,3,114,108,0,0,0,114,127,0,0,0,114,39,0, 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, - 0,114,190,0,0,0,159,3,0,0,115,4,0,0,0,0, + 0,114,191,0,0,0,165,3,0,0,115,4,0,0,0,0, 3,6,1,122,19,70,105,108,101,76,111,97,100,101,114,46, 95,95,105,110,105,116,95,95,99,2,0,0,0,0,0,0, 0,2,0,0,0,2,0,0,0,67,0,0,0,115,24,0, 0,0,124,0,106,0,124,1,106,0,107,2,111,22,124,0, 106,1,124,1,106,1,107,2,83,0,41,1,78,41,2,218, - 9,95,95,99,108,97,115,115,95,95,114,117,0,0,0,41, - 2,114,106,0,0,0,218,5,111,116,104,101,114,114,2,0, + 9,95,95,99,108,97,115,115,95,95,114,119,0,0,0,41, + 2,114,108,0,0,0,218,5,111,116,104,101,114,114,2,0, 0,0,114,2,0,0,0,114,4,0,0,0,218,6,95,95, - 101,113,95,95,165,3,0,0,115,6,0,0,0,0,1,12, + 101,113,95,95,171,3,0,0,115,6,0,0,0,0,1,12, 1,10,255,122,17,70,105,108,101,76,111,97,100,101,114,46, 95,95,101,113,95,95,99,1,0,0,0,0,0,0,0,1, 0,0,0,3,0,0,0,67,0,0,0,115,20,0,0,0, 116,0,124,0,106,1,131,1,116,0,124,0,106,2,131,1, 65,0,83,0,41,1,78,41,3,218,4,104,97,115,104,114, - 104,0,0,0,114,35,0,0,0,41,1,114,106,0,0,0, + 106,0,0,0,114,39,0,0,0,41,1,114,108,0,0,0, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 8,95,95,104,97,115,104,95,95,169,3,0,0,115,2,0, + 8,95,95,104,97,115,104,95,95,175,3,0,0,115,2,0, 0,0,0,1,122,19,70,105,108,101,76,111,97,100,101,114, 46,95,95,104,97,115,104,95,95,99,2,0,0,0,0,0, 0,0,2,0,0,0,3,0,0,0,3,0,0,0,115,16, @@ -1427,10 +1442,10 @@ const unsigned char _Py_M__importlib_external[] = { 97,116,101,100,46,32,32,85,115,101,32,101,120,101,99,95, 109,111,100,117,108,101,40,41,32,105,110,115,116,101,97,100, 46,10,10,32,32,32,32,32,32,32,32,41,3,218,5,115, - 117,112,101,114,114,216,0,0,0,114,198,0,0,0,41,2, - 114,106,0,0,0,114,125,0,0,0,41,1,114,217,0,0, - 0,114,2,0,0,0,114,4,0,0,0,114,198,0,0,0, - 172,3,0,0,115,2,0,0,0,0,10,122,22,70,105,108, + 117,112,101,114,114,217,0,0,0,114,199,0,0,0,41,2, + 114,108,0,0,0,114,127,0,0,0,41,1,114,218,0,0, + 0,114,2,0,0,0,114,4,0,0,0,114,199,0,0,0, + 178,3,0,0,115,2,0,0,0,0,10,122,22,70,105,108, 101,76,111,97,100,101,114,46,108,111,97,100,95,109,111,100, 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, 1,0,0,0,67,0,0,0,115,6,0,0,0,124,0,106, @@ -1438,9 +1453,9 @@ const unsigned char _Py_M__importlib_external[] = { 101,32,112,97,116,104,32,116,111,32,116,104,101,32,115,111, 117,114,99,101,32,102,105,108,101,32,97,115,32,102,111,117, 110,100,32,98,121,32,116,104,101,32,102,105,110,100,101,114, - 46,41,1,114,35,0,0,0,41,2,114,106,0,0,0,114, - 125,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, - 0,0,0,114,163,0,0,0,184,3,0,0,115,2,0,0, + 46,41,1,114,39,0,0,0,41,2,114,108,0,0,0,114, + 127,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, + 0,0,0,114,164,0,0,0,190,3,0,0,115,2,0,0, 0,0,3,122,23,70,105,108,101,76,111,97,100,101,114,46, 103,101,116,95,102,105,108,101,110,97,109,101,99,2,0,0, 0,0,0,0,0,3,0,0,0,10,0,0,0,67,0,0, @@ -1450,77 +1465,77 @@ const unsigned char _Py_M__importlib_external[] = { 83,0,41,3,122,39,82,101,116,117,114,110,32,116,104,101, 32,100,97,116,97,32,102,114,111,109,32,112,97,116,104,32, 97,115,32,114,97,119,32,98,121,116,101,115,46,218,1,114, - 78,41,3,114,52,0,0,0,114,53,0,0,0,90,4,114, - 101,97,100,41,3,114,106,0,0,0,114,35,0,0,0,114, - 57,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, - 0,0,0,114,204,0,0,0,189,3,0,0,115,4,0,0, + 78,41,3,114,56,0,0,0,114,57,0,0,0,90,4,114, + 101,97,100,41,3,114,108,0,0,0,114,39,0,0,0,114, + 60,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, + 0,0,0,114,205,0,0,0,195,3,0,0,115,4,0,0, 0,0,2,14,1,122,19,70,105,108,101,76,111,97,100,101, 114,46,103,101,116,95,100,97,116,97,99,2,0,0,0,0, 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, 18,0,0,0,124,0,160,0,124,1,161,1,114,14,124,0, - 83,0,100,0,83,0,41,1,78,41,1,114,165,0,0,0, - 41,2,114,106,0,0,0,114,195,0,0,0,114,2,0,0, + 83,0,100,0,83,0,41,1,78,41,1,114,166,0,0,0, + 41,2,114,108,0,0,0,114,196,0,0,0,114,2,0,0, 0,114,2,0,0,0,114,4,0,0,0,218,19,103,101,116, 95,114,101,115,111,117,114,99,101,95,114,101,97,100,101,114, - 196,3,0,0,115,6,0,0,0,0,2,10,1,4,1,122, + 202,3,0,0,115,6,0,0,0,0,2,10,1,4,1,122, 30,70,105,108,101,76,111,97,100,101,114,46,103,101,116,95, 114,101,115,111,117,114,99,101,95,114,101,97,100,101,114,99, 2,0,0,0,0,0,0,0,3,0,0,0,4,0,0,0, 67,0,0,0,115,32,0,0,0,116,0,116,1,124,0,106, 2,131,1,100,1,25,0,124,1,131,2,125,2,116,3,160, - 4,124,2,100,2,161,2,83,0,41,3,78,114,62,0,0, - 0,114,223,0,0,0,41,5,114,28,0,0,0,114,38,0, - 0,0,114,35,0,0,0,114,52,0,0,0,114,53,0,0, - 0,41,3,114,106,0,0,0,218,8,114,101,115,111,117,114, - 99,101,114,35,0,0,0,114,2,0,0,0,114,2,0,0, + 4,124,2,100,2,161,2,83,0,41,3,78,114,64,0,0, + 0,114,224,0,0,0,41,5,114,33,0,0,0,114,42,0, + 0,0,114,39,0,0,0,114,56,0,0,0,114,57,0,0, + 0,41,3,114,108,0,0,0,218,8,114,101,115,111,117,114, + 99,101,114,39,0,0,0,114,2,0,0,0,114,2,0,0, 0,114,4,0,0,0,218,13,111,112,101,110,95,114,101,115, - 111,117,114,99,101,202,3,0,0,115,4,0,0,0,0,1, + 111,117,114,99,101,208,3,0,0,115,4,0,0,0,0,1, 20,1,122,24,70,105,108,101,76,111,97,100,101,114,46,111, 112,101,110,95,114,101,115,111,117,114,99,101,99,2,0,0, 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, 0,115,38,0,0,0,124,0,160,0,124,1,161,1,115,14, 116,1,130,1,116,2,116,3,124,0,106,4,131,1,100,1, 25,0,124,1,131,2,125,2,124,2,83,0,41,2,78,114, - 62,0,0,0,41,5,218,11,105,115,95,114,101,115,111,117, + 64,0,0,0,41,5,218,11,105,115,95,114,101,115,111,117, 114,99,101,218,17,70,105,108,101,78,111,116,70,111,117,110, - 100,69,114,114,111,114,114,28,0,0,0,114,38,0,0,0, - 114,35,0,0,0,41,3,114,106,0,0,0,114,225,0,0, - 0,114,35,0,0,0,114,2,0,0,0,114,2,0,0,0, + 100,69,114,114,111,114,114,33,0,0,0,114,42,0,0,0, + 114,39,0,0,0,41,3,114,108,0,0,0,114,226,0,0, + 0,114,39,0,0,0,114,2,0,0,0,114,2,0,0,0, 114,4,0,0,0,218,13,114,101,115,111,117,114,99,101,95, - 112,97,116,104,206,3,0,0,115,8,0,0,0,0,1,10, + 112,97,116,104,212,3,0,0,115,8,0,0,0,0,1,10, 1,4,1,20,1,122,24,70,105,108,101,76,111,97,100,101, 114,46,114,101,115,111,117,114,99,101,95,112,97,116,104,99, 2,0,0,0,0,0,0,0,3,0,0,0,3,0,0,0, 67,0,0,0,115,40,0,0,0,116,0,124,1,107,6,114, 12,100,1,83,0,116,1,116,2,124,0,106,3,131,1,100, 2,25,0,124,1,131,2,125,2,116,4,124,2,131,1,83, - 0,41,3,78,70,114,62,0,0,0,41,5,114,25,0,0, - 0,114,28,0,0,0,114,38,0,0,0,114,35,0,0,0, - 114,44,0,0,0,41,3,114,106,0,0,0,114,104,0,0, - 0,114,35,0,0,0,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,114,227,0,0,0,212,3,0,0,115,8, + 0,41,3,78,70,114,64,0,0,0,41,5,114,30,0,0, + 0,114,33,0,0,0,114,42,0,0,0,114,39,0,0,0, + 114,48,0,0,0,41,3,114,108,0,0,0,114,106,0,0, + 0,114,39,0,0,0,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,114,228,0,0,0,218,3,0,0,115,8, 0,0,0,0,1,8,1,4,1,20,1,122,22,70,105,108, 101,76,111,97,100,101,114,46,105,115,95,114,101,115,111,117, 114,99,101,99,1,0,0,0,0,0,0,0,1,0,0,0, 5,0,0,0,67,0,0,0,115,24,0,0,0,116,0,116, 1,160,2,116,3,124,0,106,4,131,1,100,1,25,0,161, - 1,131,1,83,0,41,2,78,114,62,0,0,0,41,5,218, + 1,131,1,83,0,41,2,78,114,64,0,0,0,41,5,218, 4,105,116,101,114,114,1,0,0,0,218,7,108,105,115,116, - 100,105,114,114,38,0,0,0,114,35,0,0,0,41,1,114, - 106,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, - 0,0,0,218,8,99,111,110,116,101,110,116,115,218,3,0, + 100,105,114,114,42,0,0,0,114,39,0,0,0,41,1,114, + 108,0,0,0,114,2,0,0,0,114,2,0,0,0,114,4, + 0,0,0,218,8,99,111,110,116,101,110,116,115,224,3,0, 0,115,2,0,0,0,0,1,122,19,70,105,108,101,76,111, 97,100,101,114,46,99,111,110,116,101,110,116,115,41,17,114, - 111,0,0,0,114,110,0,0,0,114,112,0,0,0,114,113, - 0,0,0,114,190,0,0,0,114,219,0,0,0,114,221,0, - 0,0,114,122,0,0,0,114,198,0,0,0,114,163,0,0, - 0,114,204,0,0,0,114,224,0,0,0,114,226,0,0,0, - 114,229,0,0,0,114,227,0,0,0,114,232,0,0,0,90, + 113,0,0,0,114,112,0,0,0,114,114,0,0,0,114,115, + 0,0,0,114,191,0,0,0,114,220,0,0,0,114,222,0, + 0,0,114,124,0,0,0,114,199,0,0,0,114,164,0,0, + 0,114,205,0,0,0,114,225,0,0,0,114,227,0,0,0, + 114,230,0,0,0,114,228,0,0,0,114,233,0,0,0,90, 13,95,95,99,108,97,115,115,99,101,108,108,95,95,114,2, - 0,0,0,114,2,0,0,0,41,1,114,217,0,0,0,114, - 4,0,0,0,114,216,0,0,0,154,3,0,0,115,24,0, + 0,0,0,114,2,0,0,0,41,1,114,218,0,0,0,114, + 4,0,0,0,114,217,0,0,0,160,3,0,0,115,24,0, 0,0,8,3,4,2,8,6,8,4,8,3,16,12,12,5, - 8,7,12,6,8,4,8,6,8,6,114,216,0,0,0,99, + 8,7,12,6,8,4,8,6,8,6,114,217,0,0,0,99, 0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0, 64,0,0,0,115,46,0,0,0,101,0,90,1,100,0,90, 2,100,1,90,3,100,2,100,3,132,0,90,4,100,4,100, @@ -1536,25 +1551,25 @@ const unsigned char _Py_M__importlib_external[] = { 1,124,2,106,2,100,1,156,2,83,0,41,2,122,33,82, 101,116,117,114,110,32,116,104,101,32,109,101,116,97,100,97, 116,97,32,102,111,114,32,116,104,101,32,112,97,116,104,46, - 41,2,114,153,0,0,0,114,211,0,0,0,41,3,114,39, + 41,2,114,155,0,0,0,114,212,0,0,0,41,3,114,43, 0,0,0,218,8,115,116,95,109,116,105,109,101,90,7,115, - 116,95,115,105,122,101,41,3,114,106,0,0,0,114,35,0, - 0,0,114,215,0,0,0,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,114,201,0,0,0,226,3,0,0,115, + 116,95,115,105,122,101,41,3,114,108,0,0,0,114,39,0, + 0,0,114,216,0,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,114,202,0,0,0,232,3,0,0,115, 4,0,0,0,0,2,8,1,122,27,83,111,117,114,99,101, 70,105,108,101,76,111,97,100,101,114,46,112,97,116,104,95, 115,116,97,116,115,99,4,0,0,0,0,0,0,0,5,0, 0,0,5,0,0,0,67,0,0,0,115,24,0,0,0,116, 0,124,1,131,1,125,4,124,0,106,1,124,2,124,3,124, 4,100,1,141,3,83,0,41,2,78,41,1,218,5,95,109, - 111,100,101,41,2,114,103,0,0,0,114,202,0,0,0,41, - 5,114,106,0,0,0,114,97,0,0,0,114,96,0,0,0, - 114,56,0,0,0,114,42,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,203,0,0,0,231,3, + 111,100,101,41,2,114,105,0,0,0,114,203,0,0,0,41, + 5,114,108,0,0,0,114,99,0,0,0,114,98,0,0,0, + 114,21,0,0,0,114,46,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,204,0,0,0,237,3, 0,0,115,4,0,0,0,0,2,8,1,122,32,83,111,117, 114,99,101,70,105,108,101,76,111,97,100,101,114,46,95,99, 97,99,104,101,95,98,121,116,101,99,111,100,101,105,182,1, - 0,0,41,1,114,235,0,0,0,99,3,0,0,0,1,0, + 0,0,41,1,114,236,0,0,0,99,3,0,0,0,1,0, 0,0,9,0,0,0,11,0,0,0,67,0,0,0,115,0, 1,0,0,116,0,124,1,131,1,92,2,125,4,125,5,103, 0,125,6,124,4,114,52,116,1,124,4,131,1,115,52,116, @@ -1577,26 +1592,26 @@ const unsigned char _Py_M__importlib_external[] = { 101,46,122,27,99,111,117,108,100,32,110,111,116,32,99,114, 101,97,116,101,32,123,33,114,125,58,32,123,33,114,125,78, 122,12,99,114,101,97,116,101,100,32,123,33,114,125,41,12, - 114,38,0,0,0,114,46,0,0,0,114,169,0,0,0,114, - 33,0,0,0,114,28,0,0,0,114,1,0,0,0,90,5, + 114,42,0,0,0,114,50,0,0,0,114,170,0,0,0,114, + 37,0,0,0,114,33,0,0,0,114,1,0,0,0,90,5, 109,107,100,105,114,218,15,70,105,108,101,69,120,105,115,116, - 115,69,114,114,111,114,114,40,0,0,0,114,120,0,0,0, - 114,134,0,0,0,114,58,0,0,0,41,9,114,106,0,0, - 0,114,35,0,0,0,114,56,0,0,0,114,235,0,0,0, - 218,6,112,97,114,101,110,116,114,86,0,0,0,114,27,0, - 0,0,114,23,0,0,0,114,205,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,114,202,0,0,0, - 236,3,0,0,115,48,0,0,0,0,2,12,1,4,2,12, + 115,69,114,114,111,114,114,44,0,0,0,114,122,0,0,0, + 114,136,0,0,0,114,61,0,0,0,41,9,114,108,0,0, + 0,114,39,0,0,0,114,21,0,0,0,114,236,0,0,0, + 218,6,112,97,114,101,110,116,114,88,0,0,0,114,32,0, + 0,0,114,28,0,0,0,114,206,0,0,0,114,2,0,0, + 0,114,2,0,0,0,114,4,0,0,0,114,203,0,0,0, + 242,3,0,0,115,48,0,0,0,0,2,12,1,4,2,12, 1,12,1,12,2,12,1,10,1,2,1,14,1,14,2,8, 1,16,3,6,1,2,0,2,255,4,2,32,1,2,1,12, 1,16,1,16,2,8,1,2,255,122,25,83,111,117,114,99, 101,70,105,108,101,76,111,97,100,101,114,46,115,101,116,95, - 100,97,116,97,78,41,7,114,111,0,0,0,114,110,0,0, - 0,114,112,0,0,0,114,113,0,0,0,114,201,0,0,0, - 114,203,0,0,0,114,202,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,2,0,0,0,114,4,0,0,0,114,233, - 0,0,0,222,3,0,0,115,8,0,0,0,8,2,4,2, - 8,5,8,5,114,233,0,0,0,99,0,0,0,0,0,0, + 100,97,116,97,78,41,7,114,113,0,0,0,114,112,0,0, + 0,114,114,0,0,0,114,115,0,0,0,114,202,0,0,0, + 114,204,0,0,0,114,203,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,114,234, + 0,0,0,228,3,0,0,115,8,0,0,0,8,2,4,2, + 8,5,8,5,114,234,0,0,0,99,0,0,0,0,0,0, 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,32, 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, 2,100,3,132,0,90,4,100,4,100,5,132,0,90,5,100, @@ -1610,14 +1625,14 @@ const unsigned char _Py_M__importlib_external[] = { 160,1,124,2,161,1,125,3,124,1,124,2,100,1,156,2, 125,4,116,2,124,3,124,1,124,4,131,3,1,0,116,3, 116,4,124,3,131,1,100,2,100,0,133,2,25,0,124,1, - 124,2,100,3,141,3,83,0,41,4,78,41,2,114,104,0, - 0,0,114,35,0,0,0,114,130,0,0,0,41,2,114,104, - 0,0,0,114,96,0,0,0,41,5,114,163,0,0,0,114, - 204,0,0,0,114,137,0,0,0,114,149,0,0,0,114,212, - 0,0,0,41,5,114,106,0,0,0,114,125,0,0,0,114, - 35,0,0,0,114,56,0,0,0,114,136,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,114,192,0, - 0,0,15,4,0,0,115,22,0,0,0,0,1,10,1,10, + 124,2,100,3,141,3,83,0,41,4,78,41,2,114,106,0, + 0,0,114,39,0,0,0,114,132,0,0,0,41,2,114,106, + 0,0,0,114,98,0,0,0,41,5,114,164,0,0,0,114, + 205,0,0,0,114,139,0,0,0,114,151,0,0,0,114,213, + 0,0,0,41,5,114,108,0,0,0,114,127,0,0,0,114, + 39,0,0,0,114,21,0,0,0,114,138,0,0,0,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,193,0, + 0,0,21,4,0,0,115,22,0,0,0,0,1,10,1,10, 4,2,1,2,254,6,4,12,1,2,1,14,1,2,1,2, 253,122,29,83,111,117,114,99,101,108,101,115,115,70,105,108, 101,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, @@ -1626,16 +1641,16 @@ const unsigned char _Py_M__importlib_external[] = { 122,39,82,101,116,117,114,110,32,78,111,110,101,32,97,115, 32,116,104,101,114,101,32,105,115,32,110,111,32,115,111,117, 114,99,101,32,99,111,100,101,46,78,114,2,0,0,0,41, - 2,114,106,0,0,0,114,125,0,0,0,114,2,0,0,0, - 114,2,0,0,0,114,4,0,0,0,114,206,0,0,0,31, + 2,114,108,0,0,0,114,127,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,114,207,0,0,0,37, 4,0,0,115,2,0,0,0,0,2,122,31,83,111,117,114, 99,101,108,101,115,115,70,105,108,101,76,111,97,100,101,114, - 46,103,101,116,95,115,111,117,114,99,101,78,41,6,114,111, - 0,0,0,114,110,0,0,0,114,112,0,0,0,114,113,0, - 0,0,114,192,0,0,0,114,206,0,0,0,114,2,0,0, + 46,103,101,116,95,115,111,117,114,99,101,78,41,6,114,113, + 0,0,0,114,112,0,0,0,114,114,0,0,0,114,115,0, + 0,0,114,193,0,0,0,114,207,0,0,0,114,2,0,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 114,238,0,0,0,11,4,0,0,115,6,0,0,0,8,2, - 4,2,8,16,114,238,0,0,0,99,0,0,0,0,0,0, + 114,239,0,0,0,17,4,0,0,115,6,0,0,0,8,2, + 4,2,8,16,114,239,0,0,0,99,0,0,0,0,0,0, 0,0,0,0,0,0,3,0,0,0,64,0,0,0,115,92, 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, 2,100,3,132,0,90,4,100,4,100,5,132,0,90,5,100, @@ -1653,27 +1668,27 @@ const unsigned char _Py_M__importlib_external[] = { 10,32,32,32,32,99,3,0,0,0,0,0,0,0,3,0, 0,0,2,0,0,0,67,0,0,0,115,16,0,0,0,124, 1,124,0,95,0,124,2,124,0,95,1,100,0,83,0,41, - 1,78,41,2,114,104,0,0,0,114,35,0,0,0,41,3, - 114,106,0,0,0,114,104,0,0,0,114,35,0,0,0,114, - 2,0,0,0,114,2,0,0,0,114,4,0,0,0,114,190, - 0,0,0,48,4,0,0,115,4,0,0,0,0,1,6,1, + 1,78,41,2,114,106,0,0,0,114,39,0,0,0,41,3, + 114,108,0,0,0,114,106,0,0,0,114,39,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,114,191, + 0,0,0,54,4,0,0,115,4,0,0,0,0,1,6,1, 122,28,69,120,116,101,110,115,105,111,110,70,105,108,101,76, 111,97,100,101,114,46,95,95,105,110,105,116,95,95,99,2, 0,0,0,0,0,0,0,2,0,0,0,2,0,0,0,67, 0,0,0,115,24,0,0,0,124,0,106,0,124,1,106,0, 107,2,111,22,124,0,106,1,124,1,106,1,107,2,83,0, - 41,1,78,41,2,114,217,0,0,0,114,117,0,0,0,41, - 2,114,106,0,0,0,114,218,0,0,0,114,2,0,0,0, - 114,2,0,0,0,114,4,0,0,0,114,219,0,0,0,52, + 41,1,78,41,2,114,218,0,0,0,114,119,0,0,0,41, + 2,114,108,0,0,0,114,219,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,114,220,0,0,0,58, 4,0,0,115,6,0,0,0,0,1,12,1,10,255,122,26, 69,120,116,101,110,115,105,111,110,70,105,108,101,76,111,97, 100,101,114,46,95,95,101,113,95,95,99,1,0,0,0,0, 0,0,0,1,0,0,0,3,0,0,0,67,0,0,0,115, 20,0,0,0,116,0,124,0,106,1,131,1,116,0,124,0, - 106,2,131,1,65,0,83,0,41,1,78,41,3,114,220,0, - 0,0,114,104,0,0,0,114,35,0,0,0,41,1,114,106, + 106,2,131,1,65,0,83,0,41,1,78,41,3,114,221,0, + 0,0,114,106,0,0,0,114,39,0,0,0,41,1,114,108, 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,114,221,0,0,0,56,4,0,0,115,2,0,0,0, + 0,0,114,222,0,0,0,62,4,0,0,115,2,0,0,0, 0,1,122,28,69,120,116,101,110,115,105,111,110,70,105,108, 101,76,111,97,100,101,114,46,95,95,104,97,115,104,95,95, 99,2,0,0,0,0,0,0,0,3,0,0,0,5,0,0, @@ -1685,12 +1700,12 @@ const unsigned char _Py_M__importlib_external[] = { 110,32,109,111,100,117,108,101,122,38,101,120,116,101,110,115, 105,111,110,32,109,111,100,117,108,101,32,123,33,114,125,32, 108,111,97,100,101,100,32,102,114,111,109,32,123,33,114,125, - 41,7,114,120,0,0,0,114,193,0,0,0,114,147,0,0, + 41,7,114,122,0,0,0,114,194,0,0,0,114,149,0,0, 0,90,14,99,114,101,97,116,101,95,100,121,110,97,109,105, - 99,114,134,0,0,0,114,104,0,0,0,114,35,0,0,0, - 41,3,114,106,0,0,0,114,170,0,0,0,114,195,0,0, + 99,114,136,0,0,0,114,106,0,0,0,114,39,0,0,0, + 41,3,114,108,0,0,0,114,171,0,0,0,114,196,0,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 114,191,0,0,0,59,4,0,0,115,18,0,0,0,0,2, + 114,192,0,0,0,65,4,0,0,115,18,0,0,0,0,2, 4,1,4,0,2,255,4,2,6,1,4,0,4,255,4,2, 122,33,69,120,116,101,110,115,105,111,110,70,105,108,101,76, 111,97,100,101,114,46,99,114,101,97,116,101,95,109,111,100, @@ -1703,11 +1718,11 @@ const unsigned char _Py_M__importlib_external[] = { 117,108,101,122,40,101,120,116,101,110,115,105,111,110,32,109, 111,100,117,108,101,32,123,33,114,125,32,101,120,101,99,117, 116,101,100,32,102,114,111,109,32,123,33,114,125,78,41,7, - 114,120,0,0,0,114,193,0,0,0,114,147,0,0,0,90, - 12,101,120,101,99,95,100,121,110,97,109,105,99,114,134,0, - 0,0,114,104,0,0,0,114,35,0,0,0,41,2,114,106, - 0,0,0,114,195,0,0,0,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,114,196,0,0,0,67,4,0,0, + 114,122,0,0,0,114,194,0,0,0,114,149,0,0,0,90, + 12,101,120,101,99,95,100,121,110,97,109,105,99,114,136,0, + 0,0,114,106,0,0,0,114,39,0,0,0,41,2,114,108, + 0,0,0,114,196,0,0,0,114,2,0,0,0,114,2,0, + 0,0,114,4,0,0,0,114,197,0,0,0,73,4,0,0, 115,10,0,0,0,0,2,14,1,6,1,4,0,4,255,122, 31,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, 97,100,101,114,46,101,120,101,99,95,109,111,100,117,108,101, @@ -1718,23 +1733,23 @@ const unsigned char _Py_M__importlib_external[] = { 122,49,82,101,116,117,114,110,32,84,114,117,101,32,105,102, 32,116,104,101,32,101,120,116,101,110,115,105,111,110,32,109, 111,100,117,108,101,32,105,115,32,97,32,112,97,99,107,97, - 103,101,46,114,29,0,0,0,99,1,0,0,0,0,0,0, + 103,101,46,114,34,0,0,0,99,1,0,0,0,0,0,0, 0,2,0,0,0,4,0,0,0,51,0,0,0,115,26,0, 0,0,124,0,93,18,125,1,136,0,100,0,124,1,23,0, - 107,2,86,0,1,0,113,2,100,1,83,0,41,2,114,190, - 0,0,0,78,114,2,0,0,0,41,2,114,22,0,0,0, + 107,2,86,0,1,0,113,2,100,1,83,0,41,2,114,191, + 0,0,0,78,114,2,0,0,0,41,2,114,27,0,0,0, 218,6,115,117,102,102,105,120,41,1,218,9,102,105,108,101, 95,110,97,109,101,114,2,0,0,0,114,4,0,0,0,218, - 9,60,103,101,110,101,120,112,114,62,76,4,0,0,115,4, + 9,60,103,101,110,101,120,112,114,62,82,4,0,0,115,4, 0,0,0,4,1,2,255,122,49,69,120,116,101,110,115,105, 111,110,70,105,108,101,76,111,97,100,101,114,46,105,115,95, 112,97,99,107,97,103,101,46,60,108,111,99,97,108,115,62, - 46,60,103,101,110,101,120,112,114,62,41,4,114,38,0,0, - 0,114,35,0,0,0,218,3,97,110,121,218,18,69,88,84, + 46,60,103,101,110,101,120,112,114,62,41,4,114,42,0,0, + 0,114,39,0,0,0,218,3,97,110,121,218,18,69,88,84, 69,78,83,73,79,78,95,83,85,70,70,73,88,69,83,41, - 2,114,106,0,0,0,114,125,0,0,0,114,2,0,0,0, - 41,1,114,241,0,0,0,114,4,0,0,0,114,165,0,0, - 0,73,4,0,0,115,8,0,0,0,0,2,14,1,12,1, + 2,114,108,0,0,0,114,127,0,0,0,114,2,0,0,0, + 41,1,114,242,0,0,0,114,4,0,0,0,114,166,0,0, + 0,79,4,0,0,115,8,0,0,0,0,2,14,1,12,1, 2,255,122,30,69,120,116,101,110,115,105,111,110,70,105,108, 101,76,111,97,100,101,114,46,105,115,95,112,97,99,107,97, 103,101,99,2,0,0,0,0,0,0,0,2,0,0,0,1, @@ -1743,9 +1758,9 @@ const unsigned char _Py_M__importlib_external[] = { 97,115,32,97,110,32,101,120,116,101,110,115,105,111,110,32, 109,111,100,117,108,101,32,99,97,110,110,111,116,32,99,114, 101,97,116,101,32,97,32,99,111,100,101,32,111,98,106,101, - 99,116,46,78,114,2,0,0,0,41,2,114,106,0,0,0, - 114,125,0,0,0,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,114,192,0,0,0,79,4,0,0,115,2,0, + 99,116,46,78,114,2,0,0,0,41,2,114,108,0,0,0, + 114,127,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 4,0,0,0,114,193,0,0,0,85,4,0,0,115,2,0, 0,0,0,2,122,28,69,120,116,101,110,115,105,111,110,70, 105,108,101,76,111,97,100,101,114,46,103,101,116,95,99,111, 100,101,99,2,0,0,0,0,0,0,0,2,0,0,0,1, @@ -1754,8 +1769,8 @@ const unsigned char _Py_M__importlib_external[] = { 97,115,32,101,120,116,101,110,115,105,111,110,32,109,111,100, 117,108,101,115,32,104,97,118,101,32,110,111,32,115,111,117, 114,99,101,32,99,111,100,101,46,78,114,2,0,0,0,41, - 2,114,106,0,0,0,114,125,0,0,0,114,2,0,0,0, - 114,2,0,0,0,114,4,0,0,0,114,206,0,0,0,83, + 2,114,108,0,0,0,114,127,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,114,207,0,0,0,89, 4,0,0,115,2,0,0,0,0,2,122,30,69,120,116,101, 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, @@ -1764,20 +1779,20 @@ const unsigned char _Py_M__importlib_external[] = { 116,117,114,110,32,116,104,101,32,112,97,116,104,32,116,111, 32,116,104,101,32,115,111,117,114,99,101,32,102,105,108,101, 32,97,115,32,102,111,117,110,100,32,98,121,32,116,104,101, - 32,102,105,110,100,101,114,46,41,1,114,35,0,0,0,41, - 2,114,106,0,0,0,114,125,0,0,0,114,2,0,0,0, - 114,2,0,0,0,114,4,0,0,0,114,163,0,0,0,87, + 32,102,105,110,100,101,114,46,41,1,114,39,0,0,0,41, + 2,114,108,0,0,0,114,127,0,0,0,114,2,0,0,0, + 114,2,0,0,0,114,4,0,0,0,114,164,0,0,0,93, 4,0,0,115,2,0,0,0,0,3,122,32,69,120,116,101, 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, 103,101,116,95,102,105,108,101,110,97,109,101,78,41,14,114, - 111,0,0,0,114,110,0,0,0,114,112,0,0,0,114,113, - 0,0,0,114,190,0,0,0,114,219,0,0,0,114,221,0, - 0,0,114,191,0,0,0,114,196,0,0,0,114,165,0,0, - 0,114,192,0,0,0,114,206,0,0,0,114,122,0,0,0, - 114,163,0,0,0,114,2,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,239,0,0,0,40,4, + 113,0,0,0,114,112,0,0,0,114,114,0,0,0,114,115, + 0,0,0,114,191,0,0,0,114,220,0,0,0,114,222,0, + 0,0,114,192,0,0,0,114,197,0,0,0,114,166,0,0, + 0,114,193,0,0,0,114,207,0,0,0,114,124,0,0,0, + 114,164,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,240,0,0,0,46,4, 0,0,115,20,0,0,0,8,6,4,2,8,4,8,4,8, - 3,8,8,8,6,8,6,8,4,8,4,114,239,0,0,0, + 3,8,8,8,6,8,6,8,4,8,4,114,240,0,0,0, 99,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0, 0,64,0,0,0,115,96,0,0,0,101,0,90,1,100,0, 90,2,100,1,90,3,100,2,100,3,132,0,90,4,100,4, @@ -1810,14 +1825,14 @@ const unsigned char _Py_M__importlib_external[] = { 0,124,1,124,0,95,0,124,2,124,0,95,1,116,2,124, 0,160,3,161,0,131,1,124,0,95,4,124,3,124,0,95, 5,100,0,83,0,41,1,78,41,6,218,5,95,110,97,109, - 101,218,5,95,112,97,116,104,114,100,0,0,0,218,16,95, + 101,218,5,95,112,97,116,104,114,102,0,0,0,218,16,95, 103,101,116,95,112,97,114,101,110,116,95,112,97,116,104,218, 17,95,108,97,115,116,95,112,97,114,101,110,116,95,112,97, 116,104,218,12,95,112,97,116,104,95,102,105,110,100,101,114, - 41,4,114,106,0,0,0,114,104,0,0,0,114,35,0,0, + 41,4,114,108,0,0,0,114,106,0,0,0,114,39,0,0, 0,218,11,112,97,116,104,95,102,105,110,100,101,114,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,114,190,0, - 0,0,100,4,0,0,115,8,0,0,0,0,1,6,1,6, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,191,0, + 0,0,106,4,0,0,115,8,0,0,0,0,1,6,1,6, 1,14,1,122,23,95,78,97,109,101,115,112,97,99,101,80, 97,116,104,46,95,95,105,110,105,116,95,95,99,1,0,0, 0,0,0,0,0,4,0,0,0,3,0,0,0,67,0,0, @@ -1827,27 +1842,27 @@ const unsigned char _Py_M__importlib_external[] = { 82,101,116,117,114,110,115,32,97,32,116,117,112,108,101,32, 111,102,32,40,112,97,114,101,110,116,45,109,111,100,117,108, 101,45,110,97,109,101,44,32,112,97,114,101,110,116,45,112, - 97,116,104,45,97,116,116,114,45,110,97,109,101,41,114,61, - 0,0,0,114,30,0,0,0,41,2,114,6,0,0,0,114, - 35,0,0,0,90,8,95,95,112,97,116,104,95,95,41,2, - 114,246,0,0,0,114,32,0,0,0,41,4,114,106,0,0, - 0,114,237,0,0,0,218,3,100,111,116,90,2,109,101,114, + 97,116,104,45,97,116,116,114,45,110,97,109,101,41,114,63, + 0,0,0,114,35,0,0,0,41,2,114,6,0,0,0,114, + 39,0,0,0,90,8,95,95,112,97,116,104,95,95,41,2, + 114,247,0,0,0,114,36,0,0,0,41,4,114,108,0,0, + 0,114,238,0,0,0,218,3,100,111,116,90,2,109,101,114, 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,23, 95,102,105,110,100,95,112,97,114,101,110,116,95,112,97,116, - 104,95,110,97,109,101,115,106,4,0,0,115,8,0,0,0, + 104,95,110,97,109,101,115,112,4,0,0,115,8,0,0,0, 0,2,18,1,8,2,4,3,122,38,95,78,97,109,101,115, 112,97,99,101,80,97,116,104,46,95,102,105,110,100,95,112, 97,114,101,110,116,95,112,97,116,104,95,110,97,109,101,115, 99,1,0,0,0,0,0,0,0,3,0,0,0,3,0,0, 0,67,0,0,0,115,28,0,0,0,124,0,160,0,161,0, 92,2,125,1,125,2,116,1,116,2,106,3,124,1,25,0, - 124,2,131,2,83,0,41,1,78,41,4,114,253,0,0,0, - 114,116,0,0,0,114,6,0,0,0,218,7,109,111,100,117, - 108,101,115,41,3,114,106,0,0,0,90,18,112,97,114,101, + 124,2,131,2,83,0,41,1,78,41,4,114,254,0,0,0, + 114,118,0,0,0,114,6,0,0,0,218,7,109,111,100,117, + 108,101,115,41,3,114,108,0,0,0,90,18,112,97,114,101, 110,116,95,109,111,100,117,108,101,95,110,97,109,101,90,14, 112,97,116,104,95,97,116,116,114,95,110,97,109,101,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,114,248,0, - 0,0,116,4,0,0,115,4,0,0,0,0,1,12,1,122, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,249,0, + 0,0,122,4,0,0,115,4,0,0,0,0,1,12,1,122, 31,95,78,97,109,101,115,112,97,99,101,80,97,116,104,46, 95,103,101,116,95,112,97,114,101,110,116,95,112,97,116,104, 99,1,0,0,0,0,0,0,0,3,0,0,0,4,0,0, @@ -1857,72 +1872,72 @@ const unsigned char _Py_M__importlib_external[] = { 100,0,107,9,114,68,124,2,106,5,100,0,107,8,114,68, 124,2,106,6,114,68,124,2,106,6,124,0,95,7,124,1, 124,0,95,2,124,0,106,7,83,0,41,1,78,41,8,114, - 100,0,0,0,114,248,0,0,0,114,249,0,0,0,114,250, - 0,0,0,114,246,0,0,0,114,126,0,0,0,114,162,0, - 0,0,114,247,0,0,0,41,3,114,106,0,0,0,90,11, - 112,97,114,101,110,116,95,112,97,116,104,114,170,0,0,0, + 102,0,0,0,114,249,0,0,0,114,250,0,0,0,114,251, + 0,0,0,114,247,0,0,0,114,128,0,0,0,114,163,0, + 0,0,114,248,0,0,0,41,3,114,108,0,0,0,90,11, + 112,97,114,101,110,116,95,112,97,116,104,114,171,0,0,0, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 12,95,114,101,99,97,108,99,117,108,97,116,101,120,4,0, + 12,95,114,101,99,97,108,99,117,108,97,116,101,126,4,0, 0,115,16,0,0,0,0,2,12,1,10,1,14,3,18,1, 6,1,8,1,6,1,122,27,95,78,97,109,101,115,112,97, 99,101,80,97,116,104,46,95,114,101,99,97,108,99,117,108, 97,116,101,99,1,0,0,0,0,0,0,0,1,0,0,0, 3,0,0,0,67,0,0,0,115,12,0,0,0,116,0,124, - 0,160,1,161,0,131,1,83,0,41,1,78,41,2,114,230, - 0,0,0,114,255,0,0,0,41,1,114,106,0,0,0,114, + 0,160,1,161,0,131,1,83,0,41,1,78,41,2,114,231, + 0,0,0,114,0,1,0,0,41,1,114,108,0,0,0,114, 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,8, - 95,95,105,116,101,114,95,95,133,4,0,0,115,2,0,0, + 95,95,105,116,101,114,95,95,139,4,0,0,115,2,0,0, 0,0,1,122,23,95,78,97,109,101,115,112,97,99,101,80, 97,116,104,46,95,95,105,116,101,114,95,95,99,3,0,0, 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, 0,115,14,0,0,0,124,2,124,0,106,0,124,1,60,0, - 100,0,83,0,41,1,78,41,1,114,247,0,0,0,41,3, - 114,106,0,0,0,218,5,105,110,100,101,120,114,35,0,0, + 100,0,83,0,41,1,78,41,1,114,248,0,0,0,41,3, + 114,108,0,0,0,218,5,105,110,100,101,120,114,39,0,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 218,11,95,95,115,101,116,105,116,101,109,95,95,136,4,0, + 218,11,95,95,115,101,116,105,116,101,109,95,95,142,4,0, 0,115,2,0,0,0,0,1,122,26,95,78,97,109,101,115, 112,97,99,101,80,97,116,104,46,95,95,115,101,116,105,116, 101,109,95,95,99,1,0,0,0,0,0,0,0,1,0,0, 0,3,0,0,0,67,0,0,0,115,12,0,0,0,116,0, 124,0,160,1,161,0,131,1,83,0,41,1,78,41,2,114, - 31,0,0,0,114,255,0,0,0,41,1,114,106,0,0,0, + 18,0,0,0,114,0,1,0,0,41,1,114,108,0,0,0, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,218, - 7,95,95,108,101,110,95,95,139,4,0,0,115,2,0,0, + 7,95,95,108,101,110,95,95,145,4,0,0,115,2,0,0, 0,0,1,122,22,95,78,97,109,101,115,112,97,99,101,80, 97,116,104,46,95,95,108,101,110,95,95,99,1,0,0,0, 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, 115,12,0,0,0,100,1,160,0,124,0,106,1,161,1,83, 0,41,2,78,122,20,95,78,97,109,101,115,112,97,99,101, - 80,97,116,104,40,123,33,114,125,41,41,2,114,50,0,0, - 0,114,247,0,0,0,41,1,114,106,0,0,0,114,2,0, + 80,97,116,104,40,123,33,114,125,41,41,2,114,54,0,0, + 0,114,248,0,0,0,41,1,114,108,0,0,0,114,2,0, 0,0,114,2,0,0,0,114,4,0,0,0,218,8,95,95, - 114,101,112,114,95,95,142,4,0,0,115,2,0,0,0,0, + 114,101,112,114,95,95,148,4,0,0,115,2,0,0,0,0, 1,122,23,95,78,97,109,101,115,112,97,99,101,80,97,116, 104,46,95,95,114,101,112,114,95,95,99,2,0,0,0,0, 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, 12,0,0,0,124,1,124,0,160,0,161,0,107,6,83,0, - 41,1,78,41,1,114,255,0,0,0,41,2,114,106,0,0, + 41,1,78,41,1,114,0,1,0,0,41,2,114,108,0,0, 0,218,4,105,116,101,109,114,2,0,0,0,114,2,0,0, 0,114,4,0,0,0,218,12,95,95,99,111,110,116,97,105, - 110,115,95,95,145,4,0,0,115,2,0,0,0,0,1,122, + 110,115,95,95,151,4,0,0,115,2,0,0,0,0,1,122, 27,95,78,97,109,101,115,112,97,99,101,80,97,116,104,46, 95,95,99,111,110,116,97,105,110,115,95,95,99,2,0,0, 0,0,0,0,0,2,0,0,0,3,0,0,0,67,0,0, 0,115,16,0,0,0,124,0,106,0,160,1,124,1,161,1, - 1,0,100,0,83,0,41,1,78,41,2,114,247,0,0,0, - 114,169,0,0,0,41,2,114,106,0,0,0,114,5,1,0, + 1,0,100,0,83,0,41,1,78,41,2,114,248,0,0,0, + 114,170,0,0,0,41,2,114,108,0,0,0,114,6,1,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 114,169,0,0,0,148,4,0,0,115,2,0,0,0,0,1, + 114,170,0,0,0,154,4,0,0,115,2,0,0,0,0,1, 122,21,95,78,97,109,101,115,112,97,99,101,80,97,116,104, - 46,97,112,112,101,110,100,78,41,14,114,111,0,0,0,114, - 110,0,0,0,114,112,0,0,0,114,113,0,0,0,114,190, - 0,0,0,114,253,0,0,0,114,248,0,0,0,114,255,0, - 0,0,114,0,1,0,0,114,2,1,0,0,114,3,1,0, - 0,114,4,1,0,0,114,6,1,0,0,114,169,0,0,0, + 46,97,112,112,101,110,100,78,41,14,114,113,0,0,0,114, + 112,0,0,0,114,114,0,0,0,114,115,0,0,0,114,191, + 0,0,0,114,254,0,0,0,114,249,0,0,0,114,0,1, + 0,0,114,1,1,0,0,114,3,1,0,0,114,4,1,0, + 0,114,5,1,0,0,114,7,1,0,0,114,170,0,0,0, 114,2,0,0,0,114,2,0,0,0,114,2,0,0,0,114, - 4,0,0,0,114,245,0,0,0,93,4,0,0,115,22,0, + 4,0,0,0,114,246,0,0,0,99,4,0,0,115,22,0, 0,0,8,5,4,2,8,6,8,10,8,4,8,13,8,3, - 8,3,8,3,8,3,8,3,114,245,0,0,0,99,0,0, + 8,3,8,3,8,3,8,3,114,246,0,0,0,99,0,0, 0,0,0,0,0,0,0,0,0,0,3,0,0,0,64,0, 0,0,115,80,0,0,0,101,0,90,1,100,0,90,2,100, 1,100,2,132,0,90,3,101,4,100,3,100,4,132,0,131, @@ -1933,11 +1948,11 @@ const unsigned char _Py_M__importlib_external[] = { 115,112,97,99,101,76,111,97,100,101,114,99,4,0,0,0, 0,0,0,0,4,0,0,0,4,0,0,0,67,0,0,0, 115,18,0,0,0,116,0,124,1,124,2,124,3,131,3,124, - 0,95,1,100,0,83,0,41,1,78,41,2,114,245,0,0, - 0,114,247,0,0,0,41,4,114,106,0,0,0,114,104,0, - 0,0,114,35,0,0,0,114,251,0,0,0,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,114,190,0,0,0, - 154,4,0,0,115,2,0,0,0,0,1,122,25,95,78,97, + 0,95,1,100,0,83,0,41,1,78,41,2,114,246,0,0, + 0,114,248,0,0,0,41,4,114,108,0,0,0,114,106,0, + 0,0,114,39,0,0,0,114,252,0,0,0,114,2,0,0, + 0,114,2,0,0,0,114,4,0,0,0,114,191,0,0,0, + 160,4,0,0,115,2,0,0,0,0,1,122,25,95,78,97, 109,101,115,112,97,99,101,76,111,97,100,101,114,46,95,95, 105,110,105,116,95,95,99,2,0,0,0,0,0,0,0,2, 0,0,0,3,0,0,0,67,0,0,0,115,12,0,0,0, @@ -1951,33 +1966,33 @@ const unsigned char _Py_M__importlib_external[] = { 98,32,105,116,115,101,108,102,46,10,10,32,32,32,32,32, 32,32,32,122,25,60,109,111,100,117,108,101,32,123,33,114, 125,32,40,110,97,109,101,115,112,97,99,101,41,62,41,2, - 114,50,0,0,0,114,111,0,0,0,41,2,114,176,0,0, - 0,114,195,0,0,0,114,2,0,0,0,114,2,0,0,0, + 114,54,0,0,0,114,113,0,0,0,41,2,114,177,0,0, + 0,114,196,0,0,0,114,2,0,0,0,114,2,0,0,0, 114,4,0,0,0,218,11,109,111,100,117,108,101,95,114,101, - 112,114,157,4,0,0,115,2,0,0,0,0,7,122,28,95, + 112,114,163,4,0,0,115,2,0,0,0,0,7,122,28,95, 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, 109,111,100,117,108,101,95,114,101,112,114,99,2,0,0,0, 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, 115,4,0,0,0,100,1,83,0,41,2,78,84,114,2,0, - 0,0,41,2,114,106,0,0,0,114,125,0,0,0,114,2, - 0,0,0,114,2,0,0,0,114,4,0,0,0,114,165,0, - 0,0,166,4,0,0,115,2,0,0,0,0,1,122,27,95, + 0,0,41,2,114,108,0,0,0,114,127,0,0,0,114,2, + 0,0,0,114,2,0,0,0,114,4,0,0,0,114,166,0, + 0,0,172,4,0,0,115,2,0,0,0,0,1,122,27,95, 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, 105,115,95,112,97,99,107,97,103,101,99,2,0,0,0,0, 0,0,0,2,0,0,0,1,0,0,0,67,0,0,0,115, - 4,0,0,0,100,1,83,0,41,2,78,114,30,0,0,0, - 114,2,0,0,0,41,2,114,106,0,0,0,114,125,0,0, + 4,0,0,0,100,1,83,0,41,2,78,114,35,0,0,0, + 114,2,0,0,0,41,2,114,108,0,0,0,114,127,0,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 114,206,0,0,0,169,4,0,0,115,2,0,0,0,0,1, + 114,207,0,0,0,175,4,0,0,115,2,0,0,0,0,1, 122,27,95,78,97,109,101,115,112,97,99,101,76,111,97,100, 101,114,46,103,101,116,95,115,111,117,114,99,101,99,2,0, 0,0,0,0,0,0,2,0,0,0,6,0,0,0,67,0, 0,0,115,16,0,0,0,116,0,100,1,100,2,100,3,100, - 4,100,5,141,4,83,0,41,6,78,114,30,0,0,0,122, - 8,60,115,116,114,105,110,103,62,114,194,0,0,0,84,41, - 1,114,208,0,0,0,41,1,114,209,0,0,0,41,2,114, - 106,0,0,0,114,125,0,0,0,114,2,0,0,0,114,2, - 0,0,0,114,4,0,0,0,114,192,0,0,0,172,4,0, + 4,100,5,141,4,83,0,41,6,78,114,35,0,0,0,122, + 8,60,115,116,114,105,110,103,62,114,195,0,0,0,84,41, + 1,114,209,0,0,0,41,1,114,210,0,0,0,41,2,114, + 108,0,0,0,114,127,0,0,0,114,2,0,0,0,114,2, + 0,0,0,114,4,0,0,0,114,193,0,0,0,178,4,0, 0,115,2,0,0,0,0,1,122,25,95,78,97,109,101,115, 112,97,99,101,76,111,97,100,101,114,46,103,101,116,95,99, 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, @@ -1985,16 +2000,16 @@ const unsigned char _Py_M__importlib_external[] = { 0,41,2,122,42,85,115,101,32,100,101,102,97,117,108,116, 32,115,101,109,97,110,116,105,99,115,32,102,111,114,32,109, 111,100,117,108,101,32,99,114,101,97,116,105,111,110,46,78, - 114,2,0,0,0,41,2,114,106,0,0,0,114,170,0,0, + 114,2,0,0,0,41,2,114,108,0,0,0,114,171,0,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 114,191,0,0,0,175,4,0,0,115,2,0,0,0,0,1, + 114,192,0,0,0,181,4,0,0,115,2,0,0,0,0,1, 122,30,95,78,97,109,101,115,112,97,99,101,76,111,97,100, 101,114,46,99,114,101,97,116,101,95,109,111,100,117,108,101, 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, 0,67,0,0,0,115,4,0,0,0,100,0,83,0,41,1, - 78,114,2,0,0,0,41,2,114,106,0,0,0,114,195,0, + 78,114,2,0,0,0,41,2,114,108,0,0,0,114,196,0, 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, - 0,114,196,0,0,0,178,4,0,0,115,2,0,0,0,0, + 0,114,197,0,0,0,184,4,0,0,115,2,0,0,0,0, 1,122,28,95,78,97,109,101,115,112,97,99,101,76,111,97, 100,101,114,46,101,120,101,99,95,109,111,100,117,108,101,99, 2,0,0,0,0,0,0,0,2,0,0,0,4,0,0,0, @@ -2009,20 +2024,20 @@ const unsigned char _Py_M__importlib_external[] = { 10,32,32,32,32,32,32,32,32,122,38,110,97,109,101,115, 112,97,99,101,32,109,111,100,117,108,101,32,108,111,97,100, 101,100,32,119,105,116,104,32,112,97,116,104,32,123,33,114, - 125,41,4,114,120,0,0,0,114,134,0,0,0,114,247,0, - 0,0,114,197,0,0,0,41,2,114,106,0,0,0,114,125, + 125,41,4,114,122,0,0,0,114,136,0,0,0,114,248,0, + 0,0,114,198,0,0,0,41,2,114,108,0,0,0,114,127, 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,114,198,0,0,0,181,4,0,0,115,8,0,0,0, + 0,0,114,199,0,0,0,187,4,0,0,115,8,0,0,0, 0,7,6,1,4,255,4,2,122,28,95,78,97,109,101,115, 112,97,99,101,76,111,97,100,101,114,46,108,111,97,100,95, - 109,111,100,117,108,101,78,41,12,114,111,0,0,0,114,110, - 0,0,0,114,112,0,0,0,114,190,0,0,0,114,188,0, - 0,0,114,8,1,0,0,114,165,0,0,0,114,206,0,0, - 0,114,192,0,0,0,114,191,0,0,0,114,196,0,0,0, - 114,198,0,0,0,114,2,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,7,1,0,0,153,4, + 109,111,100,117,108,101,78,41,12,114,113,0,0,0,114,112, + 0,0,0,114,114,0,0,0,114,191,0,0,0,114,189,0, + 0,0,114,9,1,0,0,114,166,0,0,0,114,207,0,0, + 0,114,193,0,0,0,114,192,0,0,0,114,197,0,0,0, + 114,199,0,0,0,114,2,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,8,1,0,0,159,4, 0,0,115,16,0,0,0,8,1,8,3,12,9,8,3,8, - 3,8,3,8,3,8,3,114,7,1,0,0,99,0,0,0, + 3,8,3,8,3,8,3,114,8,1,0,0,99,0,0,0, 0,0,0,0,0,0,0,0,0,4,0,0,0,64,0,0, 0,115,114,0,0,0,101,0,90,1,100,0,90,2,100,1, 90,3,101,4,100,2,100,3,132,0,131,1,90,5,101,4, @@ -2053,11 +2068,11 @@ const unsigned char _Py_M__importlib_external[] = { 46,78,218,17,105,110,118,97,108,105,100,97,116,101,95,99, 97,99,104,101,115,41,6,218,4,108,105,115,116,114,6,0, 0,0,218,19,112,97,116,104,95,105,109,112,111,114,116,101, - 114,95,99,97,99,104,101,218,5,105,116,101,109,115,114,114, - 0,0,0,114,10,1,0,0,41,3,114,176,0,0,0,114, - 104,0,0,0,218,6,102,105,110,100,101,114,114,2,0,0, - 0,114,2,0,0,0,114,4,0,0,0,114,10,1,0,0, - 199,4,0,0,115,10,0,0,0,0,4,22,1,8,1,10, + 114,95,99,97,99,104,101,218,5,105,116,101,109,115,114,116, + 0,0,0,114,11,1,0,0,41,3,114,177,0,0,0,114, + 106,0,0,0,218,6,102,105,110,100,101,114,114,2,0,0, + 0,114,2,0,0,0,114,4,0,0,0,114,11,1,0,0, + 205,4,0,0,115,10,0,0,0,0,4,22,1,8,1,10, 1,10,1,122,28,80,97,116,104,70,105,110,100,101,114,46, 105,110,118,97,108,105,100,97,116,101,95,99,97,99,104,101, 115,99,2,0,0,0,0,0,0,0,3,0,0,0,9,0, @@ -2072,12 +2087,12 @@ const unsigned char _Py_M__importlib_external[] = { 105,110,100,101,114,32,102,111,114,32,39,112,97,116,104,39, 46,78,122,23,115,121,115,46,112,97,116,104,95,104,111,111, 107,115,32,105,115,32,101,109,112,116,121,41,6,114,6,0, - 0,0,218,10,112,97,116,104,95,104,111,111,107,115,114,64, - 0,0,0,114,65,0,0,0,114,124,0,0,0,114,105,0, - 0,0,41,3,114,176,0,0,0,114,35,0,0,0,90,4, + 0,0,218,10,112,97,116,104,95,104,111,111,107,115,114,66, + 0,0,0,114,67,0,0,0,114,126,0,0,0,114,107,0, + 0,0,41,3,114,177,0,0,0,114,39,0,0,0,90,4, 104,111,111,107,114,2,0,0,0,114,2,0,0,0,114,4, 0,0,0,218,11,95,112,97,116,104,95,104,111,111,107,115, - 209,4,0,0,115,16,0,0,0,0,3,16,1,12,1,10, + 215,4,0,0,115,16,0,0,0,0,3,16,1,12,1,10, 1,2,1,14,1,14,1,12,2,122,22,80,97,116,104,70, 105,110,100,101,114,46,95,112,97,116,104,95,104,111,111,107, 115,99,2,0,0,0,0,0,0,0,3,0,0,0,8,0, @@ -2101,13 +2116,13 @@ const unsigned char _Py_M__importlib_external[] = { 97,99,104,101,32,105,116,46,32,73,102,32,110,111,32,102, 105,110,100,101,114,32,105,115,32,97,118,97,105,108,97,98, 108,101,44,32,115,116,111,114,101,32,78,111,110,101,46,10, - 10,32,32,32,32,32,32,32,32,114,30,0,0,0,78,41, - 7,114,1,0,0,0,114,45,0,0,0,114,228,0,0,0, - 114,6,0,0,0,114,12,1,0,0,218,8,75,101,121,69, - 114,114,111,114,114,16,1,0,0,41,3,114,176,0,0,0, - 114,35,0,0,0,114,14,1,0,0,114,2,0,0,0,114, + 10,32,32,32,32,32,32,32,32,114,35,0,0,0,78,41, + 7,114,1,0,0,0,114,49,0,0,0,114,229,0,0,0, + 114,6,0,0,0,114,13,1,0,0,218,8,75,101,121,69, + 114,114,111,114,114,17,1,0,0,41,3,114,177,0,0,0, + 114,39,0,0,0,114,15,1,0,0,114,2,0,0,0,114, 2,0,0,0,114,4,0,0,0,218,20,95,112,97,116,104, - 95,105,109,112,111,114,116,101,114,95,99,97,99,104,101,222, + 95,105,109,112,111,114,116,101,114,95,99,97,99,104,101,228, 4,0,0,115,22,0,0,0,0,8,8,1,2,1,12,1, 14,3,8,1,2,1,14,1,14,1,10,1,16,1,122,31, 80,97,116,104,70,105,110,100,101,114,46,95,112,97,116,104, @@ -2118,14 +2133,14 @@ const unsigned char _Py_M__importlib_external[] = { 4,110,14,124,2,160,2,124,1,161,1,125,3,103,0,125, 4,124,3,100,0,107,9,114,60,116,3,160,4,124,1,124, 3,161,2,83,0,116,3,160,5,124,1,100,0,161,2,125, - 5,124,4,124,5,95,6,124,5,83,0,41,2,78,114,123, - 0,0,0,41,7,114,114,0,0,0,114,123,0,0,0,114, - 187,0,0,0,114,120,0,0,0,114,184,0,0,0,114,166, - 0,0,0,114,162,0,0,0,41,6,114,176,0,0,0,114, - 125,0,0,0,114,14,1,0,0,114,126,0,0,0,114,127, - 0,0,0,114,170,0,0,0,114,2,0,0,0,114,2,0, + 5,124,4,124,5,95,6,124,5,83,0,41,2,78,114,125, + 0,0,0,41,7,114,116,0,0,0,114,125,0,0,0,114, + 188,0,0,0,114,122,0,0,0,114,185,0,0,0,114,167, + 0,0,0,114,163,0,0,0,41,6,114,177,0,0,0,114, + 127,0,0,0,114,15,1,0,0,114,128,0,0,0,114,129, + 0,0,0,114,171,0,0,0,114,2,0,0,0,114,2,0, 0,0,114,4,0,0,0,218,16,95,108,101,103,97,99,121, - 95,103,101,116,95,115,112,101,99,244,4,0,0,115,18,0, + 95,103,101,116,95,115,112,101,99,250,4,0,0,115,18,0, 0,0,0,4,10,1,16,2,10,1,4,1,8,1,12,1, 12,1,6,1,122,27,80,97,116,104,70,105,110,100,101,114, 46,95,108,101,103,97,99,121,95,103,101,116,95,115,112,101, @@ -2145,18 +2160,18 @@ const unsigned char _Py_M__importlib_external[] = { 111,97,100,101,114,32,111,114,32,110,97,109,101,115,112,97, 99,101,95,112,97,116,104,32,102,111,114,32,116,104,105,115, 32,109,111,100,117,108,101,47,112,97,99,107,97,103,101,32, - 110,97,109,101,46,78,114,186,0,0,0,122,19,115,112,101, + 110,97,109,101,46,78,114,187,0,0,0,122,19,115,112,101, 99,32,109,105,115,115,105,110,103,32,108,111,97,100,101,114, - 41,13,114,145,0,0,0,114,74,0,0,0,218,5,98,121, - 116,101,115,114,18,1,0,0,114,114,0,0,0,114,186,0, - 0,0,114,19,1,0,0,114,126,0,0,0,114,162,0,0, - 0,114,105,0,0,0,114,151,0,0,0,114,120,0,0,0, - 114,166,0,0,0,41,9,114,176,0,0,0,114,125,0,0, - 0,114,35,0,0,0,114,185,0,0,0,218,14,110,97,109, + 41,13,114,147,0,0,0,114,76,0,0,0,218,5,98,121, + 116,101,115,114,19,1,0,0,114,116,0,0,0,114,187,0, + 0,0,114,20,1,0,0,114,128,0,0,0,114,163,0,0, + 0,114,107,0,0,0,114,153,0,0,0,114,122,0,0,0, + 114,167,0,0,0,41,9,114,177,0,0,0,114,127,0,0, + 0,114,39,0,0,0,114,186,0,0,0,218,14,110,97,109, 101,115,112,97,99,101,95,112,97,116,104,90,5,101,110,116, - 114,121,114,14,1,0,0,114,170,0,0,0,114,127,0,0, + 114,121,114,15,1,0,0,114,171,0,0,0,114,129,0,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 218,9,95,103,101,116,95,115,112,101,99,3,5,0,0,115, + 218,9,95,103,101,116,95,115,112,101,99,9,5,0,0,115, 40,0,0,0,0,5,4,1,8,1,14,1,2,1,10,1, 8,1,10,1,14,2,12,1,8,1,2,1,10,1,8,1, 6,1,8,1,8,5,12,2,12,1,6,1,122,20,80,97, @@ -2178,12 +2193,12 @@ const unsigned char _Py_M__importlib_external[] = { 46,112,97,116,104,95,104,111,111,107,115,32,97,110,100,32, 115,121,115,46,112,97,116,104,95,105,109,112,111,114,116,101, 114,95,99,97,99,104,101,46,10,32,32,32,32,32,32,32, - 32,78,41,7,114,6,0,0,0,114,35,0,0,0,114,22, - 1,0,0,114,126,0,0,0,114,162,0,0,0,114,164,0, - 0,0,114,245,0,0,0,41,6,114,176,0,0,0,114,125, - 0,0,0,114,35,0,0,0,114,185,0,0,0,114,170,0, - 0,0,114,21,1,0,0,114,2,0,0,0,114,2,0,0, - 0,114,4,0,0,0,114,186,0,0,0,35,5,0,0,115, + 32,78,41,7,114,6,0,0,0,114,39,0,0,0,114,23, + 1,0,0,114,128,0,0,0,114,163,0,0,0,114,165,0, + 0,0,114,246,0,0,0,41,6,114,177,0,0,0,114,127, + 0,0,0,114,39,0,0,0,114,186,0,0,0,114,171,0, + 0,0,114,22,1,0,0,114,2,0,0,0,114,2,0,0, + 0,114,4,0,0,0,114,187,0,0,0,41,5,0,0,115, 26,0,0,0,0,6,8,1,6,1,14,1,8,1,4,1, 10,1,6,1,4,3,6,1,16,1,4,2,6,2,122,20, 80,97,116,104,70,105,110,100,101,114,46,102,105,110,100,95, @@ -2201,21 +2216,21 @@ const unsigned char _Py_M__importlib_external[] = { 32,109,101,116,104,111,100,32,105,115,32,100,101,112,114,101, 99,97,116,101,100,46,32,32,85,115,101,32,102,105,110,100, 95,115,112,101,99,40,41,32,105,110,115,116,101,97,100,46, - 10,10,32,32,32,32,32,32,32,32,78,41,2,114,186,0, - 0,0,114,126,0,0,0,41,4,114,176,0,0,0,114,125, - 0,0,0,114,35,0,0,0,114,170,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,187,0,0, - 0,59,5,0,0,115,8,0,0,0,0,8,12,1,8,1, + 10,10,32,32,32,32,32,32,32,32,78,41,2,114,187,0, + 0,0,114,128,0,0,0,41,4,114,177,0,0,0,114,127, + 0,0,0,114,39,0,0,0,114,171,0,0,0,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,114,188,0,0, + 0,65,5,0,0,115,8,0,0,0,0,8,12,1,8,1, 4,1,122,22,80,97,116,104,70,105,110,100,101,114,46,102, - 105,110,100,95,109,111,100,117,108,101,41,12,114,111,0,0, - 0,114,110,0,0,0,114,112,0,0,0,114,113,0,0,0, - 114,188,0,0,0,114,10,1,0,0,114,16,1,0,0,114, - 18,1,0,0,114,19,1,0,0,114,22,1,0,0,114,186, - 0,0,0,114,187,0,0,0,114,2,0,0,0,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,9,1,0, - 0,195,4,0,0,115,30,0,0,0,8,2,4,2,12,10, + 105,110,100,95,109,111,100,117,108,101,41,12,114,113,0,0, + 0,114,112,0,0,0,114,114,0,0,0,114,115,0,0,0, + 114,189,0,0,0,114,11,1,0,0,114,17,1,0,0,114, + 19,1,0,0,114,20,1,0,0,114,23,1,0,0,114,187, + 0,0,0,114,188,0,0,0,114,2,0,0,0,114,2,0, + 0,0,114,2,0,0,0,114,4,0,0,0,114,10,1,0, + 0,201,4,0,0,115,30,0,0,0,8,2,4,2,12,10, 12,13,12,22,12,15,2,1,2,255,12,32,2,1,2,0, - 2,255,12,24,2,1,2,255,114,9,1,0,0,99,0,0, + 2,255,12,24,2,1,2,255,114,10,1,0,0,99,0,0, 0,0,0,0,0,0,0,0,0,0,3,0,0,0,64,0, 0,0,115,90,0,0,0,101,0,90,1,100,0,90,2,100, 1,90,3,100,2,100,3,132,0,90,4,100,4,100,5,132, @@ -2255,21 +2270,21 @@ const unsigned char _Py_M__importlib_external[] = { 0,0,2,0,0,0,3,0,0,0,51,0,0,0,115,22, 0,0,0,124,0,93,14,125,1,124,1,136,0,102,2,86, 0,1,0,113,2,100,0,83,0,41,1,78,114,2,0,0, - 0,41,2,114,22,0,0,0,114,240,0,0,0,41,1,114, - 126,0,0,0,114,2,0,0,0,114,4,0,0,0,114,242, - 0,0,0,88,5,0,0,115,4,0,0,0,4,0,2,0, + 0,41,2,114,27,0,0,0,114,241,0,0,0,41,1,114, + 128,0,0,0,114,2,0,0,0,114,4,0,0,0,114,243, + 0,0,0,94,5,0,0,115,4,0,0,0,4,0,2,0, 122,38,70,105,108,101,70,105,110,100,101,114,46,95,95,105, 110,105,116,95,95,46,60,108,111,99,97,108,115,62,46,60, - 103,101,110,101,120,112,114,62,114,61,0,0,0,114,94,0, - 0,0,78,41,7,114,151,0,0,0,218,8,95,108,111,97, - 100,101,114,115,114,35,0,0,0,218,11,95,112,97,116,104, + 103,101,110,101,120,112,114,62,114,63,0,0,0,114,96,0, + 0,0,78,41,7,114,153,0,0,0,218,8,95,108,111,97, + 100,101,114,115,114,39,0,0,0,218,11,95,112,97,116,104, 95,109,116,105,109,101,218,3,115,101,116,218,11,95,112,97, 116,104,95,99,97,99,104,101,218,19,95,114,101,108,97,120, 101,100,95,112,97,116,104,95,99,97,99,104,101,41,5,114, - 106,0,0,0,114,35,0,0,0,218,14,108,111,97,100,101, + 108,0,0,0,114,39,0,0,0,218,14,108,111,97,100,101, 114,95,100,101,116,97,105,108,115,90,7,108,111,97,100,101, - 114,115,114,172,0,0,0,114,2,0,0,0,41,1,114,126, - 0,0,0,114,4,0,0,0,114,190,0,0,0,82,5,0, + 114,115,114,173,0,0,0,114,2,0,0,0,41,1,114,128, + 0,0,0,114,4,0,0,0,114,191,0,0,0,88,5,0, 0,115,16,0,0,0,0,4,4,1,12,1,26,1,6,2, 10,1,6,1,8,1,122,19,70,105,108,101,70,105,110,100, 101,114,46,95,95,105,110,105,116,95,95,99,1,0,0,0, @@ -2277,9 +2292,9 @@ const unsigned char _Py_M__importlib_external[] = { 115,10,0,0,0,100,1,124,0,95,0,100,2,83,0,41, 3,122,31,73,110,118,97,108,105,100,97,116,101,32,116,104, 101,32,100,105,114,101,99,116,111,114,121,32,109,116,105,109, - 101,46,114,94,0,0,0,78,41,1,114,25,1,0,0,41, - 1,114,106,0,0,0,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,114,10,1,0,0,96,5,0,0,115,2, + 101,46,114,96,0,0,0,78,41,1,114,26,1,0,0,41, + 1,114,108,0,0,0,114,2,0,0,0,114,2,0,0,0, + 114,4,0,0,0,114,11,1,0,0,102,5,0,0,115,2, 0,0,0,0,2,122,28,70,105,108,101,70,105,110,100,101, 114,46,105,110,118,97,108,105,100,97,116,101,95,99,97,99, 104,101,115,99,2,0,0,0,0,0,0,0,3,0,0,0, @@ -2299,21 +2314,21 @@ const unsigned char _Py_M__importlib_external[] = { 32,100,101,112,114,101,99,97,116,101,100,46,32,32,85,115, 101,32,102,105,110,100,95,115,112,101,99,40,41,32,105,110, 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, - 78,41,3,114,186,0,0,0,114,126,0,0,0,114,162,0, - 0,0,41,3,114,106,0,0,0,114,125,0,0,0,114,170, + 78,41,3,114,187,0,0,0,114,128,0,0,0,114,163,0, + 0,0,41,3,114,108,0,0,0,114,127,0,0,0,114,171, 0,0,0,114,2,0,0,0,114,2,0,0,0,114,4,0, - 0,0,114,123,0,0,0,102,5,0,0,115,8,0,0,0, + 0,0,114,125,0,0,0,108,5,0,0,115,8,0,0,0, 0,7,10,1,8,1,8,1,122,22,70,105,108,101,70,105, 110,100,101,114,46,102,105,110,100,95,108,111,97,100,101,114, 99,6,0,0,0,0,0,0,0,7,0,0,0,6,0,0, 0,67,0,0,0,115,26,0,0,0,124,1,124,2,124,3, 131,2,125,6,116,0,124,2,124,3,124,6,124,4,100,1, - 141,4,83,0,41,2,78,41,2,114,126,0,0,0,114,162, - 0,0,0,41,1,114,173,0,0,0,41,7,114,106,0,0, - 0,114,171,0,0,0,114,125,0,0,0,114,35,0,0,0, - 90,4,115,109,115,108,114,185,0,0,0,114,126,0,0,0, + 141,4,83,0,41,2,78,41,2,114,128,0,0,0,114,163, + 0,0,0,41,1,114,174,0,0,0,41,7,114,108,0,0, + 0,114,172,0,0,0,114,127,0,0,0,114,39,0,0,0, + 90,4,115,109,115,108,114,186,0,0,0,114,128,0,0,0, 114,2,0,0,0,114,2,0,0,0,114,4,0,0,0,114, - 22,1,0,0,114,5,0,0,115,8,0,0,0,0,1,10, + 23,1,0,0,120,5,0,0,115,8,0,0,0,0,1,10, 1,8,1,2,255,122,20,70,105,108,101,70,105,110,100,101, 114,46,95,103,101,116,95,115,112,101,99,78,99,3,0,0, 0,0,0,0,0,14,0,0,0,8,0,0,0,67,0,0, @@ -2347,27 +2362,27 @@ const unsigned char _Py_M__importlib_external[] = { 101,32,109,97,116,99,104,105,110,103,32,115,112,101,99,44, 32,111,114,32,78,111,110,101,32,105,102,32,110,111,116,32, 102,111,117,110,100,46,10,32,32,32,32,32,32,32,32,70, - 114,61,0,0,0,114,59,0,0,0,114,94,0,0,0,114, - 190,0,0,0,122,9,116,114,121,105,110,103,32,123,125,41, + 114,63,0,0,0,114,23,0,0,0,114,96,0,0,0,114, + 191,0,0,0,122,9,116,114,121,105,110,103,32,123,125,41, 1,90,9,118,101,114,98,111,115,105,116,121,78,122,25,112, 111,115,115,105,98,108,101,32,110,97,109,101,115,112,97,99, - 101,32,102,111,114,32,123,125,41,22,114,32,0,0,0,114, - 39,0,0,0,114,35,0,0,0,114,1,0,0,0,114,45, - 0,0,0,114,234,0,0,0,114,40,0,0,0,114,25,1, + 101,32,102,111,114,32,123,125,41,22,114,36,0,0,0,114, + 43,0,0,0,114,39,0,0,0,114,1,0,0,0,114,49, + 0,0,0,114,235,0,0,0,114,44,0,0,0,114,26,1, 0,0,218,11,95,102,105,108,108,95,99,97,99,104,101,114, - 5,0,0,0,114,28,1,0,0,114,95,0,0,0,114,27, - 1,0,0,114,28,0,0,0,114,24,1,0,0,114,44,0, - 0,0,114,22,1,0,0,114,46,0,0,0,114,120,0,0, - 0,114,134,0,0,0,114,166,0,0,0,114,162,0,0,0, - 41,14,114,106,0,0,0,114,125,0,0,0,114,185,0,0, + 5,0,0,0,114,29,1,0,0,114,97,0,0,0,114,28, + 1,0,0,114,33,0,0,0,114,25,1,0,0,114,48,0, + 0,0,114,23,1,0,0,114,50,0,0,0,114,122,0,0, + 0,114,136,0,0,0,114,167,0,0,0,114,163,0,0,0, + 41,14,114,108,0,0,0,114,127,0,0,0,114,186,0,0, 0,90,12,105,115,95,110,97,109,101,115,112,97,99,101,90, - 11,116,97,105,108,95,109,111,100,117,108,101,114,153,0,0, + 11,116,97,105,108,95,109,111,100,117,108,101,114,155,0,0, 0,90,5,99,97,99,104,101,90,12,99,97,99,104,101,95, 109,111,100,117,108,101,90,9,98,97,115,101,95,112,97,116, - 104,114,240,0,0,0,114,171,0,0,0,90,13,105,110,105, + 104,114,241,0,0,0,114,172,0,0,0,90,13,105,110,105, 116,95,102,105,108,101,110,97,109,101,90,9,102,117,108,108, - 95,112,97,116,104,114,170,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,186,0,0,0,119,5, + 95,112,97,116,104,114,171,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,187,0,0,0,125,5, 0,0,115,74,0,0,0,0,5,4,1,14,1,2,1,24, 1,14,1,10,1,10,1,8,1,6,2,6,1,6,1,10, 2,6,1,4,2,8,1,12,1,14,1,8,1,10,1,8, @@ -2392,31 +2407,31 @@ const unsigned char _Py_M__importlib_external[] = { 101,32,111,102,32,112,111,116,101,110,116,105,97,108,32,109, 111,100,117,108,101,115,32,97,110,100,32,112,97,99,107,97, 103,101,115,32,102,111,114,32,116,104,105,115,32,100,105,114, - 101,99,116,111,114,121,46,114,0,0,0,0,114,61,0,0, + 101,99,116,111,114,121,46,114,0,0,0,0,114,63,0,0, 0,122,5,123,125,46,123,125,99,1,0,0,0,0,0,0, 0,2,0,0,0,4,0,0,0,83,0,0,0,115,20,0, 0,0,104,0,124,0,93,12,125,1,124,1,160,0,161,0, - 146,2,113,4,83,0,114,2,0,0,0,41,1,114,95,0, - 0,0,41,2,114,22,0,0,0,90,2,102,110,114,2,0, + 146,2,113,4,83,0,114,2,0,0,0,41,1,114,97,0, + 0,0,41,2,114,27,0,0,0,90,2,102,110,114,2,0, 0,0,114,2,0,0,0,114,4,0,0,0,218,9,60,115, - 101,116,99,111,109,112,62,196,5,0,0,115,4,0,0,0, + 101,116,99,111,109,112,62,202,5,0,0,115,4,0,0,0, 6,0,2,0,122,41,70,105,108,101,70,105,110,100,101,114, 46,95,102,105,108,108,95,99,97,99,104,101,46,60,108,111, 99,97,108,115,62,46,60,115,101,116,99,111,109,112,62,78, - 41,18,114,35,0,0,0,114,1,0,0,0,114,231,0,0, - 0,114,45,0,0,0,114,228,0,0,0,218,15,80,101,114, + 41,18,114,39,0,0,0,114,1,0,0,0,114,232,0,0, + 0,114,49,0,0,0,114,229,0,0,0,218,15,80,101,114, 109,105,115,115,105,111,110,69,114,114,111,114,218,18,78,111, 116,65,68,105,114,101,99,116,111,114,121,69,114,114,111,114, 114,6,0,0,0,114,7,0,0,0,114,8,0,0,0,114, - 26,1,0,0,114,27,1,0,0,114,90,0,0,0,114,50, - 0,0,0,114,95,0,0,0,218,3,97,100,100,114,9,0, - 0,0,114,28,1,0,0,41,9,114,106,0,0,0,114,35, - 0,0,0,114,232,0,0,0,90,21,108,111,119,101,114,95, + 27,1,0,0,114,28,1,0,0,114,92,0,0,0,114,54, + 0,0,0,114,97,0,0,0,218,3,97,100,100,114,9,0, + 0,0,114,29,1,0,0,41,9,114,108,0,0,0,114,39, + 0,0,0,114,233,0,0,0,90,21,108,111,119,101,114,95, 115,117,102,102,105,120,95,99,111,110,116,101,110,116,115,114, - 5,1,0,0,114,104,0,0,0,114,252,0,0,0,114,240, + 6,1,0,0,114,106,0,0,0,114,253,0,0,0,114,241, 0,0,0,90,8,110,101,119,95,110,97,109,101,114,2,0, - 0,0,114,2,0,0,0,114,4,0,0,0,114,30,1,0, - 0,167,5,0,0,115,34,0,0,0,0,2,6,1,2,1, + 0,0,114,2,0,0,0,114,4,0,0,0,114,31,1,0, + 0,173,5,0,0,115,34,0,0,0,0,2,6,1,2,1, 22,1,20,3,10,3,12,1,12,7,6,1,8,1,16,1, 4,1,18,2,4,1,12,1,6,1,12,1,122,22,70,105, 108,101,70,105,110,100,101,114,46,95,102,105,108,108,95,99, @@ -2449,38 +2464,38 @@ const unsigned char _Py_M__importlib_external[] = { 105,98,46,109,97,99,104,105,110,101,114,121,46,70,105,108, 101,70,105,110,100,101,114,46,122,30,111,110,108,121,32,100, 105,114,101,99,116,111,114,105,101,115,32,97,114,101,32,115, - 117,112,112,111,114,116,101,100,41,1,114,35,0,0,0,41, - 2,114,46,0,0,0,114,105,0,0,0,41,1,114,35,0, - 0,0,41,2,114,176,0,0,0,114,29,1,0,0,114,2, + 117,112,112,111,114,116,101,100,41,1,114,39,0,0,0,41, + 2,114,50,0,0,0,114,107,0,0,0,41,1,114,39,0, + 0,0,41,2,114,177,0,0,0,114,30,1,0,0,114,2, 0,0,0,114,4,0,0,0,218,24,112,97,116,104,95,104, 111,111,107,95,102,111,114,95,70,105,108,101,70,105,110,100, - 101,114,208,5,0,0,115,6,0,0,0,0,2,8,1,12, + 101,114,214,5,0,0,115,6,0,0,0,0,2,8,1,12, 1,122,54,70,105,108,101,70,105,110,100,101,114,46,112,97, 116,104,95,104,111,111,107,46,60,108,111,99,97,108,115,62, 46,112,97,116,104,95,104,111,111,107,95,102,111,114,95,70, 105,108,101,70,105,110,100,101,114,114,2,0,0,0,41,3, - 114,176,0,0,0,114,29,1,0,0,114,35,1,0,0,114, - 2,0,0,0,41,2,114,176,0,0,0,114,29,1,0,0, + 114,177,0,0,0,114,30,1,0,0,114,36,1,0,0,114, + 2,0,0,0,41,2,114,177,0,0,0,114,30,1,0,0, 114,4,0,0,0,218,9,112,97,116,104,95,104,111,111,107, - 198,5,0,0,115,4,0,0,0,0,10,14,6,122,20,70, + 204,5,0,0,115,4,0,0,0,0,10,14,6,122,20,70, 105,108,101,70,105,110,100,101,114,46,112,97,116,104,95,104, 111,111,107,99,1,0,0,0,0,0,0,0,1,0,0,0, 3,0,0,0,67,0,0,0,115,12,0,0,0,100,1,160, 0,124,0,106,1,161,1,83,0,41,2,78,122,16,70,105, 108,101,70,105,110,100,101,114,40,123,33,114,125,41,41,2, - 114,50,0,0,0,114,35,0,0,0,41,1,114,106,0,0, + 114,54,0,0,0,114,39,0,0,0,41,1,114,108,0,0, 0,114,2,0,0,0,114,2,0,0,0,114,4,0,0,0, - 114,4,1,0,0,216,5,0,0,115,2,0,0,0,0,1, + 114,5,1,0,0,222,5,0,0,115,2,0,0,0,0,1, 122,19,70,105,108,101,70,105,110,100,101,114,46,95,95,114, - 101,112,114,95,95,41,1,78,41,15,114,111,0,0,0,114, - 110,0,0,0,114,112,0,0,0,114,113,0,0,0,114,190, - 0,0,0,114,10,1,0,0,114,129,0,0,0,114,187,0, - 0,0,114,123,0,0,0,114,22,1,0,0,114,186,0,0, - 0,114,30,1,0,0,114,188,0,0,0,114,36,1,0,0, - 114,4,1,0,0,114,2,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,23,1,0,0,73,5, + 101,112,114,95,95,41,1,78,41,15,114,113,0,0,0,114, + 112,0,0,0,114,114,0,0,0,114,115,0,0,0,114,191, + 0,0,0,114,11,1,0,0,114,131,0,0,0,114,188,0, + 0,0,114,125,0,0,0,114,23,1,0,0,114,187,0,0, + 0,114,31,1,0,0,114,189,0,0,0,114,37,1,0,0, + 114,5,1,0,0,114,2,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,24,1,0,0,79,5, 0,0,115,20,0,0,0,8,7,4,2,8,14,8,4,4, - 2,8,12,8,5,10,48,8,31,12,18,114,23,1,0,0, + 2,8,12,8,5,10,48,8,31,12,18,114,24,1,0,0, 99,4,0,0,0,0,0,0,0,6,0,0,0,8,0,0, 0,67,0,0,0,115,146,0,0,0,124,0,160,0,100,1, 161,1,125,4,124,0,160,0,100,2,161,1,125,5,124,4, @@ -2493,18 +2508,18 @@ const unsigned char _Py_M__importlib_external[] = { 87,0,110,20,4,0,116,5,107,10,114,140,1,0,1,0, 1,0,89,0,110,2,88,0,100,0,83,0,41,6,78,218, 10,95,95,108,111,97,100,101,114,95,95,218,8,95,95,115, - 112,101,99,95,95,41,1,114,126,0,0,0,90,8,95,95, + 112,101,99,95,95,41,1,114,128,0,0,0,90,8,95,95, 102,105,108,101,95,95,90,10,95,95,99,97,99,104,101,100, - 95,95,41,6,218,3,103,101,116,114,126,0,0,0,114,238, - 0,0,0,114,233,0,0,0,114,173,0,0,0,218,9,69, - 120,99,101,112,116,105,111,110,41,6,90,2,110,115,114,104, + 95,95,41,6,218,3,103,101,116,114,128,0,0,0,114,239, + 0,0,0,114,234,0,0,0,114,174,0,0,0,218,9,69, + 120,99,101,112,116,105,111,110,41,6,90,2,110,115,114,106, 0,0,0,90,8,112,97,116,104,110,97,109,101,90,9,99, - 112,97,116,104,110,97,109,101,114,126,0,0,0,114,170,0, + 112,97,116,104,110,97,109,101,114,128,0,0,0,114,171,0, 0,0,114,2,0,0,0,114,2,0,0,0,114,4,0,0, 0,218,14,95,102,105,120,95,117,112,95,109,111,100,117,108, - 101,222,5,0,0,115,34,0,0,0,0,2,10,1,10,1, + 101,228,5,0,0,115,34,0,0,0,0,2,10,1,10,1, 4,1,4,1,8,1,8,1,12,2,10,1,4,1,14,1, - 2,1,8,1,8,1,8,1,12,1,14,2,114,41,1,0, + 2,1,8,1,8,1,8,1,12,1,14,2,114,42,1,0, 0,99,0,0,0,0,0,0,0,0,3,0,0,0,3,0, 0,0,67,0,0,0,115,38,0,0,0,116,0,116,1,160, 2,161,0,102,2,125,0,116,3,116,4,102,2,125,1,116, @@ -2515,14 +2530,14 @@ const unsigned char _Py_M__importlib_external[] = { 46,10,10,32,32,32,32,69,97,99,104,32,105,116,101,109, 32,105,115,32,97,32,116,117,112,108,101,32,40,108,111,97, 100,101,114,44,32,115,117,102,102,105,120,101,115,41,46,10, - 32,32,32,32,41,7,114,239,0,0,0,114,147,0,0,0, + 32,32,32,32,41,7,114,240,0,0,0,114,149,0,0,0, 218,18,101,120,116,101,110,115,105,111,110,95,115,117,102,102, - 105,120,101,115,114,233,0,0,0,114,91,0,0,0,114,238, - 0,0,0,114,78,0,0,0,41,3,90,10,101,120,116,101, + 105,120,101,115,114,234,0,0,0,114,93,0,0,0,114,239, + 0,0,0,114,80,0,0,0,41,3,90,10,101,120,116,101, 110,115,105,111,110,115,90,6,115,111,117,114,99,101,90,8, 98,121,116,101,99,111,100,101,114,2,0,0,0,114,2,0, - 0,0,114,4,0,0,0,114,167,0,0,0,245,5,0,0, - 115,8,0,0,0,0,5,12,1,8,1,8,1,114,167,0, + 0,0,114,4,0,0,0,114,168,0,0,0,251,5,0,0, + 115,8,0,0,0,0,5,12,1,8,1,8,1,114,168,0, 0,0,99,1,0,0,0,0,0,0,0,12,0,0,0,9, 0,0,0,67,0,0,0,115,178,1,0,0,124,0,97,0, 116,0,106,1,97,1,116,0,106,2,97,2,116,1,106,3, @@ -2565,57 +2580,57 @@ const unsigned char _Py_M__importlib_external[] = { 101,120,116,114,97,99,116,101,100,32,102,114,111,109,32,116, 104,101,32,99,111,114,101,32,98,111,111,116,115,116,114,97, 112,32,109,111,100,117,108,101,46,10,10,32,32,32,32,41, - 4,114,52,0,0,0,114,64,0,0,0,218,8,98,117,105, - 108,116,105,110,115,114,144,0,0,0,90,5,112,111,115,105, + 4,114,56,0,0,0,114,66,0,0,0,218,8,98,117,105, + 108,116,105,110,115,114,146,0,0,0,90,5,112,111,115,105, 120,250,1,47,90,2,110,116,250,1,92,99,1,0,0,0, 0,0,0,0,2,0,0,0,3,0,0,0,115,0,0,0, 115,26,0,0,0,124,0,93,18,125,1,116,0,124,1,131, 1,100,0,107,2,86,0,1,0,113,2,100,1,83,0,41, - 2,114,29,0,0,0,78,41,1,114,31,0,0,0,41,2, - 114,22,0,0,0,114,84,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,242,0,0,0,25,6, + 2,114,34,0,0,0,78,41,1,114,18,0,0,0,41,2, + 114,27,0,0,0,114,86,0,0,0,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,243,0,0,0,31,6, 0,0,115,4,0,0,0,4,0,2,0,122,25,95,115,101, 116,117,112,46,60,108,111,99,97,108,115,62,46,60,103,101, - 110,101,120,112,114,62,114,62,0,0,0,122,30,105,109,112, + 110,101,120,112,114,62,114,64,0,0,0,122,30,105,109,112, 111,114,116,108,105,98,32,114,101,113,117,105,114,101,115,32, 112,111,115,105,120,32,111,114,32,110,116,114,1,0,0,0, - 114,25,0,0,0,114,21,0,0,0,114,30,0,0,0,114, - 48,0,0,0,99,1,0,0,0,0,0,0,0,2,0,0, + 114,30,0,0,0,114,26,0,0,0,114,35,0,0,0,114, + 52,0,0,0,99,1,0,0,0,0,0,0,0,2,0,0, 0,4,0,0,0,83,0,0,0,115,22,0,0,0,104,0, 124,0,93,14,125,1,100,0,124,1,155,0,157,2,146,2, - 113,4,83,0,41,1,114,63,0,0,0,114,2,0,0,0, - 41,2,114,22,0,0,0,218,1,115,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,114,31,1,0,0,41,6, + 113,4,83,0,41,1,114,65,0,0,0,114,2,0,0,0, + 41,2,114,27,0,0,0,218,1,115,114,2,0,0,0,114, + 2,0,0,0,114,4,0,0,0,114,32,1,0,0,47,6, 0,0,115,4,0,0,0,6,0,2,0,122,25,95,115,101, 116,117,112,46,60,108,111,99,97,108,115,62,46,60,115,101, 116,99,111,109,112,62,90,7,95,116,104,114,101,97,100,90, 8,95,119,101,97,107,114,101,102,90,6,119,105,110,114,101, - 103,114,175,0,0,0,114,5,0,0,0,122,4,46,112,121, - 119,122,6,95,100,46,112,121,100,84,78,41,19,114,120,0, - 0,0,114,6,0,0,0,114,147,0,0,0,114,254,0,0, - 0,114,111,0,0,0,90,18,95,98,117,105,108,116,105,110, - 95,102,114,111,109,95,110,97,109,101,114,115,0,0,0,218, - 3,97,108,108,114,155,0,0,0,114,105,0,0,0,114,26, - 0,0,0,114,11,0,0,0,114,244,0,0,0,114,151,0, - 0,0,114,42,1,0,0,114,91,0,0,0,114,169,0,0, - 0,114,174,0,0,0,114,178,0,0,0,41,12,218,17,95, + 103,114,176,0,0,0,114,5,0,0,0,122,4,46,112,121, + 119,122,6,95,100,46,112,121,100,84,78,41,19,114,122,0, + 0,0,114,6,0,0,0,114,149,0,0,0,114,255,0,0, + 0,114,113,0,0,0,90,18,95,98,117,105,108,116,105,110, + 95,102,114,111,109,95,110,97,109,101,114,117,0,0,0,218, + 3,97,108,108,114,19,0,0,0,114,107,0,0,0,114,31, + 0,0,0,114,11,0,0,0,114,245,0,0,0,114,153,0, + 0,0,114,43,1,0,0,114,93,0,0,0,114,170,0,0, + 0,114,175,0,0,0,114,179,0,0,0,41,12,218,17,95, 98,111,111,116,115,116,114,97,112,95,109,111,100,117,108,101, 90,11,115,101,108,102,95,109,111,100,117,108,101,90,12,98, 117,105,108,116,105,110,95,110,97,109,101,90,14,98,117,105, 108,116,105,110,95,109,111,100,117,108,101,90,10,111,115,95, 100,101,116,97,105,108,115,90,10,98,117,105,108,116,105,110, - 95,111,115,114,21,0,0,0,114,25,0,0,0,90,9,111, + 95,111,115,114,26,0,0,0,114,30,0,0,0,90,9,111, 115,95,109,111,100,117,108,101,90,13,116,104,114,101,97,100, 95,109,111,100,117,108,101,90,14,119,101,97,107,114,101,102, 95,109,111,100,117,108,101,90,13,119,105,110,114,101,103,95, 109,111,100,117,108,101,114,2,0,0,0,114,2,0,0,0, - 114,4,0,0,0,218,6,95,115,101,116,117,112,0,6,0, + 114,4,0,0,0,218,6,95,115,101,116,117,112,6,6,0, 0,115,78,0,0,0,0,8,4,1,6,1,6,3,10,1, 8,1,10,1,12,2,10,1,14,3,22,1,12,2,22,1, 8,1,10,1,10,1,6,2,2,1,10,1,10,1,14,1, 12,2,8,1,12,1,12,1,18,1,22,3,10,1,12,3, 10,1,12,3,10,1,10,1,12,3,14,1,14,1,10,1, - 10,1,10,1,114,49,1,0,0,99,1,0,0,0,0,0, + 10,1,10,1,114,50,1,0,0,99,1,0,0,0,0,0, 0,0,2,0,0,0,4,0,0,0,67,0,0,0,115,50, 0,0,0,116,0,124,0,131,1,1,0,116,1,131,0,125, 1,116,2,106,3,160,4,116,5,106,6,124,1,142,0,103, @@ -2623,48 +2638,48 @@ const unsigned char _Py_M__importlib_external[] = { 0,100,1,83,0,41,2,122,41,73,110,115,116,97,108,108, 32,116,104,101,32,112,97,116,104,45,98,97,115,101,100,32, 105,109,112,111,114,116,32,99,111,109,112,111,110,101,110,116, - 115,46,78,41,10,114,49,1,0,0,114,167,0,0,0,114, - 6,0,0,0,114,15,1,0,0,114,151,0,0,0,114,23, - 1,0,0,114,36,1,0,0,218,9,109,101,116,97,95,112, - 97,116,104,114,169,0,0,0,114,9,1,0,0,41,2,114, - 48,1,0,0,90,17,115,117,112,112,111,114,116,101,100,95, + 115,46,78,41,10,114,50,1,0,0,114,168,0,0,0,114, + 6,0,0,0,114,16,1,0,0,114,153,0,0,0,114,24, + 1,0,0,114,37,1,0,0,218,9,109,101,116,97,95,112, + 97,116,104,114,170,0,0,0,114,10,1,0,0,41,2,114, + 49,1,0,0,90,17,115,117,112,112,111,114,116,101,100,95, 108,111,97,100,101,114,115,114,2,0,0,0,114,2,0,0, 0,114,4,0,0,0,218,8,95,105,110,115,116,97,108,108, - 65,6,0,0,115,8,0,0,0,0,2,8,1,6,1,20, - 1,114,51,1,0,0,41,62,114,113,0,0,0,114,10,0, + 71,6,0,0,115,8,0,0,0,0,2,8,1,6,1,20, + 1,114,52,1,0,0,41,63,114,115,0,0,0,114,10,0, 0,0,90,37,95,67,65,83,69,95,73,78,83,69,78,83, 73,84,73,86,69,95,80,76,65,84,70,79,82,77,83,95, 66,89,84,69,83,95,75,69,89,114,9,0,0,0,114,11, - 0,0,0,114,17,0,0,0,114,19,0,0,0,114,28,0, - 0,0,114,38,0,0,0,114,39,0,0,0,114,43,0,0, - 0,114,44,0,0,0,114,46,0,0,0,114,49,0,0,0, - 114,58,0,0,0,218,4,116,121,112,101,218,8,95,95,99, - 111,100,101,95,95,114,146,0,0,0,114,15,0,0,0,114, - 133,0,0,0,114,14,0,0,0,114,18,0,0,0,114,213, - 0,0,0,114,81,0,0,0,114,77,0,0,0,114,91,0, - 0,0,114,78,0,0,0,90,23,68,69,66,85,71,95,66, - 89,84,69,67,79,68,69,95,83,85,70,70,73,88,69,83, - 90,27,79,80,84,73,77,73,90,69,68,95,66,89,84,69, - 67,79,68,69,95,83,85,70,70,73,88,69,83,114,87,0, - 0,0,114,92,0,0,0,114,98,0,0,0,114,101,0,0, - 0,114,103,0,0,0,114,122,0,0,0,114,129,0,0,0, - 114,137,0,0,0,114,141,0,0,0,114,143,0,0,0,114, - 149,0,0,0,114,154,0,0,0,114,156,0,0,0,114,161, - 0,0,0,218,6,111,98,106,101,99,116,114,168,0,0,0, - 114,173,0,0,0,114,174,0,0,0,114,189,0,0,0,114, - 199,0,0,0,114,216,0,0,0,114,233,0,0,0,114,238, - 0,0,0,114,244,0,0,0,114,239,0,0,0,114,245,0, - 0,0,114,7,1,0,0,114,9,1,0,0,114,23,1,0, - 0,114,41,1,0,0,114,167,0,0,0,114,49,1,0,0, - 114,51,1,0,0,114,2,0,0,0,114,2,0,0,0,114, - 2,0,0,0,114,4,0,0,0,218,8,60,109,111,100,117, - 108,101,62,8,0,0,0,115,124,0,0,0,4,15,4,1, - 4,1,2,1,2,255,4,4,8,17,8,5,8,5,8,6, - 8,12,8,10,8,9,8,5,8,7,8,9,12,22,10,127, - 0,7,16,1,12,2,4,1,4,2,6,2,6,2,8,2, - 18,71,8,40,8,19,8,12,8,12,8,28,8,17,8,33, - 8,28,8,24,16,13,14,10,12,11,8,14,6,3,6,1, - 2,255,12,68,14,64,14,29,16,127,0,17,14,68,18,45, - 18,26,4,3,18,53,14,60,14,42,14,127,0,7,14,127, - 0,22,12,23,8,11,8,65, + 0,0,0,114,17,0,0,0,114,22,0,0,0,114,24,0, + 0,0,114,33,0,0,0,114,42,0,0,0,114,43,0,0, + 0,114,47,0,0,0,114,48,0,0,0,114,50,0,0,0, + 114,53,0,0,0,114,61,0,0,0,218,4,116,121,112,101, + 218,8,95,95,99,111,100,101,95,95,114,148,0,0,0,114, + 15,0,0,0,114,135,0,0,0,114,14,0,0,0,114,20, + 0,0,0,114,214,0,0,0,114,83,0,0,0,114,79,0, + 0,0,114,93,0,0,0,114,80,0,0,0,90,23,68,69, + 66,85,71,95,66,89,84,69,67,79,68,69,95,83,85,70, + 70,73,88,69,83,90,27,79,80,84,73,77,73,90,69,68, + 95,66,89,84,69,67,79,68,69,95,83,85,70,70,73,88, + 69,83,114,89,0,0,0,114,94,0,0,0,114,100,0,0, + 0,114,103,0,0,0,114,105,0,0,0,114,124,0,0,0, + 114,131,0,0,0,114,139,0,0,0,114,143,0,0,0,114, + 145,0,0,0,114,151,0,0,0,114,156,0,0,0,114,157, + 0,0,0,114,162,0,0,0,218,6,111,98,106,101,99,116, + 114,169,0,0,0,114,174,0,0,0,114,175,0,0,0,114, + 190,0,0,0,114,200,0,0,0,114,217,0,0,0,114,234, + 0,0,0,114,239,0,0,0,114,245,0,0,0,114,240,0, + 0,0,114,246,0,0,0,114,8,1,0,0,114,10,1,0, + 0,114,24,1,0,0,114,42,1,0,0,114,168,0,0,0, + 114,50,1,0,0,114,52,1,0,0,114,2,0,0,0,114, + 2,0,0,0,114,2,0,0,0,114,4,0,0,0,218,8, + 60,109,111,100,117,108,101,62,8,0,0,0,115,126,0,0, + 0,4,15,4,1,4,1,2,1,2,255,4,4,8,17,8, + 5,8,5,8,6,8,6,8,12,8,10,8,9,8,5,8, + 7,8,9,12,22,10,127,0,7,16,1,12,2,4,1,4, + 2,6,2,6,2,8,2,18,71,8,40,8,19,8,12,8, + 12,8,28,8,17,8,33,8,28,8,24,16,13,14,10,12, + 11,8,14,6,3,6,1,2,255,12,68,14,64,14,29,16, + 127,0,17,14,68,18,45,18,26,4,3,18,53,14,60,14, + 42,14,127,0,7,14,127,0,22,12,23,8,11,8,65, }; diff --git a/Python/importlib_zipimport.h b/Python/importlib_zipimport.h new file mode 100644 index 000000000000..6ca737a01394 --- /dev/null +++ b/Python/importlib_zipimport.h @@ -0,0 +1,900 @@ +/* Auto-generated by Programs/_freeze_importlib.c */ +const unsigned char _Py_M__zipimport[] = { + 99,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0, + 0,64,0,0,0,115,48,1,0,0,100,0,90,0,100,1, + 100,2,108,1,90,2,100,1,100,3,108,1,109,3,90,3, + 109,4,90,4,1,0,100,1,100,2,108,5,90,6,100,1, + 100,2,108,7,90,7,100,1,100,2,108,8,90,8,100,1, + 100,2,108,9,90,9,100,1,100,2,108,10,90,10,100,1, + 100,2,108,11,90,11,100,4,100,5,103,2,90,12,101,2, + 106,13,90,13,101,2,106,14,100,6,100,2,133,2,25,0, + 90,15,71,0,100,7,100,4,132,0,100,4,101,16,131,3, + 90,17,105,0,90,18,101,19,101,10,131,1,90,20,71,0, + 100,8,100,5,132,0,100,5,131,2,90,21,101,13,100,9, + 23,0,100,10,100,10,102,3,101,13,100,11,23,0,100,12, + 100,10,102,3,100,13,100,14,102,4,90,22,100,15,100,16, + 132,0,90,23,100,17,100,18,132,0,90,24,100,19,100,20, + 132,0,90,25,100,21,100,22,132,0,90,26,100,23,90,27, + 100,12,97,28,100,24,100,25,132,0,90,29,100,26,100,27, + 132,0,90,30,100,28,100,29,132,0,90,31,100,30,100,31, + 132,0,90,32,101,19,101,32,106,33,131,1,90,34,100,32, + 100,33,132,0,90,35,100,34,100,35,132,0,90,36,100,36, + 100,37,132,0,90,37,100,38,100,39,132,0,90,38,100,40, + 100,41,132,0,90,39,100,2,83,0,41,42,97,80,2,0, + 0,122,105,112,105,109,112,111,114,116,32,112,114,111,118,105, + 100,101,115,32,115,117,112,112,111,114,116,32,102,111,114,32, + 105,109,112,111,114,116,105,110,103,32,80,121,116,104,111,110, + 32,109,111,100,117,108,101,115,32,102,114,111,109,32,90,105, + 112,32,97,114,99,104,105,118,101,115,46,10,10,84,104,105, + 115,32,109,111,100,117,108,101,32,101,120,112,111,114,116,115, + 32,116,104,114,101,101,32,111,98,106,101,99,116,115,58,10, + 45,32,122,105,112,105,109,112,111,114,116,101,114,58,32,97, + 32,99,108,97,115,115,59,32,105,116,115,32,99,111,110,115, + 116,114,117,99,116,111,114,32,116,97,107,101,115,32,97,32, + 112,97,116,104,32,116,111,32,97,32,90,105,112,32,97,114, + 99,104,105,118,101,46,10,45,32,90,105,112,73,109,112,111, + 114,116,69,114,114,111,114,58,32,101,120,99,101,112,116,105, + 111,110,32,114,97,105,115,101,100,32,98,121,32,122,105,112, + 105,109,112,111,114,116,101,114,32,111,98,106,101,99,116,115, + 46,32,73,116,39,115,32,97,10,32,32,115,117,98,99,108, + 97,115,115,32,111,102,32,73,109,112,111,114,116,69,114,114, + 111,114,44,32,115,111,32,105,116,32,99,97,110,32,98,101, + 32,99,97,117,103,104,116,32,97,115,32,73,109,112,111,114, + 116,69,114,114,111,114,44,32,116,111,111,46,10,45,32,95, + 122,105,112,95,100,105,114,101,99,116,111,114,121,95,99,97, + 99,104,101,58,32,97,32,100,105,99,116,44,32,109,97,112, + 112,105,110,103,32,97,114,99,104,105,118,101,32,112,97,116, + 104,115,32,116,111,32,122,105,112,32,100,105,114,101,99,116, + 111,114,121,10,32,32,105,110,102,111,32,100,105,99,116,115, + 44,32,97,115,32,117,115,101,100,32,105,110,32,122,105,112, + 105,109,112,111,114,116,101,114,46,95,102,105,108,101,115,46, + 10,10,73,116,32,105,115,32,117,115,117,97,108,108,121,32, + 110,111,116,32,110,101,101,100,101,100,32,116,111,32,117,115, + 101,32,116,104,101,32,122,105,112,105,109,112,111,114,116,32, + 109,111,100,117,108,101,32,101,120,112,108,105,99,105,116,108, + 121,59,32,105,116,32,105,115,10,117,115,101,100,32,98,121, + 32,116,104,101,32,98,117,105,108,116,105,110,32,105,109,112, + 111,114,116,32,109,101,99,104,97,110,105,115,109,32,102,111, + 114,32,115,121,115,46,112,97,116,104,32,105,116,101,109,115, + 32,116,104,97,116,32,97,114,101,32,112,97,116,104,115,10, + 116,111,32,90,105,112,32,97,114,99,104,105,118,101,115,46, + 10,233,0,0,0,0,78,41,2,218,14,95,117,110,112,97, + 99,107,95,117,105,110,116,49,54,218,14,95,117,110,112,97, + 99,107,95,117,105,110,116,51,50,218,14,90,105,112,73,109, + 112,111,114,116,69,114,114,111,114,218,11,122,105,112,105,109, + 112,111,114,116,101,114,233,1,0,0,0,99,0,0,0,0, + 0,0,0,0,0,0,0,0,1,0,0,0,64,0,0,0, + 115,12,0,0,0,101,0,90,1,100,0,90,2,100,1,83, + 0,41,2,114,3,0,0,0,78,41,3,218,8,95,95,110, + 97,109,101,95,95,218,10,95,95,109,111,100,117,108,101,95, + 95,218,12,95,95,113,117,97,108,110,97,109,101,95,95,169, + 0,114,9,0,0,0,114,9,0,0,0,250,18,60,102,114, + 111,122,101,110,32,122,105,112,105,109,112,111,114,116,62,114, + 3,0,0,0,33,0,0,0,115,2,0,0,0,8,1,99, + 0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0, + 64,0,0,0,115,108,0,0,0,101,0,90,1,100,0,90, + 2,100,1,90,3,100,2,100,3,132,0,90,4,100,25,100, + 5,100,6,132,1,90,5,100,26,100,7,100,8,132,1,90, + 6,100,9,100,10,132,0,90,7,100,11,100,12,132,0,90, + 8,100,13,100,14,132,0,90,9,100,15,100,16,132,0,90, + 10,100,17,100,18,132,0,90,11,100,19,100,20,132,0,90, + 12,100,21,100,22,132,0,90,13,100,23,100,24,132,0,90, + 14,100,4,83,0,41,27,114,4,0,0,0,97,255,1,0, + 0,122,105,112,105,109,112,111,114,116,101,114,40,97,114,99, + 104,105,118,101,112,97,116,104,41,32,45,62,32,122,105,112, + 105,109,112,111,114,116,101,114,32,111,98,106,101,99,116,10, + 10,32,32,32,32,67,114,101,97,116,101,32,97,32,110,101, + 119,32,122,105,112,105,109,112,111,114,116,101,114,32,105,110, + 115,116,97,110,99,101,46,32,39,97,114,99,104,105,118,101, + 112,97,116,104,39,32,109,117,115,116,32,98,101,32,97,32, + 112,97,116,104,32,116,111,10,32,32,32,32,97,32,122,105, + 112,102,105,108,101,44,32,111,114,32,116,111,32,97,32,115, + 112,101,99,105,102,105,99,32,112,97,116,104,32,105,110,115, + 105,100,101,32,97,32,122,105,112,102,105,108,101,46,32,70, + 111,114,32,101,120,97,109,112,108,101,44,32,105,116,32,99, + 97,110,32,98,101,10,32,32,32,32,39,47,116,109,112,47, + 109,121,105,109,112,111,114,116,46,122,105,112,39,44,32,111, + 114,32,39,47,116,109,112,47,109,121,105,109,112,111,114,116, + 46,122,105,112,47,109,121,100,105,114,101,99,116,111,114,121, + 39,44,32,105,102,32,109,121,100,105,114,101,99,116,111,114, + 121,32,105,115,32,97,10,32,32,32,32,118,97,108,105,100, + 32,100,105,114,101,99,116,111,114,121,32,105,110,115,105,100, + 101,32,116,104,101,32,97,114,99,104,105,118,101,46,10,10, + 32,32,32,32,39,90,105,112,73,109,112,111,114,116,69,114, + 114,111,114,32,105,115,32,114,97,105,115,101,100,32,105,102, + 32,39,97,114,99,104,105,118,101,112,97,116,104,39,32,100, + 111,101,115,110,39,116,32,112,111,105,110,116,32,116,111,32, + 97,32,118,97,108,105,100,32,90,105,112,10,32,32,32,32, + 97,114,99,104,105,118,101,46,10,10,32,32,32,32,84,104, + 101,32,39,97,114,99,104,105,118,101,39,32,97,116,116,114, + 105,98,117,116,101,32,111,102,32,122,105,112,105,109,112,111, + 114,116,101,114,32,111,98,106,101,99,116,115,32,99,111,110, + 116,97,105,110,115,32,116,104,101,32,110,97,109,101,32,111, + 102,32,116,104,101,10,32,32,32,32,122,105,112,102,105,108, + 101,32,116,97,114,103,101,116,101,100,46,10,32,32,32,32, + 99,2,0,0,0,0,0,0,0,8,0,0,0,9,0,0, + 0,67,0,0,0,115,36,1,0,0,116,0,124,1,116,1, + 131,2,115,28,100,1,100,0,108,2,125,2,124,2,160,3, + 124,1,161,1,125,1,124,1,115,44,116,4,100,2,124,1, + 100,3,141,2,130,1,116,5,114,60,124,1,160,6,116,5, + 116,7,161,2,125,1,103,0,125,3,122,14,116,8,160,9, + 124,1,161,1,125,4,87,0,110,72,4,0,116,10,116,11, + 102,2,107,10,114,150,1,0,1,0,1,0,116,8,160,12, + 124,1,161,1,92,2,125,5,125,6,124,5,124,1,107,2, + 114,132,116,4,100,4,124,1,100,3,141,2,130,1,124,5, + 125,1,124,3,160,13,124,6,161,1,1,0,89,0,113,64, + 88,0,124,4,106,14,100,5,64,0,100,6,107,3,114,182, + 116,4,100,4,124,1,100,3,141,2,130,1,113,182,113,64, + 122,12,116,15,124,1,25,0,125,7,87,0,110,36,4,0, + 116,16,107,10,114,230,1,0,1,0,1,0,116,17,124,1, + 131,1,125,7,124,7,116,15,124,1,60,0,89,0,110,2, + 88,0,124,7,124,0,95,18,124,1,124,0,95,19,116,8, + 106,20,124,3,100,0,100,0,100,7,133,3,25,0,142,0, + 124,0,95,21,124,0,106,21,144,1,114,32,124,0,4,0, + 106,21,116,7,55,0,2,0,95,21,100,0,83,0,41,8, + 78,114,0,0,0,0,122,21,97,114,99,104,105,118,101,32, + 112,97,116,104,32,105,115,32,101,109,112,116,121,41,1,218, + 4,112,97,116,104,122,14,110,111,116,32,97,32,90,105,112, + 32,102,105,108,101,105,0,240,0,0,105,0,128,0,0,233, + 255,255,255,255,41,22,218,10,105,115,105,110,115,116,97,110, + 99,101,218,3,115,116,114,218,2,111,115,90,8,102,115,100, + 101,99,111,100,101,114,3,0,0,0,218,12,97,108,116,95, + 112,97,116,104,95,115,101,112,218,7,114,101,112,108,97,99, + 101,218,8,112,97,116,104,95,115,101,112,218,19,95,98,111, + 111,116,115,116,114,97,112,95,101,120,116,101,114,110,97,108, + 90,10,95,112,97,116,104,95,115,116,97,116,218,7,79,83, + 69,114,114,111,114,218,10,86,97,108,117,101,69,114,114,111, + 114,90,11,95,112,97,116,104,95,115,112,108,105,116,218,6, + 97,112,112,101,110,100,90,7,115,116,95,109,111,100,101,218, + 20,95,122,105,112,95,100,105,114,101,99,116,111,114,121,95, + 99,97,99,104,101,218,8,75,101,121,69,114,114,111,114,218, + 15,95,114,101,97,100,95,100,105,114,101,99,116,111,114,121, + 218,6,95,102,105,108,101,115,218,7,97,114,99,104,105,118, + 101,218,10,95,112,97,116,104,95,106,111,105,110,218,6,112, + 114,101,102,105,120,41,8,218,4,115,101,108,102,114,11,0, + 0,0,114,15,0,0,0,114,29,0,0,0,90,2,115,116, + 90,7,100,105,114,110,97,109,101,90,8,98,97,115,101,110, + 97,109,101,218,5,102,105,108,101,115,114,9,0,0,0,114, + 9,0,0,0,114,10,0,0,0,218,8,95,95,105,110,105, + 116,95,95,60,0,0,0,115,58,0,0,0,0,1,10,1, + 8,1,10,1,4,1,12,1,4,1,12,2,4,2,2,1, + 14,1,18,3,14,1,8,1,12,1,4,1,16,3,14,2, + 12,1,4,2,2,1,12,1,14,1,8,1,14,1,6,1, + 6,2,22,1,8,1,122,20,122,105,112,105,109,112,111,114, + 116,101,114,46,95,95,105,110,105,116,95,95,78,99,3,0, + 0,0,0,0,0,0,5,0,0,0,4,0,0,0,67,0, + 0,0,115,78,0,0,0,116,0,124,0,124,1,131,2,125, + 3,124,3,100,1,107,9,114,26,124,0,103,0,102,2,83, + 0,116,1,124,0,124,1,131,2,125,4,116,2,124,0,124, + 4,131,2,114,70,100,1,124,0,106,3,155,0,116,4,155, + 0,124,4,155,0,157,3,103,1,102,2,83,0,100,1,103, + 0,102,2,83,0,41,2,97,239,1,0,0,102,105,110,100, + 95,108,111,97,100,101,114,40,102,117,108,108,110,97,109,101, + 44,32,112,97,116,104,61,78,111,110,101,41,32,45,62,32, + 115,101,108,102,44,32,115,116,114,32,111,114,32,78,111,110, + 101,46,10,10,32,32,32,32,32,32,32,32,83,101,97,114, + 99,104,32,102,111,114,32,97,32,109,111,100,117,108,101,32, + 115,112,101,99,105,102,105,101,100,32,98,121,32,39,102,117, + 108,108,110,97,109,101,39,46,32,39,102,117,108,108,110,97, + 109,101,39,32,109,117,115,116,32,98,101,32,116,104,101,10, + 32,32,32,32,32,32,32,32,102,117,108,108,121,32,113,117, + 97,108,105,102,105,101,100,32,40,100,111,116,116,101,100,41, + 32,109,111,100,117,108,101,32,110,97,109,101,46,32,73,116, + 32,114,101,116,117,114,110,115,32,116,104,101,32,122,105,112, + 105,109,112,111,114,116,101,114,10,32,32,32,32,32,32,32, + 32,105,110,115,116,97,110,99,101,32,105,116,115,101,108,102, + 32,105,102,32,116,104,101,32,109,111,100,117,108,101,32,119, + 97,115,32,102,111,117,110,100,44,32,97,32,115,116,114,105, + 110,103,32,99,111,110,116,97,105,110,105,110,103,32,116,104, + 101,10,32,32,32,32,32,32,32,32,102,117,108,108,32,112, + 97,116,104,32,110,97,109,101,32,105,102,32,105,116,39,115, + 32,112,111,115,115,105,98,108,121,32,97,32,112,111,114,116, + 105,111,110,32,111,102,32,97,32,110,97,109,101,115,112,97, + 99,101,32,112,97,99,107,97,103,101,44,10,32,32,32,32, + 32,32,32,32,111,114,32,78,111,110,101,32,111,116,104,101, + 114,119,105,115,101,46,32,84,104,101,32,111,112,116,105,111, + 110,97,108,32,39,112,97,116,104,39,32,97,114,103,117,109, + 101,110,116,32,105,115,32,105,103,110,111,114,101,100,32,45, + 45,32,105,116,39,115,10,32,32,32,32,32,32,32,32,116, + 104,101,114,101,32,102,111,114,32,99,111,109,112,97,116,105, + 98,105,108,105,116,121,32,119,105,116,104,32,116,104,101,32, + 105,109,112,111,114,116,101,114,32,112,114,111,116,111,99,111, + 108,46,10,32,32,32,32,32,32,32,32,78,41,5,218,16, + 95,103,101,116,95,109,111,100,117,108,101,95,105,110,102,111, + 218,16,95,103,101,116,95,109,111,100,117,108,101,95,112,97, + 116,104,218,7,95,105,115,95,100,105,114,114,27,0,0,0, + 114,18,0,0,0,41,5,114,30,0,0,0,218,8,102,117, + 108,108,110,97,109,101,114,11,0,0,0,218,2,109,105,218, + 7,109,111,100,112,97,116,104,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,218,11,102,105,110,100,95,108,111, + 97,100,101,114,106,0,0,0,115,14,0,0,0,0,10,10, + 1,8,2,8,7,10,1,10,4,24,2,122,23,122,105,112, + 105,109,112,111,114,116,101,114,46,102,105,110,100,95,108,111, + 97,100,101,114,99,3,0,0,0,0,0,0,0,3,0,0, + 0,4,0,0,0,67,0,0,0,115,16,0,0,0,124,0, + 160,0,124,1,124,2,161,2,100,1,25,0,83,0,41,2, + 97,139,1,0,0,102,105,110,100,95,109,111,100,117,108,101, + 40,102,117,108,108,110,97,109,101,44,32,112,97,116,104,61, + 78,111,110,101,41,32,45,62,32,115,101,108,102,32,111,114, + 32,78,111,110,101,46,10,10,32,32,32,32,32,32,32,32, + 83,101,97,114,99,104,32,102,111,114,32,97,32,109,111,100, + 117,108,101,32,115,112,101,99,105,102,105,101,100,32,98,121, + 32,39,102,117,108,108,110,97,109,101,39,46,32,39,102,117, + 108,108,110,97,109,101,39,32,109,117,115,116,32,98,101,32, + 116,104,101,10,32,32,32,32,32,32,32,32,102,117,108,108, + 121,32,113,117,97,108,105,102,105,101,100,32,40,100,111,116, + 116,101,100,41,32,109,111,100,117,108,101,32,110,97,109,101, + 46,32,73,116,32,114,101,116,117,114,110,115,32,116,104,101, + 32,122,105,112,105,109,112,111,114,116,101,114,10,32,32,32, + 32,32,32,32,32,105,110,115,116,97,110,99,101,32,105,116, + 115,101,108,102,32,105,102,32,116,104,101,32,109,111,100,117, + 108,101,32,119,97,115,32,102,111,117,110,100,44,32,111,114, + 32,78,111,110,101,32,105,102,32,105,116,32,119,97,115,110, + 39,116,46,10,32,32,32,32,32,32,32,32,84,104,101,32, + 111,112,116,105,111,110,97,108,32,39,112,97,116,104,39,32, + 97,114,103,117,109,101,110,116,32,105,115,32,105,103,110,111, + 114,101,100,32,45,45,32,105,116,39,115,32,116,104,101,114, + 101,32,102,111,114,32,99,111,109,112,97,116,105,98,105,108, + 105,116,121,10,32,32,32,32,32,32,32,32,119,105,116,104, + 32,116,104,101,32,105,109,112,111,114,116,101,114,32,112,114, + 111,116,111,99,111,108,46,10,32,32,32,32,32,32,32,32, + 114,0,0,0,0,41,1,114,39,0,0,0,41,3,114,30, + 0,0,0,114,36,0,0,0,114,11,0,0,0,114,9,0, + 0,0,114,9,0,0,0,114,10,0,0,0,218,11,102,105, + 110,100,95,109,111,100,117,108,101,138,0,0,0,115,2,0, + 0,0,0,9,122,23,122,105,112,105,109,112,111,114,116,101, + 114,46,102,105,110,100,95,109,111,100,117,108,101,99,2,0, + 0,0,0,0,0,0,5,0,0,0,3,0,0,0,67,0, + 0,0,115,20,0,0,0,116,0,124,0,124,1,131,2,92, + 3,125,2,125,3,125,4,124,2,83,0,41,1,122,163,103, + 101,116,95,99,111,100,101,40,102,117,108,108,110,97,109,101, + 41,32,45,62,32,99,111,100,101,32,111,98,106,101,99,116, + 46,10,10,32,32,32,32,32,32,32,32,82,101,116,117,114, + 110,32,116,104,101,32,99,111,100,101,32,111,98,106,101,99, + 116,32,102,111,114,32,116,104,101,32,115,112,101,99,105,102, + 105,101,100,32,109,111,100,117,108,101,46,32,82,97,105,115, + 101,32,90,105,112,73,109,112,111,114,116,69,114,114,111,114, + 10,32,32,32,32,32,32,32,32,105,102,32,116,104,101,32, + 109,111,100,117,108,101,32,99,111,117,108,100,110,39,116,32, + 98,101,32,102,111,117,110,100,46,10,32,32,32,32,32,32, + 32,32,41,1,218,16,95,103,101,116,95,109,111,100,117,108, + 101,95,99,111,100,101,41,5,114,30,0,0,0,114,36,0, + 0,0,218,4,99,111,100,101,218,9,105,115,112,97,99,107, + 97,103,101,114,38,0,0,0,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,218,8,103,101,116,95,99,111,100, + 101,150,0,0,0,115,4,0,0,0,0,6,16,1,122,20, + 122,105,112,105,109,112,111,114,116,101,114,46,103,101,116,95, + 99,111,100,101,99,2,0,0,0,0,0,0,0,4,0,0, + 0,8,0,0,0,67,0,0,0,115,118,0,0,0,116,0, + 114,16,124,1,160,1,116,0,116,2,161,2,125,1,124,1, + 125,2,124,1,160,3,124,0,106,4,116,2,23,0,161,1, + 114,58,124,1,116,5,124,0,106,4,116,2,23,0,131,1, + 100,1,133,2,25,0,125,2,122,14,124,0,106,6,124,2, + 25,0,125,3,87,0,110,32,4,0,116,7,107,10,114,104, + 1,0,1,0,1,0,116,8,100,2,100,3,124,2,131,3, + 130,1,89,0,110,2,88,0,116,9,124,0,106,4,124,3, + 131,2,83,0,41,4,122,154,103,101,116,95,100,97,116,97, + 40,112,97,116,104,110,97,109,101,41,32,45,62,32,115,116, + 114,105,110,103,32,119,105,116,104,32,102,105,108,101,32,100, + 97,116,97,46,10,10,32,32,32,32,32,32,32,32,82,101, + 116,117,114,110,32,116,104,101,32,100,97,116,97,32,97,115, + 115,111,99,105,97,116,101,100,32,119,105,116,104,32,39,112, + 97,116,104,110,97,109,101,39,46,32,82,97,105,115,101,32, + 79,83,69,114,114,111,114,32,105,102,10,32,32,32,32,32, + 32,32,32,116,104,101,32,102,105,108,101,32,119,97,115,110, + 39,116,32,102,111,117,110,100,46,10,32,32,32,32,32,32, + 32,32,78,114,0,0,0,0,218,0,41,10,114,16,0,0, + 0,114,17,0,0,0,114,18,0,0,0,218,10,115,116,97, + 114,116,115,119,105,116,104,114,27,0,0,0,218,3,108,101, + 110,114,26,0,0,0,114,24,0,0,0,114,20,0,0,0, + 218,9,95,103,101,116,95,100,97,116,97,41,4,114,30,0, + 0,0,218,8,112,97,116,104,110,97,109,101,90,3,107,101, + 121,218,9,116,111,99,95,101,110,116,114,121,114,9,0,0, + 0,114,9,0,0,0,114,10,0,0,0,218,8,103,101,116, + 95,100,97,116,97,160,0,0,0,115,20,0,0,0,0,6, + 4,1,12,2,4,1,16,1,22,2,2,1,14,1,14,1, + 18,1,122,20,122,105,112,105,109,112,111,114,116,101,114,46, + 103,101,116,95,100,97,116,97,99,2,0,0,0,0,0,0, + 0,5,0,0,0,3,0,0,0,67,0,0,0,115,20,0, + 0,0,116,0,124,0,124,1,131,2,92,3,125,2,125,3, + 125,4,124,4,83,0,41,1,122,106,103,101,116,95,102,105, + 108,101,110,97,109,101,40,102,117,108,108,110,97,109,101,41, + 32,45,62,32,102,105,108,101,110,97,109,101,32,115,116,114, + 105,110,103,46,10,10,32,32,32,32,32,32,32,32,82,101, + 116,117,114,110,32,116,104,101,32,102,105,108,101,110,97,109, + 101,32,102,111,114,32,116,104,101,32,115,112,101,99,105,102, + 105,101,100,32,109,111,100,117,108,101,46,10,32,32,32,32, + 32,32,32,32,41,1,114,41,0,0,0,41,5,114,30,0, + 0,0,114,36,0,0,0,114,42,0,0,0,114,43,0,0, + 0,114,38,0,0,0,114,9,0,0,0,114,9,0,0,0, + 114,10,0,0,0,218,12,103,101,116,95,102,105,108,101,110, + 97,109,101,181,0,0,0,115,4,0,0,0,0,7,16,1, + 122,24,122,105,112,105,109,112,111,114,116,101,114,46,103,101, + 116,95,102,105,108,101,110,97,109,101,99,2,0,0,0,0, + 0,0,0,6,0,0,0,8,0,0,0,67,0,0,0,115, + 128,0,0,0,116,0,124,0,124,1,131,2,125,2,124,2, + 100,1,107,8,114,36,116,1,100,2,124,1,155,2,157,2, + 124,1,100,3,141,2,130,1,116,2,124,0,124,1,131,2, + 125,3,124,2,114,64,116,3,160,4,124,3,100,4,161,2, + 125,4,110,10,124,3,155,0,100,5,157,2,125,4,122,14, + 124,0,106,5,124,4,25,0,125,5,87,0,110,22,4,0, + 116,6,107,10,114,110,1,0,1,0,1,0,89,0,100,1, + 83,0,88,0,116,7,124,0,106,8,124,5,131,2,160,9, + 161,0,83,0,41,6,122,253,103,101,116,95,115,111,117,114, + 99,101,40,102,117,108,108,110,97,109,101,41,32,45,62,32, + 115,111,117,114,99,101,32,115,116,114,105,110,103,46,10,10, + 32,32,32,32,32,32,32,32,82,101,116,117,114,110,32,116, + 104,101,32,115,111,117,114,99,101,32,99,111,100,101,32,102, + 111,114,32,116,104,101,32,115,112,101,99,105,102,105,101,100, + 32,109,111,100,117,108,101,46,32,82,97,105,115,101,32,90, + 105,112,73,109,112,111,114,116,69,114,114,111,114,10,32,32, + 32,32,32,32,32,32,105,102,32,116,104,101,32,109,111,100, + 117,108,101,32,99,111,117,108,100,110,39,116,32,98,101,32, + 102,111,117,110,100,44,32,114,101,116,117,114,110,32,78,111, + 110,101,32,105,102,32,116,104,101,32,97,114,99,104,105,118, + 101,32,100,111,101,115,10,32,32,32,32,32,32,32,32,99, + 111,110,116,97,105,110,32,116,104,101,32,109,111,100,117,108, + 101,44,32,98,117,116,32,104,97,115,32,110,111,32,115,111, + 117,114,99,101,32,102,111,114,32,105,116,46,10,32,32,32, + 32,32,32,32,32,78,122,18,99,97,110,39,116,32,102,105, + 110,100,32,109,111,100,117,108,101,32,41,1,218,4,110,97, + 109,101,122,11,95,95,105,110,105,116,95,95,46,112,121,122, + 3,46,112,121,41,10,114,33,0,0,0,114,3,0,0,0, + 114,34,0,0,0,114,19,0,0,0,114,28,0,0,0,114, + 26,0,0,0,114,24,0,0,0,114,48,0,0,0,114,27, + 0,0,0,218,6,100,101,99,111,100,101,41,6,114,30,0, + 0,0,114,36,0,0,0,114,37,0,0,0,114,11,0,0, + 0,218,8,102,117,108,108,112,97,116,104,114,50,0,0,0, + 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,218, + 10,103,101,116,95,115,111,117,114,99,101,192,0,0,0,115, + 24,0,0,0,0,7,10,1,8,1,18,2,10,1,4,1, + 14,2,10,2,2,1,14,1,14,2,8,1,122,22,122,105, + 112,105,109,112,111,114,116,101,114,46,103,101,116,95,115,111, + 117,114,99,101,99,2,0,0,0,0,0,0,0,3,0,0, + 0,4,0,0,0,67,0,0,0,115,40,0,0,0,116,0, + 124,0,124,1,131,2,125,2,124,2,100,1,107,8,114,36, + 116,1,100,2,124,1,155,2,157,2,124,1,100,3,141,2, + 130,1,124,2,83,0,41,4,122,171,105,115,95,112,97,99, + 107,97,103,101,40,102,117,108,108,110,97,109,101,41,32,45, + 62,32,98,111,111,108,46,10,10,32,32,32,32,32,32,32, + 32,82,101,116,117,114,110,32,84,114,117,101,32,105,102,32, + 116,104,101,32,109,111,100,117,108,101,32,115,112,101,99,105, + 102,105,101,100,32,98,121,32,102,117,108,108,110,97,109,101, + 32,105,115,32,97,32,112,97,99,107,97,103,101,46,10,32, + 32,32,32,32,32,32,32,82,97,105,115,101,32,90,105,112, + 73,109,112,111,114,116,69,114,114,111,114,32,105,102,32,116, + 104,101,32,109,111,100,117,108,101,32,99,111,117,108,100,110, + 39,116,32,98,101,32,102,111,117,110,100,46,10,32,32,32, + 32,32,32,32,32,78,122,18,99,97,110,39,116,32,102,105, + 110,100,32,109,111,100,117,108,101,32,41,1,114,53,0,0, + 0,41,2,114,33,0,0,0,114,3,0,0,0,41,3,114, + 30,0,0,0,114,36,0,0,0,114,37,0,0,0,114,9, + 0,0,0,114,9,0,0,0,114,10,0,0,0,218,10,105, + 115,95,112,97,99,107,97,103,101,218,0,0,0,115,8,0, + 0,0,0,6,10,1,8,1,18,1,122,22,122,105,112,105, + 109,112,111,114,116,101,114,46,105,115,95,112,97,99,107,97, + 103,101,99,2,0,0,0,0,0,0,0,8,0,0,0,8, + 0,0,0,67,0,0,0,115,248,0,0,0,116,0,124,0, + 124,1,131,2,92,3,125,2,125,3,125,4,116,1,106,2, + 160,3,124,1,161,1,125,5,124,5,100,1,107,8,115,46, + 116,4,124,5,116,5,131,2,115,64,116,5,124,1,131,1, + 125,5,124,5,116,1,106,2,124,1,60,0,124,0,124,5, + 95,6,122,84,124,3,114,108,116,7,124,0,124,1,131,2, + 125,6,116,8,160,9,124,0,106,10,124,6,161,2,125,7, + 124,7,103,1,124,5,95,11,116,12,124,5,100,2,131,2, + 115,124,116,13,124,5,95,13,116,8,160,14,124,5,106,15, + 124,1,124,4,161,3,1,0,116,16,124,2,124,5,106,15, + 131,2,1,0,87,0,110,22,1,0,1,0,1,0,116,1, + 106,2,124,1,61,0,130,0,89,0,110,2,88,0,122,14, + 116,1,106,2,124,1,25,0,125,5,87,0,110,36,4,0, + 116,17,107,10,114,228,1,0,1,0,1,0,116,18,100,3, + 124,1,155,2,100,4,157,3,131,1,130,1,89,0,110,2, + 88,0,116,19,160,20,100,5,124,1,124,4,161,3,1,0, + 124,5,83,0,41,6,122,245,108,111,97,100,95,109,111,100, + 117,108,101,40,102,117,108,108,110,97,109,101,41,32,45,62, + 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, + 32,32,76,111,97,100,32,116,104,101,32,109,111,100,117,108, + 101,32,115,112,101,99,105,102,105,101,100,32,98,121,32,39, + 102,117,108,108,110,97,109,101,39,46,32,39,102,117,108,108, + 110,97,109,101,39,32,109,117,115,116,32,98,101,32,116,104, + 101,10,32,32,32,32,32,32,32,32,102,117,108,108,121,32, + 113,117,97,108,105,102,105,101,100,32,40,100,111,116,116,101, + 100,41,32,109,111,100,117,108,101,32,110,97,109,101,46,32, + 73,116,32,114,101,116,117,114,110,115,32,116,104,101,32,105, + 109,112,111,114,116,101,100,10,32,32,32,32,32,32,32,32, + 109,111,100,117,108,101,44,32,111,114,32,114,97,105,115,101, + 115,32,90,105,112,73,109,112,111,114,116,69,114,114,111,114, + 32,105,102,32,105,116,32,119,97,115,110,39,116,32,102,111, + 117,110,100,46,10,32,32,32,32,32,32,32,32,78,218,12, + 95,95,98,117,105,108,116,105,110,115,95,95,122,14,76,111, + 97,100,101,100,32,109,111,100,117,108,101,32,122,25,32,110, + 111,116,32,102,111,117,110,100,32,105,110,32,115,121,115,46, + 109,111,100,117,108,101,115,122,30,105,109,112,111,114,116,32, + 123,125,32,35,32,108,111,97,100,101,100,32,102,114,111,109, + 32,90,105,112,32,123,125,41,21,114,41,0,0,0,218,3, + 115,121,115,218,7,109,111,100,117,108,101,115,218,3,103,101, + 116,114,13,0,0,0,218,12,95,109,111,100,117,108,101,95, + 116,121,112,101,218,10,95,95,108,111,97,100,101,114,95,95, + 114,34,0,0,0,114,19,0,0,0,114,28,0,0,0,114, + 27,0,0,0,90,8,95,95,112,97,116,104,95,95,218,7, + 104,97,115,97,116,116,114,114,58,0,0,0,90,14,95,102, + 105,120,95,117,112,95,109,111,100,117,108,101,218,8,95,95, + 100,105,99,116,95,95,218,4,101,120,101,99,114,24,0,0, + 0,218,11,73,109,112,111,114,116,69,114,114,111,114,218,10, + 95,98,111,111,116,115,116,114,97,112,218,16,95,118,101,114, + 98,111,115,101,95,109,101,115,115,97,103,101,41,8,114,30, + 0,0,0,114,36,0,0,0,114,42,0,0,0,114,43,0, + 0,0,114,38,0,0,0,90,3,109,111,100,114,11,0,0, + 0,114,55,0,0,0,114,9,0,0,0,114,9,0,0,0, + 114,10,0,0,0,218,11,108,111,97,100,95,109,111,100,117, + 108,101,231,0,0,0,115,48,0,0,0,0,7,16,1,12, + 1,18,1,8,1,10,1,6,2,2,1,4,3,10,1,14, + 1,8,2,10,1,6,1,16,1,16,1,6,1,8,1,8, + 2,2,1,14,1,14,1,22,1,14,1,122,23,122,105,112, + 105,109,112,111,114,116,101,114,46,108,111,97,100,95,109,111, + 100,117,108,101,99,2,0,0,0,0,0,0,0,3,0,0, + 0,4,0,0,0,67,0,0,0,115,24,0,0,0,100,1, + 100,2,108,0,109,1,125,2,1,0,124,2,160,2,124,0, + 124,1,161,2,83,0,41,3,122,204,82,101,116,117,114,110, + 32,116,104,101,32,82,101,115,111,117,114,99,101,82,101,97, + 100,101,114,32,102,111,114,32,97,32,112,97,99,107,97,103, + 101,32,105,110,32,97,32,122,105,112,32,102,105,108,101,46, + 10,10,32,32,32,32,32,32,32,32,73,102,32,39,102,117, + 108,108,110,97,109,101,39,32,105,115,32,97,32,112,97,99, + 107,97,103,101,32,119,105,116,104,105,110,32,116,104,101,32, + 122,105,112,32,102,105,108,101,44,32,114,101,116,117,114,110, + 32,116,104,101,10,32,32,32,32,32,32,32,32,39,82,101, + 115,111,117,114,99,101,82,101,97,100,101,114,39,32,111,98, + 106,101,99,116,32,102,111,114,32,116,104,101,32,112,97,99, + 107,97,103,101,46,32,32,79,116,104,101,114,119,105,115,101, + 32,114,101,116,117,114,110,32,78,111,110,101,46,10,32,32, + 32,32,32,32,32,32,114,0,0,0,0,41,1,218,9,114, + 101,115,111,117,114,99,101,115,41,3,90,9,105,109,112,111, + 114,116,108,105,98,114,71,0,0,0,90,30,95,122,105,112, + 105,109,112,111,114,116,95,103,101,116,95,114,101,115,111,117, + 114,99,101,95,114,101,97,100,101,114,41,3,114,30,0,0, + 0,114,36,0,0,0,114,71,0,0,0,114,9,0,0,0, + 114,9,0,0,0,114,10,0,0,0,218,19,103,101,116,95, + 114,101,115,111,117,114,99,101,95,114,101,97,100,101,114,13, + 1,0,0,115,4,0,0,0,0,6,12,1,122,31,122,105, + 112,105,109,112,111,114,116,101,114,46,103,101,116,95,114,101, + 115,111,117,114,99,101,95,114,101,97,100,101,114,99,1,0, + 0,0,0,0,0,0,1,0,0,0,5,0,0,0,67,0, + 0,0,115,24,0,0,0,100,1,124,0,106,0,155,0,116, + 1,155,0,124,0,106,2,155,0,100,2,157,5,83,0,41, + 3,78,122,21,60,122,105,112,105,109,112,111,114,116,101,114, + 32,111,98,106,101,99,116,32,34,122,2,34,62,41,3,114, + 27,0,0,0,114,18,0,0,0,114,29,0,0,0,41,1, + 114,30,0,0,0,114,9,0,0,0,114,9,0,0,0,114, + 10,0,0,0,218,8,95,95,114,101,112,114,95,95,23,1, + 0,0,115,2,0,0,0,0,1,122,20,122,105,112,105,109, + 112,111,114,116,101,114,46,95,95,114,101,112,114,95,95,41, + 1,78,41,1,78,41,15,114,6,0,0,0,114,7,0,0, + 0,114,8,0,0,0,218,7,95,95,100,111,99,95,95,114, + 32,0,0,0,114,39,0,0,0,114,40,0,0,0,114,44, + 0,0,0,114,51,0,0,0,114,52,0,0,0,114,56,0, + 0,0,114,57,0,0,0,114,70,0,0,0,114,72,0,0, + 0,114,73,0,0,0,114,9,0,0,0,114,9,0,0,0, + 114,9,0,0,0,114,10,0,0,0,114,4,0,0,0,42, + 0,0,0,115,24,0,0,0,8,13,4,5,8,46,10,32, + 10,12,8,10,8,21,8,11,8,26,8,13,8,38,8,10, + 122,12,95,95,105,110,105,116,95,95,46,112,121,99,84,122, + 11,95,95,105,110,105,116,95,95,46,112,121,70,41,3,122, + 4,46,112,121,99,84,70,41,3,122,3,46,112,121,70,70, + 99,2,0,0,0,0,0,0,0,2,0,0,0,4,0,0, + 0,67,0,0,0,115,20,0,0,0,124,0,106,0,124,1, + 160,1,100,1,161,1,100,2,25,0,23,0,83,0,41,3, + 78,218,1,46,233,2,0,0,0,41,2,114,29,0,0,0, + 218,10,114,112,97,114,116,105,116,105,111,110,41,2,114,30, + 0,0,0,114,36,0,0,0,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,114,34,0,0,0,41,1,0,0, + 115,2,0,0,0,0,1,114,34,0,0,0,99,2,0,0, + 0,0,0,0,0,3,0,0,0,2,0,0,0,67,0,0, + 0,115,18,0,0,0,124,1,116,0,23,0,125,2,124,2, + 124,0,106,1,107,6,83,0,41,1,78,41,2,114,18,0, + 0,0,114,26,0,0,0,41,3,114,30,0,0,0,114,11, + 0,0,0,90,7,100,105,114,112,97,116,104,114,9,0,0, + 0,114,9,0,0,0,114,10,0,0,0,114,35,0,0,0, + 45,1,0,0,115,4,0,0,0,0,4,8,2,114,35,0, + 0,0,99,2,0,0,0,0,0,0,0,7,0,0,0,4, + 0,0,0,67,0,0,0,115,56,0,0,0,116,0,124,0, + 124,1,131,2,125,2,116,1,68,0,93,36,92,3,125,3, + 125,4,125,5,124,2,124,3,23,0,125,6,124,6,124,0, + 106,2,107,6,114,14,124,5,2,0,1,0,83,0,113,14, + 100,0,83,0,41,1,78,41,3,114,34,0,0,0,218,16, + 95,122,105,112,95,115,101,97,114,99,104,111,114,100,101,114, + 114,26,0,0,0,41,7,114,30,0,0,0,114,36,0,0, + 0,114,11,0,0,0,218,6,115,117,102,102,105,120,218,10, + 105,115,98,121,116,101,99,111,100,101,114,43,0,0,0,114, + 55,0,0,0,114,9,0,0,0,114,9,0,0,0,114,10, + 0,0,0,114,33,0,0,0,54,1,0,0,115,12,0,0, + 0,0,1,10,1,14,1,8,1,10,1,10,1,114,33,0, + 0,0,99,1,0,0,0,0,0,0,0,23,0,0,0,9, + 0,0,0,67,0,0,0,115,20,4,0,0,122,16,116,0, + 160,1,124,0,100,1,161,2,125,1,87,0,110,38,4,0, + 116,2,107,10,114,54,1,0,1,0,1,0,116,3,100,2, + 124,0,155,2,157,2,124,0,100,3,141,2,130,1,89,0, + 110,2,88,0,124,1,144,3,143,190,1,0,122,34,124,1, + 160,4,100,4,100,5,161,2,1,0,124,1,160,5,161,0, + 125,2,124,1,160,6,100,6,161,1,125,3,87,0,110,38, + 4,0,116,2,107,10,114,136,1,0,1,0,1,0,116,3, + 100,7,124,0,155,2,157,2,124,0,100,3,141,2,130,1, + 89,0,110,2,88,0,116,7,124,3,131,1,100,6,107,3, + 114,168,116,3,100,7,124,0,155,2,157,2,124,0,100,3, + 141,2,130,1,124,3,100,0,100,8,133,2,25,0,100,9, + 107,3,114,202,116,3,100,10,124,0,155,2,157,2,124,0, + 100,3,141,2,130,1,116,8,124,3,100,11,100,12,133,2, + 25,0,131,1,125,4,116,8,124,3,100,12,100,13,133,2, + 25,0,131,1,125,5,124,2,124,4,107,0,144,1,114,6, + 116,3,100,14,124,0,155,2,157,2,124,0,100,3,141,2, + 130,1,124,2,124,5,107,0,144,1,114,34,116,3,100,15, + 124,0,155,2,157,2,124,0,100,3,141,2,130,1,124,2, + 124,4,56,0,125,2,124,2,124,5,24,0,125,6,124,6, + 100,16,107,0,144,1,114,78,116,3,100,17,124,0,155,2, + 157,2,124,0,100,3,141,2,130,1,105,0,125,7,100,16, + 125,8,122,14,124,1,160,4,124,2,161,1,1,0,87,0, + 110,40,4,0,116,2,107,10,144,1,114,140,1,0,1,0, + 1,0,116,3,100,7,124,0,155,2,157,2,124,0,100,3, + 141,2,130,1,89,0,110,2,88,0,124,1,160,6,100,18, + 161,1,125,3,116,7,124,3,131,1,100,8,107,0,144,1, + 114,174,116,9,100,19,131,1,130,1,124,3,100,0,100,8, + 133,2,25,0,100,20,107,3,144,1,114,196,144,3,113,248, + 116,7,124,3,131,1,100,18,107,3,144,1,114,218,116,9, + 100,19,131,1,130,1,116,10,124,3,100,21,100,22,133,2, + 25,0,131,1,125,9,116,10,124,3,100,22,100,11,133,2, + 25,0,131,1,125,10,116,10,124,3,100,11,100,23,133,2, + 25,0,131,1,125,11,116,10,124,3,100,23,100,12,133,2, + 25,0,131,1,125,12,116,8,124,3,100,12,100,13,133,2, + 25,0,131,1,125,13,116,8,124,3,100,13,100,24,133,2, + 25,0,131,1,125,14,116,8,124,3,100,24,100,25,133,2, + 25,0,131,1,125,15,116,10,124,3,100,25,100,26,133,2, + 25,0,131,1,125,16,116,10,124,3,100,26,100,27,133,2, + 25,0,131,1,125,17,116,10,124,3,100,27,100,28,133,2, + 25,0,131,1,125,18,116,8,124,3,100,29,100,18,133,2, + 25,0,131,1,125,19,124,16,124,17,23,0,124,18,23,0, + 125,4,124,19,124,5,107,4,144,2,114,178,116,3,100,30, + 124,0,155,2,157,2,124,0,100,3,141,2,130,1,124,19, + 124,6,55,0,125,19,122,14,124,1,160,6,124,16,161,1, + 125,20,87,0,110,40,4,0,116,2,107,10,144,2,114,240, + 1,0,1,0,1,0,116,3,100,7,124,0,155,2,157,2, + 124,0,100,3,141,2,130,1,89,0,110,2,88,0,116,7, + 124,20,131,1,124,16,107,3,144,3,114,18,116,3,100,7, + 124,0,155,2,157,2,124,0,100,3,141,2,130,1,122,50, + 116,7,124,1,160,6,124,4,124,16,24,0,161,1,131,1, + 124,4,124,16,24,0,107,3,144,3,114,66,116,3,100,7, + 124,0,155,2,157,2,124,0,100,3,141,2,130,1,87,0, + 110,40,4,0,116,2,107,10,144,3,114,108,1,0,1,0, + 1,0,116,3,100,7,124,0,155,2,157,2,124,0,100,3, + 141,2,130,1,89,0,110,2,88,0,124,9,100,31,64,0, + 144,3,114,130,124,20,160,11,161,0,125,20,110,54,122,14, + 124,20,160,11,100,32,161,1,125,20,87,0,110,38,4,0, + 116,12,107,10,144,3,114,182,1,0,1,0,1,0,124,20, + 160,11,100,33,161,1,160,13,116,14,161,1,125,20,89,0, + 110,2,88,0,124,20,160,15,100,34,116,16,161,2,125,20, + 116,17,160,18,124,0,124,20,161,2,125,21,124,21,124,10, + 124,14,124,15,124,19,124,11,124,12,124,13,102,8,125,22, + 124,22,124,7,124,20,60,0,124,8,100,35,55,0,125,8, + 144,1,113,142,87,0,53,0,81,0,82,0,88,0,116,19, + 160,20,100,36,124,8,124,0,161,3,1,0,124,7,83,0, + 41,37,78,218,2,114,98,122,21,99,97,110,39,116,32,111, + 112,101,110,32,90,105,112,32,102,105,108,101,58,32,41,1, + 114,11,0,0,0,105,234,255,255,255,114,76,0,0,0,233, + 22,0,0,0,122,21,99,97,110,39,116,32,114,101,97,100, + 32,90,105,112,32,102,105,108,101,58,32,233,4,0,0,0, + 115,4,0,0,0,80,75,5,6,122,16,110,111,116,32,97, + 32,90,105,112,32,102,105,108,101,58,32,233,12,0,0,0, + 233,16,0,0,0,233,20,0,0,0,122,28,98,97,100,32, + 99,101,110,116,114,97,108,32,100,105,114,101,99,116,111,114, + 121,32,115,105,122,101,58,32,122,30,98,97,100,32,99,101, + 110,116,114,97,108,32,100,105,114,101,99,116,111,114,121,32, + 111,102,102,115,101,116,58,32,114,0,0,0,0,122,38,98, + 97,100,32,99,101,110,116,114,97,108,32,100,105,114,101,99, + 116,111,114,121,32,115,105,122,101,32,111,114,32,111,102,102, + 115,101,116,58,32,233,46,0,0,0,122,27,69,79,70,32, + 114,101,97,100,32,119,104,101,114,101,32,110,111,116,32,101, + 120,112,101,99,116,101,100,115,4,0,0,0,80,75,1,2, + 233,8,0,0,0,233,10,0,0,0,233,14,0,0,0,233, + 24,0,0,0,233,28,0,0,0,233,30,0,0,0,233,32, + 0,0,0,233,34,0,0,0,233,42,0,0,0,122,25,98, + 97,100,32,108,111,99,97,108,32,104,101,97,100,101,114,32, + 111,102,102,115,101,116,58,32,105,0,8,0,0,218,5,97, + 115,99,105,105,90,6,108,97,116,105,110,49,250,1,47,114, + 5,0,0,0,122,33,122,105,112,105,109,112,111,114,116,58, + 32,102,111,117,110,100,32,123,125,32,110,97,109,101,115,32, + 105,110,32,123,33,114,125,41,21,218,3,95,105,111,218,4, + 111,112,101,110,114,20,0,0,0,114,3,0,0,0,218,4, + 115,101,101,107,90,4,116,101,108,108,218,4,114,101,97,100, + 114,47,0,0,0,114,2,0,0,0,218,8,69,79,70,69, + 114,114,111,114,114,1,0,0,0,114,54,0,0,0,218,18, + 85,110,105,99,111,100,101,68,101,99,111,100,101,69,114,114, + 111,114,218,9,116,114,97,110,115,108,97,116,101,218,11,99, + 112,52,51,55,95,116,97,98,108,101,114,17,0,0,0,114, + 18,0,0,0,114,19,0,0,0,114,28,0,0,0,114,68, + 0,0,0,114,69,0,0,0,41,23,114,27,0,0,0,218, + 2,102,112,90,15,104,101,97,100,101,114,95,112,111,115,105, + 116,105,111,110,218,6,98,117,102,102,101,114,218,11,104,101, + 97,100,101,114,95,115,105,122,101,90,13,104,101,97,100,101, + 114,95,111,102,102,115,101,116,90,10,97,114,99,95,111,102, + 102,115,101,116,114,31,0,0,0,218,5,99,111,117,110,116, + 218,5,102,108,97,103,115,218,8,99,111,109,112,114,101,115, + 115,218,4,116,105,109,101,218,4,100,97,116,101,218,3,99, + 114,99,218,9,100,97,116,97,95,115,105,122,101,218,9,102, + 105,108,101,95,115,105,122,101,218,9,110,97,109,101,95,115, + 105,122,101,218,10,101,120,116,114,97,95,115,105,122,101,90, + 12,99,111,109,109,101,110,116,95,115,105,122,101,218,11,102, + 105,108,101,95,111,102,102,115,101,116,114,53,0,0,0,114, + 11,0,0,0,218,1,116,114,9,0,0,0,114,9,0,0, + 0,114,10,0,0,0,114,25,0,0,0,85,1,0,0,115, + 154,0,0,0,0,1,2,1,16,1,14,1,24,2,8,1, + 2,1,12,1,8,1,14,1,14,1,24,1,12,1,18,1, + 16,2,18,2,16,1,16,1,10,1,18,1,10,1,18,1, + 8,1,8,1,10,1,18,2,4,2,4,1,2,1,14,1, + 16,1,24,2,10,1,14,1,8,2,18,1,4,1,14,1, + 8,1,16,1,16,1,16,1,16,1,16,1,16,1,16,1, + 16,1,16,1,16,1,16,1,12,1,10,1,18,1,8,2, + 2,1,14,1,16,1,24,1,14,1,18,4,2,1,28,1, + 22,1,16,1,24,2,10,2,10,3,2,1,14,1,16,1, + 22,2,12,1,12,1,20,1,8,1,22,1,14,1,114,25, + 0,0,0,117,190,1,0,0,0,1,2,3,4,5,6,7, + 8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23, + 24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39, + 40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55, + 56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71, + 72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87, + 88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103, + 104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119, + 120,121,122,123,124,125,126,127,195,135,195,188,195,169,195,162, + 195,164,195,160,195,165,195,167,195,170,195,171,195,168,195,175, + 195,174,195,172,195,132,195,133,195,137,195,166,195,134,195,180, + 195,182,195,178,195,187,195,185,195,191,195,150,195,156,194,162, + 194,163,194,165,226,130,167,198,146,195,161,195,173,195,179,195, + 186,195,177,195,145,194,170,194,186,194,191,226,140,144,194,172, + 194,189,194,188,194,161,194,171,194,187,226,150,145,226,150,146, + 226,150,147,226,148,130,226,148,164,226,149,161,226,149,162,226, + 149,150,226,149,149,226,149,163,226,149,145,226,149,151,226,149, + 157,226,149,156,226,149,155,226,148,144,226,148,148,226,148,180, + 226,148,172,226,148,156,226,148,128,226,148,188,226,149,158,226, + 149,159,226,149,154,226,149,148,226,149,169,226,149,166,226,149, + 160,226,149,144,226,149,172,226,149,167,226,149,168,226,149,164, + 226,149,165,226,149,153,226,149,152,226,149,146,226,149,147,226, + 149,171,226,149,170,226,148,152,226,148,140,226,150,136,226,150, + 132,226,150,140,226,150,144,226,150,128,206,177,195,159,206,147, + 207,128,206,163,207,131,194,181,207,132,206,166,206,152,206,169, + 206,180,226,136,158,207,134,206,181,226,136,169,226,137,161,194, + 177,226,137,165,226,137,164,226,140,160,226,140,161,195,183,226, + 137,136,194,176,226,136,153,194,183,226,136,154,226,129,191,194, + 178,226,150,160,194,160,99,0,0,0,0,0,0,0,0,1, + 0,0,0,8,0,0,0,67,0,0,0,115,108,0,0,0, + 116,0,114,22,116,1,160,2,100,1,161,1,1,0,116,3, + 100,2,131,1,130,1,100,3,97,0,122,60,122,16,100,4, + 100,5,108,4,109,5,125,0,1,0,87,0,110,38,4,0, + 116,6,107,10,114,82,1,0,1,0,1,0,116,1,160,2, + 100,1,161,1,1,0,116,3,100,2,131,1,130,1,89,0, + 110,2,88,0,87,0,53,0,100,6,97,0,88,0,116,1, + 160,2,100,7,161,1,1,0,124,0,83,0,41,8,78,122, + 27,122,105,112,105,109,112,111,114,116,58,32,122,108,105,98, + 32,85,78,65,86,65,73,76,65,66,76,69,122,41,99,97, + 110,39,116,32,100,101,99,111,109,112,114,101,115,115,32,100, + 97,116,97,59,32,122,108,105,98,32,110,111,116,32,97,118, + 97,105,108,97,98,108,101,84,114,0,0,0,0,41,1,218, + 10,100,101,99,111,109,112,114,101,115,115,70,122,25,122,105, + 112,105,109,112,111,114,116,58,32,122,108,105,98,32,97,118, + 97,105,108,97,98,108,101,41,7,218,15,95,105,109,112,111, + 114,116,105,110,103,95,122,108,105,98,114,68,0,0,0,114, + 69,0,0,0,114,3,0,0,0,90,4,122,108,105,98,114, + 122,0,0,0,218,9,69,120,99,101,112,116,105,111,110,41, + 1,114,122,0,0,0,114,9,0,0,0,114,9,0,0,0, + 114,10,0,0,0,218,20,95,103,101,116,95,100,101,99,111, + 109,112,114,101,115,115,95,102,117,110,99,220,1,0,0,115, + 24,0,0,0,0,2,4,3,10,1,8,2,4,1,4,1, + 16,1,14,1,10,1,18,2,6,2,10,1,114,125,0,0, + 0,99,2,0,0,0,0,0,0,0,17,0,0,0,9,0, + 0,0,67,0,0,0,115,130,1,0,0,124,1,92,8,125, + 2,125,3,125,4,125,5,125,6,125,7,125,8,125,9,124, + 4,100,1,107,0,114,36,116,0,100,2,131,1,130,1,116, + 1,160,2,124,0,100,3,161,2,144,1,143,8,125,10,122, + 14,124,10,160,3,124,6,161,1,1,0,87,0,110,38,4, + 0,116,4,107,10,114,104,1,0,1,0,1,0,116,0,100, + 4,124,0,155,2,157,2,124,0,100,5,141,2,130,1,89, + 0,110,2,88,0,124,10,160,5,100,6,161,1,125,11,116, + 6,124,11,131,1,100,6,107,3,114,136,116,7,100,7,131, + 1,130,1,124,11,100,0,100,8,133,2,25,0,100,9,107, + 3,114,170,116,0,100,10,124,0,155,2,157,2,124,0,100, + 5,141,2,130,1,116,8,124,11,100,11,100,12,133,2,25, + 0,131,1,125,12,116,8,124,11,100,12,100,6,133,2,25, + 0,131,1,125,13,100,6,124,12,23,0,124,13,23,0,125, + 14,124,6,124,14,55,0,125,6,122,14,124,10,160,3,124, + 6,161,1,1,0,87,0,110,40,4,0,116,4,107,10,144, + 1,114,20,1,0,1,0,1,0,116,0,100,4,124,0,155, + 2,157,2,124,0,100,5,141,2,130,1,89,0,110,2,88, + 0,124,10,160,5,124,4,161,1,125,15,116,6,124,15,131, + 1,124,4,107,3,144,1,114,54,116,4,100,13,131,1,130, + 1,87,0,53,0,81,0,82,0,88,0,124,3,100,1,107, + 2,144,1,114,78,124,15,83,0,122,10,116,9,131,0,125, + 16,87,0,110,30,4,0,116,10,107,10,144,1,114,118,1, + 0,1,0,1,0,116,0,100,14,131,1,130,1,89,0,110, + 2,88,0,124,16,124,15,100,15,131,2,83,0,41,16,78, + 114,0,0,0,0,122,18,110,101,103,97,116,105,118,101,32, + 100,97,116,97,32,115,105,122,101,114,81,0,0,0,122,21, + 99,97,110,39,116,32,114,101,97,100,32,90,105,112,32,102, + 105,108,101,58,32,41,1,114,11,0,0,0,114,93,0,0, + 0,122,27,69,79,70,32,114,101,97,100,32,119,104,101,114, + 101,32,110,111,116,32,101,120,112,101,99,116,101,100,114,83, + 0,0,0,115,4,0,0,0,80,75,3,4,122,23,98,97, + 100,32,108,111,99,97,108,32,102,105,108,101,32,104,101,97, + 100,101,114,58,32,233,26,0,0,0,114,92,0,0,0,122, + 26,122,105,112,105,109,112,111,114,116,58,32,99,97,110,39, + 116,32,114,101,97,100,32,100,97,116,97,122,41,99,97,110, + 39,116,32,100,101,99,111,109,112,114,101,115,115,32,100,97, + 116,97,59,32,122,108,105,98,32,110,111,116,32,97,118,97, + 105,108,97,98,108,101,105,241,255,255,255,41,11,114,3,0, + 0,0,114,99,0,0,0,114,100,0,0,0,114,101,0,0, + 0,114,20,0,0,0,114,102,0,0,0,114,47,0,0,0, + 114,103,0,0,0,114,1,0,0,0,114,125,0,0,0,114, + 124,0,0,0,41,17,114,27,0,0,0,114,50,0,0,0, + 90,8,100,97,116,97,112,97,116,104,114,112,0,0,0,114, + 116,0,0,0,114,117,0,0,0,114,120,0,0,0,114,113, + 0,0,0,114,114,0,0,0,114,115,0,0,0,114,107,0, + 0,0,114,108,0,0,0,114,118,0,0,0,114,119,0,0, + 0,114,109,0,0,0,90,8,114,97,119,95,100,97,116,97, + 114,122,0,0,0,114,9,0,0,0,114,9,0,0,0,114, + 10,0,0,0,114,48,0,0,0,241,1,0,0,115,62,0, + 0,0,0,1,20,1,8,1,8,2,16,2,2,1,14,1, + 14,1,24,1,10,1,12,1,8,2,16,2,18,2,16,1, + 16,1,12,1,8,1,2,1,14,1,16,1,24,1,10,1, + 14,1,18,2,10,2,4,3,2,1,10,1,16,1,14,1, + 114,48,0,0,0,99,2,0,0,0,0,0,0,0,2,0, + 0,0,3,0,0,0,67,0,0,0,115,16,0,0,0,116, + 0,124,0,124,1,24,0,131,1,100,1,107,1,83,0,41, + 2,78,114,5,0,0,0,41,1,218,3,97,98,115,41,2, + 90,2,116,49,90,2,116,50,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,218,9,95,101,113,95,109,116,105, + 109,101,31,2,0,0,115,2,0,0,0,0,2,114,128,0, + 0,0,99,3,0,0,0,0,0,0,0,5,0,0,0,5, + 0,0,0,67,0,0,0,115,206,0,0,0,116,0,124,1, + 131,1,100,1,107,0,114,20,116,1,100,2,131,1,130,1, + 124,1,100,0,100,3,133,2,25,0,116,2,106,3,107,3, + 114,54,116,4,160,5,100,4,124,0,161,2,1,0,100,0, + 83,0,116,6,124,1,100,3,100,5,133,2,25,0,131,1, + 125,3,124,3,100,6,107,3,114,112,116,7,106,8,100,7, + 107,3,114,110,124,3,100,8,107,3,115,106,116,7,106,8, + 100,9,107,2,114,110,100,0,83,0,110,46,124,2,100,6, + 107,3,114,158,116,9,116,6,124,1,100,5,100,10,133,2, + 25,0,131,1,124,2,131,2,115,158,116,4,160,5,100,11, + 124,0,161,2,1,0,100,0,83,0,116,10,160,11,124,1, + 100,1,100,0,133,2,25,0,161,1,125,4,116,12,124,4, + 116,13,131,2,115,202,116,14,100,12,124,0,155,2,100,13, + 157,3,131,1,130,1,124,4,83,0,41,14,78,114,85,0, + 0,0,122,12,98,97,100,32,112,121,99,32,100,97,116,97, + 114,83,0,0,0,122,18,123,33,114,125,32,104,97,115,32, + 98,97,100,32,109,97,103,105,99,114,88,0,0,0,114,0, + 0,0,0,90,5,110,101,118,101,114,114,5,0,0,0,90, + 6,97,108,119,97,121,115,114,84,0,0,0,122,18,123,33, + 114,125,32,104,97,115,32,98,97,100,32,109,116,105,109,101, + 122,16,99,111,109,112,105,108,101,100,32,109,111,100,117,108, + 101,32,122,21,32,105,115,32,110,111,116,32,97,32,99,111, + 100,101,32,111,98,106,101,99,116,41,15,114,47,0,0,0, + 114,3,0,0,0,114,19,0,0,0,90,12,77,65,71,73, + 67,95,78,85,77,66,69,82,114,68,0,0,0,114,69,0, + 0,0,114,2,0,0,0,218,4,95,105,109,112,90,21,99, + 104,101,99,107,95,104,97,115,104,95,98,97,115,101,100,95, + 112,121,99,115,114,128,0,0,0,218,7,109,97,114,115,104, + 97,108,90,5,108,111,97,100,115,114,13,0,0,0,218,10, + 95,99,111,100,101,95,116,121,112,101,218,9,84,121,112,101, + 69,114,114,111,114,41,5,114,49,0,0,0,218,4,100,97, + 116,97,218,5,109,116,105,109,101,114,111,0,0,0,114,42, + 0,0,0,114,9,0,0,0,114,9,0,0,0,114,10,0, + 0,0,218,15,95,117,110,109,97,114,115,104,97,108,95,99, + 111,100,101,39,2,0,0,115,40,0,0,0,0,1,12,1, + 8,2,18,1,12,1,4,2,16,1,8,5,10,1,6,255, + 2,1,8,255,2,2,6,1,30,1,12,1,4,4,18,1, + 10,1,16,1,114,135,0,0,0,99,1,0,0,0,0,0, + 0,0,1,0,0,0,4,0,0,0,67,0,0,0,115,28, + 0,0,0,124,0,160,0,100,1,100,2,161,2,125,0,124, + 0,160,0,100,3,100,2,161,2,125,0,124,0,83,0,41, + 4,78,115,2,0,0,0,13,10,243,1,0,0,0,10,243, + 1,0,0,0,13,41,1,114,17,0,0,0,41,1,218,6, + 115,111,117,114,99,101,114,9,0,0,0,114,9,0,0,0, + 114,10,0,0,0,218,23,95,110,111,114,109,97,108,105,122, + 101,95,108,105,110,101,95,101,110,100,105,110,103,115,72,2, + 0,0,115,6,0,0,0,0,1,12,1,12,1,114,139,0, + 0,0,99,2,0,0,0,0,0,0,0,2,0,0,0,6, + 0,0,0,67,0,0,0,115,24,0,0,0,116,0,124,1, + 131,1,125,1,116,1,124,1,124,0,100,1,100,2,100,3, + 141,4,83,0,41,4,78,114,66,0,0,0,84,41,1,90, + 12,100,111,110,116,95,105,110,104,101,114,105,116,41,2,114, + 139,0,0,0,218,7,99,111,109,112,105,108,101,41,2,114, + 49,0,0,0,114,138,0,0,0,114,9,0,0,0,114,9, + 0,0,0,114,10,0,0,0,218,15,95,99,111,109,112,105, + 108,101,95,115,111,117,114,99,101,79,2,0,0,115,4,0, + 0,0,0,1,8,1,114,141,0,0,0,99,2,0,0,0, + 0,0,0,0,2,0,0,0,11,0,0,0,67,0,0,0, + 115,68,0,0,0,116,0,160,1,124,0,100,1,63,0,100, + 2,23,0,124,0,100,3,63,0,100,4,64,0,124,0,100, + 5,64,0,124,1,100,6,63,0,124,1,100,3,63,0,100, + 7,64,0,124,1,100,5,64,0,100,8,20,0,100,9,100, + 9,100,9,102,9,161,1,83,0,41,10,78,233,9,0,0, + 0,105,188,7,0,0,233,5,0,0,0,233,15,0,0,0, + 233,31,0,0,0,233,11,0,0,0,233,63,0,0,0,114, + 76,0,0,0,114,12,0,0,0,41,2,114,113,0,0,0, + 90,6,109,107,116,105,109,101,41,2,218,1,100,114,121,0, + 0,0,114,9,0,0,0,114,9,0,0,0,114,10,0,0, + 0,218,14,95,112,97,114,115,101,95,100,111,115,116,105,109, + 101,85,2,0,0,115,24,0,0,0,0,1,4,1,10,1, + 10,1,6,1,6,1,10,1,10,1,2,0,2,0,2,250, + 2,255,114,149,0,0,0,99,2,0,0,0,0,0,0,0, + 5,0,0,0,10,0,0,0,67,0,0,0,115,104,0,0, + 0,122,70,124,1,100,1,100,0,133,2,25,0,100,2,107, + 6,115,22,116,0,130,1,124,1,100,0,100,1,133,2,25, + 0,125,1,124,0,106,1,124,1,25,0,125,2,124,2,100, + 3,25,0,125,3,124,2,100,4,25,0,125,4,116,2,124, + 4,124,3,131,2,87,0,83,0,4,0,116,3,116,4,116, + 5,102,3,107,10,114,98,1,0,1,0,1,0,89,0,100, + 5,83,0,88,0,100,0,83,0,41,6,78,114,12,0,0, + 0,41,2,218,1,99,218,1,111,114,143,0,0,0,233,6, + 0,0,0,114,0,0,0,0,41,6,218,14,65,115,115,101, + 114,116,105,111,110,69,114,114,111,114,114,26,0,0,0,114, + 149,0,0,0,114,24,0,0,0,218,10,73,110,100,101,120, + 69,114,114,111,114,114,132,0,0,0,41,5,114,30,0,0, + 0,114,11,0,0,0,114,50,0,0,0,114,113,0,0,0, + 114,114,0,0,0,114,9,0,0,0,114,9,0,0,0,114, + 10,0,0,0,218,20,95,103,101,116,95,109,116,105,109,101, + 95,111,102,95,115,111,117,114,99,101,98,2,0,0,115,18, + 0,0,0,0,1,2,2,20,1,12,1,10,3,8,1,8, + 1,12,1,20,1,114,155,0,0,0,99,2,0,0,0,0, + 0,0,0,12,0,0,0,9,0,0,0,67,0,0,0,115, + 204,0,0,0,116,0,124,0,124,1,131,2,125,2,116,1, + 68,0,93,166,92,3,125,3,125,4,125,5,124,2,124,3, + 23,0,125,6,116,2,106,3,100,1,124,0,106,4,116,5, + 124,6,100,2,100,3,141,5,1,0,122,14,124,0,106,6, + 124,6,25,0,125,7,87,0,110,20,4,0,116,7,107,10, + 114,88,1,0,1,0,1,0,89,0,113,14,88,0,124,7, + 100,4,25,0,125,8,116,8,124,0,106,4,124,7,131,2, + 125,9,124,4,114,138,116,9,124,0,124,6,131,2,125,10, + 116,10,124,8,124,9,124,10,131,3,125,11,110,10,116,11, + 124,8,124,9,131,2,125,11,124,11,100,0,107,8,114,158, + 113,14,124,7,100,4,25,0,125,8,124,11,124,5,124,8, + 102,3,2,0,1,0,83,0,113,14,116,12,100,5,124,1, + 155,2,157,2,124,1,100,6,141,2,130,1,100,0,83,0, + 41,7,78,122,13,116,114,121,105,110,103,32,123,125,123,125, + 123,125,114,76,0,0,0,41,1,90,9,118,101,114,98,111, + 115,105,116,121,114,0,0,0,0,122,18,99,97,110,39,116, + 32,102,105,110,100,32,109,111,100,117,108,101,32,41,1,114, + 53,0,0,0,41,13,114,34,0,0,0,114,78,0,0,0, + 114,68,0,0,0,114,69,0,0,0,114,27,0,0,0,114, + 18,0,0,0,114,26,0,0,0,114,24,0,0,0,114,48, + 0,0,0,114,155,0,0,0,114,135,0,0,0,114,141,0, + 0,0,114,3,0,0,0,41,12,114,30,0,0,0,114,36, + 0,0,0,114,11,0,0,0,114,79,0,0,0,114,80,0, + 0,0,114,43,0,0,0,114,55,0,0,0,114,50,0,0, + 0,114,38,0,0,0,114,133,0,0,0,114,134,0,0,0, + 114,42,0,0,0,114,9,0,0,0,114,9,0,0,0,114, + 10,0,0,0,114,41,0,0,0,114,2,0,0,115,38,0, + 0,0,0,1,10,1,14,1,8,1,22,1,2,1,14,1, + 14,1,6,2,8,1,12,1,4,1,10,1,14,2,10,1, + 8,3,2,1,8,1,16,2,114,41,0,0,0,41,40,114, + 74,0,0,0,90,26,95,102,114,111,122,101,110,95,105,109, + 112,111,114,116,108,105,98,95,101,120,116,101,114,110,97,108, + 114,19,0,0,0,114,1,0,0,0,114,2,0,0,0,90, + 17,95,102,114,111,122,101,110,95,105,109,112,111,114,116,108, + 105,98,114,68,0,0,0,114,129,0,0,0,114,99,0,0, + 0,114,130,0,0,0,114,59,0,0,0,114,113,0,0,0, + 90,7,95,95,97,108,108,95,95,114,18,0,0,0,90,15, + 112,97,116,104,95,115,101,112,97,114,97,116,111,114,115,114, + 16,0,0,0,114,67,0,0,0,114,3,0,0,0,114,23, + 0,0,0,218,4,116,121,112,101,114,62,0,0,0,114,4, + 0,0,0,114,78,0,0,0,114,34,0,0,0,114,35,0, + 0,0,114,33,0,0,0,114,25,0,0,0,114,106,0,0, + 0,114,123,0,0,0,114,125,0,0,0,114,48,0,0,0, + 114,128,0,0,0,114,135,0,0,0,218,8,95,95,99,111, + 100,101,95,95,114,131,0,0,0,114,139,0,0,0,114,141, + 0,0,0,114,149,0,0,0,114,155,0,0,0,114,41,0, + 0,0,114,9,0,0,0,114,9,0,0,0,114,9,0,0, + 0,114,10,0,0,0,218,8,60,109,111,100,117,108,101,62, + 13,0,0,0,115,78,0,0,0,4,4,8,1,16,1,8, + 1,8,1,8,1,8,1,8,1,8,2,8,3,6,1,14, + 3,16,4,4,2,8,3,14,127,0,120,12,1,12,1,2, + 1,2,253,2,255,2,9,8,4,8,9,8,31,8,103,2, + 254,2,29,4,5,8,21,8,46,8,8,8,28,10,5,8, + 7,8,6,8,13,8,16, +}; diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index b735228ea56d..06ea6b4dcd7d 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -191,7 +191,6 @@ initimport(PyInterpreterState *interp, PyObject *sysmod) PyObject *importlib; PyObject *impmod; PyObject *value; - _PyInitError err; /* Import _importlib through its frozen version, _frozen_importlib. */ if (PyImport_ImportFrozenModule("_frozen_importlib") <= 0) { @@ -233,11 +232,6 @@ initimport(PyInterpreterState *interp, PyObject *sysmod) Py_DECREF(value); Py_DECREF(impmod); - err = _PyImportZip_Init(); - if (_Py_INIT_FAILED(err)) { - return err; - } - return _Py_INIT_OK(); } @@ -252,7 +246,7 @@ initexternalimport(PyInterpreterState *interp) return _Py_INIT_ERR("external importer setup failed"); } Py_DECREF(value); - return _Py_INIT_OK(); + return _PyImportZip_Init(); } /* Helper functions to better handle the legacy C locale From webhook-mailer at python.org Tue Sep 18 16:28:38 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 18 Sep 2018 20:28:38 -0000 Subject: [Python-checkins] [2.7] bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683). (GH-9400) Message-ID: https://github.com/python/cpython/commit/29034baf58156e2462b0680e9092c0a3cccb0798 commit: 29034baf58156e2462b0680e9092c0a3cccb0798 branch: 2.7 author: Serhiy Storchaka committer: GitHub date: 2018-09-18T23:28:34+03:00 summary: [2.7] bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683). (GH-9400) (cherry picked from commit 9bdb7be482aef8f60daa1d36606568a132dcb616) files: A Misc/NEWS.d/next/Library/2018-08-06-11-01-18.bpo-34341.E0b9p2.rst M Lib/test/test_zipfile.py M Lib/zipfile.py diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index 9c63aebbbe0e..4e545f140a83 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -812,6 +812,20 @@ def test_too_many_files_append(self): self.assertEqual(content, "%d" % (i**3 % 57)) zipf2.close() + def test_append(self): + # Test that appending to the Zip64 archive doesn't change + # extra fields of existing entries. + with zipfile.ZipFile(TESTFN2, "w", allowZip64=True) as zipfp: + zipfp.writestr("strfile", self.data) + with zipfile.ZipFile(TESTFN2, "r", allowZip64=True) as zipfp: + zinfo = zipfp.getinfo("strfile") + extra = zinfo.extra + with zipfile.ZipFile(TESTFN2, "a", allowZip64=True) as zipfp: + zipfp.writestr("strfile2", self.data) + with zipfile.ZipFile(TESTFN2, "r", allowZip64=True) as zipfp: + zinfo = zipfp.getinfo("strfile") + self.assertEqual(zinfo.extra, extra) + def tearDown(self): zipfile.ZIP64_LIMIT = self._limit zipfile.ZIP_FILECOUNT_LIMIT = self._filecount_limit diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 3ab66cea69d8..0f890ac6f3ab 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -131,6 +131,27 @@ class LargeZipFile(Exception): _CD64_DIRECTORY_SIZE = 8 _CD64_OFFSET_START_CENTDIR = 9 +_EXTRA_FIELD_STRUCT = struct.Struct(' https://github.com/python/cpython/commit/a3c88ef12c7b8993912750b56a1e095652fe47c0 commit: a3c88ef12c7b8993912750b56a1e095652fe47c0 branch: master author: Nathaniel J. Smith committer: Carol Willing date: 2018-09-18T14:27:59-07:00 summary: Clarify that AsyncExitStack works with coroutine functions (GH-9405) The docs were ambiguous about whether you pass in a coroutine function or a coroutine object, e.g. is it: aestack.push_async_exit(some_async_func) or aestack.push_async_exit(some_async_func()) (It's the first one.) files: M Doc/library/contextlib.rst diff --git a/Doc/library/contextlib.rst b/Doc/library/contextlib.rst index 7dc5b2989f9c..930c97358e08 100644 --- a/Doc/library/contextlib.rst +++ b/Doc/library/contextlib.rst @@ -471,11 +471,11 @@ Functions and classes provided: .. method:: push_async_exit(exit) Similar to :meth:`push` but expects either an asynchronous context manager - or a coroutine. + or a coroutine function. .. method:: push_async_callback(callback, *args, **kwds) - Similar to :meth:`callback` but expects a coroutine. + Similar to :meth:`callback` but expects a coroutine function. .. method:: aclose() From webhook-mailer at python.org Tue Sep 18 17:55:48 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 18 Sep 2018 21:55:48 -0000 Subject: [Python-checkins] bpo-33649: Add a high-level section about Futures; few quick fixes (GH-9403) Message-ID: https://github.com/python/cpython/commit/471503954a91d86cf04228c38134108c67a263b0 commit: 471503954a91d86cf04228c38134108c67a263b0 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-18T17:55:44-04:00 summary: bpo-33649: Add a high-level section about Futures; few quick fixes (GH-9403) Co-authored-by: Elvis Pranskevichus files: M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-future.rst M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst M Doc/tools/extensions/pyspecific.py diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 20d18c02976b..92637327ae40 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -989,7 +989,7 @@ Availability: Unix. Executing code in thread or process pools ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. coroutinemethod:: loop.run_in_executor(executor, func, \*args) +.. awaitablemethod:: loop.run_in_executor(executor, func, \*args) Arrange for *func* to be called in the specified executor. diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index d6c5335c0e18..6e6e0137c1bd 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -7,7 +7,7 @@ Futures ======= -*Future* objects are used to bridge low-level callback-based code +*Future* objects are used to bridge **low-level callback-based code** with high-level async/await code. diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 4f37296f4a3e..85292a61ecd6 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -103,6 +103,31 @@ To actually run a coroutine asyncio provides three main mechanisms: world finished at 17:14:34 + +.. _asyncio-awaitables: + +Awaitables +========== + +We say that an object is an *awaitable* object if it can be used +in an :keyword:`await` expression. + + +.. rubric:: Coroutines and Tasks + +Python coroutines are *awaitables*:: + + async def nested(): + return 42 + + async def main(): + # Will print "42": + print(await nested()) + +*Tasks* are used to schedule coroutines *concurrently*. +See the previous :ref:`section ` for an introduction +to coroutines and tasks. + Note that in this documentation the term "coroutine" can be used for two closely related concepts: @@ -112,14 +137,41 @@ two closely related concepts: *coroutine function*. +.. rubric:: Futures + +There is a dedicated section about the :ref:`asyncio Future object +`, but the concept is fundamental to asyncio so +it needs a brief introduction in this section. + +A Future is a special **low-level** awaitable object that represents +an **eventual result** of an asynchronous operation. +Future objects in asyncio are needed to allow callback-based code +to be used with async/await. + +Normally, **there is no need** to create Future objects at the +application level code. + +Future objects, sometimes exposed by libraries and some asyncio +APIs, should be awaited:: + + async def main(): + await function_that_returns_a_future_object() + + # this is also valid: + await asyncio.gather( + function_that_returns_a_future_object(), + some_python_coroutine() + ) + + Running an asyncio Program ========================== .. function:: run(coro, \*, debug=False) This function runs the passed coroutine, taking care of - managing the asyncio event loop and finalizing asynchronous - generators. + managing the asyncio event loop and *finalizing asynchronous + generators*. This function cannot be called when another asyncio event loop is running in the same thread. @@ -140,8 +192,8 @@ Creating Tasks .. function:: create_task(coro, \*, name=None) - Wrap the *coro* :ref:`coroutine ` into a task and schedule - its execution. Return the task object. + Wrap the *coro* :ref:`coroutine ` into a Task and + schedule its execution. Return the Task object. If *name* is not ``None``, it is set as the name of the task using :meth:`Task.set_name`. @@ -150,6 +202,21 @@ Creating Tasks :exc:`RuntimeError` is raised if there is no running loop in current thread. + This function has been **added in Python 3.7**. Prior to + Python 3.7, the low-level :func:`asyncio.ensure_future` function + can be used instead:: + + async def coro(): + ... + + # In Python 3.7+ + task = asyncio.create_task(coro()) + ... + + # This works in all Python versions but is less readable + task = asyncio.ensure_future(coro()) + ... + .. versionadded:: 3.7 .. versionchanged:: 3.8 @@ -166,6 +233,9 @@ Sleeping If *result* is provided, it is returned to the caller when the coroutine completes. + The *loop* argument is deprecated and scheduled for removal + in Python 4.0. + .. _asyncio_example_sleep: Example of coroutine displaying the current date every second @@ -189,36 +259,31 @@ Sleeping Running Tasks Concurrently ========================== -.. function:: gather(\*fs, loop=None, return_exceptions=False) +.. awaitablefunction:: gather(\*fs, loop=None, return_exceptions=False) - Return a Future aggregating results from the given coroutine objects, - Tasks, or Futures. + Run :ref:`awaitable objects ` in the *fs* + sequence *concurrently*. - If all Tasks/Futures are completed successfully, the result is an - aggregate list of returned values. The result values are in the - order of the original *fs* sequence. + If any awaitable in *fs* is a coroutine, it is automatically + scheduled as a Task. - All coroutines in the *fs* list are automatically - scheduled as :class:`Tasks `. + If all awaitables are completed successfully, the result is an + aggregate list of returned values. The order of result values + corresponds to the order of awaitables in *fs*. - If *return_exceptions* is ``True``, exceptions in the Tasks/Futures - are treated the same as successful results, and gathered in the - result list. Otherwise, the first raised exception is immediately - propagated to the returned Future. + If *return_exceptions* is ``True``, exceptions are treated the + same as successful results, and aggregated in the result list. + Otherwise, the first raised exception is immediately propagated + to the task that awaits on ``gather()``. - If the outer Future is *cancelled*, all submitted Tasks/Futures + If ``gather`` is *cancelled*, all submitted awaitables (that have not completed yet) are also *cancelled*. - If any child is *cancelled*, it is treated as if it raised - :exc:`CancelledError` -- the outer Future is **not** cancelled in - this case. This is to prevent the cancellation of one submitted - Task/Future to cause other Tasks/Futures to be cancelled. - - All futures must share the same event loop. - - .. versionchanged:: 3.7 - If the *gather* itself is cancelled, the cancellation is - propagated regardless of *return_exceptions*. + If any Task or Future from the *fs* sequence is *cancelled*, it is + treated as if it raised :exc:`CancelledError` -- the ``gather()`` + call is **not** cancelled in this case. This is to prevent the + cancellation of one submitted Task/Future to cause other + Tasks/Futures to be cancelled. .. _asyncio_example_gather: @@ -235,6 +300,7 @@ Running Tasks Concurrently print(f"Task {name}: factorial({number}) = {f}") async def main(): + # Schedule three calls *concurrently*: await asyncio.gather( factorial("A", 2), factorial("B", 3), @@ -255,17 +321,21 @@ Running Tasks Concurrently # Task C: Compute factorial(4)... # Task C: factorial(4) = 24 + .. versionchanged:: 3.7 + If the *gather* itself is cancelled, the cancellation is + propagated regardless of *return_exceptions*. + Shielding Tasks From Cancellation ================================= -.. coroutinefunction:: shield(fut, \*, loop=None) +.. awaitablefunction:: shield(fut, \*, loop=None) - Wait for a Future/Task while protecting it from being cancelled. + Protect an :ref:`awaitable object ` + from being :meth:`cancelled `. *fut* can be a coroutine, a Task, or a Future-like object. If - *fut* is a coroutine it is automatically scheduled as a - :class:`Task`. + *fut* is a coroutine it is automatically scheduled as a Task. The statement:: @@ -299,11 +369,10 @@ Timeouts .. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) - Wait for a coroutine, Task, or Future to complete with timeout. + Wait for the *fut* :ref:`awaitable ` + to complete with a timeout. - *fut* can be a coroutine, a Task, or a Future-like object. If - *fut* is a coroutine it is automatically scheduled as a - :class:`Task`. + If *fut* is a coroutine it is automatically scheduled as a Task. *timeout* can either be ``None`` or a float or int number of seconds to wait for. If *timeout* is ``None``, block until the future @@ -312,13 +381,17 @@ Timeouts If a timeout occurs, it cancels the task and raises :exc:`asyncio.TimeoutError`. - To avoid the task cancellation, wrap it in :func:`shield`. + To avoid the task :meth:`cancellation `, + wrap it in :func:`shield`. The function will wait until the future is actually cancelled, so the total wait time may exceed the *timeout*. If the wait is cancelled, the future *fut* is also cancelled. + The *loop* argument is deprecated and scheduled for removal + in Python 4.0. + .. _asyncio_example_waitfor: Example:: @@ -353,13 +426,18 @@ Waiting Primitives .. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\ return_when=ALL_COMPLETED) - Wait for a set of coroutines, Tasks, or Futures to complete. + Run :ref:`awaitable objects ` in the *fs* + sequence concurrently and block until the condition specified + by *return_when*. - *fs* is a list of coroutines, Futures, and/or Tasks. Coroutines - are automatically scheduled as :class:`Tasks `. + If any awaitable in *fs* is a coroutine, it is automatically + scheduled as a Task. Returns two sets of Tasks/Futures: ``(done, pending)``. + The *loop* argument is deprecated and scheduled for removal + in Python 4.0. + *timeout* (a float or int), if specified, can be used to control the maximum number of seconds to wait before returning. @@ -398,8 +476,10 @@ Waiting Primitives .. function:: as_completed(fs, \*, loop=None, timeout=None) - Return an iterator of awaitables which return - :class:`Future` instances. + Run :ref:`awaitable objects ` in the *fs* + set concurrently. Return an iterator of :class:`Future` objects. + Each Future object returned represents the earliest result + from the set of the remaining awaitables. Raises :exc:`asyncio.TimeoutError` if the timeout occurs before all Futures are done. @@ -407,7 +487,7 @@ Waiting Primitives Example:: for f in as_completed(fs): - result = await f + earliest_result = await f # ... @@ -418,7 +498,8 @@ Scheduling From Other Threads Submit a coroutine to the given event loop. Thread-safe. - Return a :class:`concurrent.futures.Future` to access the result. + Return a :class:`concurrent.futures.Future` to wait for the result + from another OS thread. This function is meant to be called from a different OS thread than the one where the event loop is running. Example:: diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 1511b2f75a89..6990adb21e36 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -17,6 +17,7 @@ await asyncio.sleep(1) print('... World!') + # Python 3.7+ asyncio.run(main()) asyncio is a library to write **concurrent** code using diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py index b6618f929d4e..8036daa7d1de 100644 --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -163,6 +163,13 @@ def handle_signature(self, sig, signode): return ret +class PyAwaitableMixin(object): + def handle_signature(self, sig, signode): + ret = super(PyAwaitableMixin, self).handle_signature(sig, signode) + signode.insert(0, addnodes.desc_annotation('awaitable ', 'awaitable ')) + return ret + + class PyCoroutineFunction(PyCoroutineMixin, PyModulelevel): def run(self): self.name = 'py:function' @@ -175,6 +182,18 @@ def run(self): return PyClassmember.run(self) +class PyAwaitableFunction(PyAwaitableMixin, PyClassmember): + def run(self): + self.name = 'py:function' + return PyClassmember.run(self) + + +class PyAwaitableMethod(PyAwaitableMixin, PyClassmember): + def run(self): + self.name = 'py:method' + return PyClassmember.run(self) + + class PyAbstractMethod(PyClassmember): def handle_signature(self, sig, signode): @@ -394,6 +413,8 @@ def setup(app): app.add_directive_to_domain('py', 'decoratormethod', PyDecoratorMethod) app.add_directive_to_domain('py', 'coroutinefunction', PyCoroutineFunction) app.add_directive_to_domain('py', 'coroutinemethod', PyCoroutineMethod) + app.add_directive_to_domain('py', 'awaitablefunction', PyAwaitableFunction) + app.add_directive_to_domain('py', 'awaitablemethod', PyAwaitableMethod) app.add_directive_to_domain('py', 'abstractmethod', PyAbstractMethod) app.add_directive('miscnews', MiscNews) return {'version': '1.0', 'parallel_read_safe': True} From webhook-mailer at python.org Tue Sep 18 18:09:55 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 18 Sep 2018 22:09:55 -0000 Subject: [Python-checkins] bpo-33649: Add a high-level section about Futures; few quick fixes (GH-9403) Message-ID: https://github.com/python/cpython/commit/73c0006e71683b7d5b28192f18a2b9796e4195ef commit: 73c0006e71683b7d5b28192f18a2b9796e4195ef branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-18T15:09:51-07:00 summary: bpo-33649: Add a high-level section about Futures; few quick fixes (GH-9403) Co-authored-by: Elvis Pranskevichus (cherry picked from commit 471503954a91d86cf04228c38134108c67a263b0) Co-authored-by: Yury Selivanov files: M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-future.rst M Doc/library/asyncio-task.rst M Doc/library/asyncio.rst M Doc/tools/extensions/pyspecific.py diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 252224218572..30996308c80d 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -983,7 +983,7 @@ Availability: Unix. Executing code in thread or process pools ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. coroutinemethod:: loop.run_in_executor(executor, func, \*args) +.. awaitablemethod:: loop.run_in_executor(executor, func, \*args) Arrange for *func* to be called in the specified executor. diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index d6c5335c0e18..6e6e0137c1bd 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -7,7 +7,7 @@ Futures ======= -*Future* objects are used to bridge low-level callback-based code +*Future* objects are used to bridge **low-level callback-based code** with high-level async/await code. diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 45b8b604200e..4b079e81a292 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -103,6 +103,31 @@ To actually run a coroutine asyncio provides three main mechanisms: world finished at 17:14:34 + +.. _asyncio-awaitables: + +Awaitables +========== + +We say that an object is an *awaitable* object if it can be used +in an :keyword:`await` expression. + + +.. rubric:: Coroutines and Tasks + +Python coroutines are *awaitables*:: + + async def nested(): + return 42 + + async def main(): + # Will print "42": + print(await nested()) + +*Tasks* are used to schedule coroutines *concurrently*. +See the previous :ref:`section ` for an introduction +to coroutines and tasks. + Note that in this documentation the term "coroutine" can be used for two closely related concepts: @@ -112,14 +137,41 @@ two closely related concepts: *coroutine function*. +.. rubric:: Futures + +There is a dedicated section about the :ref:`asyncio Future object +`, but the concept is fundamental to asyncio so +it needs a brief introduction in this section. + +A Future is a special **low-level** awaitable object that represents +an **eventual result** of an asynchronous operation. +Future objects in asyncio are needed to allow callback-based code +to be used with async/await. + +Normally, **there is no need** to create Future objects at the +application level code. + +Future objects, sometimes exposed by libraries and some asyncio +APIs, should be awaited:: + + async def main(): + await function_that_returns_a_future_object() + + # this is also valid: + await asyncio.gather( + function_that_returns_a_future_object(), + some_python_coroutine() + ) + + Running an asyncio Program ========================== .. function:: run(coro, \*, debug=False) This function runs the passed coroutine, taking care of - managing the asyncio event loop and finalizing asynchronous - generators. + managing the asyncio event loop and *finalizing asynchronous + generators*. This function cannot be called when another asyncio event loop is running in the same thread. @@ -140,13 +192,28 @@ Creating Tasks .. function:: create_task(coro) - Wrap the *coro* :ref:`coroutine ` into a task and schedule - its execution. Return the task object. + Wrap the *coro* :ref:`coroutine ` into a Task and + schedule its execution. Return the Task object. The task is executed in the loop returned by :func:`get_running_loop`, :exc:`RuntimeError` is raised if there is no running loop in current thread. + This function has been **added in Python 3.7**. Prior to + Python 3.7, the low-level :func:`asyncio.ensure_future` function + can be used instead:: + + async def coro(): + ... + + # In Python 3.7+ + task = asyncio.create_task(coro()) + ... + + # This works in all Python versions but is less readable + task = asyncio.ensure_future(coro()) + ... + .. versionadded:: 3.7 @@ -160,6 +227,9 @@ Sleeping If *result* is provided, it is returned to the caller when the coroutine completes. + The *loop* argument is deprecated and scheduled for removal + in Python 4.0. + .. _asyncio_example_sleep: Example of coroutine displaying the current date every second @@ -183,36 +253,31 @@ Sleeping Running Tasks Concurrently ========================== -.. function:: gather(\*fs, loop=None, return_exceptions=False) +.. awaitablefunction:: gather(\*fs, loop=None, return_exceptions=False) - Return a Future aggregating results from the given coroutine objects, - Tasks, or Futures. + Run :ref:`awaitable objects ` in the *fs* + sequence *concurrently*. - If all Tasks/Futures are completed successfully, the result is an - aggregate list of returned values. The result values are in the - order of the original *fs* sequence. + If any awaitable in *fs* is a coroutine, it is automatically + scheduled as a Task. - All coroutines in the *fs* list are automatically - scheduled as :class:`Tasks `. + If all awaitables are completed successfully, the result is an + aggregate list of returned values. The order of result values + corresponds to the order of awaitables in *fs*. - If *return_exceptions* is ``True``, exceptions in the Tasks/Futures - are treated the same as successful results, and gathered in the - result list. Otherwise, the first raised exception is immediately - propagated to the returned Future. + If *return_exceptions* is ``True``, exceptions are treated the + same as successful results, and aggregated in the result list. + Otherwise, the first raised exception is immediately propagated + to the task that awaits on ``gather()``. - If the outer Future is *cancelled*, all submitted Tasks/Futures + If ``gather`` is *cancelled*, all submitted awaitables (that have not completed yet) are also *cancelled*. - If any child is *cancelled*, it is treated as if it raised - :exc:`CancelledError` -- the outer Future is **not** cancelled in - this case. This is to prevent the cancellation of one submitted - Task/Future to cause other Tasks/Futures to be cancelled. - - All futures must share the same event loop. - - .. versionchanged:: 3.7 - If the *gather* itself is cancelled, the cancellation is - propagated regardless of *return_exceptions*. + If any Task or Future from the *fs* sequence is *cancelled*, it is + treated as if it raised :exc:`CancelledError` -- the ``gather()`` + call is **not** cancelled in this case. This is to prevent the + cancellation of one submitted Task/Future to cause other + Tasks/Futures to be cancelled. .. _asyncio_example_gather: @@ -229,6 +294,7 @@ Running Tasks Concurrently print(f"Task {name}: factorial({number}) = {f}") async def main(): + # Schedule three calls *concurrently*: await asyncio.gather( factorial("A", 2), factorial("B", 3), @@ -249,17 +315,21 @@ Running Tasks Concurrently # Task C: Compute factorial(4)... # Task C: factorial(4) = 24 + .. versionchanged:: 3.7 + If the *gather* itself is cancelled, the cancellation is + propagated regardless of *return_exceptions*. + Shielding Tasks From Cancellation ================================= -.. coroutinefunction:: shield(fut, \*, loop=None) +.. awaitablefunction:: shield(fut, \*, loop=None) - Wait for a Future/Task while protecting it from being cancelled. + Protect an :ref:`awaitable object ` + from being :meth:`cancelled `. *fut* can be a coroutine, a Task, or a Future-like object. If - *fut* is a coroutine it is automatically scheduled as a - :class:`Task`. + *fut* is a coroutine it is automatically scheduled as a Task. The statement:: @@ -293,11 +363,10 @@ Timeouts .. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) - Wait for a coroutine, Task, or Future to complete with timeout. + Wait for the *fut* :ref:`awaitable ` + to complete with a timeout. - *fut* can be a coroutine, a Task, or a Future-like object. If - *fut* is a coroutine it is automatically scheduled as a - :class:`Task`. + If *fut* is a coroutine it is automatically scheduled as a Task. *timeout* can either be ``None`` or a float or int number of seconds to wait for. If *timeout* is ``None``, block until the future @@ -306,13 +375,17 @@ Timeouts If a timeout occurs, it cancels the task and raises :exc:`asyncio.TimeoutError`. - To avoid the task cancellation, wrap it in :func:`shield`. + To avoid the task :meth:`cancellation `, + wrap it in :func:`shield`. The function will wait until the future is actually cancelled, so the total wait time may exceed the *timeout*. If the wait is cancelled, the future *fut* is also cancelled. + The *loop* argument is deprecated and scheduled for removal + in Python 4.0. + .. _asyncio_example_waitfor: Example:: @@ -347,13 +420,18 @@ Waiting Primitives .. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\ return_when=ALL_COMPLETED) - Wait for a set of coroutines, Tasks, or Futures to complete. + Run :ref:`awaitable objects ` in the *fs* + sequence concurrently and block until the condition specified + by *return_when*. - *fs* is a list of coroutines, Futures, and/or Tasks. Coroutines - are automatically scheduled as :class:`Tasks `. + If any awaitable in *fs* is a coroutine, it is automatically + scheduled as a Task. Returns two sets of Tasks/Futures: ``(done, pending)``. + The *loop* argument is deprecated and scheduled for removal + in Python 4.0. + *timeout* (a float or int), if specified, can be used to control the maximum number of seconds to wait before returning. @@ -392,8 +470,10 @@ Waiting Primitives .. function:: as_completed(fs, \*, loop=None, timeout=None) - Return an iterator of awaitables which return - :class:`Future` instances. + Run :ref:`awaitable objects ` in the *fs* + set concurrently. Return an iterator of :class:`Future` objects. + Each Future object returned represents the earliest result + from the set of the remaining awaitables. Raises :exc:`asyncio.TimeoutError` if the timeout occurs before all Futures are done. @@ -401,7 +481,7 @@ Waiting Primitives Example:: for f in as_completed(fs): - result = await f + earliest_result = await f # ... @@ -412,7 +492,8 @@ Scheduling From Other Threads Submit a coroutine to the given event loop. Thread-safe. - Return a :class:`concurrent.futures.Future` to access the result. + Return a :class:`concurrent.futures.Future` to wait for the result + from another OS thread. This function is meant to be called from a different OS thread than the one where the event loop is running. Example:: diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 1511b2f75a89..6990adb21e36 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -17,6 +17,7 @@ await asyncio.sleep(1) print('... World!') + # Python 3.7+ asyncio.run(main()) asyncio is a library to write **concurrent** code using diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py index 92506e7b3270..90d7ceac568c 100644 --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -163,6 +163,13 @@ def handle_signature(self, sig, signode): return ret +class PyAwaitableMixin(object): + def handle_signature(self, sig, signode): + ret = super(PyAwaitableMixin, self).handle_signature(sig, signode) + signode.insert(0, addnodes.desc_annotation('awaitable ', 'awaitable ')) + return ret + + class PyCoroutineFunction(PyCoroutineMixin, PyModulelevel): def run(self): self.name = 'py:function' @@ -175,6 +182,18 @@ def run(self): return PyClassmember.run(self) +class PyAwaitableFunction(PyAwaitableMixin, PyClassmember): + def run(self): + self.name = 'py:function' + return PyClassmember.run(self) + + +class PyAwaitableMethod(PyAwaitableMixin, PyClassmember): + def run(self): + self.name = 'py:method' + return PyClassmember.run(self) + + class PyAbstractMethod(PyClassmember): def handle_signature(self, sig, signode): @@ -394,6 +413,8 @@ def setup(app): app.add_directive_to_domain('py', 'decoratormethod', PyDecoratorMethod) app.add_directive_to_domain('py', 'coroutinefunction', PyCoroutineFunction) app.add_directive_to_domain('py', 'coroutinemethod', PyCoroutineMethod) + app.add_directive_to_domain('py', 'awaitablefunction', PyAwaitableFunction) + app.add_directive_to_domain('py', 'awaitablemethod', PyAwaitableMethod) app.add_directive_to_domain('py', 'abstractmethod', PyAbstractMethod) app.add_directive('miscnews', MiscNews) return {'version': '1.0', 'parallel_read_safe': True} From webhook-mailer at python.org Wed Sep 19 02:25:55 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 19 Sep 2018 06:25:55 -0000 Subject: [Python-checkins] closes bpo-34585: Don't do runtime test to get float byte order. (GH-9085) Message-ID: https://github.com/python/cpython/commit/2a9c3805ddedf282881ef7811a561c70b74f80b1 commit: 2a9c3805ddedf282881ef7811a561c70b74f80b1 branch: master author: Ross Burton committer: Benjamin Peterson date: 2018-09-18T23:25:48-07:00 summary: closes bpo-34585: Don't do runtime test to get float byte order. (GH-9085) Currently configure.ac uses AC_RUN_IFELSE to determine the byte order of doubles, but this silently fails under cross compilation and Python doesn't do floats properly. Instead, steal a macro from autoconf-archive which compiles code using magic doubles (which encode to ASCII) and grep for the representation in the binary. RFC because this doesn't yet handle the weird ancient ARMv4 OABI 'mixed-endian' encoding properly. This encoding is ancient and I don't believe the union of "Python 3.8 users" and "OABI users" has anything in. Should the support for this just be dropped too? Alternatively, someone will need to find an OABI toolchain to verify the encoding of the magic double. files: A Misc/NEWS.d/next/Build/2018-09-18-16-28-31.bpo-34585.CGMu0h.rst A m4/ax_c_float_words_bigendian.m4 M configure.ac diff --git a/Misc/NEWS.d/next/Build/2018-09-18-16-28-31.bpo-34585.CGMu0h.rst b/Misc/NEWS.d/next/Build/2018-09-18-16-28-31.bpo-34585.CGMu0h.rst new file mode 100644 index 000000000000..01318e6e46a3 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2018-09-18-16-28-31.bpo-34585.CGMu0h.rst @@ -0,0 +1,3 @@ +Check for floating-point byte order in configure.ac using compilation tests +instead of executing code, so that these checks work in cross-compiled +builds. diff --git a/configure.ac b/configure.ac index 03638f8ae9bc..96331ec221be 100644 --- a/configure.ac +++ b/configure.ac @@ -4206,74 +4206,24 @@ fi # * Check for various properties of floating point * # ************************************************** -AC_MSG_CHECKING(whether C doubles are little-endian IEEE 754 binary64) -AC_CACHE_VAL(ac_cv_little_endian_double, [ -AC_RUN_IFELSE([AC_LANG_SOURCE([[ -#include -int main() { - double x = 9006104071832581.0; - if (memcmp(&x, "\x05\x04\x03\x02\x01\xff\x3f\x43", 8) == 0) - return 0; - else - return 1; -} -]])], -[ac_cv_little_endian_double=yes], -[ac_cv_little_endian_double=no], -[ac_cv_little_endian_double=no])]) -AC_MSG_RESULT($ac_cv_little_endian_double) -if test "$ac_cv_little_endian_double" = yes -then - AC_DEFINE(DOUBLE_IS_LITTLE_ENDIAN_IEEE754, 1, - [Define if C doubles are 64-bit IEEE 754 binary format, stored - with the least significant byte first]) -fi - -AC_MSG_CHECKING(whether C doubles are big-endian IEEE 754 binary64) -AC_CACHE_VAL(ac_cv_big_endian_double, [ -AC_RUN_IFELSE([AC_LANG_SOURCE([[ -#include -int main() { - double x = 9006104071832581.0; - if (memcmp(&x, "\x43\x3f\xff\x01\x02\x03\x04\x05", 8) == 0) - return 0; - else - return 1; -} -]])], -[ac_cv_big_endian_double=yes], -[ac_cv_big_endian_double=no], -[ac_cv_big_endian_double=no])]) -AC_MSG_RESULT($ac_cv_big_endian_double) -if test "$ac_cv_big_endian_double" = yes +AX_C_FLOAT_WORDS_BIGENDIAN +if test "$ax_cv_c_float_words_bigendian" = "yes" then AC_DEFINE(DOUBLE_IS_BIG_ENDIAN_IEEE754, 1, [Define if C doubles are 64-bit IEEE 754 binary format, stored with the most significant byte first]) -fi - -# Some ARM platforms use a mixed-endian representation for doubles. -# While Python doesn't currently have full support for these platforms -# (see e.g., issue 1762561), we can at least make sure that float <-> string -# conversions work. -AC_MSG_CHECKING(whether C doubles are ARM mixed-endian IEEE 754 binary64) -AC_CACHE_VAL(ac_cv_mixed_endian_double, [ -AC_RUN_IFELSE([AC_LANG_SOURCE([[ -#include -int main() { - double x = 9006104071832581.0; - if (memcmp(&x, "\x01\xff\x3f\x43\x05\x04\x03\x02", 8) == 0) - return 0; - else - return 1; -} -]])], -[ac_cv_mixed_endian_double=yes], -[ac_cv_mixed_endian_double=no], -[ac_cv_mixed_endian_double=no])]) -AC_MSG_RESULT($ac_cv_mixed_endian_double) -if test "$ac_cv_mixed_endian_double" = yes +elif test "$ax_cv_c_float_words_bigendian" = "no" then + AC_DEFINE(DOUBLE_IS_LITTLE_ENDIAN_IEEE754, 1, + [Define if C doubles are 64-bit IEEE 754 binary format, stored + with the least significant byte first]) +else + # Some ARM platforms use a mixed-endian representation for doubles. + # While Python doesn't currently have full support for these platforms + # (see e.g., issue 1762561), we can at least make sure that float <-> string + # conversions work. + # FLOAT_WORDS_BIGENDIAN doesnt actually detect this case, but if it's not big + # or little, then it must be this? AC_DEFINE(DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754, 1, [Define if C doubles are 64-bit IEEE 754 binary format, stored in ARM mixed-endian order (byte order 45670123)]) diff --git a/m4/ax_c_float_words_bigendian.m4 b/m4/ax_c_float_words_bigendian.m4 new file mode 100644 index 000000000000..216b90d80318 --- /dev/null +++ b/m4/ax_c_float_words_bigendian.m4 @@ -0,0 +1,83 @@ +# =============================================================================== +# https://www.gnu.org/software/autoconf-archive/ax_c_float_words_bigendian.html +# =============================================================================== +# +# SYNOPSIS +# +# AX_C_FLOAT_WORDS_BIGENDIAN([ACTION-IF-TRUE], [ACTION-IF-FALSE], [ACTION-IF-UNKNOWN]) +# +# DESCRIPTION +# +# Checks the ordering of words within a multi-word float. This check is +# necessary because on some systems (e.g. certain ARM systems), the float +# word ordering can be different from the byte ordering. In a multi-word +# float context, "big-endian" implies that the word containing the sign +# bit is found in the memory location with the lowest address. This +# implementation was inspired by the AC_C_BIGENDIAN macro in autoconf. +# +# The endianness is detected by first compiling C code that contains a +# special double float value, then grepping the resulting object file for +# certain strings of ASCII values. The double is specially crafted to have +# a binary representation that corresponds with a simple string. In this +# implementation, the string "noonsees" was selected because the +# individual word values ("noon" and "sees") are palindromes, thus making +# this test byte-order agnostic. If grep finds the string "noonsees" in +# the object file, the target platform stores float words in big-endian +# order. If grep finds "seesnoon", float words are in little-endian order. +# If neither value is found, the user is instructed to specify the +# ordering. +# +# LICENSE +# +# Copyright (c) 2008 Daniel Amelang +# +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice +# and this notice are preserved. This file is offered as-is, without any +# warranty. + +#serial 11 + +AC_DEFUN([AX_C_FLOAT_WORDS_BIGENDIAN], + [AC_CACHE_CHECK(whether float word ordering is bigendian, + ax_cv_c_float_words_bigendian, [ + +ax_cv_c_float_words_bigendian=unknown +AC_COMPILE_IFELSE([AC_LANG_SOURCE([[ + +double d = 90904234967036810337470478905505011476211692735615632014797120844053488865816695273723469097858056257517020191247487429516932130503560650002327564517570778480236724525140520121371739201496540132640109977779420565776568942592.0; + +]])], [ + +if grep noonsees conftest.$ac_objext >/dev/null ; then + ax_cv_c_float_words_bigendian=yes +fi +if grep seesnoon conftest.$ac_objext >/dev/null ; then + if test "$ax_cv_c_float_words_bigendian" = unknown; then + ax_cv_c_float_words_bigendian=no + else + ax_cv_c_float_words_bigendian=unknown + fi +fi + +])]) + +case $ax_cv_c_float_words_bigendian in + yes) + m4_default([$1], + [AC_DEFINE([FLOAT_WORDS_BIGENDIAN], 1, + [Define to 1 if your system stores words within floats + with the most significant word first])]) ;; + no) + $2 ;; + *) + m4_default([$3], + [AC_MSG_ERROR([ + +Unknown float word ordering. You need to manually preset +ax_cv_c_float_words_bigendian=no (or yes) according to your system. + + ])]) ;; +esac + +])# AX_C_FLOAT_WORDS_BIGENDIAN From webhook-mailer at python.org Wed Sep 19 02:28:11 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Wed, 19 Sep 2018 06:28:11 -0000 Subject: [Python-checkins] bpo-25711: Move _ZipImportResourceReader from importlib to zipimport. (GH-9406) Message-ID: https://github.com/python/cpython/commit/9da3961f364da2a3ced740230b85ffb4309238d3 commit: 9da3961f364da2a3ced740230b85ffb4309238d3 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-19T09:28:06+03:00 summary: bpo-25711: Move _ZipImportResourceReader from importlib to zipimport. (GH-9406) files: M Lib/importlib/resources.py M Lib/zipimport.py M Python/importlib_zipimport.h diff --git a/Lib/importlib/resources.py b/Lib/importlib/resources.py index cbefdd540e9e..fc3a1c9cabe6 100644 --- a/Lib/importlib/resources.py +++ b/Lib/importlib/resources.py @@ -257,87 +257,3 @@ def contents(package: Package) -> Iterable[str]: else: package_directory = Path(package.__spec__.origin).parent return os.listdir(package_directory) - - -# Private implementation of ResourceReader and get_resource_reader() called -# from zipimport.c. Don't use these directly! We're implementing these in -# Python because 1) it's easier, 2) zipimport may get rewritten in Python -# itself at some point, so doing this all in C would difficult and a waste of -# effort. - -class _ZipImportResourceReader(resources_abc.ResourceReader): - """Private class used to support ZipImport.get_resource_reader(). - - This class is allowed to reference all the innards and private parts of - the zipimporter. - """ - - def __init__(self, zipimporter, fullname): - self.zipimporter = zipimporter - self.fullname = fullname - - def open_resource(self, resource): - fullname_as_path = self.fullname.replace('.', '/') - path = f'{fullname_as_path}/{resource}' - try: - return BytesIO(self.zipimporter.get_data(path)) - except OSError: - raise FileNotFoundError(path) - - def resource_path(self, resource): - # All resources are in the zip file, so there is no path to the file. - # Raising FileNotFoundError tells the higher level API to extract the - # binary data and create a temporary file. - raise FileNotFoundError - - def is_resource(self, name): - # Maybe we could do better, but if we can get the data, it's a - # resource. Otherwise it isn't. - fullname_as_path = self.fullname.replace('.', '/') - path = f'{fullname_as_path}/{name}' - try: - self.zipimporter.get_data(path) - except OSError: - return False - return True - - def contents(self): - # This is a bit convoluted, because fullname will be a module path, - # but _files is a list of file names relative to the top of the - # archive's namespace. We want to compare file paths to find all the - # names of things inside the module represented by fullname. So we - # turn the module path of fullname into a file path relative to the - # top of the archive, and then we iterate through _files looking for - # names inside that "directory". - fullname_path = Path(self.zipimporter.get_filename(self.fullname)) - relative_path = fullname_path.relative_to(self.zipimporter.archive) - # Don't forget that fullname names a package, so its path will include - # __init__.py, which we want to ignore. - assert relative_path.name == '__init__.py' - package_path = relative_path.parent - subdirs_seen = set() - for filename in self.zipimporter._files: - try: - relative = Path(filename).relative_to(package_path) - except ValueError: - continue - # If the path of the file (which is relative to the top of the zip - # namespace), relative to the package given when the resource - # reader was created, has a parent, then it's a name in a - # subdirectory and thus we skip it. - parent_name = relative.parent.name - if len(parent_name) == 0: - yield relative.name - elif parent_name not in subdirs_seen: - subdirs_seen.add(parent_name) - yield parent_name - - -# Called from zipimport.c -def _zipimport_get_resource_reader(zipimporter, fullname): - try: - if not zipimporter.is_package(fullname): - return None - except ZipImportError: - return None - return _ZipImportResourceReader(zipimporter, fullname) diff --git a/Lib/zipimport.py b/Lib/zipimport.py index 059f124512dc..4017340d1184 100644 --- a/Lib/zipimport.py +++ b/Lib/zipimport.py @@ -272,8 +272,16 @@ def get_resource_reader(self, fullname): If 'fullname' is a package within the zip file, return the 'ResourceReader' object for the package. Otherwise return None. """ - from importlib import resources - return resources._zipimport_get_resource_reader(self, fullname) + try: + if not self.is_package(fullname): + return None + except ZipImportError: + return None + if not _ZipImportResourceReader._registered: + from importlib.abc import ResourceReader + ResourceReader.register(_ZipImportResourceReader) + _ZipImportResourceReader._registered = True + return _ZipImportResourceReader(self, fullname) def __repr__(self): @@ -648,3 +656,74 @@ def _get_module_code(self, fullname): return code, ispackage, modpath else: raise ZipImportError(f"can't find module {fullname!r}", name=fullname) + + +class _ZipImportResourceReader: + """Private class used to support ZipImport.get_resource_reader(). + + This class is allowed to reference all the innards and private parts of + the zipimporter. + """ + _registered = False + + def __init__(self, zipimporter, fullname): + self.zipimporter = zipimporter + self.fullname = fullname + + def open_resource(self, resource): + fullname_as_path = self.fullname.replace('.', '/') + path = f'{fullname_as_path}/{resource}' + from io import BytesIO + try: + return BytesIO(self.zipimporter.get_data(path)) + except OSError: + raise FileNotFoundError(path) + + def resource_path(self, resource): + # All resources are in the zip file, so there is no path to the file. + # Raising FileNotFoundError tells the higher level API to extract the + # binary data and create a temporary file. + raise FileNotFoundError + + def is_resource(self, name): + # Maybe we could do better, but if we can get the data, it's a + # resource. Otherwise it isn't. + fullname_as_path = self.fullname.replace('.', '/') + path = f'{fullname_as_path}/{name}' + try: + self.zipimporter.get_data(path) + except OSError: + return False + return True + + def contents(self): + # This is a bit convoluted, because fullname will be a module path, + # but _files is a list of file names relative to the top of the + # archive's namespace. We want to compare file paths to find all the + # names of things inside the module represented by fullname. So we + # turn the module path of fullname into a file path relative to the + # top of the archive, and then we iterate through _files looking for + # names inside that "directory". + from pathlib import Path + fullname_path = Path(self.zipimporter.get_filename(self.fullname)) + relative_path = fullname_path.relative_to(self.zipimporter.archive) + # Don't forget that fullname names a package, so its path will include + # __init__.py, which we want to ignore. + assert relative_path.name == '__init__.py' + package_path = relative_path.parent + subdirs_seen = set() + for filename in self.zipimporter._files: + try: + relative = Path(filename).relative_to(package_path) + except ValueError: + continue + # If the path of the file (which is relative to the top of the zip + # namespace), relative to the package given when the resource + # reader was created, has a parent, then it's a name in a + # subdirectory and thus we skip it. + parent_name = relative.parent.name + if len(parent_name) == 0: + yield relative.name + elif parent_name not in subdirs_seen: + subdirs_seen.add(parent_name) + yield parent_name diff --git a/Python/importlib_zipimport.h b/Python/importlib_zipimport.h index 6ca737a01394..5384154aaf57 100644 --- a/Python/importlib_zipimport.h +++ b/Python/importlib_zipimport.h @@ -1,7 +1,7 @@ /* Auto-generated by Programs/_freeze_importlib.c */ const unsigned char _Py_M__zipimport[] = { 99,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0, - 0,64,0,0,0,115,48,1,0,0,100,0,90,0,100,1, + 0,64,0,0,0,115,62,1,0,0,100,0,90,0,100,1, 100,2,108,1,90,2,100,1,100,3,108,1,109,3,90,3, 109,4,90,4,1,0,100,1,100,2,108,5,90,6,100,1, 100,2,108,7,90,7,100,1,100,2,108,8,90,8,100,1, @@ -20,881 +20,1001 @@ const unsigned char _Py_M__zipimport[] = { 132,0,90,32,101,19,101,32,106,33,131,1,90,34,100,32, 100,33,132,0,90,35,100,34,100,35,132,0,90,36,100,36, 100,37,132,0,90,37,100,38,100,39,132,0,90,38,100,40, - 100,41,132,0,90,39,100,2,83,0,41,42,97,80,2,0, - 0,122,105,112,105,109,112,111,114,116,32,112,114,111,118,105, - 100,101,115,32,115,117,112,112,111,114,116,32,102,111,114,32, - 105,109,112,111,114,116,105,110,103,32,80,121,116,104,111,110, - 32,109,111,100,117,108,101,115,32,102,114,111,109,32,90,105, - 112,32,97,114,99,104,105,118,101,115,46,10,10,84,104,105, - 115,32,109,111,100,117,108,101,32,101,120,112,111,114,116,115, - 32,116,104,114,101,101,32,111,98,106,101,99,116,115,58,10, - 45,32,122,105,112,105,109,112,111,114,116,101,114,58,32,97, - 32,99,108,97,115,115,59,32,105,116,115,32,99,111,110,115, - 116,114,117,99,116,111,114,32,116,97,107,101,115,32,97,32, - 112,97,116,104,32,116,111,32,97,32,90,105,112,32,97,114, - 99,104,105,118,101,46,10,45,32,90,105,112,73,109,112,111, - 114,116,69,114,114,111,114,58,32,101,120,99,101,112,116,105, - 111,110,32,114,97,105,115,101,100,32,98,121,32,122,105,112, - 105,109,112,111,114,116,101,114,32,111,98,106,101,99,116,115, - 46,32,73,116,39,115,32,97,10,32,32,115,117,98,99,108, - 97,115,115,32,111,102,32,73,109,112,111,114,116,69,114,114, - 111,114,44,32,115,111,32,105,116,32,99,97,110,32,98,101, - 32,99,97,117,103,104,116,32,97,115,32,73,109,112,111,114, - 116,69,114,114,111,114,44,32,116,111,111,46,10,45,32,95, + 100,41,132,0,90,39,71,0,100,42,100,43,132,0,100,43, + 131,2,90,40,100,2,83,0,41,44,97,80,2,0,0,122, + 105,112,105,109,112,111,114,116,32,112,114,111,118,105,100,101, + 115,32,115,117,112,112,111,114,116,32,102,111,114,32,105,109, + 112,111,114,116,105,110,103,32,80,121,116,104,111,110,32,109, + 111,100,117,108,101,115,32,102,114,111,109,32,90,105,112,32, + 97,114,99,104,105,118,101,115,46,10,10,84,104,105,115,32, + 109,111,100,117,108,101,32,101,120,112,111,114,116,115,32,116, + 104,114,101,101,32,111,98,106,101,99,116,115,58,10,45,32, + 122,105,112,105,109,112,111,114,116,101,114,58,32,97,32,99, + 108,97,115,115,59,32,105,116,115,32,99,111,110,115,116,114, + 117,99,116,111,114,32,116,97,107,101,115,32,97,32,112,97, + 116,104,32,116,111,32,97,32,90,105,112,32,97,114,99,104, + 105,118,101,46,10,45,32,90,105,112,73,109,112,111,114,116, + 69,114,114,111,114,58,32,101,120,99,101,112,116,105,111,110, + 32,114,97,105,115,101,100,32,98,121,32,122,105,112,105,109, + 112,111,114,116,101,114,32,111,98,106,101,99,116,115,46,32, + 73,116,39,115,32,97,10,32,32,115,117,98,99,108,97,115, + 115,32,111,102,32,73,109,112,111,114,116,69,114,114,111,114, + 44,32,115,111,32,105,116,32,99,97,110,32,98,101,32,99, + 97,117,103,104,116,32,97,115,32,73,109,112,111,114,116,69, + 114,114,111,114,44,32,116,111,111,46,10,45,32,95,122,105, + 112,95,100,105,114,101,99,116,111,114,121,95,99,97,99,104, + 101,58,32,97,32,100,105,99,116,44,32,109,97,112,112,105, + 110,103,32,97,114,99,104,105,118,101,32,112,97,116,104,115, + 32,116,111,32,122,105,112,32,100,105,114,101,99,116,111,114, + 121,10,32,32,105,110,102,111,32,100,105,99,116,115,44,32, + 97,115,32,117,115,101,100,32,105,110,32,122,105,112,105,109, + 112,111,114,116,101,114,46,95,102,105,108,101,115,46,10,10, + 73,116,32,105,115,32,117,115,117,97,108,108,121,32,110,111, + 116,32,110,101,101,100,101,100,32,116,111,32,117,115,101,32, + 116,104,101,32,122,105,112,105,109,112,111,114,116,32,109,111, + 100,117,108,101,32,101,120,112,108,105,99,105,116,108,121,59, + 32,105,116,32,105,115,10,117,115,101,100,32,98,121,32,116, + 104,101,32,98,117,105,108,116,105,110,32,105,109,112,111,114, + 116,32,109,101,99,104,97,110,105,115,109,32,102,111,114,32, + 115,121,115,46,112,97,116,104,32,105,116,101,109,115,32,116, + 104,97,116,32,97,114,101,32,112,97,116,104,115,10,116,111, + 32,90,105,112,32,97,114,99,104,105,118,101,115,46,10,233, + 0,0,0,0,78,41,2,218,14,95,117,110,112,97,99,107, + 95,117,105,110,116,49,54,218,14,95,117,110,112,97,99,107, + 95,117,105,110,116,51,50,218,14,90,105,112,73,109,112,111, + 114,116,69,114,114,111,114,218,11,122,105,112,105,109,112,111, + 114,116,101,114,233,1,0,0,0,99,0,0,0,0,0,0, + 0,0,0,0,0,0,1,0,0,0,64,0,0,0,115,12, + 0,0,0,101,0,90,1,100,0,90,2,100,1,83,0,41, + 2,114,3,0,0,0,78,41,3,218,8,95,95,110,97,109, + 101,95,95,218,10,95,95,109,111,100,117,108,101,95,95,218, + 12,95,95,113,117,97,108,110,97,109,101,95,95,169,0,114, + 9,0,0,0,114,9,0,0,0,250,18,60,102,114,111,122, + 101,110,32,122,105,112,105,109,112,111,114,116,62,114,3,0, + 0,0,33,0,0,0,115,2,0,0,0,8,1,99,0,0, + 0,0,0,0,0,0,0,0,0,0,3,0,0,0,64,0, + 0,0,115,108,0,0,0,101,0,90,1,100,0,90,2,100, + 1,90,3,100,2,100,3,132,0,90,4,100,25,100,5,100, + 6,132,1,90,5,100,26,100,7,100,8,132,1,90,6,100, + 9,100,10,132,0,90,7,100,11,100,12,132,0,90,8,100, + 13,100,14,132,0,90,9,100,15,100,16,132,0,90,10,100, + 17,100,18,132,0,90,11,100,19,100,20,132,0,90,12,100, + 21,100,22,132,0,90,13,100,23,100,24,132,0,90,14,100, + 4,83,0,41,27,114,4,0,0,0,97,255,1,0,0,122, + 105,112,105,109,112,111,114,116,101,114,40,97,114,99,104,105, + 118,101,112,97,116,104,41,32,45,62,32,122,105,112,105,109, + 112,111,114,116,101,114,32,111,98,106,101,99,116,10,10,32, + 32,32,32,67,114,101,97,116,101,32,97,32,110,101,119,32, + 122,105,112,105,109,112,111,114,116,101,114,32,105,110,115,116, + 97,110,99,101,46,32,39,97,114,99,104,105,118,101,112,97, + 116,104,39,32,109,117,115,116,32,98,101,32,97,32,112,97, + 116,104,32,116,111,10,32,32,32,32,97,32,122,105,112,102, + 105,108,101,44,32,111,114,32,116,111,32,97,32,115,112,101, + 99,105,102,105,99,32,112,97,116,104,32,105,110,115,105,100, + 101,32,97,32,122,105,112,102,105,108,101,46,32,70,111,114, + 32,101,120,97,109,112,108,101,44,32,105,116,32,99,97,110, + 32,98,101,10,32,32,32,32,39,47,116,109,112,47,109,121, + 105,109,112,111,114,116,46,122,105,112,39,44,32,111,114,32, + 39,47,116,109,112,47,109,121,105,109,112,111,114,116,46,122, + 105,112,47,109,121,100,105,114,101,99,116,111,114,121,39,44, + 32,105,102,32,109,121,100,105,114,101,99,116,111,114,121,32, + 105,115,32,97,10,32,32,32,32,118,97,108,105,100,32,100, + 105,114,101,99,116,111,114,121,32,105,110,115,105,100,101,32, + 116,104,101,32,97,114,99,104,105,118,101,46,10,10,32,32, + 32,32,39,90,105,112,73,109,112,111,114,116,69,114,114,111, + 114,32,105,115,32,114,97,105,115,101,100,32,105,102,32,39, + 97,114,99,104,105,118,101,112,97,116,104,39,32,100,111,101, + 115,110,39,116,32,112,111,105,110,116,32,116,111,32,97,32, + 118,97,108,105,100,32,90,105,112,10,32,32,32,32,97,114, + 99,104,105,118,101,46,10,10,32,32,32,32,84,104,101,32, + 39,97,114,99,104,105,118,101,39,32,97,116,116,114,105,98, + 117,116,101,32,111,102,32,122,105,112,105,109,112,111,114,116, + 101,114,32,111,98,106,101,99,116,115,32,99,111,110,116,97, + 105,110,115,32,116,104,101,32,110,97,109,101,32,111,102,32, + 116,104,101,10,32,32,32,32,122,105,112,102,105,108,101,32, + 116,97,114,103,101,116,101,100,46,10,32,32,32,32,99,2, + 0,0,0,0,0,0,0,8,0,0,0,9,0,0,0,67, + 0,0,0,115,36,1,0,0,116,0,124,1,116,1,131,2, + 115,28,100,1,100,0,108,2,125,2,124,2,160,3,124,1, + 161,1,125,1,124,1,115,44,116,4,100,2,124,1,100,3, + 141,2,130,1,116,5,114,60,124,1,160,6,116,5,116,7, + 161,2,125,1,103,0,125,3,122,14,116,8,160,9,124,1, + 161,1,125,4,87,0,110,72,4,0,116,10,116,11,102,2, + 107,10,114,150,1,0,1,0,1,0,116,8,160,12,124,1, + 161,1,92,2,125,5,125,6,124,5,124,1,107,2,114,132, + 116,4,100,4,124,1,100,3,141,2,130,1,124,5,125,1, + 124,3,160,13,124,6,161,1,1,0,89,0,113,64,88,0, + 124,4,106,14,100,5,64,0,100,6,107,3,114,182,116,4, + 100,4,124,1,100,3,141,2,130,1,113,182,113,64,122,12, + 116,15,124,1,25,0,125,7,87,0,110,36,4,0,116,16, + 107,10,114,230,1,0,1,0,1,0,116,17,124,1,131,1, + 125,7,124,7,116,15,124,1,60,0,89,0,110,2,88,0, + 124,7,124,0,95,18,124,1,124,0,95,19,116,8,106,20, + 124,3,100,0,100,0,100,7,133,3,25,0,142,0,124,0, + 95,21,124,0,106,21,144,1,114,32,124,0,4,0,106,21, + 116,7,55,0,2,0,95,21,100,0,83,0,41,8,78,114, + 0,0,0,0,122,21,97,114,99,104,105,118,101,32,112,97, + 116,104,32,105,115,32,101,109,112,116,121,41,1,218,4,112, + 97,116,104,122,14,110,111,116,32,97,32,90,105,112,32,102, + 105,108,101,105,0,240,0,0,105,0,128,0,0,233,255,255, + 255,255,41,22,218,10,105,115,105,110,115,116,97,110,99,101, + 218,3,115,116,114,218,2,111,115,90,8,102,115,100,101,99, + 111,100,101,114,3,0,0,0,218,12,97,108,116,95,112,97, + 116,104,95,115,101,112,218,7,114,101,112,108,97,99,101,218, + 8,112,97,116,104,95,115,101,112,218,19,95,98,111,111,116, + 115,116,114,97,112,95,101,120,116,101,114,110,97,108,90,10, + 95,112,97,116,104,95,115,116,97,116,218,7,79,83,69,114, + 114,111,114,218,10,86,97,108,117,101,69,114,114,111,114,90, + 11,95,112,97,116,104,95,115,112,108,105,116,218,6,97,112, + 112,101,110,100,90,7,115,116,95,109,111,100,101,218,20,95, 122,105,112,95,100,105,114,101,99,116,111,114,121,95,99,97, - 99,104,101,58,32,97,32,100,105,99,116,44,32,109,97,112, - 112,105,110,103,32,97,114,99,104,105,118,101,32,112,97,116, - 104,115,32,116,111,32,122,105,112,32,100,105,114,101,99,116, - 111,114,121,10,32,32,105,110,102,111,32,100,105,99,116,115, - 44,32,97,115,32,117,115,101,100,32,105,110,32,122,105,112, - 105,109,112,111,114,116,101,114,46,95,102,105,108,101,115,46, - 10,10,73,116,32,105,115,32,117,115,117,97,108,108,121,32, - 110,111,116,32,110,101,101,100,101,100,32,116,111,32,117,115, - 101,32,116,104,101,32,122,105,112,105,109,112,111,114,116,32, - 109,111,100,117,108,101,32,101,120,112,108,105,99,105,116,108, - 121,59,32,105,116,32,105,115,10,117,115,101,100,32,98,121, - 32,116,104,101,32,98,117,105,108,116,105,110,32,105,109,112, - 111,114,116,32,109,101,99,104,97,110,105,115,109,32,102,111, - 114,32,115,121,115,46,112,97,116,104,32,105,116,101,109,115, - 32,116,104,97,116,32,97,114,101,32,112,97,116,104,115,10, - 116,111,32,90,105,112,32,97,114,99,104,105,118,101,115,46, - 10,233,0,0,0,0,78,41,2,218,14,95,117,110,112,97, - 99,107,95,117,105,110,116,49,54,218,14,95,117,110,112,97, - 99,107,95,117,105,110,116,51,50,218,14,90,105,112,73,109, - 112,111,114,116,69,114,114,111,114,218,11,122,105,112,105,109, - 112,111,114,116,101,114,233,1,0,0,0,99,0,0,0,0, - 0,0,0,0,0,0,0,0,1,0,0,0,64,0,0,0, - 115,12,0,0,0,101,0,90,1,100,0,90,2,100,1,83, - 0,41,2,114,3,0,0,0,78,41,3,218,8,95,95,110, - 97,109,101,95,95,218,10,95,95,109,111,100,117,108,101,95, - 95,218,12,95,95,113,117,97,108,110,97,109,101,95,95,169, - 0,114,9,0,0,0,114,9,0,0,0,250,18,60,102,114, - 111,122,101,110,32,122,105,112,105,109,112,111,114,116,62,114, - 3,0,0,0,33,0,0,0,115,2,0,0,0,8,1,99, - 0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0, - 64,0,0,0,115,108,0,0,0,101,0,90,1,100,0,90, - 2,100,1,90,3,100,2,100,3,132,0,90,4,100,25,100, - 5,100,6,132,1,90,5,100,26,100,7,100,8,132,1,90, - 6,100,9,100,10,132,0,90,7,100,11,100,12,132,0,90, - 8,100,13,100,14,132,0,90,9,100,15,100,16,132,0,90, - 10,100,17,100,18,132,0,90,11,100,19,100,20,132,0,90, - 12,100,21,100,22,132,0,90,13,100,23,100,24,132,0,90, - 14,100,4,83,0,41,27,114,4,0,0,0,97,255,1,0, - 0,122,105,112,105,109,112,111,114,116,101,114,40,97,114,99, - 104,105,118,101,112,97,116,104,41,32,45,62,32,122,105,112, - 105,109,112,111,114,116,101,114,32,111,98,106,101,99,116,10, - 10,32,32,32,32,67,114,101,97,116,101,32,97,32,110,101, - 119,32,122,105,112,105,109,112,111,114,116,101,114,32,105,110, - 115,116,97,110,99,101,46,32,39,97,114,99,104,105,118,101, - 112,97,116,104,39,32,109,117,115,116,32,98,101,32,97,32, - 112,97,116,104,32,116,111,10,32,32,32,32,97,32,122,105, - 112,102,105,108,101,44,32,111,114,32,116,111,32,97,32,115, - 112,101,99,105,102,105,99,32,112,97,116,104,32,105,110,115, - 105,100,101,32,97,32,122,105,112,102,105,108,101,46,32,70, - 111,114,32,101,120,97,109,112,108,101,44,32,105,116,32,99, - 97,110,32,98,101,10,32,32,32,32,39,47,116,109,112,47, - 109,121,105,109,112,111,114,116,46,122,105,112,39,44,32,111, - 114,32,39,47,116,109,112,47,109,121,105,109,112,111,114,116, - 46,122,105,112,47,109,121,100,105,114,101,99,116,111,114,121, - 39,44,32,105,102,32,109,121,100,105,114,101,99,116,111,114, - 121,32,105,115,32,97,10,32,32,32,32,118,97,108,105,100, - 32,100,105,114,101,99,116,111,114,121,32,105,110,115,105,100, - 101,32,116,104,101,32,97,114,99,104,105,118,101,46,10,10, - 32,32,32,32,39,90,105,112,73,109,112,111,114,116,69,114, - 114,111,114,32,105,115,32,114,97,105,115,101,100,32,105,102, - 32,39,97,114,99,104,105,118,101,112,97,116,104,39,32,100, - 111,101,115,110,39,116,32,112,111,105,110,116,32,116,111,32, - 97,32,118,97,108,105,100,32,90,105,112,10,32,32,32,32, - 97,114,99,104,105,118,101,46,10,10,32,32,32,32,84,104, - 101,32,39,97,114,99,104,105,118,101,39,32,97,116,116,114, - 105,98,117,116,101,32,111,102,32,122,105,112,105,109,112,111, - 114,116,101,114,32,111,98,106,101,99,116,115,32,99,111,110, - 116,97,105,110,115,32,116,104,101,32,110,97,109,101,32,111, - 102,32,116,104,101,10,32,32,32,32,122,105,112,102,105,108, - 101,32,116,97,114,103,101,116,101,100,46,10,32,32,32,32, - 99,2,0,0,0,0,0,0,0,8,0,0,0,9,0,0, - 0,67,0,0,0,115,36,1,0,0,116,0,124,1,116,1, - 131,2,115,28,100,1,100,0,108,2,125,2,124,2,160,3, - 124,1,161,1,125,1,124,1,115,44,116,4,100,2,124,1, - 100,3,141,2,130,1,116,5,114,60,124,1,160,6,116,5, - 116,7,161,2,125,1,103,0,125,3,122,14,116,8,160,9, - 124,1,161,1,125,4,87,0,110,72,4,0,116,10,116,11, - 102,2,107,10,114,150,1,0,1,0,1,0,116,8,160,12, - 124,1,161,1,92,2,125,5,125,6,124,5,124,1,107,2, - 114,132,116,4,100,4,124,1,100,3,141,2,130,1,124,5, - 125,1,124,3,160,13,124,6,161,1,1,0,89,0,113,64, - 88,0,124,4,106,14,100,5,64,0,100,6,107,3,114,182, - 116,4,100,4,124,1,100,3,141,2,130,1,113,182,113,64, - 122,12,116,15,124,1,25,0,125,7,87,0,110,36,4,0, - 116,16,107,10,114,230,1,0,1,0,1,0,116,17,124,1, - 131,1,125,7,124,7,116,15,124,1,60,0,89,0,110,2, - 88,0,124,7,124,0,95,18,124,1,124,0,95,19,116,8, - 106,20,124,3,100,0,100,0,100,7,133,3,25,0,142,0, - 124,0,95,21,124,0,106,21,144,1,114,32,124,0,4,0, - 106,21,116,7,55,0,2,0,95,21,100,0,83,0,41,8, - 78,114,0,0,0,0,122,21,97,114,99,104,105,118,101,32, - 112,97,116,104,32,105,115,32,101,109,112,116,121,41,1,218, - 4,112,97,116,104,122,14,110,111,116,32,97,32,90,105,112, - 32,102,105,108,101,105,0,240,0,0,105,0,128,0,0,233, - 255,255,255,255,41,22,218,10,105,115,105,110,115,116,97,110, - 99,101,218,3,115,116,114,218,2,111,115,90,8,102,115,100, - 101,99,111,100,101,114,3,0,0,0,218,12,97,108,116,95, - 112,97,116,104,95,115,101,112,218,7,114,101,112,108,97,99, - 101,218,8,112,97,116,104,95,115,101,112,218,19,95,98,111, - 111,116,115,116,114,97,112,95,101,120,116,101,114,110,97,108, - 90,10,95,112,97,116,104,95,115,116,97,116,218,7,79,83, - 69,114,114,111,114,218,10,86,97,108,117,101,69,114,114,111, - 114,90,11,95,112,97,116,104,95,115,112,108,105,116,218,6, - 97,112,112,101,110,100,90,7,115,116,95,109,111,100,101,218, - 20,95,122,105,112,95,100,105,114,101,99,116,111,114,121,95, - 99,97,99,104,101,218,8,75,101,121,69,114,114,111,114,218, - 15,95,114,101,97,100,95,100,105,114,101,99,116,111,114,121, - 218,6,95,102,105,108,101,115,218,7,97,114,99,104,105,118, - 101,218,10,95,112,97,116,104,95,106,111,105,110,218,6,112, - 114,101,102,105,120,41,8,218,4,115,101,108,102,114,11,0, - 0,0,114,15,0,0,0,114,29,0,0,0,90,2,115,116, - 90,7,100,105,114,110,97,109,101,90,8,98,97,115,101,110, - 97,109,101,218,5,102,105,108,101,115,114,9,0,0,0,114, - 9,0,0,0,114,10,0,0,0,218,8,95,95,105,110,105, - 116,95,95,60,0,0,0,115,58,0,0,0,0,1,10,1, - 8,1,10,1,4,1,12,1,4,1,12,2,4,2,2,1, - 14,1,18,3,14,1,8,1,12,1,4,1,16,3,14,2, - 12,1,4,2,2,1,12,1,14,1,8,1,14,1,6,1, - 6,2,22,1,8,1,122,20,122,105,112,105,109,112,111,114, - 116,101,114,46,95,95,105,110,105,116,95,95,78,99,3,0, - 0,0,0,0,0,0,5,0,0,0,4,0,0,0,67,0, - 0,0,115,78,0,0,0,116,0,124,0,124,1,131,2,125, - 3,124,3,100,1,107,9,114,26,124,0,103,0,102,2,83, - 0,116,1,124,0,124,1,131,2,125,4,116,2,124,0,124, - 4,131,2,114,70,100,1,124,0,106,3,155,0,116,4,155, - 0,124,4,155,0,157,3,103,1,102,2,83,0,100,1,103, - 0,102,2,83,0,41,2,97,239,1,0,0,102,105,110,100, - 95,108,111,97,100,101,114,40,102,117,108,108,110,97,109,101, - 44,32,112,97,116,104,61,78,111,110,101,41,32,45,62,32, - 115,101,108,102,44,32,115,116,114,32,111,114,32,78,111,110, - 101,46,10,10,32,32,32,32,32,32,32,32,83,101,97,114, - 99,104,32,102,111,114,32,97,32,109,111,100,117,108,101,32, - 115,112,101,99,105,102,105,101,100,32,98,121,32,39,102,117, - 108,108,110,97,109,101,39,46,32,39,102,117,108,108,110,97, - 109,101,39,32,109,117,115,116,32,98,101,32,116,104,101,10, - 32,32,32,32,32,32,32,32,102,117,108,108,121,32,113,117, - 97,108,105,102,105,101,100,32,40,100,111,116,116,101,100,41, - 32,109,111,100,117,108,101,32,110,97,109,101,46,32,73,116, - 32,114,101,116,117,114,110,115,32,116,104,101,32,122,105,112, - 105,109,112,111,114,116,101,114,10,32,32,32,32,32,32,32, - 32,105,110,115,116,97,110,99,101,32,105,116,115,101,108,102, - 32,105,102,32,116,104,101,32,109,111,100,117,108,101,32,119, - 97,115,32,102,111,117,110,100,44,32,97,32,115,116,114,105, - 110,103,32,99,111,110,116,97,105,110,105,110,103,32,116,104, - 101,10,32,32,32,32,32,32,32,32,102,117,108,108,32,112, - 97,116,104,32,110,97,109,101,32,105,102,32,105,116,39,115, - 32,112,111,115,115,105,98,108,121,32,97,32,112,111,114,116, - 105,111,110,32,111,102,32,97,32,110,97,109,101,115,112,97, - 99,101,32,112,97,99,107,97,103,101,44,10,32,32,32,32, - 32,32,32,32,111,114,32,78,111,110,101,32,111,116,104,101, - 114,119,105,115,101,46,32,84,104,101,32,111,112,116,105,111, - 110,97,108,32,39,112,97,116,104,39,32,97,114,103,117,109, - 101,110,116,32,105,115,32,105,103,110,111,114,101,100,32,45, - 45,32,105,116,39,115,10,32,32,32,32,32,32,32,32,116, - 104,101,114,101,32,102,111,114,32,99,111,109,112,97,116,105, - 98,105,108,105,116,121,32,119,105,116,104,32,116,104,101,32, - 105,109,112,111,114,116,101,114,32,112,114,111,116,111,99,111, - 108,46,10,32,32,32,32,32,32,32,32,78,41,5,218,16, - 95,103,101,116,95,109,111,100,117,108,101,95,105,110,102,111, - 218,16,95,103,101,116,95,109,111,100,117,108,101,95,112,97, - 116,104,218,7,95,105,115,95,100,105,114,114,27,0,0,0, - 114,18,0,0,0,41,5,114,30,0,0,0,218,8,102,117, - 108,108,110,97,109,101,114,11,0,0,0,218,2,109,105,218, - 7,109,111,100,112,97,116,104,114,9,0,0,0,114,9,0, - 0,0,114,10,0,0,0,218,11,102,105,110,100,95,108,111, - 97,100,101,114,106,0,0,0,115,14,0,0,0,0,10,10, - 1,8,2,8,7,10,1,10,4,24,2,122,23,122,105,112, - 105,109,112,111,114,116,101,114,46,102,105,110,100,95,108,111, - 97,100,101,114,99,3,0,0,0,0,0,0,0,3,0,0, - 0,4,0,0,0,67,0,0,0,115,16,0,0,0,124,0, - 160,0,124,1,124,2,161,2,100,1,25,0,83,0,41,2, - 97,139,1,0,0,102,105,110,100,95,109,111,100,117,108,101, - 40,102,117,108,108,110,97,109,101,44,32,112,97,116,104,61, - 78,111,110,101,41,32,45,62,32,115,101,108,102,32,111,114, - 32,78,111,110,101,46,10,10,32,32,32,32,32,32,32,32, - 83,101,97,114,99,104,32,102,111,114,32,97,32,109,111,100, - 117,108,101,32,115,112,101,99,105,102,105,101,100,32,98,121, - 32,39,102,117,108,108,110,97,109,101,39,46,32,39,102,117, - 108,108,110,97,109,101,39,32,109,117,115,116,32,98,101,32, - 116,104,101,10,32,32,32,32,32,32,32,32,102,117,108,108, - 121,32,113,117,97,108,105,102,105,101,100,32,40,100,111,116, - 116,101,100,41,32,109,111,100,117,108,101,32,110,97,109,101, - 46,32,73,116,32,114,101,116,117,114,110,115,32,116,104,101, - 32,122,105,112,105,109,112,111,114,116,101,114,10,32,32,32, - 32,32,32,32,32,105,110,115,116,97,110,99,101,32,105,116, - 115,101,108,102,32,105,102,32,116,104,101,32,109,111,100,117, - 108,101,32,119,97,115,32,102,111,117,110,100,44,32,111,114, - 32,78,111,110,101,32,105,102,32,105,116,32,119,97,115,110, - 39,116,46,10,32,32,32,32,32,32,32,32,84,104,101,32, - 111,112,116,105,111,110,97,108,32,39,112,97,116,104,39,32, - 97,114,103,117,109,101,110,116,32,105,115,32,105,103,110,111, - 114,101,100,32,45,45,32,105,116,39,115,32,116,104,101,114, - 101,32,102,111,114,32,99,111,109,112,97,116,105,98,105,108, - 105,116,121,10,32,32,32,32,32,32,32,32,119,105,116,104, - 32,116,104,101,32,105,109,112,111,114,116,101,114,32,112,114, - 111,116,111,99,111,108,46,10,32,32,32,32,32,32,32,32, - 114,0,0,0,0,41,1,114,39,0,0,0,41,3,114,30, - 0,0,0,114,36,0,0,0,114,11,0,0,0,114,9,0, - 0,0,114,9,0,0,0,114,10,0,0,0,218,11,102,105, - 110,100,95,109,111,100,117,108,101,138,0,0,0,115,2,0, - 0,0,0,9,122,23,122,105,112,105,109,112,111,114,116,101, - 114,46,102,105,110,100,95,109,111,100,117,108,101,99,2,0, - 0,0,0,0,0,0,5,0,0,0,3,0,0,0,67,0, - 0,0,115,20,0,0,0,116,0,124,0,124,1,131,2,92, - 3,125,2,125,3,125,4,124,2,83,0,41,1,122,163,103, - 101,116,95,99,111,100,101,40,102,117,108,108,110,97,109,101, - 41,32,45,62,32,99,111,100,101,32,111,98,106,101,99,116, - 46,10,10,32,32,32,32,32,32,32,32,82,101,116,117,114, - 110,32,116,104,101,32,99,111,100,101,32,111,98,106,101,99, - 116,32,102,111,114,32,116,104,101,32,115,112,101,99,105,102, - 105,101,100,32,109,111,100,117,108,101,46,32,82,97,105,115, - 101,32,90,105,112,73,109,112,111,114,116,69,114,114,111,114, - 10,32,32,32,32,32,32,32,32,105,102,32,116,104,101,32, - 109,111,100,117,108,101,32,99,111,117,108,100,110,39,116,32, - 98,101,32,102,111,117,110,100,46,10,32,32,32,32,32,32, - 32,32,41,1,218,16,95,103,101,116,95,109,111,100,117,108, - 101,95,99,111,100,101,41,5,114,30,0,0,0,114,36,0, - 0,0,218,4,99,111,100,101,218,9,105,115,112,97,99,107, - 97,103,101,114,38,0,0,0,114,9,0,0,0,114,9,0, - 0,0,114,10,0,0,0,218,8,103,101,116,95,99,111,100, - 101,150,0,0,0,115,4,0,0,0,0,6,16,1,122,20, - 122,105,112,105,109,112,111,114,116,101,114,46,103,101,116,95, - 99,111,100,101,99,2,0,0,0,0,0,0,0,4,0,0, - 0,8,0,0,0,67,0,0,0,115,118,0,0,0,116,0, - 114,16,124,1,160,1,116,0,116,2,161,2,125,1,124,1, - 125,2,124,1,160,3,124,0,106,4,116,2,23,0,161,1, - 114,58,124,1,116,5,124,0,106,4,116,2,23,0,131,1, - 100,1,133,2,25,0,125,2,122,14,124,0,106,6,124,2, - 25,0,125,3,87,0,110,32,4,0,116,7,107,10,114,104, - 1,0,1,0,1,0,116,8,100,2,100,3,124,2,131,3, - 130,1,89,0,110,2,88,0,116,9,124,0,106,4,124,3, - 131,2,83,0,41,4,122,154,103,101,116,95,100,97,116,97, - 40,112,97,116,104,110,97,109,101,41,32,45,62,32,115,116, - 114,105,110,103,32,119,105,116,104,32,102,105,108,101,32,100, - 97,116,97,46,10,10,32,32,32,32,32,32,32,32,82,101, - 116,117,114,110,32,116,104,101,32,100,97,116,97,32,97,115, - 115,111,99,105,97,116,101,100,32,119,105,116,104,32,39,112, - 97,116,104,110,97,109,101,39,46,32,82,97,105,115,101,32, - 79,83,69,114,114,111,114,32,105,102,10,32,32,32,32,32, - 32,32,32,116,104,101,32,102,105,108,101,32,119,97,115,110, - 39,116,32,102,111,117,110,100,46,10,32,32,32,32,32,32, - 32,32,78,114,0,0,0,0,218,0,41,10,114,16,0,0, - 0,114,17,0,0,0,114,18,0,0,0,218,10,115,116,97, - 114,116,115,119,105,116,104,114,27,0,0,0,218,3,108,101, - 110,114,26,0,0,0,114,24,0,0,0,114,20,0,0,0, - 218,9,95,103,101,116,95,100,97,116,97,41,4,114,30,0, - 0,0,218,8,112,97,116,104,110,97,109,101,90,3,107,101, - 121,218,9,116,111,99,95,101,110,116,114,121,114,9,0,0, - 0,114,9,0,0,0,114,10,0,0,0,218,8,103,101,116, - 95,100,97,116,97,160,0,0,0,115,20,0,0,0,0,6, - 4,1,12,2,4,1,16,1,22,2,2,1,14,1,14,1, - 18,1,122,20,122,105,112,105,109,112,111,114,116,101,114,46, - 103,101,116,95,100,97,116,97,99,2,0,0,0,0,0,0, - 0,5,0,0,0,3,0,0,0,67,0,0,0,115,20,0, - 0,0,116,0,124,0,124,1,131,2,92,3,125,2,125,3, - 125,4,124,4,83,0,41,1,122,106,103,101,116,95,102,105, - 108,101,110,97,109,101,40,102,117,108,108,110,97,109,101,41, - 32,45,62,32,102,105,108,101,110,97,109,101,32,115,116,114, - 105,110,103,46,10,10,32,32,32,32,32,32,32,32,82,101, - 116,117,114,110,32,116,104,101,32,102,105,108,101,110,97,109, - 101,32,102,111,114,32,116,104,101,32,115,112,101,99,105,102, - 105,101,100,32,109,111,100,117,108,101,46,10,32,32,32,32, - 32,32,32,32,41,1,114,41,0,0,0,41,5,114,30,0, - 0,0,114,36,0,0,0,114,42,0,0,0,114,43,0,0, - 0,114,38,0,0,0,114,9,0,0,0,114,9,0,0,0, - 114,10,0,0,0,218,12,103,101,116,95,102,105,108,101,110, - 97,109,101,181,0,0,0,115,4,0,0,0,0,7,16,1, - 122,24,122,105,112,105,109,112,111,114,116,101,114,46,103,101, - 116,95,102,105,108,101,110,97,109,101,99,2,0,0,0,0, - 0,0,0,6,0,0,0,8,0,0,0,67,0,0,0,115, - 128,0,0,0,116,0,124,0,124,1,131,2,125,2,124,2, - 100,1,107,8,114,36,116,1,100,2,124,1,155,2,157,2, - 124,1,100,3,141,2,130,1,116,2,124,0,124,1,131,2, - 125,3,124,2,114,64,116,3,160,4,124,3,100,4,161,2, - 125,4,110,10,124,3,155,0,100,5,157,2,125,4,122,14, - 124,0,106,5,124,4,25,0,125,5,87,0,110,22,4,0, - 116,6,107,10,114,110,1,0,1,0,1,0,89,0,100,1, - 83,0,88,0,116,7,124,0,106,8,124,5,131,2,160,9, - 161,0,83,0,41,6,122,253,103,101,116,95,115,111,117,114, - 99,101,40,102,117,108,108,110,97,109,101,41,32,45,62,32, - 115,111,117,114,99,101,32,115,116,114,105,110,103,46,10,10, - 32,32,32,32,32,32,32,32,82,101,116,117,114,110,32,116, - 104,101,32,115,111,117,114,99,101,32,99,111,100,101,32,102, - 111,114,32,116,104,101,32,115,112,101,99,105,102,105,101,100, - 32,109,111,100,117,108,101,46,32,82,97,105,115,101,32,90, - 105,112,73,109,112,111,114,116,69,114,114,111,114,10,32,32, - 32,32,32,32,32,32,105,102,32,116,104,101,32,109,111,100, - 117,108,101,32,99,111,117,108,100,110,39,116,32,98,101,32, - 102,111,117,110,100,44,32,114,101,116,117,114,110,32,78,111, - 110,101,32,105,102,32,116,104,101,32,97,114,99,104,105,118, - 101,32,100,111,101,115,10,32,32,32,32,32,32,32,32,99, - 111,110,116,97,105,110,32,116,104,101,32,109,111,100,117,108, - 101,44,32,98,117,116,32,104,97,115,32,110,111,32,115,111, - 117,114,99,101,32,102,111,114,32,105,116,46,10,32,32,32, - 32,32,32,32,32,78,122,18,99,97,110,39,116,32,102,105, - 110,100,32,109,111,100,117,108,101,32,41,1,218,4,110,97, - 109,101,122,11,95,95,105,110,105,116,95,95,46,112,121,122, - 3,46,112,121,41,10,114,33,0,0,0,114,3,0,0,0, - 114,34,0,0,0,114,19,0,0,0,114,28,0,0,0,114, - 26,0,0,0,114,24,0,0,0,114,48,0,0,0,114,27, - 0,0,0,218,6,100,101,99,111,100,101,41,6,114,30,0, - 0,0,114,36,0,0,0,114,37,0,0,0,114,11,0,0, - 0,218,8,102,117,108,108,112,97,116,104,114,50,0,0,0, - 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,218, - 10,103,101,116,95,115,111,117,114,99,101,192,0,0,0,115, - 24,0,0,0,0,7,10,1,8,1,18,2,10,1,4,1, - 14,2,10,2,2,1,14,1,14,2,8,1,122,22,122,105, - 112,105,109,112,111,114,116,101,114,46,103,101,116,95,115,111, - 117,114,99,101,99,2,0,0,0,0,0,0,0,3,0,0, - 0,4,0,0,0,67,0,0,0,115,40,0,0,0,116,0, - 124,0,124,1,131,2,125,2,124,2,100,1,107,8,114,36, - 116,1,100,2,124,1,155,2,157,2,124,1,100,3,141,2, - 130,1,124,2,83,0,41,4,122,171,105,115,95,112,97,99, - 107,97,103,101,40,102,117,108,108,110,97,109,101,41,32,45, - 62,32,98,111,111,108,46,10,10,32,32,32,32,32,32,32, - 32,82,101,116,117,114,110,32,84,114,117,101,32,105,102,32, - 116,104,101,32,109,111,100,117,108,101,32,115,112,101,99,105, - 102,105,101,100,32,98,121,32,102,117,108,108,110,97,109,101, - 32,105,115,32,97,32,112,97,99,107,97,103,101,46,10,32, - 32,32,32,32,32,32,32,82,97,105,115,101,32,90,105,112, - 73,109,112,111,114,116,69,114,114,111,114,32,105,102,32,116, - 104,101,32,109,111,100,117,108,101,32,99,111,117,108,100,110, - 39,116,32,98,101,32,102,111,117,110,100,46,10,32,32,32, - 32,32,32,32,32,78,122,18,99,97,110,39,116,32,102,105, - 110,100,32,109,111,100,117,108,101,32,41,1,114,53,0,0, - 0,41,2,114,33,0,0,0,114,3,0,0,0,41,3,114, - 30,0,0,0,114,36,0,0,0,114,37,0,0,0,114,9, - 0,0,0,114,9,0,0,0,114,10,0,0,0,218,10,105, - 115,95,112,97,99,107,97,103,101,218,0,0,0,115,8,0, - 0,0,0,6,10,1,8,1,18,1,122,22,122,105,112,105, - 109,112,111,114,116,101,114,46,105,115,95,112,97,99,107,97, - 103,101,99,2,0,0,0,0,0,0,0,8,0,0,0,8, - 0,0,0,67,0,0,0,115,248,0,0,0,116,0,124,0, - 124,1,131,2,92,3,125,2,125,3,125,4,116,1,106,2, - 160,3,124,1,161,1,125,5,124,5,100,1,107,8,115,46, - 116,4,124,5,116,5,131,2,115,64,116,5,124,1,131,1, - 125,5,124,5,116,1,106,2,124,1,60,0,124,0,124,5, - 95,6,122,84,124,3,114,108,116,7,124,0,124,1,131,2, - 125,6,116,8,160,9,124,0,106,10,124,6,161,2,125,7, - 124,7,103,1,124,5,95,11,116,12,124,5,100,2,131,2, - 115,124,116,13,124,5,95,13,116,8,160,14,124,5,106,15, - 124,1,124,4,161,3,1,0,116,16,124,2,124,5,106,15, - 131,2,1,0,87,0,110,22,1,0,1,0,1,0,116,1, - 106,2,124,1,61,0,130,0,89,0,110,2,88,0,122,14, - 116,1,106,2,124,1,25,0,125,5,87,0,110,36,4,0, - 116,17,107,10,114,228,1,0,1,0,1,0,116,18,100,3, - 124,1,155,2,100,4,157,3,131,1,130,1,89,0,110,2, - 88,0,116,19,160,20,100,5,124,1,124,4,161,3,1,0, - 124,5,83,0,41,6,122,245,108,111,97,100,95,109,111,100, - 117,108,101,40,102,117,108,108,110,97,109,101,41,32,45,62, - 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, - 32,32,76,111,97,100,32,116,104,101,32,109,111,100,117,108, + 99,104,101,218,8,75,101,121,69,114,114,111,114,218,15,95, + 114,101,97,100,95,100,105,114,101,99,116,111,114,121,218,6, + 95,102,105,108,101,115,218,7,97,114,99,104,105,118,101,218, + 10,95,112,97,116,104,95,106,111,105,110,218,6,112,114,101, + 102,105,120,41,8,218,4,115,101,108,102,114,11,0,0,0, + 114,15,0,0,0,114,29,0,0,0,90,2,115,116,90,7, + 100,105,114,110,97,109,101,90,8,98,97,115,101,110,97,109, + 101,218,5,102,105,108,101,115,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,218,8,95,95,105,110,105,116,95, + 95,60,0,0,0,115,58,0,0,0,0,1,10,1,8,1, + 10,1,4,1,12,1,4,1,12,2,4,2,2,1,14,1, + 18,3,14,1,8,1,12,1,4,1,16,3,14,2,12,1, + 4,2,2,1,12,1,14,1,8,1,14,1,6,1,6,2, + 22,1,8,1,122,20,122,105,112,105,109,112,111,114,116,101, + 114,46,95,95,105,110,105,116,95,95,78,99,3,0,0,0, + 0,0,0,0,5,0,0,0,4,0,0,0,67,0,0,0, + 115,78,0,0,0,116,0,124,0,124,1,131,2,125,3,124, + 3,100,1,107,9,114,26,124,0,103,0,102,2,83,0,116, + 1,124,0,124,1,131,2,125,4,116,2,124,0,124,4,131, + 2,114,70,100,1,124,0,106,3,155,0,116,4,155,0,124, + 4,155,0,157,3,103,1,102,2,83,0,100,1,103,0,102, + 2,83,0,41,2,97,239,1,0,0,102,105,110,100,95,108, + 111,97,100,101,114,40,102,117,108,108,110,97,109,101,44,32, + 112,97,116,104,61,78,111,110,101,41,32,45,62,32,115,101, + 108,102,44,32,115,116,114,32,111,114,32,78,111,110,101,46, + 10,10,32,32,32,32,32,32,32,32,83,101,97,114,99,104, + 32,102,111,114,32,97,32,109,111,100,117,108,101,32,115,112, + 101,99,105,102,105,101,100,32,98,121,32,39,102,117,108,108, + 110,97,109,101,39,46,32,39,102,117,108,108,110,97,109,101, + 39,32,109,117,115,116,32,98,101,32,116,104,101,10,32,32, + 32,32,32,32,32,32,102,117,108,108,121,32,113,117,97,108, + 105,102,105,101,100,32,40,100,111,116,116,101,100,41,32,109, + 111,100,117,108,101,32,110,97,109,101,46,32,73,116,32,114, + 101,116,117,114,110,115,32,116,104,101,32,122,105,112,105,109, + 112,111,114,116,101,114,10,32,32,32,32,32,32,32,32,105, + 110,115,116,97,110,99,101,32,105,116,115,101,108,102,32,105, + 102,32,116,104,101,32,109,111,100,117,108,101,32,119,97,115, + 32,102,111,117,110,100,44,32,97,32,115,116,114,105,110,103, + 32,99,111,110,116,97,105,110,105,110,103,32,116,104,101,10, + 32,32,32,32,32,32,32,32,102,117,108,108,32,112,97,116, + 104,32,110,97,109,101,32,105,102,32,105,116,39,115,32,112, + 111,115,115,105,98,108,121,32,97,32,112,111,114,116,105,111, + 110,32,111,102,32,97,32,110,97,109,101,115,112,97,99,101, + 32,112,97,99,107,97,103,101,44,10,32,32,32,32,32,32, + 32,32,111,114,32,78,111,110,101,32,111,116,104,101,114,119, + 105,115,101,46,32,84,104,101,32,111,112,116,105,111,110,97, + 108,32,39,112,97,116,104,39,32,97,114,103,117,109,101,110, + 116,32,105,115,32,105,103,110,111,114,101,100,32,45,45,32, + 105,116,39,115,10,32,32,32,32,32,32,32,32,116,104,101, + 114,101,32,102,111,114,32,99,111,109,112,97,116,105,98,105, + 108,105,116,121,32,119,105,116,104,32,116,104,101,32,105,109, + 112,111,114,116,101,114,32,112,114,111,116,111,99,111,108,46, + 10,32,32,32,32,32,32,32,32,78,41,5,218,16,95,103, + 101,116,95,109,111,100,117,108,101,95,105,110,102,111,218,16, + 95,103,101,116,95,109,111,100,117,108,101,95,112,97,116,104, + 218,7,95,105,115,95,100,105,114,114,27,0,0,0,114,18, + 0,0,0,41,5,114,30,0,0,0,218,8,102,117,108,108, + 110,97,109,101,114,11,0,0,0,218,2,109,105,218,7,109, + 111,100,112,97,116,104,114,9,0,0,0,114,9,0,0,0, + 114,10,0,0,0,218,11,102,105,110,100,95,108,111,97,100, + 101,114,106,0,0,0,115,14,0,0,0,0,10,10,1,8, + 2,8,7,10,1,10,4,24,2,122,23,122,105,112,105,109, + 112,111,114,116,101,114,46,102,105,110,100,95,108,111,97,100, + 101,114,99,3,0,0,0,0,0,0,0,3,0,0,0,4, + 0,0,0,67,0,0,0,115,16,0,0,0,124,0,160,0, + 124,1,124,2,161,2,100,1,25,0,83,0,41,2,97,139, + 1,0,0,102,105,110,100,95,109,111,100,117,108,101,40,102, + 117,108,108,110,97,109,101,44,32,112,97,116,104,61,78,111, + 110,101,41,32,45,62,32,115,101,108,102,32,111,114,32,78, + 111,110,101,46,10,10,32,32,32,32,32,32,32,32,83,101, + 97,114,99,104,32,102,111,114,32,97,32,109,111,100,117,108, 101,32,115,112,101,99,105,102,105,101,100,32,98,121,32,39, 102,117,108,108,110,97,109,101,39,46,32,39,102,117,108,108, 110,97,109,101,39,32,109,117,115,116,32,98,101,32,116,104, 101,10,32,32,32,32,32,32,32,32,102,117,108,108,121,32, 113,117,97,108,105,102,105,101,100,32,40,100,111,116,116,101, 100,41,32,109,111,100,117,108,101,32,110,97,109,101,46,32, - 73,116,32,114,101,116,117,114,110,115,32,116,104,101,32,105, - 109,112,111,114,116,101,100,10,32,32,32,32,32,32,32,32, - 109,111,100,117,108,101,44,32,111,114,32,114,97,105,115,101, - 115,32,90,105,112,73,109,112,111,114,116,69,114,114,111,114, - 32,105,102,32,105,116,32,119,97,115,110,39,116,32,102,111, - 117,110,100,46,10,32,32,32,32,32,32,32,32,78,218,12, - 95,95,98,117,105,108,116,105,110,115,95,95,122,14,76,111, - 97,100,101,100,32,109,111,100,117,108,101,32,122,25,32,110, - 111,116,32,102,111,117,110,100,32,105,110,32,115,121,115,46, - 109,111,100,117,108,101,115,122,30,105,109,112,111,114,116,32, - 123,125,32,35,32,108,111,97,100,101,100,32,102,114,111,109, - 32,90,105,112,32,123,125,41,21,114,41,0,0,0,218,3, - 115,121,115,218,7,109,111,100,117,108,101,115,218,3,103,101, - 116,114,13,0,0,0,218,12,95,109,111,100,117,108,101,95, - 116,121,112,101,218,10,95,95,108,111,97,100,101,114,95,95, - 114,34,0,0,0,114,19,0,0,0,114,28,0,0,0,114, - 27,0,0,0,90,8,95,95,112,97,116,104,95,95,218,7, - 104,97,115,97,116,116,114,114,58,0,0,0,90,14,95,102, - 105,120,95,117,112,95,109,111,100,117,108,101,218,8,95,95, - 100,105,99,116,95,95,218,4,101,120,101,99,114,24,0,0, - 0,218,11,73,109,112,111,114,116,69,114,114,111,114,218,10, - 95,98,111,111,116,115,116,114,97,112,218,16,95,118,101,114, - 98,111,115,101,95,109,101,115,115,97,103,101,41,8,114,30, - 0,0,0,114,36,0,0,0,114,42,0,0,0,114,43,0, - 0,0,114,38,0,0,0,90,3,109,111,100,114,11,0,0, - 0,114,55,0,0,0,114,9,0,0,0,114,9,0,0,0, - 114,10,0,0,0,218,11,108,111,97,100,95,109,111,100,117, - 108,101,231,0,0,0,115,48,0,0,0,0,7,16,1,12, - 1,18,1,8,1,10,1,6,2,2,1,4,3,10,1,14, - 1,8,2,10,1,6,1,16,1,16,1,6,1,8,1,8, - 2,2,1,14,1,14,1,22,1,14,1,122,23,122,105,112, - 105,109,112,111,114,116,101,114,46,108,111,97,100,95,109,111, - 100,117,108,101,99,2,0,0,0,0,0,0,0,3,0,0, - 0,4,0,0,0,67,0,0,0,115,24,0,0,0,100,1, - 100,2,108,0,109,1,125,2,1,0,124,2,160,2,124,0, - 124,1,161,2,83,0,41,3,122,204,82,101,116,117,114,110, - 32,116,104,101,32,82,101,115,111,117,114,99,101,82,101,97, - 100,101,114,32,102,111,114,32,97,32,112,97,99,107,97,103, - 101,32,105,110,32,97,32,122,105,112,32,102,105,108,101,46, - 10,10,32,32,32,32,32,32,32,32,73,102,32,39,102,117, - 108,108,110,97,109,101,39,32,105,115,32,97,32,112,97,99, - 107,97,103,101,32,119,105,116,104,105,110,32,116,104,101,32, - 122,105,112,32,102,105,108,101,44,32,114,101,116,117,114,110, - 32,116,104,101,10,32,32,32,32,32,32,32,32,39,82,101, - 115,111,117,114,99,101,82,101,97,100,101,114,39,32,111,98, - 106,101,99,116,32,102,111,114,32,116,104,101,32,112,97,99, - 107,97,103,101,46,32,32,79,116,104,101,114,119,105,115,101, - 32,114,101,116,117,114,110,32,78,111,110,101,46,10,32,32, - 32,32,32,32,32,32,114,0,0,0,0,41,1,218,9,114, - 101,115,111,117,114,99,101,115,41,3,90,9,105,109,112,111, - 114,116,108,105,98,114,71,0,0,0,90,30,95,122,105,112, - 105,109,112,111,114,116,95,103,101,116,95,114,101,115,111,117, - 114,99,101,95,114,101,97,100,101,114,41,3,114,30,0,0, - 0,114,36,0,0,0,114,71,0,0,0,114,9,0,0,0, - 114,9,0,0,0,114,10,0,0,0,218,19,103,101,116,95, - 114,101,115,111,117,114,99,101,95,114,101,97,100,101,114,13, - 1,0,0,115,4,0,0,0,0,6,12,1,122,31,122,105, - 112,105,109,112,111,114,116,101,114,46,103,101,116,95,114,101, - 115,111,117,114,99,101,95,114,101,97,100,101,114,99,1,0, - 0,0,0,0,0,0,1,0,0,0,5,0,0,0,67,0, - 0,0,115,24,0,0,0,100,1,124,0,106,0,155,0,116, - 1,155,0,124,0,106,2,155,0,100,2,157,5,83,0,41, - 3,78,122,21,60,122,105,112,105,109,112,111,114,116,101,114, - 32,111,98,106,101,99,116,32,34,122,2,34,62,41,3,114, - 27,0,0,0,114,18,0,0,0,114,29,0,0,0,41,1, - 114,30,0,0,0,114,9,0,0,0,114,9,0,0,0,114, - 10,0,0,0,218,8,95,95,114,101,112,114,95,95,23,1, - 0,0,115,2,0,0,0,0,1,122,20,122,105,112,105,109, - 112,111,114,116,101,114,46,95,95,114,101,112,114,95,95,41, - 1,78,41,1,78,41,15,114,6,0,0,0,114,7,0,0, - 0,114,8,0,0,0,218,7,95,95,100,111,99,95,95,114, - 32,0,0,0,114,39,0,0,0,114,40,0,0,0,114,44, - 0,0,0,114,51,0,0,0,114,52,0,0,0,114,56,0, - 0,0,114,57,0,0,0,114,70,0,0,0,114,72,0,0, - 0,114,73,0,0,0,114,9,0,0,0,114,9,0,0,0, - 114,9,0,0,0,114,10,0,0,0,114,4,0,0,0,42, - 0,0,0,115,24,0,0,0,8,13,4,5,8,46,10,32, - 10,12,8,10,8,21,8,11,8,26,8,13,8,38,8,10, - 122,12,95,95,105,110,105,116,95,95,46,112,121,99,84,122, - 11,95,95,105,110,105,116,95,95,46,112,121,70,41,3,122, - 4,46,112,121,99,84,70,41,3,122,3,46,112,121,70,70, - 99,2,0,0,0,0,0,0,0,2,0,0,0,4,0,0, - 0,67,0,0,0,115,20,0,0,0,124,0,106,0,124,1, - 160,1,100,1,161,1,100,2,25,0,23,0,83,0,41,3, - 78,218,1,46,233,2,0,0,0,41,2,114,29,0,0,0, - 218,10,114,112,97,114,116,105,116,105,111,110,41,2,114,30, - 0,0,0,114,36,0,0,0,114,9,0,0,0,114,9,0, - 0,0,114,10,0,0,0,114,34,0,0,0,41,1,0,0, - 115,2,0,0,0,0,1,114,34,0,0,0,99,2,0,0, - 0,0,0,0,0,3,0,0,0,2,0,0,0,67,0,0, - 0,115,18,0,0,0,124,1,116,0,23,0,125,2,124,2, - 124,0,106,1,107,6,83,0,41,1,78,41,2,114,18,0, - 0,0,114,26,0,0,0,41,3,114,30,0,0,0,114,11, - 0,0,0,90,7,100,105,114,112,97,116,104,114,9,0,0, - 0,114,9,0,0,0,114,10,0,0,0,114,35,0,0,0, - 45,1,0,0,115,4,0,0,0,0,4,8,2,114,35,0, - 0,0,99,2,0,0,0,0,0,0,0,7,0,0,0,4, - 0,0,0,67,0,0,0,115,56,0,0,0,116,0,124,0, - 124,1,131,2,125,2,116,1,68,0,93,36,92,3,125,3, - 125,4,125,5,124,2,124,3,23,0,125,6,124,6,124,0, - 106,2,107,6,114,14,124,5,2,0,1,0,83,0,113,14, - 100,0,83,0,41,1,78,41,3,114,34,0,0,0,218,16, - 95,122,105,112,95,115,101,97,114,99,104,111,114,100,101,114, - 114,26,0,0,0,41,7,114,30,0,0,0,114,36,0,0, - 0,114,11,0,0,0,218,6,115,117,102,102,105,120,218,10, - 105,115,98,121,116,101,99,111,100,101,114,43,0,0,0,114, + 73,116,32,114,101,116,117,114,110,115,32,116,104,101,32,122, + 105,112,105,109,112,111,114,116,101,114,10,32,32,32,32,32, + 32,32,32,105,110,115,116,97,110,99,101,32,105,116,115,101, + 108,102,32,105,102,32,116,104,101,32,109,111,100,117,108,101, + 32,119,97,115,32,102,111,117,110,100,44,32,111,114,32,78, + 111,110,101,32,105,102,32,105,116,32,119,97,115,110,39,116, + 46,10,32,32,32,32,32,32,32,32,84,104,101,32,111,112, + 116,105,111,110,97,108,32,39,112,97,116,104,39,32,97,114, + 103,117,109,101,110,116,32,105,115,32,105,103,110,111,114,101, + 100,32,45,45,32,105,116,39,115,32,116,104,101,114,101,32, + 102,111,114,32,99,111,109,112,97,116,105,98,105,108,105,116, + 121,10,32,32,32,32,32,32,32,32,119,105,116,104,32,116, + 104,101,32,105,109,112,111,114,116,101,114,32,112,114,111,116, + 111,99,111,108,46,10,32,32,32,32,32,32,32,32,114,0, + 0,0,0,41,1,114,39,0,0,0,41,3,114,30,0,0, + 0,114,36,0,0,0,114,11,0,0,0,114,9,0,0,0, + 114,9,0,0,0,114,10,0,0,0,218,11,102,105,110,100, + 95,109,111,100,117,108,101,138,0,0,0,115,2,0,0,0, + 0,9,122,23,122,105,112,105,109,112,111,114,116,101,114,46, + 102,105,110,100,95,109,111,100,117,108,101,99,2,0,0,0, + 0,0,0,0,5,0,0,0,3,0,0,0,67,0,0,0, + 115,20,0,0,0,116,0,124,0,124,1,131,2,92,3,125, + 2,125,3,125,4,124,2,83,0,41,1,122,163,103,101,116, + 95,99,111,100,101,40,102,117,108,108,110,97,109,101,41,32, + 45,62,32,99,111,100,101,32,111,98,106,101,99,116,46,10, + 10,32,32,32,32,32,32,32,32,82,101,116,117,114,110,32, + 116,104,101,32,99,111,100,101,32,111,98,106,101,99,116,32, + 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, + 100,32,109,111,100,117,108,101,46,32,82,97,105,115,101,32, + 90,105,112,73,109,112,111,114,116,69,114,114,111,114,10,32, + 32,32,32,32,32,32,32,105,102,32,116,104,101,32,109,111, + 100,117,108,101,32,99,111,117,108,100,110,39,116,32,98,101, + 32,102,111,117,110,100,46,10,32,32,32,32,32,32,32,32, + 41,1,218,16,95,103,101,116,95,109,111,100,117,108,101,95, + 99,111,100,101,41,5,114,30,0,0,0,114,36,0,0,0, + 218,4,99,111,100,101,218,9,105,115,112,97,99,107,97,103, + 101,114,38,0,0,0,114,9,0,0,0,114,9,0,0,0, + 114,10,0,0,0,218,8,103,101,116,95,99,111,100,101,150, + 0,0,0,115,4,0,0,0,0,6,16,1,122,20,122,105, + 112,105,109,112,111,114,116,101,114,46,103,101,116,95,99,111, + 100,101,99,2,0,0,0,0,0,0,0,4,0,0,0,8, + 0,0,0,67,0,0,0,115,118,0,0,0,116,0,114,16, + 124,1,160,1,116,0,116,2,161,2,125,1,124,1,125,2, + 124,1,160,3,124,0,106,4,116,2,23,0,161,1,114,58, + 124,1,116,5,124,0,106,4,116,2,23,0,131,1,100,1, + 133,2,25,0,125,2,122,14,124,0,106,6,124,2,25,0, + 125,3,87,0,110,32,4,0,116,7,107,10,114,104,1,0, + 1,0,1,0,116,8,100,2,100,3,124,2,131,3,130,1, + 89,0,110,2,88,0,116,9,124,0,106,4,124,3,131,2, + 83,0,41,4,122,154,103,101,116,95,100,97,116,97,40,112, + 97,116,104,110,97,109,101,41,32,45,62,32,115,116,114,105, + 110,103,32,119,105,116,104,32,102,105,108,101,32,100,97,116, + 97,46,10,10,32,32,32,32,32,32,32,32,82,101,116,117, + 114,110,32,116,104,101,32,100,97,116,97,32,97,115,115,111, + 99,105,97,116,101,100,32,119,105,116,104,32,39,112,97,116, + 104,110,97,109,101,39,46,32,82,97,105,115,101,32,79,83, + 69,114,114,111,114,32,105,102,10,32,32,32,32,32,32,32, + 32,116,104,101,32,102,105,108,101,32,119,97,115,110,39,116, + 32,102,111,117,110,100,46,10,32,32,32,32,32,32,32,32, + 78,114,0,0,0,0,218,0,41,10,114,16,0,0,0,114, + 17,0,0,0,114,18,0,0,0,218,10,115,116,97,114,116, + 115,119,105,116,104,114,27,0,0,0,218,3,108,101,110,114, + 26,0,0,0,114,24,0,0,0,114,20,0,0,0,218,9, + 95,103,101,116,95,100,97,116,97,41,4,114,30,0,0,0, + 218,8,112,97,116,104,110,97,109,101,90,3,107,101,121,218, + 9,116,111,99,95,101,110,116,114,121,114,9,0,0,0,114, + 9,0,0,0,114,10,0,0,0,218,8,103,101,116,95,100, + 97,116,97,160,0,0,0,115,20,0,0,0,0,6,4,1, + 12,2,4,1,16,1,22,2,2,1,14,1,14,1,18,1, + 122,20,122,105,112,105,109,112,111,114,116,101,114,46,103,101, + 116,95,100,97,116,97,99,2,0,0,0,0,0,0,0,5, + 0,0,0,3,0,0,0,67,0,0,0,115,20,0,0,0, + 116,0,124,0,124,1,131,2,92,3,125,2,125,3,125,4, + 124,4,83,0,41,1,122,106,103,101,116,95,102,105,108,101, + 110,97,109,101,40,102,117,108,108,110,97,109,101,41,32,45, + 62,32,102,105,108,101,110,97,109,101,32,115,116,114,105,110, + 103,46,10,10,32,32,32,32,32,32,32,32,82,101,116,117, + 114,110,32,116,104,101,32,102,105,108,101,110,97,109,101,32, + 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, + 100,32,109,111,100,117,108,101,46,10,32,32,32,32,32,32, + 32,32,41,1,114,41,0,0,0,41,5,114,30,0,0,0, + 114,36,0,0,0,114,42,0,0,0,114,43,0,0,0,114, + 38,0,0,0,114,9,0,0,0,114,9,0,0,0,114,10, + 0,0,0,218,12,103,101,116,95,102,105,108,101,110,97,109, + 101,181,0,0,0,115,4,0,0,0,0,7,16,1,122,24, + 122,105,112,105,109,112,111,114,116,101,114,46,103,101,116,95, + 102,105,108,101,110,97,109,101,99,2,0,0,0,0,0,0, + 0,6,0,0,0,8,0,0,0,67,0,0,0,115,128,0, + 0,0,116,0,124,0,124,1,131,2,125,2,124,2,100,1, + 107,8,114,36,116,1,100,2,124,1,155,2,157,2,124,1, + 100,3,141,2,130,1,116,2,124,0,124,1,131,2,125,3, + 124,2,114,64,116,3,160,4,124,3,100,4,161,2,125,4, + 110,10,124,3,155,0,100,5,157,2,125,4,122,14,124,0, + 106,5,124,4,25,0,125,5,87,0,110,22,4,0,116,6, + 107,10,114,110,1,0,1,0,1,0,89,0,100,1,83,0, + 88,0,116,7,124,0,106,8,124,5,131,2,160,9,161,0, + 83,0,41,6,122,253,103,101,116,95,115,111,117,114,99,101, + 40,102,117,108,108,110,97,109,101,41,32,45,62,32,115,111, + 117,114,99,101,32,115,116,114,105,110,103,46,10,10,32,32, + 32,32,32,32,32,32,82,101,116,117,114,110,32,116,104,101, + 32,115,111,117,114,99,101,32,99,111,100,101,32,102,111,114, + 32,116,104,101,32,115,112,101,99,105,102,105,101,100,32,109, + 111,100,117,108,101,46,32,82,97,105,115,101,32,90,105,112, + 73,109,112,111,114,116,69,114,114,111,114,10,32,32,32,32, + 32,32,32,32,105,102,32,116,104,101,32,109,111,100,117,108, + 101,32,99,111,117,108,100,110,39,116,32,98,101,32,102,111, + 117,110,100,44,32,114,101,116,117,114,110,32,78,111,110,101, + 32,105,102,32,116,104,101,32,97,114,99,104,105,118,101,32, + 100,111,101,115,10,32,32,32,32,32,32,32,32,99,111,110, + 116,97,105,110,32,116,104,101,32,109,111,100,117,108,101,44, + 32,98,117,116,32,104,97,115,32,110,111,32,115,111,117,114, + 99,101,32,102,111,114,32,105,116,46,10,32,32,32,32,32, + 32,32,32,78,122,18,99,97,110,39,116,32,102,105,110,100, + 32,109,111,100,117,108,101,32,41,1,218,4,110,97,109,101, + 122,11,95,95,105,110,105,116,95,95,46,112,121,122,3,46, + 112,121,41,10,114,33,0,0,0,114,3,0,0,0,114,34, + 0,0,0,114,19,0,0,0,114,28,0,0,0,114,26,0, + 0,0,114,24,0,0,0,114,48,0,0,0,114,27,0,0, + 0,218,6,100,101,99,111,100,101,41,6,114,30,0,0,0, + 114,36,0,0,0,114,37,0,0,0,114,11,0,0,0,218, + 8,102,117,108,108,112,97,116,104,114,50,0,0,0,114,9, + 0,0,0,114,9,0,0,0,114,10,0,0,0,218,10,103, + 101,116,95,115,111,117,114,99,101,192,0,0,0,115,24,0, + 0,0,0,7,10,1,8,1,18,2,10,1,4,1,14,2, + 10,2,2,1,14,1,14,2,8,1,122,22,122,105,112,105, + 109,112,111,114,116,101,114,46,103,101,116,95,115,111,117,114, + 99,101,99,2,0,0,0,0,0,0,0,3,0,0,0,4, + 0,0,0,67,0,0,0,115,40,0,0,0,116,0,124,0, + 124,1,131,2,125,2,124,2,100,1,107,8,114,36,116,1, + 100,2,124,1,155,2,157,2,124,1,100,3,141,2,130,1, + 124,2,83,0,41,4,122,171,105,115,95,112,97,99,107,97, + 103,101,40,102,117,108,108,110,97,109,101,41,32,45,62,32, + 98,111,111,108,46,10,10,32,32,32,32,32,32,32,32,82, + 101,116,117,114,110,32,84,114,117,101,32,105,102,32,116,104, + 101,32,109,111,100,117,108,101,32,115,112,101,99,105,102,105, + 101,100,32,98,121,32,102,117,108,108,110,97,109,101,32,105, + 115,32,97,32,112,97,99,107,97,103,101,46,10,32,32,32, + 32,32,32,32,32,82,97,105,115,101,32,90,105,112,73,109, + 112,111,114,116,69,114,114,111,114,32,105,102,32,116,104,101, + 32,109,111,100,117,108,101,32,99,111,117,108,100,110,39,116, + 32,98,101,32,102,111,117,110,100,46,10,32,32,32,32,32, + 32,32,32,78,122,18,99,97,110,39,116,32,102,105,110,100, + 32,109,111,100,117,108,101,32,41,1,114,53,0,0,0,41, + 2,114,33,0,0,0,114,3,0,0,0,41,3,114,30,0, + 0,0,114,36,0,0,0,114,37,0,0,0,114,9,0,0, + 0,114,9,0,0,0,114,10,0,0,0,218,10,105,115,95, + 112,97,99,107,97,103,101,218,0,0,0,115,8,0,0,0, + 0,6,10,1,8,1,18,1,122,22,122,105,112,105,109,112, + 111,114,116,101,114,46,105,115,95,112,97,99,107,97,103,101, + 99,2,0,0,0,0,0,0,0,8,0,0,0,8,0,0, + 0,67,0,0,0,115,248,0,0,0,116,0,124,0,124,1, + 131,2,92,3,125,2,125,3,125,4,116,1,106,2,160,3, + 124,1,161,1,125,5,124,5,100,1,107,8,115,46,116,4, + 124,5,116,5,131,2,115,64,116,5,124,1,131,1,125,5, + 124,5,116,1,106,2,124,1,60,0,124,0,124,5,95,6, + 122,84,124,3,114,108,116,7,124,0,124,1,131,2,125,6, + 116,8,160,9,124,0,106,10,124,6,161,2,125,7,124,7, + 103,1,124,5,95,11,116,12,124,5,100,2,131,2,115,124, + 116,13,124,5,95,13,116,8,160,14,124,5,106,15,124,1, + 124,4,161,3,1,0,116,16,124,2,124,5,106,15,131,2, + 1,0,87,0,110,22,1,0,1,0,1,0,116,1,106,2, + 124,1,61,0,130,0,89,0,110,2,88,0,122,14,116,1, + 106,2,124,1,25,0,125,5,87,0,110,36,4,0,116,17, + 107,10,114,228,1,0,1,0,1,0,116,18,100,3,124,1, + 155,2,100,4,157,3,131,1,130,1,89,0,110,2,88,0, + 116,19,160,20,100,5,124,1,124,4,161,3,1,0,124,5, + 83,0,41,6,122,245,108,111,97,100,95,109,111,100,117,108, + 101,40,102,117,108,108,110,97,109,101,41,32,45,62,32,109, + 111,100,117,108,101,46,10,10,32,32,32,32,32,32,32,32, + 76,111,97,100,32,116,104,101,32,109,111,100,117,108,101,32, + 115,112,101,99,105,102,105,101,100,32,98,121,32,39,102,117, + 108,108,110,97,109,101,39,46,32,39,102,117,108,108,110,97, + 109,101,39,32,109,117,115,116,32,98,101,32,116,104,101,10, + 32,32,32,32,32,32,32,32,102,117,108,108,121,32,113,117, + 97,108,105,102,105,101,100,32,40,100,111,116,116,101,100,41, + 32,109,111,100,117,108,101,32,110,97,109,101,46,32,73,116, + 32,114,101,116,117,114,110,115,32,116,104,101,32,105,109,112, + 111,114,116,101,100,10,32,32,32,32,32,32,32,32,109,111, + 100,117,108,101,44,32,111,114,32,114,97,105,115,101,115,32, + 90,105,112,73,109,112,111,114,116,69,114,114,111,114,32,105, + 102,32,105,116,32,119,97,115,110,39,116,32,102,111,117,110, + 100,46,10,32,32,32,32,32,32,32,32,78,218,12,95,95, + 98,117,105,108,116,105,110,115,95,95,122,14,76,111,97,100, + 101,100,32,109,111,100,117,108,101,32,122,25,32,110,111,116, + 32,102,111,117,110,100,32,105,110,32,115,121,115,46,109,111, + 100,117,108,101,115,122,30,105,109,112,111,114,116,32,123,125, + 32,35,32,108,111,97,100,101,100,32,102,114,111,109,32,90, + 105,112,32,123,125,41,21,114,41,0,0,0,218,3,115,121, + 115,218,7,109,111,100,117,108,101,115,218,3,103,101,116,114, + 13,0,0,0,218,12,95,109,111,100,117,108,101,95,116,121, + 112,101,218,10,95,95,108,111,97,100,101,114,95,95,114,34, + 0,0,0,114,19,0,0,0,114,28,0,0,0,114,27,0, + 0,0,90,8,95,95,112,97,116,104,95,95,218,7,104,97, + 115,97,116,116,114,114,58,0,0,0,90,14,95,102,105,120, + 95,117,112,95,109,111,100,117,108,101,218,8,95,95,100,105, + 99,116,95,95,218,4,101,120,101,99,114,24,0,0,0,218, + 11,73,109,112,111,114,116,69,114,114,111,114,218,10,95,98, + 111,111,116,115,116,114,97,112,218,16,95,118,101,114,98,111, + 115,101,95,109,101,115,115,97,103,101,41,8,114,30,0,0, + 0,114,36,0,0,0,114,42,0,0,0,114,43,0,0,0, + 114,38,0,0,0,90,3,109,111,100,114,11,0,0,0,114, 55,0,0,0,114,9,0,0,0,114,9,0,0,0,114,10, - 0,0,0,114,33,0,0,0,54,1,0,0,115,12,0,0, - 0,0,1,10,1,14,1,8,1,10,1,10,1,114,33,0, - 0,0,99,1,0,0,0,0,0,0,0,23,0,0,0,9, - 0,0,0,67,0,0,0,115,20,4,0,0,122,16,116,0, - 160,1,124,0,100,1,161,2,125,1,87,0,110,38,4,0, - 116,2,107,10,114,54,1,0,1,0,1,0,116,3,100,2, - 124,0,155,2,157,2,124,0,100,3,141,2,130,1,89,0, - 110,2,88,0,124,1,144,3,143,190,1,0,122,34,124,1, - 160,4,100,4,100,5,161,2,1,0,124,1,160,5,161,0, - 125,2,124,1,160,6,100,6,161,1,125,3,87,0,110,38, - 4,0,116,2,107,10,114,136,1,0,1,0,1,0,116,3, - 100,7,124,0,155,2,157,2,124,0,100,3,141,2,130,1, - 89,0,110,2,88,0,116,7,124,3,131,1,100,6,107,3, - 114,168,116,3,100,7,124,0,155,2,157,2,124,0,100,3, - 141,2,130,1,124,3,100,0,100,8,133,2,25,0,100,9, - 107,3,114,202,116,3,100,10,124,0,155,2,157,2,124,0, - 100,3,141,2,130,1,116,8,124,3,100,11,100,12,133,2, - 25,0,131,1,125,4,116,8,124,3,100,12,100,13,133,2, - 25,0,131,1,125,5,124,2,124,4,107,0,144,1,114,6, - 116,3,100,14,124,0,155,2,157,2,124,0,100,3,141,2, - 130,1,124,2,124,5,107,0,144,1,114,34,116,3,100,15, - 124,0,155,2,157,2,124,0,100,3,141,2,130,1,124,2, - 124,4,56,0,125,2,124,2,124,5,24,0,125,6,124,6, - 100,16,107,0,144,1,114,78,116,3,100,17,124,0,155,2, - 157,2,124,0,100,3,141,2,130,1,105,0,125,7,100,16, - 125,8,122,14,124,1,160,4,124,2,161,1,1,0,87,0, - 110,40,4,0,116,2,107,10,144,1,114,140,1,0,1,0, + 0,0,0,218,11,108,111,97,100,95,109,111,100,117,108,101, + 231,0,0,0,115,48,0,0,0,0,7,16,1,12,1,18, + 1,8,1,10,1,6,2,2,1,4,3,10,1,14,1,8, + 2,10,1,6,1,16,1,16,1,6,1,8,1,8,2,2, + 1,14,1,14,1,22,1,14,1,122,23,122,105,112,105,109, + 112,111,114,116,101,114,46,108,111,97,100,95,109,111,100,117, + 108,101,99,2,0,0,0,0,0,0,0,3,0,0,0,8, + 0,0,0,67,0,0,0,115,88,0,0,0,122,20,124,0, + 160,0,124,1,161,1,115,18,87,0,100,1,83,0,87,0, + 110,22,4,0,116,1,107,10,114,42,1,0,1,0,1,0, + 89,0,100,1,83,0,88,0,116,2,106,3,115,78,100,2, + 100,3,108,4,109,5,125,2,1,0,124,2,160,6,116,2, + 161,1,1,0,100,4,116,2,95,3,116,2,124,0,124,1, + 131,2,83,0,41,5,122,204,82,101,116,117,114,110,32,116, + 104,101,32,82,101,115,111,117,114,99,101,82,101,97,100,101, + 114,32,102,111,114,32,97,32,112,97,99,107,97,103,101,32, + 105,110,32,97,32,122,105,112,32,102,105,108,101,46,10,10, + 32,32,32,32,32,32,32,32,73,102,32,39,102,117,108,108, + 110,97,109,101,39,32,105,115,32,97,32,112,97,99,107,97, + 103,101,32,119,105,116,104,105,110,32,116,104,101,32,122,105, + 112,32,102,105,108,101,44,32,114,101,116,117,114,110,32,116, + 104,101,10,32,32,32,32,32,32,32,32,39,82,101,115,111, + 117,114,99,101,82,101,97,100,101,114,39,32,111,98,106,101, + 99,116,32,102,111,114,32,116,104,101,32,112,97,99,107,97, + 103,101,46,32,32,79,116,104,101,114,119,105,115,101,32,114, + 101,116,117,114,110,32,78,111,110,101,46,10,32,32,32,32, + 32,32,32,32,78,114,0,0,0,0,41,1,218,14,82,101, + 115,111,117,114,99,101,82,101,97,100,101,114,84,41,7,114, + 57,0,0,0,114,3,0,0,0,218,24,95,90,105,112,73, + 109,112,111,114,116,82,101,115,111,117,114,99,101,82,101,97, + 100,101,114,218,11,95,114,101,103,105,115,116,101,114,101,100, + 90,13,105,109,112,111,114,116,108,105,98,46,97,98,99,114, + 71,0,0,0,90,8,114,101,103,105,115,116,101,114,41,3, + 114,30,0,0,0,114,36,0,0,0,114,71,0,0,0,114, + 9,0,0,0,114,9,0,0,0,114,10,0,0,0,218,19, + 103,101,116,95,114,101,115,111,117,114,99,101,95,114,101,97, + 100,101,114,13,1,0,0,115,20,0,0,0,0,6,2,1, + 10,1,10,1,14,1,8,1,6,1,12,1,10,1,6,1, + 122,31,122,105,112,105,109,112,111,114,116,101,114,46,103,101, + 116,95,114,101,115,111,117,114,99,101,95,114,101,97,100,101, + 114,99,1,0,0,0,0,0,0,0,1,0,0,0,5,0, + 0,0,67,0,0,0,115,24,0,0,0,100,1,124,0,106, + 0,155,0,116,1,155,0,124,0,106,2,155,0,100,2,157, + 5,83,0,41,3,78,122,21,60,122,105,112,105,109,112,111, + 114,116,101,114,32,111,98,106,101,99,116,32,34,122,2,34, + 62,41,3,114,27,0,0,0,114,18,0,0,0,114,29,0, + 0,0,41,1,114,30,0,0,0,114,9,0,0,0,114,9, + 0,0,0,114,10,0,0,0,218,8,95,95,114,101,112,114, + 95,95,31,1,0,0,115,2,0,0,0,0,1,122,20,122, + 105,112,105,109,112,111,114,116,101,114,46,95,95,114,101,112, + 114,95,95,41,1,78,41,1,78,41,15,114,6,0,0,0, + 114,7,0,0,0,114,8,0,0,0,218,7,95,95,100,111, + 99,95,95,114,32,0,0,0,114,39,0,0,0,114,40,0, + 0,0,114,44,0,0,0,114,51,0,0,0,114,52,0,0, + 0,114,56,0,0,0,114,57,0,0,0,114,70,0,0,0, + 114,74,0,0,0,114,75,0,0,0,114,9,0,0,0,114, + 9,0,0,0,114,9,0,0,0,114,10,0,0,0,114,4, + 0,0,0,42,0,0,0,115,24,0,0,0,8,13,4,5, + 8,46,10,32,10,12,8,10,8,21,8,11,8,26,8,13, + 8,38,8,18,122,12,95,95,105,110,105,116,95,95,46,112, + 121,99,84,122,11,95,95,105,110,105,116,95,95,46,112,121, + 70,41,3,122,4,46,112,121,99,84,70,41,3,122,3,46, + 112,121,70,70,99,2,0,0,0,0,0,0,0,2,0,0, + 0,4,0,0,0,67,0,0,0,115,20,0,0,0,124,0, + 106,0,124,1,160,1,100,1,161,1,100,2,25,0,23,0, + 83,0,41,3,78,218,1,46,233,2,0,0,0,41,2,114, + 29,0,0,0,218,10,114,112,97,114,116,105,116,105,111,110, + 41,2,114,30,0,0,0,114,36,0,0,0,114,9,0,0, + 0,114,9,0,0,0,114,10,0,0,0,114,34,0,0,0, + 49,1,0,0,115,2,0,0,0,0,1,114,34,0,0,0, + 99,2,0,0,0,0,0,0,0,3,0,0,0,2,0,0, + 0,67,0,0,0,115,18,0,0,0,124,1,116,0,23,0, + 125,2,124,2,124,0,106,1,107,6,83,0,41,1,78,41, + 2,114,18,0,0,0,114,26,0,0,0,41,3,114,30,0, + 0,0,114,11,0,0,0,90,7,100,105,114,112,97,116,104, + 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,114, + 35,0,0,0,53,1,0,0,115,4,0,0,0,0,4,8, + 2,114,35,0,0,0,99,2,0,0,0,0,0,0,0,7, + 0,0,0,4,0,0,0,67,0,0,0,115,56,0,0,0, + 116,0,124,0,124,1,131,2,125,2,116,1,68,0,93,36, + 92,3,125,3,125,4,125,5,124,2,124,3,23,0,125,6, + 124,6,124,0,106,2,107,6,114,14,124,5,2,0,1,0, + 83,0,113,14,100,0,83,0,41,1,78,41,3,114,34,0, + 0,0,218,16,95,122,105,112,95,115,101,97,114,99,104,111, + 114,100,101,114,114,26,0,0,0,41,7,114,30,0,0,0, + 114,36,0,0,0,114,11,0,0,0,218,6,115,117,102,102, + 105,120,218,10,105,115,98,121,116,101,99,111,100,101,114,43, + 0,0,0,114,55,0,0,0,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,114,33,0,0,0,62,1,0,0, + 115,12,0,0,0,0,1,10,1,14,1,8,1,10,1,10, + 1,114,33,0,0,0,99,1,0,0,0,0,0,0,0,23, + 0,0,0,9,0,0,0,67,0,0,0,115,20,4,0,0, + 122,16,116,0,160,1,124,0,100,1,161,2,125,1,87,0, + 110,38,4,0,116,2,107,10,114,54,1,0,1,0,1,0, + 116,3,100,2,124,0,155,2,157,2,124,0,100,3,141,2, + 130,1,89,0,110,2,88,0,124,1,144,3,143,190,1,0, + 122,34,124,1,160,4,100,4,100,5,161,2,1,0,124,1, + 160,5,161,0,125,2,124,1,160,6,100,6,161,1,125,3, + 87,0,110,38,4,0,116,2,107,10,114,136,1,0,1,0, 1,0,116,3,100,7,124,0,155,2,157,2,124,0,100,3, - 141,2,130,1,89,0,110,2,88,0,124,1,160,6,100,18, - 161,1,125,3,116,7,124,3,131,1,100,8,107,0,144,1, - 114,174,116,9,100,19,131,1,130,1,124,3,100,0,100,8, - 133,2,25,0,100,20,107,3,144,1,114,196,144,3,113,248, - 116,7,124,3,131,1,100,18,107,3,144,1,114,218,116,9, - 100,19,131,1,130,1,116,10,124,3,100,21,100,22,133,2, - 25,0,131,1,125,9,116,10,124,3,100,22,100,11,133,2, - 25,0,131,1,125,10,116,10,124,3,100,11,100,23,133,2, - 25,0,131,1,125,11,116,10,124,3,100,23,100,12,133,2, - 25,0,131,1,125,12,116,8,124,3,100,12,100,13,133,2, - 25,0,131,1,125,13,116,8,124,3,100,13,100,24,133,2, - 25,0,131,1,125,14,116,8,124,3,100,24,100,25,133,2, - 25,0,131,1,125,15,116,10,124,3,100,25,100,26,133,2, - 25,0,131,1,125,16,116,10,124,3,100,26,100,27,133,2, - 25,0,131,1,125,17,116,10,124,3,100,27,100,28,133,2, - 25,0,131,1,125,18,116,8,124,3,100,29,100,18,133,2, - 25,0,131,1,125,19,124,16,124,17,23,0,124,18,23,0, - 125,4,124,19,124,5,107,4,144,2,114,178,116,3,100,30, - 124,0,155,2,157,2,124,0,100,3,141,2,130,1,124,19, - 124,6,55,0,125,19,122,14,124,1,160,6,124,16,161,1, - 125,20,87,0,110,40,4,0,116,2,107,10,144,2,114,240, + 141,2,130,1,89,0,110,2,88,0,116,7,124,3,131,1, + 100,6,107,3,114,168,116,3,100,7,124,0,155,2,157,2, + 124,0,100,3,141,2,130,1,124,3,100,0,100,8,133,2, + 25,0,100,9,107,3,114,202,116,3,100,10,124,0,155,2, + 157,2,124,0,100,3,141,2,130,1,116,8,124,3,100,11, + 100,12,133,2,25,0,131,1,125,4,116,8,124,3,100,12, + 100,13,133,2,25,0,131,1,125,5,124,2,124,4,107,0, + 144,1,114,6,116,3,100,14,124,0,155,2,157,2,124,0, + 100,3,141,2,130,1,124,2,124,5,107,0,144,1,114,34, + 116,3,100,15,124,0,155,2,157,2,124,0,100,3,141,2, + 130,1,124,2,124,4,56,0,125,2,124,2,124,5,24,0, + 125,6,124,6,100,16,107,0,144,1,114,78,116,3,100,17, + 124,0,155,2,157,2,124,0,100,3,141,2,130,1,105,0, + 125,7,100,16,125,8,122,14,124,1,160,4,124,2,161,1, + 1,0,87,0,110,40,4,0,116,2,107,10,144,1,114,140, 1,0,1,0,1,0,116,3,100,7,124,0,155,2,157,2, - 124,0,100,3,141,2,130,1,89,0,110,2,88,0,116,7, - 124,20,131,1,124,16,107,3,144,3,114,18,116,3,100,7, - 124,0,155,2,157,2,124,0,100,3,141,2,130,1,122,50, - 116,7,124,1,160,6,124,4,124,16,24,0,161,1,131,1, - 124,4,124,16,24,0,107,3,144,3,114,66,116,3,100,7, - 124,0,155,2,157,2,124,0,100,3,141,2,130,1,87,0, - 110,40,4,0,116,2,107,10,144,3,114,108,1,0,1,0, - 1,0,116,3,100,7,124,0,155,2,157,2,124,0,100,3, - 141,2,130,1,89,0,110,2,88,0,124,9,100,31,64,0, - 144,3,114,130,124,20,160,11,161,0,125,20,110,54,122,14, - 124,20,160,11,100,32,161,1,125,20,87,0,110,38,4,0, - 116,12,107,10,144,3,114,182,1,0,1,0,1,0,124,20, - 160,11,100,33,161,1,160,13,116,14,161,1,125,20,89,0, - 110,2,88,0,124,20,160,15,100,34,116,16,161,2,125,20, - 116,17,160,18,124,0,124,20,161,2,125,21,124,21,124,10, - 124,14,124,15,124,19,124,11,124,12,124,13,102,8,125,22, - 124,22,124,7,124,20,60,0,124,8,100,35,55,0,125,8, - 144,1,113,142,87,0,53,0,81,0,82,0,88,0,116,19, - 160,20,100,36,124,8,124,0,161,3,1,0,124,7,83,0, - 41,37,78,218,2,114,98,122,21,99,97,110,39,116,32,111, - 112,101,110,32,90,105,112,32,102,105,108,101,58,32,41,1, - 114,11,0,0,0,105,234,255,255,255,114,76,0,0,0,233, - 22,0,0,0,122,21,99,97,110,39,116,32,114,101,97,100, - 32,90,105,112,32,102,105,108,101,58,32,233,4,0,0,0, - 115,4,0,0,0,80,75,5,6,122,16,110,111,116,32,97, - 32,90,105,112,32,102,105,108,101,58,32,233,12,0,0,0, - 233,16,0,0,0,233,20,0,0,0,122,28,98,97,100,32, - 99,101,110,116,114,97,108,32,100,105,114,101,99,116,111,114, - 121,32,115,105,122,101,58,32,122,30,98,97,100,32,99,101, - 110,116,114,97,108,32,100,105,114,101,99,116,111,114,121,32, - 111,102,102,115,101,116,58,32,114,0,0,0,0,122,38,98, - 97,100,32,99,101,110,116,114,97,108,32,100,105,114,101,99, - 116,111,114,121,32,115,105,122,101,32,111,114,32,111,102,102, - 115,101,116,58,32,233,46,0,0,0,122,27,69,79,70,32, - 114,101,97,100,32,119,104,101,114,101,32,110,111,116,32,101, - 120,112,101,99,116,101,100,115,4,0,0,0,80,75,1,2, - 233,8,0,0,0,233,10,0,0,0,233,14,0,0,0,233, - 24,0,0,0,233,28,0,0,0,233,30,0,0,0,233,32, - 0,0,0,233,34,0,0,0,233,42,0,0,0,122,25,98, - 97,100,32,108,111,99,97,108,32,104,101,97,100,101,114,32, - 111,102,102,115,101,116,58,32,105,0,8,0,0,218,5,97, - 115,99,105,105,90,6,108,97,116,105,110,49,250,1,47,114, - 5,0,0,0,122,33,122,105,112,105,109,112,111,114,116,58, - 32,102,111,117,110,100,32,123,125,32,110,97,109,101,115,32, - 105,110,32,123,33,114,125,41,21,218,3,95,105,111,218,4, - 111,112,101,110,114,20,0,0,0,114,3,0,0,0,218,4, - 115,101,101,107,90,4,116,101,108,108,218,4,114,101,97,100, - 114,47,0,0,0,114,2,0,0,0,218,8,69,79,70,69, - 114,114,111,114,114,1,0,0,0,114,54,0,0,0,218,18, - 85,110,105,99,111,100,101,68,101,99,111,100,101,69,114,114, - 111,114,218,9,116,114,97,110,115,108,97,116,101,218,11,99, - 112,52,51,55,95,116,97,98,108,101,114,17,0,0,0,114, - 18,0,0,0,114,19,0,0,0,114,28,0,0,0,114,68, - 0,0,0,114,69,0,0,0,41,23,114,27,0,0,0,218, - 2,102,112,90,15,104,101,97,100,101,114,95,112,111,115,105, - 116,105,111,110,218,6,98,117,102,102,101,114,218,11,104,101, - 97,100,101,114,95,115,105,122,101,90,13,104,101,97,100,101, - 114,95,111,102,102,115,101,116,90,10,97,114,99,95,111,102, - 102,115,101,116,114,31,0,0,0,218,5,99,111,117,110,116, - 218,5,102,108,97,103,115,218,8,99,111,109,112,114,101,115, - 115,218,4,116,105,109,101,218,4,100,97,116,101,218,3,99, - 114,99,218,9,100,97,116,97,95,115,105,122,101,218,9,102, - 105,108,101,95,115,105,122,101,218,9,110,97,109,101,95,115, - 105,122,101,218,10,101,120,116,114,97,95,115,105,122,101,90, - 12,99,111,109,109,101,110,116,95,115,105,122,101,218,11,102, - 105,108,101,95,111,102,102,115,101,116,114,53,0,0,0,114, - 11,0,0,0,218,1,116,114,9,0,0,0,114,9,0,0, - 0,114,10,0,0,0,114,25,0,0,0,85,1,0,0,115, - 154,0,0,0,0,1,2,1,16,1,14,1,24,2,8,1, - 2,1,12,1,8,1,14,1,14,1,24,1,12,1,18,1, - 16,2,18,2,16,1,16,1,10,1,18,1,10,1,18,1, - 8,1,8,1,10,1,18,2,4,2,4,1,2,1,14,1, - 16,1,24,2,10,1,14,1,8,2,18,1,4,1,14,1, - 8,1,16,1,16,1,16,1,16,1,16,1,16,1,16,1, - 16,1,16,1,16,1,16,1,12,1,10,1,18,1,8,2, - 2,1,14,1,16,1,24,1,14,1,18,4,2,1,28,1, - 22,1,16,1,24,2,10,2,10,3,2,1,14,1,16,1, - 22,2,12,1,12,1,20,1,8,1,22,1,14,1,114,25, - 0,0,0,117,190,1,0,0,0,1,2,3,4,5,6,7, - 8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23, - 24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39, - 40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55, - 56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71, - 72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87, - 88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103, - 104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119, - 120,121,122,123,124,125,126,127,195,135,195,188,195,169,195,162, - 195,164,195,160,195,165,195,167,195,170,195,171,195,168,195,175, - 195,174,195,172,195,132,195,133,195,137,195,166,195,134,195,180, - 195,182,195,178,195,187,195,185,195,191,195,150,195,156,194,162, - 194,163,194,165,226,130,167,198,146,195,161,195,173,195,179,195, - 186,195,177,195,145,194,170,194,186,194,191,226,140,144,194,172, - 194,189,194,188,194,161,194,171,194,187,226,150,145,226,150,146, - 226,150,147,226,148,130,226,148,164,226,149,161,226,149,162,226, - 149,150,226,149,149,226,149,163,226,149,145,226,149,151,226,149, - 157,226,149,156,226,149,155,226,148,144,226,148,148,226,148,180, - 226,148,172,226,148,156,226,148,128,226,148,188,226,149,158,226, - 149,159,226,149,154,226,149,148,226,149,169,226,149,166,226,149, - 160,226,149,144,226,149,172,226,149,167,226,149,168,226,149,164, - 226,149,165,226,149,153,226,149,152,226,149,146,226,149,147,226, - 149,171,226,149,170,226,148,152,226,148,140,226,150,136,226,150, - 132,226,150,140,226,150,144,226,150,128,206,177,195,159,206,147, - 207,128,206,163,207,131,194,181,207,132,206,166,206,152,206,169, - 206,180,226,136,158,207,134,206,181,226,136,169,226,137,161,194, - 177,226,137,165,226,137,164,226,140,160,226,140,161,195,183,226, - 137,136,194,176,226,136,153,194,183,226,136,154,226,129,191,194, - 178,226,150,160,194,160,99,0,0,0,0,0,0,0,0,1, - 0,0,0,8,0,0,0,67,0,0,0,115,108,0,0,0, - 116,0,114,22,116,1,160,2,100,1,161,1,1,0,116,3, - 100,2,131,1,130,1,100,3,97,0,122,60,122,16,100,4, - 100,5,108,4,109,5,125,0,1,0,87,0,110,38,4,0, - 116,6,107,10,114,82,1,0,1,0,1,0,116,1,160,2, - 100,1,161,1,1,0,116,3,100,2,131,1,130,1,89,0, - 110,2,88,0,87,0,53,0,100,6,97,0,88,0,116,1, - 160,2,100,7,161,1,1,0,124,0,83,0,41,8,78,122, - 27,122,105,112,105,109,112,111,114,116,58,32,122,108,105,98, - 32,85,78,65,86,65,73,76,65,66,76,69,122,41,99,97, - 110,39,116,32,100,101,99,111,109,112,114,101,115,115,32,100, - 97,116,97,59,32,122,108,105,98,32,110,111,116,32,97,118, - 97,105,108,97,98,108,101,84,114,0,0,0,0,41,1,218, - 10,100,101,99,111,109,112,114,101,115,115,70,122,25,122,105, - 112,105,109,112,111,114,116,58,32,122,108,105,98,32,97,118, - 97,105,108,97,98,108,101,41,7,218,15,95,105,109,112,111, - 114,116,105,110,103,95,122,108,105,98,114,68,0,0,0,114, - 69,0,0,0,114,3,0,0,0,90,4,122,108,105,98,114, - 122,0,0,0,218,9,69,120,99,101,112,116,105,111,110,41, - 1,114,122,0,0,0,114,9,0,0,0,114,9,0,0,0, - 114,10,0,0,0,218,20,95,103,101,116,95,100,101,99,111, - 109,112,114,101,115,115,95,102,117,110,99,220,1,0,0,115, - 24,0,0,0,0,2,4,3,10,1,8,2,4,1,4,1, - 16,1,14,1,10,1,18,2,6,2,10,1,114,125,0,0, - 0,99,2,0,0,0,0,0,0,0,17,0,0,0,9,0, - 0,0,67,0,0,0,115,130,1,0,0,124,1,92,8,125, - 2,125,3,125,4,125,5,125,6,125,7,125,8,125,9,124, - 4,100,1,107,0,114,36,116,0,100,2,131,1,130,1,116, - 1,160,2,124,0,100,3,161,2,144,1,143,8,125,10,122, - 14,124,10,160,3,124,6,161,1,1,0,87,0,110,38,4, - 0,116,4,107,10,114,104,1,0,1,0,1,0,116,0,100, + 124,0,100,3,141,2,130,1,89,0,110,2,88,0,124,1, + 160,6,100,18,161,1,125,3,116,7,124,3,131,1,100,8, + 107,0,144,1,114,174,116,9,100,19,131,1,130,1,124,3, + 100,0,100,8,133,2,25,0,100,20,107,3,144,1,114,196, + 144,3,113,248,116,7,124,3,131,1,100,18,107,3,144,1, + 114,218,116,9,100,19,131,1,130,1,116,10,124,3,100,21, + 100,22,133,2,25,0,131,1,125,9,116,10,124,3,100,22, + 100,11,133,2,25,0,131,1,125,10,116,10,124,3,100,11, + 100,23,133,2,25,0,131,1,125,11,116,10,124,3,100,23, + 100,12,133,2,25,0,131,1,125,12,116,8,124,3,100,12, + 100,13,133,2,25,0,131,1,125,13,116,8,124,3,100,13, + 100,24,133,2,25,0,131,1,125,14,116,8,124,3,100,24, + 100,25,133,2,25,0,131,1,125,15,116,10,124,3,100,25, + 100,26,133,2,25,0,131,1,125,16,116,10,124,3,100,26, + 100,27,133,2,25,0,131,1,125,17,116,10,124,3,100,27, + 100,28,133,2,25,0,131,1,125,18,116,8,124,3,100,29, + 100,18,133,2,25,0,131,1,125,19,124,16,124,17,23,0, + 124,18,23,0,125,4,124,19,124,5,107,4,144,2,114,178, + 116,3,100,30,124,0,155,2,157,2,124,0,100,3,141,2, + 130,1,124,19,124,6,55,0,125,19,122,14,124,1,160,6, + 124,16,161,1,125,20,87,0,110,40,4,0,116,2,107,10, + 144,2,114,240,1,0,1,0,1,0,116,3,100,7,124,0, + 155,2,157,2,124,0,100,3,141,2,130,1,89,0,110,2, + 88,0,116,7,124,20,131,1,124,16,107,3,144,3,114,18, + 116,3,100,7,124,0,155,2,157,2,124,0,100,3,141,2, + 130,1,122,50,116,7,124,1,160,6,124,4,124,16,24,0, + 161,1,131,1,124,4,124,16,24,0,107,3,144,3,114,66, + 116,3,100,7,124,0,155,2,157,2,124,0,100,3,141,2, + 130,1,87,0,110,40,4,0,116,2,107,10,144,3,114,108, + 1,0,1,0,1,0,116,3,100,7,124,0,155,2,157,2, + 124,0,100,3,141,2,130,1,89,0,110,2,88,0,124,9, + 100,31,64,0,144,3,114,130,124,20,160,11,161,0,125,20, + 110,54,122,14,124,20,160,11,100,32,161,1,125,20,87,0, + 110,38,4,0,116,12,107,10,144,3,114,182,1,0,1,0, + 1,0,124,20,160,11,100,33,161,1,160,13,116,14,161,1, + 125,20,89,0,110,2,88,0,124,20,160,15,100,34,116,16, + 161,2,125,20,116,17,160,18,124,0,124,20,161,2,125,21, + 124,21,124,10,124,14,124,15,124,19,124,11,124,12,124,13, + 102,8,125,22,124,22,124,7,124,20,60,0,124,8,100,35, + 55,0,125,8,144,1,113,142,87,0,53,0,81,0,82,0, + 88,0,116,19,160,20,100,36,124,8,124,0,161,3,1,0, + 124,7,83,0,41,37,78,218,2,114,98,122,21,99,97,110, + 39,116,32,111,112,101,110,32,90,105,112,32,102,105,108,101, + 58,32,41,1,114,11,0,0,0,105,234,255,255,255,114,78, + 0,0,0,233,22,0,0,0,122,21,99,97,110,39,116,32, + 114,101,97,100,32,90,105,112,32,102,105,108,101,58,32,233, + 4,0,0,0,115,4,0,0,0,80,75,5,6,122,16,110, + 111,116,32,97,32,90,105,112,32,102,105,108,101,58,32,233, + 12,0,0,0,233,16,0,0,0,233,20,0,0,0,122,28, + 98,97,100,32,99,101,110,116,114,97,108,32,100,105,114,101, + 99,116,111,114,121,32,115,105,122,101,58,32,122,30,98,97, + 100,32,99,101,110,116,114,97,108,32,100,105,114,101,99,116, + 111,114,121,32,111,102,102,115,101,116,58,32,114,0,0,0, + 0,122,38,98,97,100,32,99,101,110,116,114,97,108,32,100, + 105,114,101,99,116,111,114,121,32,115,105,122,101,32,111,114, + 32,111,102,102,115,101,116,58,32,233,46,0,0,0,122,27, + 69,79,70,32,114,101,97,100,32,119,104,101,114,101,32,110, + 111,116,32,101,120,112,101,99,116,101,100,115,4,0,0,0, + 80,75,1,2,233,8,0,0,0,233,10,0,0,0,233,14, + 0,0,0,233,24,0,0,0,233,28,0,0,0,233,30,0, + 0,0,233,32,0,0,0,233,34,0,0,0,233,42,0,0, + 0,122,25,98,97,100,32,108,111,99,97,108,32,104,101,97, + 100,101,114,32,111,102,102,115,101,116,58,32,105,0,8,0, + 0,218,5,97,115,99,105,105,90,6,108,97,116,105,110,49, + 250,1,47,114,5,0,0,0,122,33,122,105,112,105,109,112, + 111,114,116,58,32,102,111,117,110,100,32,123,125,32,110,97, + 109,101,115,32,105,110,32,123,33,114,125,41,21,218,3,95, + 105,111,218,4,111,112,101,110,114,20,0,0,0,114,3,0, + 0,0,218,4,115,101,101,107,90,4,116,101,108,108,218,4, + 114,101,97,100,114,47,0,0,0,114,2,0,0,0,218,8, + 69,79,70,69,114,114,111,114,114,1,0,0,0,114,54,0, + 0,0,218,18,85,110,105,99,111,100,101,68,101,99,111,100, + 101,69,114,114,111,114,218,9,116,114,97,110,115,108,97,116, + 101,218,11,99,112,52,51,55,95,116,97,98,108,101,114,17, + 0,0,0,114,18,0,0,0,114,19,0,0,0,114,28,0, + 0,0,114,68,0,0,0,114,69,0,0,0,41,23,114,27, + 0,0,0,218,2,102,112,90,15,104,101,97,100,101,114,95, + 112,111,115,105,116,105,111,110,218,6,98,117,102,102,101,114, + 218,11,104,101,97,100,101,114,95,115,105,122,101,90,13,104, + 101,97,100,101,114,95,111,102,102,115,101,116,90,10,97,114, + 99,95,111,102,102,115,101,116,114,31,0,0,0,218,5,99, + 111,117,110,116,218,5,102,108,97,103,115,218,8,99,111,109, + 112,114,101,115,115,218,4,116,105,109,101,218,4,100,97,116, + 101,218,3,99,114,99,218,9,100,97,116,97,95,115,105,122, + 101,218,9,102,105,108,101,95,115,105,122,101,218,9,110,97, + 109,101,95,115,105,122,101,218,10,101,120,116,114,97,95,115, + 105,122,101,90,12,99,111,109,109,101,110,116,95,115,105,122, + 101,218,11,102,105,108,101,95,111,102,102,115,101,116,114,53, + 0,0,0,114,11,0,0,0,218,1,116,114,9,0,0,0, + 114,9,0,0,0,114,10,0,0,0,114,25,0,0,0,93, + 1,0,0,115,154,0,0,0,0,1,2,1,16,1,14,1, + 24,2,8,1,2,1,12,1,8,1,14,1,14,1,24,1, + 12,1,18,1,16,2,18,2,16,1,16,1,10,1,18,1, + 10,1,18,1,8,1,8,1,10,1,18,2,4,2,4,1, + 2,1,14,1,16,1,24,2,10,1,14,1,8,2,18,1, + 4,1,14,1,8,1,16,1,16,1,16,1,16,1,16,1, + 16,1,16,1,16,1,16,1,16,1,16,1,12,1,10,1, + 18,1,8,2,2,1,14,1,16,1,24,1,14,1,18,4, + 2,1,28,1,22,1,16,1,24,2,10,2,10,3,2,1, + 14,1,16,1,22,2,12,1,12,1,20,1,8,1,22,1, + 14,1,114,25,0,0,0,117,190,1,0,0,0,1,2,3, + 4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19, + 20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35, + 36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51, + 52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67, + 68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83, + 84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99, + 100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115, + 116,117,118,119,120,121,122,123,124,125,126,127,195,135,195,188, + 195,169,195,162,195,164,195,160,195,165,195,167,195,170,195,171, + 195,168,195,175,195,174,195,172,195,132,195,133,195,137,195,166, + 195,134,195,180,195,182,195,178,195,187,195,185,195,191,195,150, + 195,156,194,162,194,163,194,165,226,130,167,198,146,195,161,195, + 173,195,179,195,186,195,177,195,145,194,170,194,186,194,191,226, + 140,144,194,172,194,189,194,188,194,161,194,171,194,187,226,150, + 145,226,150,146,226,150,147,226,148,130,226,148,164,226,149,161, + 226,149,162,226,149,150,226,149,149,226,149,163,226,149,145,226, + 149,151,226,149,157,226,149,156,226,149,155,226,148,144,226,148, + 148,226,148,180,226,148,172,226,148,156,226,148,128,226,148,188, + 226,149,158,226,149,159,226,149,154,226,149,148,226,149,169,226, + 149,166,226,149,160,226,149,144,226,149,172,226,149,167,226,149, + 168,226,149,164,226,149,165,226,149,153,226,149,152,226,149,146, + 226,149,147,226,149,171,226,149,170,226,148,152,226,148,140,226, + 150,136,226,150,132,226,150,140,226,150,144,226,150,128,206,177, + 195,159,206,147,207,128,206,163,207,131,194,181,207,132,206,166, + 206,152,206,169,206,180,226,136,158,207,134,206,181,226,136,169, + 226,137,161,194,177,226,137,165,226,137,164,226,140,160,226,140, + 161,195,183,226,137,136,194,176,226,136,153,194,183,226,136,154, + 226,129,191,194,178,226,150,160,194,160,99,0,0,0,0,0, + 0,0,0,1,0,0,0,8,0,0,0,67,0,0,0,115, + 108,0,0,0,116,0,114,22,116,1,160,2,100,1,161,1, + 1,0,116,3,100,2,131,1,130,1,100,3,97,0,122,60, + 122,16,100,4,100,5,108,4,109,5,125,0,1,0,87,0, + 110,38,4,0,116,6,107,10,114,82,1,0,1,0,1,0, + 116,1,160,2,100,1,161,1,1,0,116,3,100,2,131,1, + 130,1,89,0,110,2,88,0,87,0,53,0,100,6,97,0, + 88,0,116,1,160,2,100,7,161,1,1,0,124,0,83,0, + 41,8,78,122,27,122,105,112,105,109,112,111,114,116,58,32, + 122,108,105,98,32,85,78,65,86,65,73,76,65,66,76,69, + 122,41,99,97,110,39,116,32,100,101,99,111,109,112,114,101, + 115,115,32,100,97,116,97,59,32,122,108,105,98,32,110,111, + 116,32,97,118,97,105,108,97,98,108,101,84,114,0,0,0, + 0,41,1,218,10,100,101,99,111,109,112,114,101,115,115,70, + 122,25,122,105,112,105,109,112,111,114,116,58,32,122,108,105, + 98,32,97,118,97,105,108,97,98,108,101,41,7,218,15,95, + 105,109,112,111,114,116,105,110,103,95,122,108,105,98,114,68, + 0,0,0,114,69,0,0,0,114,3,0,0,0,90,4,122, + 108,105,98,114,124,0,0,0,218,9,69,120,99,101,112,116, + 105,111,110,41,1,114,124,0,0,0,114,9,0,0,0,114, + 9,0,0,0,114,10,0,0,0,218,20,95,103,101,116,95, + 100,101,99,111,109,112,114,101,115,115,95,102,117,110,99,228, + 1,0,0,115,24,0,0,0,0,2,4,3,10,1,8,2, + 4,1,4,1,16,1,14,1,10,1,18,2,6,2,10,1, + 114,127,0,0,0,99,2,0,0,0,0,0,0,0,17,0, + 0,0,9,0,0,0,67,0,0,0,115,130,1,0,0,124, + 1,92,8,125,2,125,3,125,4,125,5,125,6,125,7,125, + 8,125,9,124,4,100,1,107,0,114,36,116,0,100,2,131, + 1,130,1,116,1,160,2,124,0,100,3,161,2,144,1,143, + 8,125,10,122,14,124,10,160,3,124,6,161,1,1,0,87, + 0,110,38,4,0,116,4,107,10,114,104,1,0,1,0,1, + 0,116,0,100,4,124,0,155,2,157,2,124,0,100,5,141, + 2,130,1,89,0,110,2,88,0,124,10,160,5,100,6,161, + 1,125,11,116,6,124,11,131,1,100,6,107,3,114,136,116, + 7,100,7,131,1,130,1,124,11,100,0,100,8,133,2,25, + 0,100,9,107,3,114,170,116,0,100,10,124,0,155,2,157, + 2,124,0,100,5,141,2,130,1,116,8,124,11,100,11,100, + 12,133,2,25,0,131,1,125,12,116,8,124,11,100,12,100, + 6,133,2,25,0,131,1,125,13,100,6,124,12,23,0,124, + 13,23,0,125,14,124,6,124,14,55,0,125,6,122,14,124, + 10,160,3,124,6,161,1,1,0,87,0,110,40,4,0,116, + 4,107,10,144,1,114,20,1,0,1,0,1,0,116,0,100, 4,124,0,155,2,157,2,124,0,100,5,141,2,130,1,89, - 0,110,2,88,0,124,10,160,5,100,6,161,1,125,11,116, - 6,124,11,131,1,100,6,107,3,114,136,116,7,100,7,131, - 1,130,1,124,11,100,0,100,8,133,2,25,0,100,9,107, - 3,114,170,116,0,100,10,124,0,155,2,157,2,124,0,100, - 5,141,2,130,1,116,8,124,11,100,11,100,12,133,2,25, - 0,131,1,125,12,116,8,124,11,100,12,100,6,133,2,25, - 0,131,1,125,13,100,6,124,12,23,0,124,13,23,0,125, - 14,124,6,124,14,55,0,125,6,122,14,124,10,160,3,124, - 6,161,1,1,0,87,0,110,40,4,0,116,4,107,10,144, - 1,114,20,1,0,1,0,1,0,116,0,100,4,124,0,155, - 2,157,2,124,0,100,5,141,2,130,1,89,0,110,2,88, - 0,124,10,160,5,124,4,161,1,125,15,116,6,124,15,131, - 1,124,4,107,3,144,1,114,54,116,4,100,13,131,1,130, - 1,87,0,53,0,81,0,82,0,88,0,124,3,100,1,107, - 2,144,1,114,78,124,15,83,0,122,10,116,9,131,0,125, - 16,87,0,110,30,4,0,116,10,107,10,144,1,114,118,1, - 0,1,0,1,0,116,0,100,14,131,1,130,1,89,0,110, - 2,88,0,124,16,124,15,100,15,131,2,83,0,41,16,78, - 114,0,0,0,0,122,18,110,101,103,97,116,105,118,101,32, - 100,97,116,97,32,115,105,122,101,114,81,0,0,0,122,21, - 99,97,110,39,116,32,114,101,97,100,32,90,105,112,32,102, - 105,108,101,58,32,41,1,114,11,0,0,0,114,93,0,0, - 0,122,27,69,79,70,32,114,101,97,100,32,119,104,101,114, - 101,32,110,111,116,32,101,120,112,101,99,116,101,100,114,83, - 0,0,0,115,4,0,0,0,80,75,3,4,122,23,98,97, - 100,32,108,111,99,97,108,32,102,105,108,101,32,104,101,97, - 100,101,114,58,32,233,26,0,0,0,114,92,0,0,0,122, - 26,122,105,112,105,109,112,111,114,116,58,32,99,97,110,39, - 116,32,114,101,97,100,32,100,97,116,97,122,41,99,97,110, - 39,116,32,100,101,99,111,109,112,114,101,115,115,32,100,97, - 116,97,59,32,122,108,105,98,32,110,111,116,32,97,118,97, - 105,108,97,98,108,101,105,241,255,255,255,41,11,114,3,0, - 0,0,114,99,0,0,0,114,100,0,0,0,114,101,0,0, - 0,114,20,0,0,0,114,102,0,0,0,114,47,0,0,0, - 114,103,0,0,0,114,1,0,0,0,114,125,0,0,0,114, - 124,0,0,0,41,17,114,27,0,0,0,114,50,0,0,0, - 90,8,100,97,116,97,112,97,116,104,114,112,0,0,0,114, - 116,0,0,0,114,117,0,0,0,114,120,0,0,0,114,113, - 0,0,0,114,114,0,0,0,114,115,0,0,0,114,107,0, - 0,0,114,108,0,0,0,114,118,0,0,0,114,119,0,0, - 0,114,109,0,0,0,90,8,114,97,119,95,100,97,116,97, - 114,122,0,0,0,114,9,0,0,0,114,9,0,0,0,114, - 10,0,0,0,114,48,0,0,0,241,1,0,0,115,62,0, - 0,0,0,1,20,1,8,1,8,2,16,2,2,1,14,1, - 14,1,24,1,10,1,12,1,8,2,16,2,18,2,16,1, - 16,1,12,1,8,1,2,1,14,1,16,1,24,1,10,1, - 14,1,18,2,10,2,4,3,2,1,10,1,16,1,14,1, - 114,48,0,0,0,99,2,0,0,0,0,0,0,0,2,0, - 0,0,3,0,0,0,67,0,0,0,115,16,0,0,0,116, - 0,124,0,124,1,24,0,131,1,100,1,107,1,83,0,41, - 2,78,114,5,0,0,0,41,1,218,3,97,98,115,41,2, - 90,2,116,49,90,2,116,50,114,9,0,0,0,114,9,0, - 0,0,114,10,0,0,0,218,9,95,101,113,95,109,116,105, - 109,101,31,2,0,0,115,2,0,0,0,0,2,114,128,0, - 0,0,99,3,0,0,0,0,0,0,0,5,0,0,0,5, - 0,0,0,67,0,0,0,115,206,0,0,0,116,0,124,1, - 131,1,100,1,107,0,114,20,116,1,100,2,131,1,130,1, - 124,1,100,0,100,3,133,2,25,0,116,2,106,3,107,3, - 114,54,116,4,160,5,100,4,124,0,161,2,1,0,100,0, - 83,0,116,6,124,1,100,3,100,5,133,2,25,0,131,1, - 125,3,124,3,100,6,107,3,114,112,116,7,106,8,100,7, - 107,3,114,110,124,3,100,8,107,3,115,106,116,7,106,8, - 100,9,107,2,114,110,100,0,83,0,110,46,124,2,100,6, - 107,3,114,158,116,9,116,6,124,1,100,5,100,10,133,2, - 25,0,131,1,124,2,131,2,115,158,116,4,160,5,100,11, - 124,0,161,2,1,0,100,0,83,0,116,10,160,11,124,1, - 100,1,100,0,133,2,25,0,161,1,125,4,116,12,124,4, - 116,13,131,2,115,202,116,14,100,12,124,0,155,2,100,13, - 157,3,131,1,130,1,124,4,83,0,41,14,78,114,85,0, - 0,0,122,12,98,97,100,32,112,121,99,32,100,97,116,97, - 114,83,0,0,0,122,18,123,33,114,125,32,104,97,115,32, - 98,97,100,32,109,97,103,105,99,114,88,0,0,0,114,0, - 0,0,0,90,5,110,101,118,101,114,114,5,0,0,0,90, - 6,97,108,119,97,121,115,114,84,0,0,0,122,18,123,33, - 114,125,32,104,97,115,32,98,97,100,32,109,116,105,109,101, - 122,16,99,111,109,112,105,108,101,100,32,109,111,100,117,108, - 101,32,122,21,32,105,115,32,110,111,116,32,97,32,99,111, - 100,101,32,111,98,106,101,99,116,41,15,114,47,0,0,0, - 114,3,0,0,0,114,19,0,0,0,90,12,77,65,71,73, - 67,95,78,85,77,66,69,82,114,68,0,0,0,114,69,0, - 0,0,114,2,0,0,0,218,4,95,105,109,112,90,21,99, - 104,101,99,107,95,104,97,115,104,95,98,97,115,101,100,95, - 112,121,99,115,114,128,0,0,0,218,7,109,97,114,115,104, - 97,108,90,5,108,111,97,100,115,114,13,0,0,0,218,10, - 95,99,111,100,101,95,116,121,112,101,218,9,84,121,112,101, - 69,114,114,111,114,41,5,114,49,0,0,0,218,4,100,97, - 116,97,218,5,109,116,105,109,101,114,111,0,0,0,114,42, + 0,110,2,88,0,124,10,160,5,124,4,161,1,125,15,116, + 6,124,15,131,1,124,4,107,3,144,1,114,54,116,4,100, + 13,131,1,130,1,87,0,53,0,81,0,82,0,88,0,124, + 3,100,1,107,2,144,1,114,78,124,15,83,0,122,10,116, + 9,131,0,125,16,87,0,110,30,4,0,116,10,107,10,144, + 1,114,118,1,0,1,0,1,0,116,0,100,14,131,1,130, + 1,89,0,110,2,88,0,124,16,124,15,100,15,131,2,83, + 0,41,16,78,114,0,0,0,0,122,18,110,101,103,97,116, + 105,118,101,32,100,97,116,97,32,115,105,122,101,114,83,0, + 0,0,122,21,99,97,110,39,116,32,114,101,97,100,32,90, + 105,112,32,102,105,108,101,58,32,41,1,114,11,0,0,0, + 114,95,0,0,0,122,27,69,79,70,32,114,101,97,100,32, + 119,104,101,114,101,32,110,111,116,32,101,120,112,101,99,116, + 101,100,114,85,0,0,0,115,4,0,0,0,80,75,3,4, + 122,23,98,97,100,32,108,111,99,97,108,32,102,105,108,101, + 32,104,101,97,100,101,114,58,32,233,26,0,0,0,114,94, + 0,0,0,122,26,122,105,112,105,109,112,111,114,116,58,32, + 99,97,110,39,116,32,114,101,97,100,32,100,97,116,97,122, + 41,99,97,110,39,116,32,100,101,99,111,109,112,114,101,115, + 115,32,100,97,116,97,59,32,122,108,105,98,32,110,111,116, + 32,97,118,97,105,108,97,98,108,101,105,241,255,255,255,41, + 11,114,3,0,0,0,114,101,0,0,0,114,102,0,0,0, + 114,103,0,0,0,114,20,0,0,0,114,104,0,0,0,114, + 47,0,0,0,114,105,0,0,0,114,1,0,0,0,114,127, + 0,0,0,114,126,0,0,0,41,17,114,27,0,0,0,114, + 50,0,0,0,90,8,100,97,116,97,112,97,116,104,114,114, + 0,0,0,114,118,0,0,0,114,119,0,0,0,114,122,0, + 0,0,114,115,0,0,0,114,116,0,0,0,114,117,0,0, + 0,114,109,0,0,0,114,110,0,0,0,114,120,0,0,0, + 114,121,0,0,0,114,111,0,0,0,90,8,114,97,119,95, + 100,97,116,97,114,124,0,0,0,114,9,0,0,0,114,9, + 0,0,0,114,10,0,0,0,114,48,0,0,0,249,1,0, + 0,115,62,0,0,0,0,1,20,1,8,1,8,2,16,2, + 2,1,14,1,14,1,24,1,10,1,12,1,8,2,16,2, + 18,2,16,1,16,1,12,1,8,1,2,1,14,1,16,1, + 24,1,10,1,14,1,18,2,10,2,4,3,2,1,10,1, + 16,1,14,1,114,48,0,0,0,99,2,0,0,0,0,0, + 0,0,2,0,0,0,3,0,0,0,67,0,0,0,115,16, + 0,0,0,116,0,124,0,124,1,24,0,131,1,100,1,107, + 1,83,0,41,2,78,114,5,0,0,0,41,1,218,3,97, + 98,115,41,2,90,2,116,49,90,2,116,50,114,9,0,0, + 0,114,9,0,0,0,114,10,0,0,0,218,9,95,101,113, + 95,109,116,105,109,101,39,2,0,0,115,2,0,0,0,0, + 2,114,130,0,0,0,99,3,0,0,0,0,0,0,0,5, + 0,0,0,5,0,0,0,67,0,0,0,115,206,0,0,0, + 116,0,124,1,131,1,100,1,107,0,114,20,116,1,100,2, + 131,1,130,1,124,1,100,0,100,3,133,2,25,0,116,2, + 106,3,107,3,114,54,116,4,160,5,100,4,124,0,161,2, + 1,0,100,0,83,0,116,6,124,1,100,3,100,5,133,2, + 25,0,131,1,125,3,124,3,100,6,107,3,114,112,116,7, + 106,8,100,7,107,3,114,110,124,3,100,8,107,3,115,106, + 116,7,106,8,100,9,107,2,114,110,100,0,83,0,110,46, + 124,2,100,6,107,3,114,158,116,9,116,6,124,1,100,5, + 100,10,133,2,25,0,131,1,124,2,131,2,115,158,116,4, + 160,5,100,11,124,0,161,2,1,0,100,0,83,0,116,10, + 160,11,124,1,100,1,100,0,133,2,25,0,161,1,125,4, + 116,12,124,4,116,13,131,2,115,202,116,14,100,12,124,0, + 155,2,100,13,157,3,131,1,130,1,124,4,83,0,41,14, + 78,114,87,0,0,0,122,12,98,97,100,32,112,121,99,32, + 100,97,116,97,114,85,0,0,0,122,18,123,33,114,125,32, + 104,97,115,32,98,97,100,32,109,97,103,105,99,114,90,0, + 0,0,114,0,0,0,0,90,5,110,101,118,101,114,114,5, + 0,0,0,90,6,97,108,119,97,121,115,114,86,0,0,0, + 122,18,123,33,114,125,32,104,97,115,32,98,97,100,32,109, + 116,105,109,101,122,16,99,111,109,112,105,108,101,100,32,109, + 111,100,117,108,101,32,122,21,32,105,115,32,110,111,116,32, + 97,32,99,111,100,101,32,111,98,106,101,99,116,41,15,114, + 47,0,0,0,114,3,0,0,0,114,19,0,0,0,90,12, + 77,65,71,73,67,95,78,85,77,66,69,82,114,68,0,0, + 0,114,69,0,0,0,114,2,0,0,0,218,4,95,105,109, + 112,90,21,99,104,101,99,107,95,104,97,115,104,95,98,97, + 115,101,100,95,112,121,99,115,114,130,0,0,0,218,7,109, + 97,114,115,104,97,108,90,5,108,111,97,100,115,114,13,0, + 0,0,218,10,95,99,111,100,101,95,116,121,112,101,218,9, + 84,121,112,101,69,114,114,111,114,41,5,114,49,0,0,0, + 218,4,100,97,116,97,218,5,109,116,105,109,101,114,113,0, + 0,0,114,42,0,0,0,114,9,0,0,0,114,9,0,0, + 0,114,10,0,0,0,218,15,95,117,110,109,97,114,115,104, + 97,108,95,99,111,100,101,47,2,0,0,115,40,0,0,0, + 0,1,12,1,8,2,18,1,12,1,4,2,16,1,8,5, + 10,1,6,255,2,1,8,255,2,2,6,1,30,1,12,1, + 4,4,18,1,10,1,16,1,114,137,0,0,0,99,1,0, + 0,0,0,0,0,0,1,0,0,0,4,0,0,0,67,0, + 0,0,115,28,0,0,0,124,0,160,0,100,1,100,2,161, + 2,125,0,124,0,160,0,100,3,100,2,161,2,125,0,124, + 0,83,0,41,4,78,115,2,0,0,0,13,10,243,1,0, + 0,0,10,243,1,0,0,0,13,41,1,114,17,0,0,0, + 41,1,218,6,115,111,117,114,99,101,114,9,0,0,0,114, + 9,0,0,0,114,10,0,0,0,218,23,95,110,111,114,109, + 97,108,105,122,101,95,108,105,110,101,95,101,110,100,105,110, + 103,115,80,2,0,0,115,6,0,0,0,0,1,12,1,12, + 1,114,141,0,0,0,99,2,0,0,0,0,0,0,0,2, + 0,0,0,6,0,0,0,67,0,0,0,115,24,0,0,0, + 116,0,124,1,131,1,125,1,116,1,124,1,124,0,100,1, + 100,2,100,3,141,4,83,0,41,4,78,114,66,0,0,0, + 84,41,1,90,12,100,111,110,116,95,105,110,104,101,114,105, + 116,41,2,114,141,0,0,0,218,7,99,111,109,112,105,108, + 101,41,2,114,49,0,0,0,114,140,0,0,0,114,9,0, + 0,0,114,9,0,0,0,114,10,0,0,0,218,15,95,99, + 111,109,112,105,108,101,95,115,111,117,114,99,101,87,2,0, + 0,115,4,0,0,0,0,1,8,1,114,143,0,0,0,99, + 2,0,0,0,0,0,0,0,2,0,0,0,11,0,0,0, + 67,0,0,0,115,68,0,0,0,116,0,160,1,124,0,100, + 1,63,0,100,2,23,0,124,0,100,3,63,0,100,4,64, + 0,124,0,100,5,64,0,124,1,100,6,63,0,124,1,100, + 3,63,0,100,7,64,0,124,1,100,5,64,0,100,8,20, + 0,100,9,100,9,100,9,102,9,161,1,83,0,41,10,78, + 233,9,0,0,0,105,188,7,0,0,233,5,0,0,0,233, + 15,0,0,0,233,31,0,0,0,233,11,0,0,0,233,63, + 0,0,0,114,78,0,0,0,114,12,0,0,0,41,2,114, + 115,0,0,0,90,6,109,107,116,105,109,101,41,2,218,1, + 100,114,123,0,0,0,114,9,0,0,0,114,9,0,0,0, + 114,10,0,0,0,218,14,95,112,97,114,115,101,95,100,111, + 115,116,105,109,101,93,2,0,0,115,24,0,0,0,0,1, + 4,1,10,1,10,1,6,1,6,1,10,1,10,1,2,0, + 2,0,2,250,2,255,114,151,0,0,0,99,2,0,0,0, + 0,0,0,0,5,0,0,0,10,0,0,0,67,0,0,0, + 115,104,0,0,0,122,70,124,1,100,1,100,0,133,2,25, + 0,100,2,107,6,115,22,116,0,130,1,124,1,100,0,100, + 1,133,2,25,0,125,1,124,0,106,1,124,1,25,0,125, + 2,124,2,100,3,25,0,125,3,124,2,100,4,25,0,125, + 4,116,2,124,4,124,3,131,2,87,0,83,0,4,0,116, + 3,116,4,116,5,102,3,107,10,114,98,1,0,1,0,1, + 0,89,0,100,5,83,0,88,0,100,0,83,0,41,6,78, + 114,12,0,0,0,41,2,218,1,99,218,1,111,114,145,0, + 0,0,233,6,0,0,0,114,0,0,0,0,41,6,218,14, + 65,115,115,101,114,116,105,111,110,69,114,114,111,114,114,26, + 0,0,0,114,151,0,0,0,114,24,0,0,0,218,10,73, + 110,100,101,120,69,114,114,111,114,114,134,0,0,0,41,5, + 114,30,0,0,0,114,11,0,0,0,114,50,0,0,0,114, + 115,0,0,0,114,116,0,0,0,114,9,0,0,0,114,9, + 0,0,0,114,10,0,0,0,218,20,95,103,101,116,95,109, + 116,105,109,101,95,111,102,95,115,111,117,114,99,101,106,2, + 0,0,115,18,0,0,0,0,1,2,2,20,1,12,1,10, + 3,8,1,8,1,12,1,20,1,114,157,0,0,0,99,2, + 0,0,0,0,0,0,0,12,0,0,0,9,0,0,0,67, + 0,0,0,115,204,0,0,0,116,0,124,0,124,1,131,2, + 125,2,116,1,68,0,93,166,92,3,125,3,125,4,125,5, + 124,2,124,3,23,0,125,6,116,2,106,3,100,1,124,0, + 106,4,116,5,124,6,100,2,100,3,141,5,1,0,122,14, + 124,0,106,6,124,6,25,0,125,7,87,0,110,20,4,0, + 116,7,107,10,114,88,1,0,1,0,1,0,89,0,113,14, + 88,0,124,7,100,4,25,0,125,8,116,8,124,0,106,4, + 124,7,131,2,125,9,124,4,114,138,116,9,124,0,124,6, + 131,2,125,10,116,10,124,8,124,9,124,10,131,3,125,11, + 110,10,116,11,124,8,124,9,131,2,125,11,124,11,100,0, + 107,8,114,158,113,14,124,7,100,4,25,0,125,8,124,11, + 124,5,124,8,102,3,2,0,1,0,83,0,113,14,116,12, + 100,5,124,1,155,2,157,2,124,1,100,6,141,2,130,1, + 100,0,83,0,41,7,78,122,13,116,114,121,105,110,103,32, + 123,125,123,125,123,125,114,78,0,0,0,41,1,90,9,118, + 101,114,98,111,115,105,116,121,114,0,0,0,0,122,18,99, + 97,110,39,116,32,102,105,110,100,32,109,111,100,117,108,101, + 32,41,1,114,53,0,0,0,41,13,114,34,0,0,0,114, + 80,0,0,0,114,68,0,0,0,114,69,0,0,0,114,27, + 0,0,0,114,18,0,0,0,114,26,0,0,0,114,24,0, + 0,0,114,48,0,0,0,114,157,0,0,0,114,137,0,0, + 0,114,143,0,0,0,114,3,0,0,0,41,12,114,30,0, + 0,0,114,36,0,0,0,114,11,0,0,0,114,81,0,0, + 0,114,82,0,0,0,114,43,0,0,0,114,55,0,0,0, + 114,50,0,0,0,114,38,0,0,0,114,135,0,0,0,114, + 136,0,0,0,114,42,0,0,0,114,9,0,0,0,114,9, + 0,0,0,114,10,0,0,0,114,41,0,0,0,122,2,0, + 0,115,38,0,0,0,0,1,10,1,14,1,8,1,22,1, + 2,1,14,1,14,1,6,2,8,1,12,1,4,1,10,1, + 14,2,10,1,8,3,2,1,8,1,16,2,114,41,0,0, + 0,99,0,0,0,0,0,0,0,0,0,0,0,0,2,0, + 0,0,64,0,0,0,115,60,0,0,0,101,0,90,1,100, + 0,90,2,100,1,90,3,100,2,90,4,100,3,100,4,132, + 0,90,5,100,5,100,6,132,0,90,6,100,7,100,8,132, + 0,90,7,100,9,100,10,132,0,90,8,100,11,100,12,132, + 0,90,9,100,13,83,0,41,14,114,72,0,0,0,122,165, + 80,114,105,118,97,116,101,32,99,108,97,115,115,32,117,115, + 101,100,32,116,111,32,115,117,112,112,111,114,116,32,90,105, + 112,73,109,112,111,114,116,46,103,101,116,95,114,101,115,111, + 117,114,99,101,95,114,101,97,100,101,114,40,41,46,10,10, + 32,32,32,32,84,104,105,115,32,99,108,97,115,115,32,105, + 115,32,97,108,108,111,119,101,100,32,116,111,32,114,101,102, + 101,114,101,110,99,101,32,97,108,108,32,116,104,101,32,105, + 110,110,97,114,100,115,32,97,110,100,32,112,114,105,118,97, + 116,101,32,112,97,114,116,115,32,111,102,10,32,32,32,32, + 116,104,101,32,122,105,112,105,109,112,111,114,116,101,114,46, + 10,32,32,32,32,70,99,3,0,0,0,0,0,0,0,3, + 0,0,0,2,0,0,0,67,0,0,0,115,16,0,0,0, + 124,1,124,0,95,0,124,2,124,0,95,1,100,0,83,0, + 41,1,78,41,2,114,4,0,0,0,114,36,0,0,0,41, + 3,114,30,0,0,0,114,4,0,0,0,114,36,0,0,0, + 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,114, + 32,0,0,0,157,2,0,0,115,4,0,0,0,0,1,6, + 1,122,33,95,90,105,112,73,109,112,111,114,116,82,101,115, + 111,117,114,99,101,82,101,97,100,101,114,46,95,95,105,110, + 105,116,95,95,99,2,0,0,0,0,0,0,0,5,0,0, + 0,8,0,0,0,67,0,0,0,115,92,0,0,0,124,0, + 106,0,160,1,100,1,100,2,161,2,125,2,124,2,155,0, + 100,2,124,1,155,0,157,3,125,3,100,3,100,4,108,2, + 109,3,125,4,1,0,122,18,124,4,124,0,106,4,160,5, + 124,3,161,1,131,1,87,0,83,0,4,0,116,6,107,10, + 114,86,1,0,1,0,1,0,116,7,124,3,131,1,130,1, + 89,0,110,2,88,0,100,0,83,0,41,5,78,114,77,0, + 0,0,114,100,0,0,0,114,0,0,0,0,41,1,218,7, + 66,121,116,101,115,73,79,41,8,114,36,0,0,0,114,17, + 0,0,0,90,2,105,111,114,158,0,0,0,114,4,0,0, + 0,114,51,0,0,0,114,20,0,0,0,218,17,70,105,108, + 101,78,111,116,70,111,117,110,100,69,114,114,111,114,41,5, + 114,30,0,0,0,218,8,114,101,115,111,117,114,99,101,218, + 16,102,117,108,108,110,97,109,101,95,97,115,95,112,97,116, + 104,114,11,0,0,0,114,158,0,0,0,114,9,0,0,0, + 114,9,0,0,0,114,10,0,0,0,218,13,111,112,101,110, + 95,114,101,115,111,117,114,99,101,161,2,0,0,115,14,0, + 0,0,0,1,14,1,14,1,12,1,2,1,18,1,14,1, + 122,38,95,90,105,112,73,109,112,111,114,116,82,101,115,111, + 117,114,99,101,82,101,97,100,101,114,46,111,112,101,110,95, + 114,101,115,111,117,114,99,101,99,2,0,0,0,0,0,0, + 0,2,0,0,0,1,0,0,0,67,0,0,0,115,8,0, + 0,0,116,0,130,1,100,0,83,0,41,1,78,41,1,114, + 159,0,0,0,41,2,114,30,0,0,0,114,160,0,0,0, + 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,218, + 13,114,101,115,111,117,114,99,101,95,112,97,116,104,170,2, + 0,0,115,2,0,0,0,0,4,122,38,95,90,105,112,73, + 109,112,111,114,116,82,101,115,111,117,114,99,101,82,101,97, + 100,101,114,46,114,101,115,111,117,114,99,101,95,112,97,116, + 104,99,2,0,0,0,0,0,0,0,4,0,0,0,8,0, + 0,0,67,0,0,0,115,72,0,0,0,124,0,106,0,160, + 1,100,1,100,2,161,2,125,2,124,2,155,0,100,2,124, + 1,155,0,157,3,125,3,122,16,124,0,106,2,160,3,124, + 3,161,1,1,0,87,0,110,22,4,0,116,4,107,10,114, + 66,1,0,1,0,1,0,89,0,100,3,83,0,88,0,100, + 4,83,0,41,5,78,114,77,0,0,0,114,100,0,0,0, + 70,84,41,5,114,36,0,0,0,114,17,0,0,0,114,4, + 0,0,0,114,51,0,0,0,114,20,0,0,0,41,4,114, + 30,0,0,0,114,53,0,0,0,114,161,0,0,0,114,11, 0,0,0,114,9,0,0,0,114,9,0,0,0,114,10,0, - 0,0,218,15,95,117,110,109,97,114,115,104,97,108,95,99, - 111,100,101,39,2,0,0,115,40,0,0,0,0,1,12,1, - 8,2,18,1,12,1,4,2,16,1,8,5,10,1,6,255, - 2,1,8,255,2,2,6,1,30,1,12,1,4,4,18,1, - 10,1,16,1,114,135,0,0,0,99,1,0,0,0,0,0, - 0,0,1,0,0,0,4,0,0,0,67,0,0,0,115,28, - 0,0,0,124,0,160,0,100,1,100,2,161,2,125,0,124, - 0,160,0,100,3,100,2,161,2,125,0,124,0,83,0,41, - 4,78,115,2,0,0,0,13,10,243,1,0,0,0,10,243, - 1,0,0,0,13,41,1,114,17,0,0,0,41,1,218,6, - 115,111,117,114,99,101,114,9,0,0,0,114,9,0,0,0, - 114,10,0,0,0,218,23,95,110,111,114,109,97,108,105,122, - 101,95,108,105,110,101,95,101,110,100,105,110,103,115,72,2, - 0,0,115,6,0,0,0,0,1,12,1,12,1,114,139,0, - 0,0,99,2,0,0,0,0,0,0,0,2,0,0,0,6, - 0,0,0,67,0,0,0,115,24,0,0,0,116,0,124,1, - 131,1,125,1,116,1,124,1,124,0,100,1,100,2,100,3, - 141,4,83,0,41,4,78,114,66,0,0,0,84,41,1,90, - 12,100,111,110,116,95,105,110,104,101,114,105,116,41,2,114, - 139,0,0,0,218,7,99,111,109,112,105,108,101,41,2,114, - 49,0,0,0,114,138,0,0,0,114,9,0,0,0,114,9, - 0,0,0,114,10,0,0,0,218,15,95,99,111,109,112,105, - 108,101,95,115,111,117,114,99,101,79,2,0,0,115,4,0, - 0,0,0,1,8,1,114,141,0,0,0,99,2,0,0,0, - 0,0,0,0,2,0,0,0,11,0,0,0,67,0,0,0, - 115,68,0,0,0,116,0,160,1,124,0,100,1,63,0,100, - 2,23,0,124,0,100,3,63,0,100,4,64,0,124,0,100, - 5,64,0,124,1,100,6,63,0,124,1,100,3,63,0,100, - 7,64,0,124,1,100,5,64,0,100,8,20,0,100,9,100, - 9,100,9,102,9,161,1,83,0,41,10,78,233,9,0,0, - 0,105,188,7,0,0,233,5,0,0,0,233,15,0,0,0, - 233,31,0,0,0,233,11,0,0,0,233,63,0,0,0,114, - 76,0,0,0,114,12,0,0,0,41,2,114,113,0,0,0, - 90,6,109,107,116,105,109,101,41,2,218,1,100,114,121,0, - 0,0,114,9,0,0,0,114,9,0,0,0,114,10,0,0, - 0,218,14,95,112,97,114,115,101,95,100,111,115,116,105,109, - 101,85,2,0,0,115,24,0,0,0,0,1,4,1,10,1, - 10,1,6,1,6,1,10,1,10,1,2,0,2,0,2,250, - 2,255,114,149,0,0,0,99,2,0,0,0,0,0,0,0, - 5,0,0,0,10,0,0,0,67,0,0,0,115,104,0,0, - 0,122,70,124,1,100,1,100,0,133,2,25,0,100,2,107, - 6,115,22,116,0,130,1,124,1,100,0,100,1,133,2,25, - 0,125,1,124,0,106,1,124,1,25,0,125,2,124,2,100, - 3,25,0,125,3,124,2,100,4,25,0,125,4,116,2,124, - 4,124,3,131,2,87,0,83,0,4,0,116,3,116,4,116, - 5,102,3,107,10,114,98,1,0,1,0,1,0,89,0,100, - 5,83,0,88,0,100,0,83,0,41,6,78,114,12,0,0, - 0,41,2,218,1,99,218,1,111,114,143,0,0,0,233,6, - 0,0,0,114,0,0,0,0,41,6,218,14,65,115,115,101, - 114,116,105,111,110,69,114,114,111,114,114,26,0,0,0,114, - 149,0,0,0,114,24,0,0,0,218,10,73,110,100,101,120, - 69,114,114,111,114,114,132,0,0,0,41,5,114,30,0,0, - 0,114,11,0,0,0,114,50,0,0,0,114,113,0,0,0, - 114,114,0,0,0,114,9,0,0,0,114,9,0,0,0,114, - 10,0,0,0,218,20,95,103,101,116,95,109,116,105,109,101, - 95,111,102,95,115,111,117,114,99,101,98,2,0,0,115,18, - 0,0,0,0,1,2,2,20,1,12,1,10,3,8,1,8, - 1,12,1,20,1,114,155,0,0,0,99,2,0,0,0,0, - 0,0,0,12,0,0,0,9,0,0,0,67,0,0,0,115, - 204,0,0,0,116,0,124,0,124,1,131,2,125,2,116,1, - 68,0,93,166,92,3,125,3,125,4,125,5,124,2,124,3, - 23,0,125,6,116,2,106,3,100,1,124,0,106,4,116,5, - 124,6,100,2,100,3,141,5,1,0,122,14,124,0,106,6, - 124,6,25,0,125,7,87,0,110,20,4,0,116,7,107,10, - 114,88,1,0,1,0,1,0,89,0,113,14,88,0,124,7, - 100,4,25,0,125,8,116,8,124,0,106,4,124,7,131,2, - 125,9,124,4,114,138,116,9,124,0,124,6,131,2,125,10, - 116,10,124,8,124,9,124,10,131,3,125,11,110,10,116,11, - 124,8,124,9,131,2,125,11,124,11,100,0,107,8,114,158, - 113,14,124,7,100,4,25,0,125,8,124,11,124,5,124,8, - 102,3,2,0,1,0,83,0,113,14,116,12,100,5,124,1, - 155,2,157,2,124,1,100,6,141,2,130,1,100,0,83,0, - 41,7,78,122,13,116,114,121,105,110,103,32,123,125,123,125, - 123,125,114,76,0,0,0,41,1,90,9,118,101,114,98,111, - 115,105,116,121,114,0,0,0,0,122,18,99,97,110,39,116, - 32,102,105,110,100,32,109,111,100,117,108,101,32,41,1,114, - 53,0,0,0,41,13,114,34,0,0,0,114,78,0,0,0, - 114,68,0,0,0,114,69,0,0,0,114,27,0,0,0,114, - 18,0,0,0,114,26,0,0,0,114,24,0,0,0,114,48, - 0,0,0,114,155,0,0,0,114,135,0,0,0,114,141,0, - 0,0,114,3,0,0,0,41,12,114,30,0,0,0,114,36, - 0,0,0,114,11,0,0,0,114,79,0,0,0,114,80,0, - 0,0,114,43,0,0,0,114,55,0,0,0,114,50,0,0, - 0,114,38,0,0,0,114,133,0,0,0,114,134,0,0,0, - 114,42,0,0,0,114,9,0,0,0,114,9,0,0,0,114, - 10,0,0,0,114,41,0,0,0,114,2,0,0,115,38,0, - 0,0,0,1,10,1,14,1,8,1,22,1,2,1,14,1, - 14,1,6,2,8,1,12,1,4,1,10,1,14,2,10,1, - 8,3,2,1,8,1,16,2,114,41,0,0,0,41,40,114, - 74,0,0,0,90,26,95,102,114,111,122,101,110,95,105,109, - 112,111,114,116,108,105,98,95,101,120,116,101,114,110,97,108, - 114,19,0,0,0,114,1,0,0,0,114,2,0,0,0,90, - 17,95,102,114,111,122,101,110,95,105,109,112,111,114,116,108, - 105,98,114,68,0,0,0,114,129,0,0,0,114,99,0,0, - 0,114,130,0,0,0,114,59,0,0,0,114,113,0,0,0, - 90,7,95,95,97,108,108,95,95,114,18,0,0,0,90,15, - 112,97,116,104,95,115,101,112,97,114,97,116,111,114,115,114, - 16,0,0,0,114,67,0,0,0,114,3,0,0,0,114,23, - 0,0,0,218,4,116,121,112,101,114,62,0,0,0,114,4, - 0,0,0,114,78,0,0,0,114,34,0,0,0,114,35,0, - 0,0,114,33,0,0,0,114,25,0,0,0,114,106,0,0, - 0,114,123,0,0,0,114,125,0,0,0,114,48,0,0,0, - 114,128,0,0,0,114,135,0,0,0,218,8,95,95,99,111, - 100,101,95,95,114,131,0,0,0,114,139,0,0,0,114,141, - 0,0,0,114,149,0,0,0,114,155,0,0,0,114,41,0, - 0,0,114,9,0,0,0,114,9,0,0,0,114,9,0,0, - 0,114,10,0,0,0,218,8,60,109,111,100,117,108,101,62, - 13,0,0,0,115,78,0,0,0,4,4,8,1,16,1,8, - 1,8,1,8,1,8,1,8,1,8,2,8,3,6,1,14, - 3,16,4,4,2,8,3,14,127,0,120,12,1,12,1,2, - 1,2,253,2,255,2,9,8,4,8,9,8,31,8,103,2, - 254,2,29,4,5,8,21,8,46,8,8,8,28,10,5,8, - 7,8,6,8,13,8,16, + 0,0,218,11,105,115,95,114,101,115,111,117,114,99,101,176, + 2,0,0,115,14,0,0,0,0,3,14,1,14,1,2,1, + 16,1,14,1,8,1,122,36,95,90,105,112,73,109,112,111, + 114,116,82,101,115,111,117,114,99,101,82,101,97,100,101,114, + 46,105,115,95,114,101,115,111,117,114,99,101,99,1,0,0, + 0,0,0,0,0,9,0,0,0,9,0,0,0,99,0,0, + 0,115,186,0,0,0,100,1,100,2,108,0,109,1,125,1, + 1,0,124,1,124,0,106,2,160,3,124,0,106,4,161,1, + 131,1,125,2,124,2,160,5,124,0,106,2,106,6,161,1, + 125,3,124,3,106,7,100,3,107,2,115,58,116,8,130,1, + 124,3,106,9,125,4,116,10,131,0,125,5,124,0,106,2, + 106,11,68,0,93,102,125,6,122,18,124,1,124,6,131,1, + 160,5,124,4,161,1,125,7,87,0,110,24,4,0,116,12, + 107,10,114,124,1,0,1,0,1,0,89,0,113,78,89,0, + 110,2,88,0,124,7,106,9,106,7,125,8,116,13,124,8, + 131,1,100,1,107,2,114,156,124,7,106,7,86,0,1,0, + 113,78,124,8,124,5,107,7,114,78,124,5,160,14,124,8, + 161,1,1,0,124,8,86,0,1,0,113,78,100,0,83,0, + 41,4,78,114,0,0,0,0,41,1,218,4,80,97,116,104, + 122,11,95,95,105,110,105,116,95,95,46,112,121,41,15,90, + 7,112,97,116,104,108,105,98,114,165,0,0,0,114,4,0, + 0,0,114,52,0,0,0,114,36,0,0,0,90,11,114,101, + 108,97,116,105,118,101,95,116,111,114,27,0,0,0,114,53, + 0,0,0,114,155,0,0,0,90,6,112,97,114,101,110,116, + 218,3,115,101,116,114,26,0,0,0,114,21,0,0,0,114, + 47,0,0,0,218,3,97,100,100,41,9,114,30,0,0,0, + 114,165,0,0,0,90,13,102,117,108,108,110,97,109,101,95, + 112,97,116,104,90,13,114,101,108,97,116,105,118,101,95,112, + 97,116,104,90,12,112,97,99,107,97,103,101,95,112,97,116, + 104,90,12,115,117,98,100,105,114,115,95,115,101,101,110,218, + 8,102,105,108,101,110,97,109,101,90,8,114,101,108,97,116, + 105,118,101,90,11,112,97,114,101,110,116,95,110,97,109,101, + 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,218, + 8,99,111,110,116,101,110,116,115,187,2,0,0,115,34,0, + 0,0,0,8,12,1,18,1,14,3,14,1,6,1,6,1, + 12,1,2,1,18,1,14,1,10,5,8,1,12,1,10,1, + 8,1,10,1,122,33,95,90,105,112,73,109,112,111,114,116, + 82,101,115,111,117,114,99,101,82,101,97,100,101,114,46,99, + 111,110,116,101,110,116,115,78,41,10,114,6,0,0,0,114, + 7,0,0,0,114,8,0,0,0,114,76,0,0,0,114,73, + 0,0,0,114,32,0,0,0,114,162,0,0,0,114,163,0, + 0,0,114,164,0,0,0,114,169,0,0,0,114,9,0,0, + 0,114,9,0,0,0,114,9,0,0,0,114,10,0,0,0, + 114,72,0,0,0,149,2,0,0,115,14,0,0,0,8,5, + 4,1,4,2,8,4,8,9,8,6,8,11,114,72,0,0, + 0,41,41,114,76,0,0,0,90,26,95,102,114,111,122,101, + 110,95,105,109,112,111,114,116,108,105,98,95,101,120,116,101, + 114,110,97,108,114,19,0,0,0,114,1,0,0,0,114,2, + 0,0,0,90,17,95,102,114,111,122,101,110,95,105,109,112, + 111,114,116,108,105,98,114,68,0,0,0,114,131,0,0,0, + 114,101,0,0,0,114,132,0,0,0,114,59,0,0,0,114, + 115,0,0,0,90,7,95,95,97,108,108,95,95,114,18,0, + 0,0,90,15,112,97,116,104,95,115,101,112,97,114,97,116, + 111,114,115,114,16,0,0,0,114,67,0,0,0,114,3,0, + 0,0,114,23,0,0,0,218,4,116,121,112,101,114,62,0, + 0,0,114,4,0,0,0,114,80,0,0,0,114,34,0,0, + 0,114,35,0,0,0,114,33,0,0,0,114,25,0,0,0, + 114,108,0,0,0,114,125,0,0,0,114,127,0,0,0,114, + 48,0,0,0,114,130,0,0,0,114,137,0,0,0,218,8, + 95,95,99,111,100,101,95,95,114,133,0,0,0,114,141,0, + 0,0,114,143,0,0,0,114,151,0,0,0,114,157,0,0, + 0,114,41,0,0,0,114,72,0,0,0,114,9,0,0,0, + 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,218, + 8,60,109,111,100,117,108,101,62,13,0,0,0,115,82,0, + 0,0,4,4,8,1,16,1,8,1,8,1,8,1,8,1, + 8,1,8,2,8,3,6,1,14,3,16,4,4,2,8,3, + 14,127,0,127,0,1,12,1,12,1,2,1,2,253,2,255, + 2,9,8,4,8,9,8,31,8,103,2,254,2,29,4,5, + 8,21,8,46,8,8,8,28,10,5,8,7,8,6,8,13, + 8,16,8,27, }; From webhook-mailer at python.org Wed Sep 19 02:49:08 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 19 Sep 2018 06:49:08 -0000 Subject: [Python-checkins] run autoconf (GH-9411) Message-ID: https://github.com/python/cpython/commit/b3b8cb419e496629873fa7dda82a01863f58617a commit: b3b8cb419e496629873fa7dda82a01863f58617a branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-18T23:49:05-07:00 summary: run autoconf (GH-9411) Follow up to 2a9c3805ddedf282881ef7811a561c70b74f80b1 (bpo-34585). files: M aclocal.m4 M configure M pyconfig.h.in diff --git a/aclocal.m4 b/aclocal.m4 index 6a24d8e6b9c0..030e6877de9f 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -288,4 +288,5 @@ AS_VAR_COPY([$1], [pkg_cv_][$1]) AS_VAR_IF([$1], [""], [$5], [$4])dnl ])dnl PKG_CHECK_VAR +m4_include([m4/ax_c_float_words_bigendian.m4]) m4_include([m4/ax_check_openssl.m4]) diff --git a/configure b/configure index 7b0c734b5e25..38546d6ca7b4 100755 --- a/configure +++ b/configure @@ -13853,131 +13853,77 @@ fi # * Check for various properties of floating point * # ************************************************** -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether C doubles are little-endian IEEE 754 binary64" >&5 -$as_echo_n "checking whether C doubles are little-endian IEEE 754 binary64... " >&6; } -if ${ac_cv_little_endian_double+:} false; then : +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether float word ordering is bigendian" >&5 +$as_echo_n "checking whether float word ordering is bigendian... " >&6; } +if ${ax_cv_c_float_words_bigendian+:} false; then : $as_echo_n "(cached) " >&6 else -if test "$cross_compiling" = yes; then : - ac_cv_little_endian_double=no -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext + +ax_cv_c_float_words_bigendian=unknown +cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ -#include -int main() { - double x = 9006104071832581.0; - if (memcmp(&x, "\x05\x04\x03\x02\x01\xff\x3f\x43", 8) == 0) - return 0; - else - return 1; -} -_ACEOF -if ac_fn_c_try_run "$LINENO"; then : - ac_cv_little_endian_double=yes -else - ac_cv_little_endian_double=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ - conftest.$ac_objext conftest.beam conftest.$ac_ext -fi +double d = 90904234967036810337470478905505011476211692735615632014797120844053488865816695273723469097858056257517020191247487429516932130503560650002327564517570778480236724525140520121371739201496540132640109977779420565776568942592.0; -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_little_endian_double" >&5 -$as_echo "$ac_cv_little_endian_double" >&6; } -if test "$ac_cv_little_endian_double" = yes -then +_ACEOF +if ac_fn_c_try_compile "$LINENO"; then : -$as_echo "#define DOUBLE_IS_LITTLE_ENDIAN_IEEE754 1" >>confdefs.h +if grep noonsees conftest.$ac_objext >/dev/null ; then + ax_cv_c_float_words_bigendian=yes +fi +if grep seesnoon conftest.$ac_objext >/dev/null ; then + if test "$ax_cv_c_float_words_bigendian" = unknown; then + ax_cv_c_float_words_bigendian=no + else + ax_cv_c_float_words_bigendian=unknown + fi fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether C doubles are big-endian IEEE 754 binary64" >&5 -$as_echo_n "checking whether C doubles are big-endian IEEE 754 binary64... " >&6; } -if ${ac_cv_big_endian_double+:} false; then : - $as_echo_n "(cached) " >&6 -else - -if test "$cross_compiling" = yes; then : - ac_cv_big_endian_double=no -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -#include -int main() { - double x = 9006104071832581.0; - if (memcmp(&x, "\x43\x3f\xff\x01\x02\x03\x04\x05", 8) == 0) - return 0; - else - return 1; -} -_ACEOF -if ac_fn_c_try_run "$LINENO"; then : - ac_cv_big_endian_double=yes -else - ac_cv_big_endian_double=no fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ - conftest.$ac_objext conftest.beam conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ax_cv_c_float_words_bigendian" >&5 +$as_echo "$ax_cv_c_float_words_bigendian" >&6; } -fi +case $ax_cv_c_float_words_bigendian in + yes) -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_big_endian_double" >&5 -$as_echo "$ac_cv_big_endian_double" >&6; } -if test "$ac_cv_big_endian_double" = yes -then +$as_echo "#define FLOAT_WORDS_BIGENDIAN 1" >>confdefs.h + ;; + no) + ;; + *) + as_fn_error $? " -$as_echo "#define DOUBLE_IS_BIG_ENDIAN_IEEE754 1" >>confdefs.h +Unknown float word ordering. You need to manually preset +ax_cv_c_float_words_bigendian=no (or yes) according to your system. -fi + " "$LINENO" 5 ;; +esac -# Some ARM platforms use a mixed-endian representation for doubles. -# While Python doesn't currently have full support for these platforms -# (see e.g., issue 1762561), we can at least make sure that float <-> string -# conversions work. -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether C doubles are ARM mixed-endian IEEE 754 binary64" >&5 -$as_echo_n "checking whether C doubles are ARM mixed-endian IEEE 754 binary64... " >&6; } -if ${ac_cv_mixed_endian_double+:} false; then : - $as_echo_n "(cached) " >&6 -else -if test "$cross_compiling" = yes; then : - ac_cv_mixed_endian_double=no -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ +if test "$ax_cv_c_float_words_bigendian" = "yes" +then -#include -int main() { - double x = 9006104071832581.0; - if (memcmp(&x, "\x01\xff\x3f\x43\x05\x04\x03\x02", 8) == 0) - return 0; - else - return 1; -} +$as_echo "#define DOUBLE_IS_BIG_ENDIAN_IEEE754 1" >>confdefs.h -_ACEOF -if ac_fn_c_try_run "$LINENO"; then : - ac_cv_mixed_endian_double=yes -else - ac_cv_mixed_endian_double=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ - conftest.$ac_objext conftest.beam conftest.$ac_ext -fi +elif test "$ax_cv_c_float_words_bigendian" = "no" +then -fi +$as_echo "#define DOUBLE_IS_LITTLE_ENDIAN_IEEE754 1" >>confdefs.h -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_mixed_endian_double" >&5 -$as_echo "$ac_cv_mixed_endian_double" >&6; } -if test "$ac_cv_mixed_endian_double" = yes -then +else + # Some ARM platforms use a mixed-endian representation for doubles. + # While Python doesn't currently have full support for these platforms + # (see e.g., issue 1762561), we can at least make sure that float <-> string + # conversions work. + # FLOAT_WORDS_BIGENDIAN doesnt actually detect this case, but if it's not big + # or little, then it must be this? $as_echo "#define DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754 1" >>confdefs.h diff --git a/pyconfig.h.in b/pyconfig.h.in index 360f79994faf..41e0479cad2e 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -30,6 +30,10 @@ /* Define if --enable-ipv6 is specified */ #undef ENABLE_IPV6 +/* Define to 1 if your system stores words within floats with the most + significant word first */ +#undef FLOAT_WORDS_BIGENDIAN + /* Define if flock needs to be linked with bsd library. */ #undef FLOCK_NEEDS_LIBBSD From solipsis at pitrou.net Wed Sep 19 05:17:12 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 19 Sep 2018 09:17:12 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=71 Message-ID: <20180919091712.1.D5293E9666D205EF@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [0, -7, 8] memory blocks, sum=1 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_spawn leaked [44, 0, 0] references, sum=44 test_multiprocessing_spawn leaked [20, 0, 0] memory blocks, sum=20 test_multiprocessing_spawn leaked [2, 0, 0] file descriptors, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogxzJ_Ve', '--timeout', '7200'] From webhook-mailer at python.org Wed Sep 19 06:06:32 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 19 Sep 2018 10:06:32 -0000 Subject: [Python-checkins] bpo-34125: Enable profiling of method_descriptor in all cases (GH-8416) Message-ID: https://github.com/python/cpython/commit/e89de7398718f6e68848b6340830aeb90b7d582c commit: e89de7398718f6e68848b6340830aeb90b7d582c branch: master author: jdemeyer committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-19T03:06:20-07:00 summary: bpo-34125: Enable profiling of method_descriptor in all cases (GH-8416) `list.append([], None)` was profiled but `list.append([], None, **{})` was not profiled. Enable profiling for later case. https://bugs.python.org/issue34125 files: A Misc/NEWS.d/next/Core and Builtins/2018-07-23-16-34-03.bpo-34125.jCl2Q2.rst M Lib/test/test_sys_setprofile.py M Python/ceval.c diff --git a/Lib/test/test_sys_setprofile.py b/Lib/test/test_sys_setprofile.py index 16467e7f7184..c2ecf8eeed9f 100644 --- a/Lib/test/test_sys_setprofile.py +++ b/Lib/test/test_sys_setprofile.py @@ -350,6 +350,24 @@ def f(p): self.check_events(f, [(1, 'call', f_ident), (1, 'return', f_ident)]) + # Test an invalid call (bpo-34125) + def test_unbound_method_no_args(self): + kwargs = {} + def f(p): + dict.get(**kwargs) + f_ident = ident(f) + self.check_events(f, [(1, 'call', f_ident), + (1, 'return', f_ident)]) + + # Test an invalid call (bpo-34125) + def test_unbound_method_invalid_args(self): + kwargs = {} + def f(p): + dict.get(print, 42, **kwargs) + f_ident = ident(f) + self.check_events(f, [(1, 'call', f_ident), + (1, 'return', f_ident)]) + def ident(function): if hasattr(function, "f_code"): diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-07-23-16-34-03.bpo-34125.jCl2Q2.rst b/Misc/NEWS.d/next/Core and Builtins/2018-07-23-16-34-03.bpo-34125.jCl2Q2.rst new file mode 100644 index 000000000000..e6036b4d41e5 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-07-23-16-34-03.bpo-34125.jCl2Q2.rst @@ -0,0 +1 @@ +Profiling of unbound built-in methods now works when ``**kwargs`` is given. diff --git a/Python/ceval.c b/Python/ceval.c index 4a82bd3198af..d0f9915b4f0a 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4642,15 +4642,39 @@ call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *kwnames) static PyObject * do_call_core(PyObject *func, PyObject *callargs, PyObject *kwdict) { + PyObject *result; + if (PyCFunction_Check(func)) { - PyObject *result; PyThreadState *tstate = PyThreadState_GET(); C_TRACE(result, PyCFunction_Call(func, callargs, kwdict)); return result; } - else { - return PyObject_Call(func, callargs, kwdict); + else if (Py_TYPE(func) == &PyMethodDescr_Type) { + PyThreadState *tstate = PyThreadState_GET(); + Py_ssize_t nargs = PyTuple_GET_SIZE(callargs); + if (nargs > 0 && tstate->use_tracing) { + /* We need to create a temporary bound method as argument + for profiling. + + If nargs == 0, then this cannot work because we have no + "self". In any case, the call itself would raise + TypeError (foo needs an argument), so we just skip + profiling. */ + PyObject *self = PyTuple_GET_ITEM(callargs, 0); + func = Py_TYPE(func)->tp_descr_get(func, self, (PyObject*)Py_TYPE(self)); + if (func == NULL) { + return NULL; + } + + C_TRACE(result, _PyCFunction_FastCallDict(func, + &PyTuple_GET_ITEM(callargs, 1), + nargs - 1, + kwdict)); + Py_DECREF(func); + return result; + } } + return PyObject_Call(func, callargs, kwdict); } /* Extract a slice index from a PyLong or an object with the From webhook-mailer at python.org Wed Sep 19 06:28:31 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 19 Sep 2018 10:28:31 -0000 Subject: [Python-checkins] bpo-34712: Fix style in examples in "Input and Output" (GH-9361) Message-ID: https://github.com/python/cpython/commit/3705b9862025705ea60041a9e310f99a164db722 commit: 3705b9862025705ea60041a9e310f99a164db722 branch: master author: Ben Hoyt committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-19T03:28:28-07:00 summary: bpo-34712: Fix style in examples in "Input and Output" (GH-9361) A couple of fixes here to make this more PEP-8: * Avoid multiple statements on one line with `;` statement separator -- this is very rare in Python and is "generally discouraged" in PEP 8 (and if used, per PEP 8 there shouldn't be a space before the `;`) * Add output for the first "Formatted String Literals" example. (Side note: are the doctests for this being run? If so, why didn't it fail?) * Avoid space before `!r`. I have generally not seen spaces before the `!`, and this also matches the style used in the docs here: https://docs.python.org/3/library/string.html#format-string-syntax https://bugs.python.org/issue34712 files: M Doc/tutorial/inputoutput.rst diff --git a/Doc/tutorial/inputoutput.rst b/Doc/tutorial/inputoutput.rst index a92c26681596..785de29ac9d8 100644 --- a/Doc/tutorial/inputoutput.rst +++ b/Doc/tutorial/inputoutput.rst @@ -29,7 +29,8 @@ printing space-separated values. There are several ways to format output. :: - >>> year = 2016 ; event = 'Referendum' + >>> year = 2016 + >>> event = 'Referendum' >>> f'Results of the {year} {event}' 'Results of the 2016 Referendum' @@ -40,8 +41,9 @@ printing space-separated values. There are several ways to format output. :: - >>> yes_votes = 42_572_654 ; no_votes = 43_132_495 - >>> percentage = yes_votes/(yes_votes+no_votes) + >>> yes_votes = 42_572_654 + >>> no_votes = 43_132_495 + >>> percentage = yes_votes / (yes_votes + no_votes) >>> '{:-9} YES votes {:2.2%}'.format(yes_votes, percentage) ' 42572654 YES votes 49.67%' @@ -108,6 +110,7 @@ three places after the decimal:: >>> import math >>> print(f'The value of pi is approximately {math.pi:.3f}.') + The value of pi is approximately 3.142. Passing an integer after the ``':'`` will cause that field to be a minimum number of characters wide. This is useful for making columns line up. :: @@ -127,7 +130,7 @@ applies :func:`repr`:: >>> animals = 'eels' >>> print(f'My hovercraft is full of {animals}.') My hovercraft is full of eels. - >>> print(f'My hovercraft is full of {animals !r}.') + >>> print(f'My hovercraft is full of {animals!r}.') My hovercraft is full of 'eels'. For a reference on these format specifications, see From webhook-mailer at python.org Wed Sep 19 10:43:39 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Wed, 19 Sep 2018 14:43:39 -0000 Subject: [Python-checkins] bpo-25711: Remove outdated zipimport tests. (GH-9404) Message-ID: https://github.com/python/cpython/commit/b2984ab9a7c458f8b7ed8978c0c95b109116895d commit: b2984ab9a7c458f8b7ed8978c0c95b109116895d branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-19T17:43:33+03:00 summary: bpo-25711: Remove outdated zipimport tests. (GH-9404) They were specific to the C implementation. files: M Lib/test/test_zipimport.py diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py index cad73ea3f146..63e567254cd9 100644 --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -539,28 +539,6 @@ def testGetData(self): z.close() os.remove(TEMP_ZIP) - def test_issue31291(self): - # There shouldn't be an assertion failure in get_data(). - class FunnyStr(str): - def replace(self, old, new): - return 42 - z = ZipFile(TEMP_ZIP, "w") - try: - name = "test31291.dat" - data = b'foo' - z.writestr(name, data) - z.close() - zi = zipimport.zipimporter(TEMP_ZIP) - try: - data2 = zi.get_data(FunnyStr(name)) - except AttributeError: - pass - else: - self.assertEqual(data2, data) - finally: - z.close() - os.remove(TEMP_ZIP) - def testImporterAttr(self): src = """if 1: # indent hack def get_file(): @@ -687,38 +665,11 @@ def testBytesPath(self): with self.assertRaises(TypeError): zipimport.zipimporter(memoryview(os.fsencode(filename))) - @support.cpython_only - def testUninitializedZipimporter(self): - # The interpreter shouldn't crash in case of calling methods of an - # uninitialized zipimport.zipimporter object. - zi = zipimport.zipimporter.__new__(zipimport.zipimporter) - self.assertRaises((ValueError, AttributeError), zi.find_module, 'foo') - self.assertRaises((ValueError, AttributeError), zi.find_loader, 'foo') - self.assertRaises((ValueError, AttributeError), zi.load_module, 'foo') - self.assertRaises((ValueError, AttributeError), zi.get_filename, 'foo') - self.assertRaises((ValueError, AttributeError), zi.is_package, 'foo') - self.assertRaises((ValueError, AttributeError), zi.get_data, 'foo') - self.assertRaises((ValueError, AttributeError), zi.get_code, 'foo') - self.assertRaises((ValueError, AttributeError), zi.get_source, 'foo') - @support.requires_zlib class CompressedZipImportTestCase(UncompressedZipImportTestCase): compression = ZIP_DEFLATED - @support.cpython_only - def test_issue31602(self): - # There shouldn't be an assertion failure in zipimporter.get_source() - # in case of a bad zlib.decompress(). - def bad_decompress(*args): - return None - with ZipFile(TEMP_ZIP, 'w') as zip_file: - self.addCleanup(support.unlink, TEMP_ZIP) - zip_file.writestr('bar.py', b'print("hello world")', ZIP_DEFLATED) - zi = zipimport.zipimporter(TEMP_ZIP) - with support.swap_attr(zlib, 'decompress', bad_decompress): - self.assertRaises((TypeError, AttributeError), zi.get_source, 'bar') - class BadFileZipImportTestCase(unittest.TestCase): def assertZipFailure(self, filename): From webhook-mailer at python.org Wed Sep 19 15:01:56 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 19 Sep 2018 19:01:56 -0000 Subject: [Python-checkins] Revert "[3.7] bpo-34589: Add -X coerce_c_locale option; C locale coercion off by default (GH-9379)" (GH-9416) Message-ID: https://github.com/python/cpython/commit/95cc3ee00cfa079751ae2bb9a8d3387053b50489 commit: 95cc3ee00cfa079751ae2bb9a8d3387053b50489 branch: 3.7 author: Victor Stinner committer: GitHub date: 2018-09-19T12:01:52-07:00 summary: Revert "[3.7] bpo-34589: Add -X coerce_c_locale option; C locale coercion off by default (GH-9379)" (GH-9416) This reverts commit 144f1e2c6f4a24bd288c045986842c65cc289684. files: D Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst D Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst M Doc/using/cmdline.rst M Doc/whatsnew/3.7.rst M Include/pylifecycle.h M Include/pystate.h M Lib/test/test_c_locale_coercion.py M Lib/test/test_cmd_line.py M Lib/test/test_embed.py M Lib/test/test_sys.py M Lib/test/test_utf8_mode.py M Modules/main.c M Programs/_testembed.c M Programs/python.c M Python/pylifecycle.c diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 29de155f2d1e..70e8d1afb42a 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -438,19 +438,10 @@ Miscellaneous options * Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to ``True`` - * ``-X utf8`` enables UTF-8 mode (:pep:`540`) for operating system interfaces, overriding + * ``-X utf8`` enables UTF-8 mode for operating system interfaces, overriding the default locale-aware mode. ``-X utf8=0`` explicitly disables UTF-8 mode (even when it would otherwise activate automatically). See :envvar:`PYTHONUTF8` for more details. - * ``-X coerce_c_locale`` or ``-X coerce_c_locale=1`` tries to coerce the C - locale (:pep:`538`). - ``-X coerce_c_locale=0`` skips coercing the legacy ASCII-based C and POSIX - locales to a more capable UTF-8 based alternative. - ``-X coerce_c_locale=warn`` will cause Python to emit warning messages on - ``stderr`` if either the locale coercion activates, or else if a locale - that *would* have triggered coercion is still active when the Python - runtime is initialized. - See :envvar:`PYTHONCOERCECLOCALE` for more details. It also allows passing arbitrary values and retrieving them through the :data:`sys._xoptions` dictionary. @@ -470,9 +461,6 @@ Miscellaneous options .. versionadded:: 3.7 The ``-X importtime``, ``-X dev`` and ``-X utf8`` options. - .. versionadded:: 3.7.1 - The ``-X coerce_c_locale`` option. - Options you shouldn't use ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -846,8 +834,6 @@ conflict. order to force the interpreter to use ``ASCII`` instead of ``UTF-8`` for system interfaces. - Also available as the :option:`-X` ``coerce_c_locale`` option. - Availability: \*nix .. versionadded:: 3.7 diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 6cd9d46a42b0..f53a0268738a 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2494,10 +2494,3 @@ versions, it respected an ill-defined subset of those environment variables, while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before calling :c:func:`Py_Initialize`. - -:c:func:`Py_Initialize` and :c:func:`Py_Main` cannot enable the C locale -coercion (:pep:`538`) anymore: it is always disabled. It can now only be -enabled by the Python program ("python3). - -New :option:`-X` ``coerce_c_locale`` command line option to control C locale -coercion (:pep:`538`). diff --git a/Include/pylifecycle.h b/Include/pylifecycle.h index 68fc036479fd..119296194934 100644 --- a/Include/pylifecycle.h +++ b/Include/pylifecycle.h @@ -119,11 +119,7 @@ PyAPI_FUNC(int) Py_FdIsInteractive(FILE *, const char *); /* Bootstrap __main__ (defined in Modules/main.c) */ PyAPI_FUNC(int) Py_Main(int argc, wchar_t **argv); #ifdef Py_BUILD_CORE -# ifdef MS_WINDOWS -PyAPI_FUNC(int) _Py_WindowsMain(int argc, wchar_t **argv); -# else PyAPI_FUNC(int) _Py_UnixMain(int argc, char **argv); -# endif #endif /* In getpath.c */ diff --git a/Include/pystate.h b/Include/pystate.h index c2ccb203db20..f16ffb8fd2ae 100644 --- a/Include/pystate.h +++ b/Include/pystate.h @@ -41,6 +41,8 @@ typedef struct { int show_alloc_count; /* -X showalloccount */ int dump_refs; /* PYTHONDUMPREFS */ int malloc_stats; /* PYTHONMALLOCSTATS */ + int coerce_c_locale; /* PYTHONCOERCECLOCALE, -1 means unknown */ + int coerce_c_locale_warn; /* PYTHONCOERCECLOCALE=warn */ int utf8_mode; /* PYTHONUTF8, -X utf8; -1 means unknown */ wchar_t *program_name; /* Program name, see also Py_GetProgramName() */ @@ -72,8 +74,6 @@ typedef struct { /* Private fields */ int _disable_importlib; /* Needed by freeze_importlib */ - int _coerce_c_locale; /* PYTHONCOERCECLOCALE, -1 means unknown */ - int _coerce_c_locale_warn; /* PYTHONCOERCECLOCALE=warn */ } _PyCoreConfig; #define _PyCoreConfig_INIT \ @@ -81,8 +81,7 @@ typedef struct { .install_signal_handlers = -1, \ .ignore_environment = -1, \ .use_hash_seed = -1, \ - ._coerce_c_locale = 0, \ - ._coerce_c_locale_warn = 0, \ + .coerce_c_locale = -1, \ .faulthandler = -1, \ .tracemalloc = -1, \ .utf8_mode = -1, \ diff --git a/Lib/test/test_c_locale_coercion.py b/Lib/test/test_c_locale_coercion.py index f62208ab2006..1db293b9c373 100644 --- a/Lib/test/test_c_locale_coercion.py +++ b/Lib/test/test_c_locale_coercion.py @@ -139,7 +139,7 @@ def _handle_output_variations(data): return data @classmethod - def get_child_details(cls, env_vars, xoption=None): + def get_child_details(cls, env_vars): """Retrieves fsencoding and standard stream details from a child process Returns (encoding_details, stderr_lines): @@ -150,11 +150,10 @@ def get_child_details(cls, env_vars, xoption=None): The child is run in isolated mode if the current interpreter supports that. """ - args = [] - if xoption: - args.extend(("-X", f"coerce_c_locale={xoption}")) - args.extend(("-X", "utf8=0", "-c", cls.CHILD_PROCESS_SCRIPT)) - result, py_cmd = run_python_until_end(*args, **env_vars) + result, py_cmd = run_python_until_end( + "-X", "utf8=0", "-c", cls.CHILD_PROCESS_SCRIPT, + **env_vars + ) if not result.rc == 0: result.fail(py_cmd) # All subprocess outputs in this test case should be pure ASCII @@ -213,8 +212,7 @@ def _check_child_encoding_details(self, expected_fs_encoding, expected_stream_encoding, expected_warnings, - coercion_expected, - xoption=None): + coercion_expected): """Check the C locale handling for the given process environment Parameters: @@ -222,7 +220,7 @@ def _check_child_encoding_details(self, expected_stream_encoding: expected encoding for standard streams expected_warning: stderr output to expect (if any) """ - result = EncodingDetails.get_child_details(env_vars, xoption) + result = EncodingDetails.get_child_details(env_vars) encoding_details, stderr_lines = result expected_details = EncodingDetails.get_expected_details( coercion_expected, @@ -292,7 +290,6 @@ def _check_c_locale_coercion(self, coerce_c_locale, expected_warnings=None, coercion_expected=True, - use_xoption=False, **extra_vars): """Check the C locale handling for various configurations @@ -322,12 +319,8 @@ def _check_c_locale_coercion(self, "PYTHONCOERCECLOCALE": "", } base_var_dict.update(extra_vars) - xoption = None if coerce_c_locale is not None: - if use_xoption: - xoption = coerce_c_locale - else: - base_var_dict["PYTHONCOERCECLOCALE"] = coerce_c_locale + base_var_dict["PYTHONCOERCECLOCALE"] = coerce_c_locale # Check behaviour for the default locale with self.subTest(default_locale=True, @@ -349,8 +342,7 @@ def _check_c_locale_coercion(self, fs_encoding, stream_encoding, _expected_warnings, - _coercion_expected, - xoption=xoption) + _coercion_expected) # Check behaviour for explicitly configured locales for locale_to_set in EXPECTED_C_LOCALE_EQUIVALENTS: @@ -365,8 +357,7 @@ def _check_c_locale_coercion(self, fs_encoding, stream_encoding, expected_warnings, - coercion_expected, - xoption=xoption) + coercion_expected) def test_PYTHONCOERCECLOCALE_not_set(self): # This should coerce to the first available target locale by default @@ -413,32 +404,6 @@ def test_LC_ALL_set_to_C(self): expected_warnings=[LEGACY_LOCALE_WARNING], coercion_expected=False) - def test_xoption_set_to_1(self): - self._check_c_locale_coercion("utf-8", "utf-8", coerce_c_locale="1", - use_xoption=True) - - def test_xoption_set_to_zero(self): - # The setting "0" should result in the locale coercion being disabled - self._check_c_locale_coercion(EXPECTED_C_LOCALE_FS_ENCODING, - EXPECTED_C_LOCALE_STREAM_ENCODING, - coerce_c_locale="0", - coercion_expected=False, - use_xoption=True) - # Setting LC_ALL=C shouldn't make any difference to the behaviour - self._check_c_locale_coercion(EXPECTED_C_LOCALE_FS_ENCODING, - EXPECTED_C_LOCALE_STREAM_ENCODING, - coerce_c_locale="0", - LC_ALL="C", - coercion_expected=False, - use_xoption=True) - - def test_xoption_set_to_warn(self): - # -X coerce_c_locale=warn enables runtime warnings for legacy locales - self._check_c_locale_coercion("utf-8", "utf-8", - coerce_c_locale="warn", - expected_warnings=[CLI_COERCION_WARNING], - use_xoption=True) - def test_main(): test.support.run_unittest( LocaleConfigurationTests, diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index 0424f75b3b8c..95cdc8db7efb 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -159,16 +159,13 @@ def test_undecodable_code(self): env = os.environ.copy() # Use C locale to get ascii for the locale encoding env['LC_ALL'] = 'C' + env['PYTHONCOERCECLOCALE'] = '0' code = ( b'import locale; ' b'print(ascii("' + undecodable + b'"), ' b'locale.getpreferredencoding())') p = subprocess.Popen( - [sys.executable, - # Disable C locale coercion and UTF-8 Mode to not use UTF-8 - "-X", "coerce_c_locale=0", - "-X", "utf8=0", - "-c", code], + [sys.executable, "-c", code], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=env) stdout, stderr = p.communicate() diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 79c34ec591a0..29274580339f 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -267,6 +267,9 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): 'malloc_stats': 0, 'utf8_mode': 0, + 'coerce_c_locale': 0, + 'coerce_c_locale_warn': 0, + 'program_name': './_testembed', 'argc': 0, 'argv': '[]', @@ -287,8 +290,6 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): '_disable_importlib': 0, 'Py_FrozenFlag': 0, - '_coerce_c_locale': 0, - '_coerce_c_locale_warn': 0, } def check_config(self, testname, expected): diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index 9155afc069e1..27f75901c63f 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -656,8 +656,9 @@ def test_getfilesystemencoding(self): def c_locale_get_error_handler(self, locale, isolated=False, encoding=None): # Force the POSIX locale - env = dict(os.environ) + env = os.environ.copy() env["LC_ALL"] = locale + env["PYTHONCOERCECLOCALE"] = "0" code = '\n'.join(( 'import sys', 'def dump(name):', @@ -667,10 +668,7 @@ def c_locale_get_error_handler(self, locale, isolated=False, encoding=None): 'dump("stdout")', 'dump("stderr")', )) - args = [sys.executable, - "-X", "utf8=0", - "-X", "coerce_c_locale=0", - "-c", code] + args = [sys.executable, "-c", code] if isolated: args.append("-I") if encoding is not None: diff --git a/Lib/test/test_utf8_mode.py b/Lib/test/test_utf8_mode.py index 8c64276bcf68..554abfab3163 100644 --- a/Lib/test/test_utf8_mode.py +++ b/Lib/test/test_utf8_mode.py @@ -27,8 +27,6 @@ def posix_locale(self): return (loc in POSIX_LOCALES) def get_output(self, *args, failure=False, **kw): - # Always disable the C locale coercion (PEP 538) - args = ('-X', 'coerce_c_locale=0', *args) kw = dict(self.DEFAULT_ENV, **kw) if failure: out = assert_python_failure(*args, **kw) @@ -118,6 +116,7 @@ def test_filesystemencoding(self): # PYTHONLEGACYWINDOWSFSENCODING disables the UTF-8 mode # and has the priority over -X utf8 and PYTHONUTF8 out = self.get_output('-X', 'utf8', '-c', code, + PYTHONUTF8='strict', PYTHONLEGACYWINDOWSFSENCODING='1') self.assertEqual(out, 'mbcs/replace') diff --git a/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst b/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst deleted file mode 100644 index 27b6a6e0017f..000000000000 --- a/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst +++ /dev/null @@ -1,3 +0,0 @@ -Py_Initialize() and Py_Main() cannot enable the C locale coercion (PEP 538) -anymore: it is always disabled. It can now only be enabled by the Python -program ("python3). diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst deleted file mode 100644 index 618092d192c4..000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst +++ /dev/null @@ -1,2 +0,0 @@ -Add a new :option:`-X` ``coerce_c_locale`` command line option to control C -locale coercion (:pep:`538`). diff --git a/Modules/main.c b/Modules/main.c index c6ffb15d9f99..0f7498d6104f 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -1834,17 +1834,6 @@ config_init_utf8_mode(_PyCoreConfig *config) return _Py_INIT_OK(); } -#ifndef MS_WINDOWS - /* The C locale and the POSIX locale enable the UTF-8 Mode (PEP 540) */ - const char *ctype_loc = setlocale(LC_CTYPE, NULL); - if (ctype_loc != NULL - && (strcmp(ctype_loc, "C") == 0 || strcmp(ctype_loc, "POSIX") == 0)) - { - config->utf8_mode = 1; - return _Py_INIT_OK(); - } -#endif - return _Py_INIT_OK(); } @@ -1868,18 +1857,16 @@ config_read_env_vars(_PyCoreConfig *config) const char *env = config_get_env_var("PYTHONCOERCECLOCALE"); if (env) { if (strcmp(env, "0") == 0) { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 0; + if (config->coerce_c_locale < 0) { + config->coerce_c_locale = 0; } } else if (strcmp(env, "warn") == 0) { - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 1; - } + config->coerce_c_locale_warn = 1; } else { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 1; + if (config->coerce_c_locale < 0) { + config->coerce_c_locale = 1; } } } @@ -2059,7 +2046,7 @@ pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) * See the documentation of the PYTHONCOERCECLOCALE setting for more * details. */ - if (config->_coerce_c_locale && !locale_coerced) { + if (config->coerce_c_locale && !locale_coerced) { locale_coerced = 1; _Py_CoerceLegacyLocale(config); encoding_changed = 1; @@ -2086,7 +2073,7 @@ pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) pymain_read_conf_impl(). Reset Py_IsolatedFlag and Py_NoSiteFlag modified by _PyCoreConfig_Read(). */ int new_utf8_mode = config->utf8_mode; - int new_coerce_c_locale = config->_coerce_c_locale; + int new_coerce_c_locale = config->coerce_c_locale; Py_IgnoreEnvironmentFlag = init_ignore_env; if (_PyCoreConfig_Copy(config, &save_config) < 0) { pymain->err = _Py_INIT_NO_MEMORY(); @@ -2098,7 +2085,7 @@ pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) cmdline_get_global_config(cmdline); _PyCoreConfig_GetGlobalConfig(config); config->utf8_mode = new_utf8_mode; - config->_coerce_c_locale = new_coerce_c_locale; + config->coerce_c_locale = new_coerce_c_locale; /* The encoding changed: read again the configuration with the new encoding */ @@ -2116,76 +2103,28 @@ pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) } -static _PyInitError -config_init_coerce_c_locale(_PyCoreConfig *config) +static void +config_init_locale(_PyCoreConfig *config) { - const wchar_t *xopt = config_get_xoption(config, L"coerce_c_locale"); - if (xopt) { - wchar_t *sep = wcschr(xopt, L'='); - if (sep) { - xopt = sep + 1; - if (wcscmp(xopt, L"1") == 0) { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 1; - } - } - else if (wcscmp(xopt, L"0") == 0) { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 0; - } - } - else if (wcscmp(xopt, L"warn") == 0) { - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 1; - } - } - else { - return _Py_INIT_USER_ERR("invalid -X coerce_c_locale option value"); - } - } - else { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 1; - } - } - - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 0; - } - } - - const char *env = config_get_env_var("PYTHONCOERCECLOCALE"); - if (env) { - if (strcmp(env, "0") == 0) { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 0; - } - } - else if (strcmp(env, "warn") == 0) { - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 1; - } - } - else { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 1; - } - } - - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 0; - } - } - - if (config->_coerce_c_locale < 0) { + if (config->coerce_c_locale < 0) { /* The C locale enables the C locale coercion (PEP 538) */ if (_Py_LegacyLocaleDetected()) { - config->_coerce_c_locale = 1; - return _Py_INIT_OK(); + config->coerce_c_locale = 1; } } - return _Py_INIT_OK(); +#ifndef MS_WINDOWS + if (config->utf8_mode < 0) { + /* The C locale and the POSIX locale enable the UTF-8 Mode (PEP 540) */ + const char *ctype_loc = setlocale(LC_CTYPE, NULL); + if (ctype_loc != NULL + && (strcmp(ctype_loc, "C") == 0 + || strcmp(ctype_loc, "POSIX") == 0)) + { + config->utf8_mode = 1; + } + } +#endif } @@ -2345,11 +2284,8 @@ _PyCoreConfig_Read(_PyCoreConfig *config) } } - if (config->_coerce_c_locale < 0 || config->_coerce_c_locale_warn < 0) { - err = config_init_coerce_c_locale(config); - if (_Py_INIT_FAILED(err)) { - return err; - } + if (config->utf8_mode < 0 || config->coerce_c_locale < 0) { + config_init_locale(config); } if (!config->_disable_importlib) { @@ -2381,11 +2317,8 @@ _PyCoreConfig_Read(_PyCoreConfig *config) if (config->tracemalloc < 0) { config->tracemalloc = 0; } - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 0; - } - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 0; + if (config->coerce_c_locale < 0) { + config->coerce_c_locale = 0; } if (config->utf8_mode < 0) { config->utf8_mode = 0; @@ -2394,10 +2327,6 @@ _PyCoreConfig_Read(_PyCoreConfig *config) config->argc = 0; } - assert(config->_coerce_c_locale >= 0); - assert(config->_coerce_c_locale_warn >= 0); - assert(config->ignore_environment >= 0); - return _Py_INIT_OK(); } @@ -2481,8 +2410,8 @@ _PyCoreConfig_Copy(_PyCoreConfig *config, const _PyCoreConfig *config2) COPY_ATTR(dump_refs); COPY_ATTR(malloc_stats); - COPY_ATTR(_coerce_c_locale); - COPY_ATTR(_coerce_c_locale_warn); + COPY_ATTR(coerce_c_locale); + COPY_ATTR(coerce_c_locale_warn); COPY_ATTR(utf8_mode); COPY_STR_ATTR(module_search_path_env); @@ -2709,7 +2638,7 @@ pymain_run_python(_PyMain *pymain) static void -pymain_init(_PyMain *pymain, int use_c_locale_coercion) +pymain_init(_PyMain *pymain) { /* 754 requires that FP exceptions run in "no stop" mode by default, * and until C vendors implement C99's ways to control FP exceptions, @@ -2722,11 +2651,6 @@ pymain_init(_PyMain *pymain, int use_c_locale_coercion) pymain->config._disable_importlib = 0; pymain->config.install_signal_handlers = 1; - if (use_c_locale_coercion) { - /* set to -1 to be able to enable the feature */ - pymain->config._coerce_c_locale = -1; - pymain->config._coerce_c_locale_warn = -1; - } } @@ -2827,9 +2751,9 @@ pymain_cmdline(_PyMain *pymain) static int -pymain_main(_PyMain *pymain, int use_c_locale_coercion) +pymain_main(_PyMain *pymain) { - pymain_init(pymain, use_c_locale_coercion); + pymain_init(pymain); int res = pymain_cmdline(pymain); if (res < 0) { @@ -2878,22 +2802,10 @@ Py_Main(int argc, wchar_t **argv) pymain.argc = argc; pymain.wchar_argv = argv; - return pymain_main(&pymain, 0); + return pymain_main(&pymain); } -#ifdef MS_WINDOWS -int -_Py_WindowsMain(int argc, wchar_t **argv) -{ - _PyMain pymain = _PyMain_INIT; - pymain.use_bytes_argv = 0; - pymain.argc = argc; - pymain.wchar_argv = argv; - - return pymain_main(&pymain, 1); -} -#else int _Py_UnixMain(int argc, char **argv) { @@ -2902,9 +2814,8 @@ _Py_UnixMain(int argc, char **argv) pymain.argc = argc; pymain.bytes_argv = argv; - return pymain_main(&pymain, 1); + return pymain_main(&pymain); } -#endif /* this is gonna seem *real weird*, but if you put some other code between diff --git a/Programs/_testembed.c b/Programs/_testembed.c index 029f6ad4b680..6c35f9586bfe 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -328,8 +328,8 @@ dump_config(void) printf("dump_refs = %i\n", config->dump_refs); printf("malloc_stats = %i\n", config->malloc_stats); - printf("_coerce_c_locale = %i\n", config->_coerce_c_locale); - printf("_coerce_c_locale_warn = %i\n", config->_coerce_c_locale_warn); + printf("coerce_c_locale = %i\n", config->coerce_c_locale); + printf("coerce_c_locale_warn = %i\n", config->coerce_c_locale_warn); printf("utf8_mode = %i\n", config->utf8_mode); printf("program_name = %ls\n", config->program_name); @@ -473,6 +473,8 @@ static int test_init_from_config(void) putenv("PYTHONMALLOCSTATS=0"); config.malloc_stats = 1; + /* FIXME: test coerce_c_locale and coerce_c_locale_warn */ + putenv("PYTHONUTF8=0"); Py_UTF8Mode = 0; config.utf8_mode = 1; @@ -542,7 +544,8 @@ static int test_init_isolated(void) /* Test _PyCoreConfig.isolated=1 */ _PyCoreConfig config = _PyCoreConfig_INIT; - /* Set utf8_mode to not depend on the locale */ + /* Set coerce_c_locale and utf8_mode to not depend on the locale */ + config.coerce_c_locale = 0; config.utf8_mode = 0; /* Use path starting with "./" avoids a search along the PATH */ config.program_name = L"./_testembed"; diff --git a/Programs/python.c b/Programs/python.c index c7697facbe3b..78e48f800c95 100644 --- a/Programs/python.c +++ b/Programs/python.c @@ -6,7 +6,7 @@ int wmain(int argc, wchar_t **argv) { - return _Py_WindowsMain(argc, argv); + return Py_Main(argc, argv); } #else int diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index ef742c16c014..ba4b54864fd8 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -400,7 +400,7 @@ static const char *_C_LOCALE_WARNING = static void _emit_stderr_warning_for_legacy_locale(const _PyCoreConfig *core_config) { - if (core_config->_coerce_c_locale_warn) { + if (core_config->coerce_c_locale_warn) { if (_Py_LegacyLocaleDetected()) { fprintf(stderr, "%s", _C_LOCALE_WARNING); } @@ -462,7 +462,7 @@ _coerce_default_locale_settings(const _PyCoreConfig *config, const _LocaleCoerci "Error setting LC_CTYPE, skipping C locale coercion\n"); return; } - if (config->_coerce_c_locale_warn) { + if (config->coerce_c_locale_warn) { fprintf(stderr, C_LOCALE_COERCION_WARNING, newloc); } From webhook-mailer at python.org Wed Sep 19 16:23:37 2018 From: webhook-mailer at python.org (Steve Dower) Date: Wed, 19 Sep 2018 20:23:37 -0000 Subject: [Python-checkins] Ignores failure to update lists (GH-9424) Message-ID: https://github.com/python/cpython/commit/76531e2e82319a487d659bc469441bd4b8251608 commit: 76531e2e82319a487d659bc469441bd4b8251608 branch: master author: Steve Dower committer: GitHub date: 2018-09-19T13:23:25-07:00 summary: Ignores failure to update lists (GH-9424) files: M .vsts/install_deps.sh diff --git a/.vsts/install_deps.sh b/.vsts/install_deps.sh index 7b98cfddb6c4..b1fa576047e6 100755 --- a/.vsts/install_deps.sh +++ b/.vsts/install_deps.sh @@ -1,4 +1,4 @@ -sudo apt-get update +sudo apt-get update || true sudo apt-get -yq install \ build-essential \ From webhook-mailer at python.org Wed Sep 19 16:30:47 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 19 Sep 2018 20:30:47 -0000 Subject: [Python-checkins] bpo-34712: Fix style in examples in "Input and Output" (GH-9361) Message-ID: https://github.com/python/cpython/commit/d9c89111bd82979ce8716f7ca68c69cf0f46e679 commit: d9c89111bd82979ce8716f7ca68c69cf0f46e679 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-19T13:30:40-07:00 summary: bpo-34712: Fix style in examples in "Input and Output" (GH-9361) A couple of fixes here to make this more PEP-8: * Avoid multiple statements on one line with `;` statement separator -- this is very rare in Python and is "generally discouraged" in PEP 8 (and if used, per PEP 8 there shouldn't be a space before the `;`) * Add output for the first "Formatted String Literals" example. (Side note: are the doctests for this being run? If so, why didn't it fail?) * Avoid space before `!r`. I have generally not seen spaces before the `!`, and this also matches the style used in the docs here: https://docs.python.org/3/library/string.htmlGH-format-string-syntax https://bugs.python.org/issue34712 (cherry picked from commit 3705b9862025705ea60041a9e310f99a164db722) Co-authored-by: Ben Hoyt files: M Doc/tutorial/inputoutput.rst diff --git a/Doc/tutorial/inputoutput.rst b/Doc/tutorial/inputoutput.rst index a92c26681596..785de29ac9d8 100644 --- a/Doc/tutorial/inputoutput.rst +++ b/Doc/tutorial/inputoutput.rst @@ -29,7 +29,8 @@ printing space-separated values. There are several ways to format output. :: - >>> year = 2016 ; event = 'Referendum' + >>> year = 2016 + >>> event = 'Referendum' >>> f'Results of the {year} {event}' 'Results of the 2016 Referendum' @@ -40,8 +41,9 @@ printing space-separated values. There are several ways to format output. :: - >>> yes_votes = 42_572_654 ; no_votes = 43_132_495 - >>> percentage = yes_votes/(yes_votes+no_votes) + >>> yes_votes = 42_572_654 + >>> no_votes = 43_132_495 + >>> percentage = yes_votes / (yes_votes + no_votes) >>> '{:-9} YES votes {:2.2%}'.format(yes_votes, percentage) ' 42572654 YES votes 49.67%' @@ -108,6 +110,7 @@ three places after the decimal:: >>> import math >>> print(f'The value of pi is approximately {math.pi:.3f}.') + The value of pi is approximately 3.142. Passing an integer after the ``':'`` will cause that field to be a minimum number of characters wide. This is useful for making columns line up. :: @@ -127,7 +130,7 @@ applies :func:`repr`:: >>> animals = 'eels' >>> print(f'My hovercraft is full of {animals}.') My hovercraft is full of eels. - >>> print(f'My hovercraft is full of {animals !r}.') + >>> print(f'My hovercraft is full of {animals!r}.') My hovercraft is full of 'eels'. For a reference on these format specifications, see From webhook-mailer at python.org Wed Sep 19 16:48:26 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 19 Sep 2018 20:48:26 -0000 Subject: [Python-checkins] [2.7] bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) (GH-9425) Message-ID: https://github.com/python/cpython/commit/b63a16febbd1c943c9dbc5c651326b410aa50698 commit: b63a16febbd1c943c9dbc5c651326b410aa50698 branch: 2.7 author: Vladimir Matveev committer: Victor Stinner date: 2018-09-19T13:48:21-07:00 summary: [2.7] bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) (GH-9425) Co-authored-by: Vladimir Matveev files: A Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst M Lib/ctypes/test/test_win32.py M Modules/_ctypes/_ctypes_test.c M Modules/_ctypes/callproc.c M Modules/_ctypes/libffi_msvc/ffi.c M Modules/_ctypes/libffi_msvc/ffi.h M Modules/_ctypes/libffi_msvc/prep_cif.c diff --git a/Lib/ctypes/test/test_win32.py b/Lib/ctypes/test/test_win32.py index d22e139a3f97..13a986359fa4 100644 --- a/Lib/ctypes/test/test_win32.py +++ b/Lib/ctypes/test/test_win32.py @@ -52,6 +52,24 @@ def test_noargs(self): # This is a special case on win32 x64 windll.user32.GetDesktopWindow() + at unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') +class ReturnStructSizesTestCase(unittest.TestCase): + def test_sizes(self): + dll = CDLL(_ctypes_test.__file__) + for i in range(1, 11): + fields = [ ("f%d" % f, c_char) for f in range(1, i + 1)] + class S(Structure): + _fields_ = fields + f = getattr(dll, "TestSize%d" % i) + f.restype = S + res = f() + for i, f in enumerate(fields): + value = getattr(res, f[0]) + expected = chr(ord('a') + i) + self.assertEquals(value, expected) + + + @unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') class TestWintypes(unittest.TestCase): def test_HWND(self): diff --git a/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst b/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst new file mode 100644 index 000000000000..86ae1cd06171 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-13-08-29-04.bpo-34603.2AB7sc.rst @@ -0,0 +1 @@ +Fix returning structs from functions produced by MSVC diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c index 94678f3189ce..93876d3d01a6 100644 --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -655,6 +655,200 @@ EXPORT(void) TwoOutArgs(int a, int *pi, int b, int *pj) *pj += b; } +#ifdef MS_WIN32 + +typedef struct { + char f1; +} Size1; + +typedef struct { + char f1; + char f2; +} Size2; + +typedef struct { + char f1; + char f2; + char f3; +} Size3; + +typedef struct { + char f1; + char f2; + char f3; + char f4; +} Size4; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; +} Size5; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; +} Size6; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; +} Size7; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; +} Size8; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; + char f9; +} Size9; + +typedef struct { + char f1; + char f2; + char f3; + char f4; + char f5; + char f6; + char f7; + char f8; + char f9; + char f10; +} Size10; + +EXPORT(Size1) TestSize1() { + Size1 f; + f.f1 = 'a'; + return f; +} + +EXPORT(Size2) TestSize2() { + Size2 f; + f.f1 = 'a'; + f.f2 = 'b'; + return f; +} + +EXPORT(Size3) TestSize3() { + Size3 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + return f; +} + +EXPORT(Size4) TestSize4() { + Size4 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + return f; +} + +EXPORT(Size5) TestSize5() { + Size5 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + return f; +} + +EXPORT(Size6) TestSize6() { + Size6 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + return f; +} + +EXPORT(Size7) TestSize7() { + Size7 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + return f; +} + +EXPORT(Size8) TestSize8() { + Size8 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + return f; +} + +EXPORT(Size9) TestSize9() { + Size9 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + f.f9 = 'i'; + return f; +} + +EXPORT(Size10) TestSize10() { + Size10 f; + f.f1 = 'a'; + f.f2 = 'b'; + f.f3 = 'c'; + f.f4 = 'd'; + f.f5 = 'e'; + f.f6 = 'f'; + f.f7 = 'g'; + f.f8 = 'h'; + f.f9 = 'i'; + f.f10 = 'j'; + return f; +} + +#endif + #ifdef MS_WIN32 EXPORT(S2H) __stdcall s_ret_2h_func(S2H inp) { return ret_2h_func(inp); } EXPORT(S8I) __stdcall s_ret_8i_func(S8I inp) { return ret_8i_func(inp); } diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index 91233d508c57..63f3c21784d5 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -747,9 +747,9 @@ ffi_type *_ctypes_get_ffi_type(PyObject *obj) It returns small structures in registers */ if (dict->ffi_type_pointer.type == FFI_TYPE_STRUCT) { - if (dict->ffi_type_pointer.size <= 4) + if (can_return_struct_as_int(dict->ffi_type_pointer.size)) return &ffi_type_sint32; - else if (dict->ffi_type_pointer.size <= 8) + else if (can_return_struct_as_sint64 (dict->ffi_type_pointer.size)) return &ffi_type_sint64; } #endif diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c index 587c94b7e61f..15a100fe3467 100644 --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -126,6 +126,21 @@ void ffi_prep_args(char *stack, extended_cif *ecif) return; } +/* +Per: https://msdn.microsoft.com/en-us/library/7572ztz4.aspx +To be returned by value in RAX, user-defined types must have a length +of 1, 2, 4, 8, 16, 32, or 64 bits +*/ +int can_return_struct_as_int(size_t s) +{ + return s == 1 || s == 2 || s == 4; +} + +int can_return_struct_as_sint64(size_t s) +{ + return s == 8; +} + /* Perform machine dependent cif processing */ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) { @@ -144,9 +159,9 @@ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) /* MSVC returns small structures in registers. Put in cif->flags the value FFI_TYPE_STRUCT only if the structure is big enough; otherwise, put the 4- or 8-bytes integer type. */ - if (cif->rtype->size <= 4) + if (can_return_struct_as_int(cif->rtype->size)) cif->flags = FFI_TYPE_INT; - else if (cif->rtype->size <= 8) + else if (can_return_struct_as_sint64(cif->rtype->size)) cif->flags = FFI_TYPE_SINT64; else cif->flags = FFI_TYPE_STRUCT; diff --git a/Modules/_ctypes/libffi_msvc/ffi.h b/Modules/_ctypes/libffi_msvc/ffi.h index efb14c5f6f3a..ba74202720a6 100644 --- a/Modules/_ctypes/libffi_msvc/ffi.h +++ b/Modules/_ctypes/libffi_msvc/ffi.h @@ -136,6 +136,9 @@ typedef struct _ffi_type /*@null@*/ struct _ffi_type **elements; } ffi_type; +int can_return_struct_as_int(size_t); +int can_return_struct_as_sint64(size_t); + /* These are defined in types.c */ extern ffi_type ffi_type_void; extern ffi_type ffi_type_uint8; diff --git a/Modules/_ctypes/libffi_msvc/prep_cif.c b/Modules/_ctypes/libffi_msvc/prep_cif.c index b28f895a1e46..80c5a608188d 100644 --- a/Modules/_ctypes/libffi_msvc/prep_cif.c +++ b/Modules/_ctypes/libffi_msvc/prep_cif.c @@ -117,7 +117,8 @@ ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, /* Make space for the return structure pointer */ if (cif->rtype->type == FFI_TYPE_STRUCT #ifdef _WIN32 - && (cif->rtype->size > 8) /* MSVC returns small structs in registers */ + && !can_return_struct_as_int(cif->rtype->size) /* MSVC returns small structs in registers */ + && !can_return_struct_as_sint64(cif->rtype->size) #endif #ifdef SPARC && (cif->abi != FFI_V9 || cif->rtype->size > 32) @@ -146,7 +147,9 @@ ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, bytes += sizeof(void*); else #elif defined (_WIN64) - if ((*ptr)->type == FFI_TYPE_STRUCT && ((*ptr)->size > 8)) + if ((*ptr)->type == FFI_TYPE_STRUCT && + !can_return_struct_as_int((*ptr)->size) && + !can_return_struct_as_sint64((*ptr)->size)) bytes += sizeof(void*); else #endif From webhook-mailer at python.org Wed Sep 19 17:34:18 2018 From: webhook-mailer at python.org (Steve Dower) Date: Wed, 19 Sep 2018 21:34:18 -0000 Subject: [Python-checkins] Ignores failure to update lists (GH-9424) Message-ID: https://github.com/python/cpython/commit/cccbb9b23db56f32638ef192d73e72e13ddc63b7 commit: cccbb9b23db56f32638ef192d73e72e13ddc63b7 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Steve Dower date: 2018-09-19T14:34:14-07:00 summary: Ignores failure to update lists (GH-9424) (cherry picked from commit 76531e2e82319a487d659bc469441bd4b8251608) Co-authored-by: Steve Dower files: M .vsts/install_deps.sh diff --git a/.vsts/install_deps.sh b/.vsts/install_deps.sh index 7b98cfddb6c4..b1fa576047e6 100755 --- a/.vsts/install_deps.sh +++ b/.vsts/install_deps.sh @@ -1,4 +1,4 @@ -sudo apt-get update +sudo apt-get update || true sudo apt-get -yq install \ build-essential \ From webhook-mailer at python.org Wed Sep 19 17:34:32 2018 From: webhook-mailer at python.org (Steve Dower) Date: Wed, 19 Sep 2018 21:34:32 -0000 Subject: [Python-checkins] Ignores failure to update lists (GH-9424) Message-ID: https://github.com/python/cpython/commit/71bc694f380fded55ac15aafaff41a12f329a90e commit: 71bc694f380fded55ac15aafaff41a12f329a90e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Steve Dower date: 2018-09-19T14:34:29-07:00 summary: Ignores failure to update lists (GH-9424) (cherry picked from commit 76531e2e82319a487d659bc469441bd4b8251608) Co-authored-by: Steve Dower files: M .vsts/install_deps.sh diff --git a/.vsts/install_deps.sh b/.vsts/install_deps.sh index 7b98cfddb6c4..b1fa576047e6 100755 --- a/.vsts/install_deps.sh +++ b/.vsts/install_deps.sh @@ -1,4 +1,4 @@ -sudo apt-get update +sudo apt-get update || true sudo apt-get -yq install \ build-essential \ From webhook-mailer at python.org Wed Sep 19 17:51:21 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Wed, 19 Sep 2018 21:51:21 -0000 Subject: [Python-checkins] bpo-34733: Return of the docs search bar (GH-9431) Message-ID: https://github.com/python/cpython/commit/581890cda36f60cd46185c6e184abe35a95813a2 commit: 581890cda36f60cd46185c6e184abe35a95813a2 branch: 3.7 author: Yury Selivanov committer: GitHub date: 2018-09-19T17:51:17-04:00 summary: bpo-34733: Return of the docs search bar (GH-9431) Partially revert changes to Doc/tools/templates/layout.html accidentally committed in 512d7101098b971837cbb406942215244f636547. files: M Doc/tools/templates/layout.html diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index c39922456140..8cf903dec6ef 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -1,27 +1,106 @@ {% extends "!layout.html" %} - {% block rootrellink %} -{{ super() }} -
  • - {%- if switchers is defined %} - {{ language or 'en' }} - {{ release }} - {% trans %}Documentation {% endtrans %}{{ reldelim1 }} - {%- else %} - {{ shorttitle }}{{ reldelim1 }} - {%- endif %} +
    • +
  • Python{{ reldelim1 }}
    • +
  • + {%- if switchers is defined %} + {{ language or 'en' }} + {{ release }} + {% trans %}Documentation {% endtrans %}{{ reldelim1 }} + {%- else %} + {{ shorttitle }}{{ reldelim1 }} + {%- endif %} +
    • +{% endblock %} +{%- macro searchbox() %} +{# modified from sphinx/themes/basic/searchbox.html #} + {%- if builder != "htmlhelp" %} + + + {%- endif %} +{%- endmacro %} +{% block relbar1 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} +{% block relbar2 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} +{% block relbaritems %} + {%- if pagename != "search" and builder != "singlehtml" and builder != "htmlhelp" %} +
  • + {{ searchbox() }} + {{ reldelim2 }}
    • + {%- endif %} {% endblock %} - {% block extrahead %} + {% if builder != "htmlhelp" %} - {% if switchers is defined and not embedded %} - {% endif %} - {% if pagename == 'whatsnew/changelog' and not embedded %} - {% endif %} - {% endif %} + {% if not embedded %}{% endif %} + {% if switchers is defined and not embedded %}{% endif %} + {% if pagename == 'whatsnew/changelog' and not embedded %} + + {% endif %} + {% endif %} {# custom CSS; used in asyncio docs! #} {{ super() }} {% endblock %} +{% block footer %} +
    + © {% trans %}Copyright{% endtrans %} {{ copyright|e }}. +
    + {% trans %}The Python Software Foundation is a non-profit corporation.{% endtrans %} + {% trans %}Please donate.{% endtrans %} +
    + {% trans last_updated=last_updated|e %}Last updated on {{ last_updated }}.{% endtrans %} + {% trans pathto_bugs=pathto('bugs') %}Found a bug?{% endtrans %} +
    + {% trans sphinx_version=sphinx_version|e %}Created using Sphinx {{ sphinx_version }}.{% endtrans %} +
    +{% endblock %} From webhook-mailer at python.org Wed Sep 19 17:56:40 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 19 Sep 2018 21:56:40 -0000 Subject: [Python-checkins] Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)" (GH-9430) Message-ID: https://github.com/python/cpython/commit/06e7608207daab9fb82d13ccf2d3664535442f11 commit: 06e7608207daab9fb82d13ccf2d3664535442f11 branch: master author: Victor Stinner committer: GitHub date: 2018-09-19T14:56:36-07:00 summary: Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)" (GH-9430) * Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)" This reverts commit dbdee0073cf0b88fe541980ace1f650900f455cc. * Revert "bpo-34589: C locale coercion off by default (GH-9073)" This reverts commit 7a0791b6992d420dc52536257f2f093851ed7215. * Revert "bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)" This reverts commit 188ebfa475a6f6aa2d0ea14ca8e1fbe7865b6d27. files: D Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst D Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst M Doc/using/cmdline.rst M Doc/whatsnew/3.7.rst M Include/coreconfig.h M Include/pylifecycle.h M Lib/test/test_c_locale_coercion.py M Lib/test/test_cmd_line.py M Lib/test/test_embed.py M Lib/test/test_sys.py M Lib/test/test_utf8_mode.py M Modules/_testcapimodule.c M Modules/main.c M Programs/_testembed.c M Programs/python.c M Python/coreconfig.c M Python/pylifecycle.c diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index cd3b2410c84d..b61df8a4b77d 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -438,22 +438,13 @@ Miscellaneous options * Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to ``True`` - * ``-X utf8`` enables UTF-8 mode (:pep:`540`) for operating system interfaces, overriding + * ``-X utf8`` enables UTF-8 mode for operating system interfaces, overriding the default locale-aware mode. ``-X utf8=0`` explicitly disables UTF-8 mode (even when it would otherwise activate automatically). See :envvar:`PYTHONUTF8` for more details. * ``-X pycache_prefix=PATH`` enables writing ``.pyc`` files to a parallel tree rooted at the given directory instead of to the code tree. See also :envvar:`PYTHONPYCACHEPREFIX`. - * ``-X coerce_c_locale`` or ``-X coerce_c_locale=1`` tries to coerce the C - locale (:pep:`538`). - ``-X coerce_c_locale=0`` skips coercing the legacy ASCII-based C and POSIX - locales to a more capable UTF-8 based alternative. - ``-X coerce_c_locale=warn`` will cause Python to emit warning messages on - ``stderr`` if either the locale coercion activates, or else if a locale - that *would* have triggered coercion is still active when the Python - runtime is initialized. - See :envvar:`PYTHONCOERCECLOCALE` for more details. It also allows passing arbitrary values and retrieving them through the :data:`sys._xoptions` dictionary. @@ -473,9 +464,6 @@ Miscellaneous options .. versionadded:: 3.7 The ``-X importtime``, ``-X dev`` and ``-X utf8`` options. - .. versionadded:: 3.7.1 - The ``-X coerce_c_locale`` option. - .. versionadded:: 3.8 The ``-X pycache_prefix`` option. @@ -862,8 +850,6 @@ conflict. order to force the interpreter to use ``ASCII`` instead of ``UTF-8`` for system interfaces. - Also available as the :option:`-X` ``coerce_c_locale`` option. - Availability: \*nix .. versionadded:: 3.7 diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 6cd9d46a42b0..f53a0268738a 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2494,10 +2494,3 @@ versions, it respected an ill-defined subset of those environment variables, while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before calling :c:func:`Py_Initialize`. - -:c:func:`Py_Initialize` and :c:func:`Py_Main` cannot enable the C locale -coercion (:pep:`538`) anymore: it is always disabled. It can now only be -enabled by the Python program ("python3). - -New :option:`-X` ``coerce_c_locale`` command line option to control C locale -coercion (:pep:`538`). diff --git a/Include/coreconfig.h b/Include/coreconfig.h index 293d8ed16604..ef043ab02df6 100644 --- a/Include/coreconfig.h +++ b/Include/coreconfig.h @@ -63,6 +63,8 @@ typedef struct { int show_alloc_count; /* -X showalloccount */ int dump_refs; /* PYTHONDUMPREFS */ int malloc_stats; /* PYTHONMALLOCSTATS */ + int coerce_c_locale; /* PYTHONCOERCECLOCALE, -1 means unknown */ + int coerce_c_locale_warn; /* PYTHONCOERCECLOCALE=warn */ /* Python filesystem encoding and error handler: sys.getfilesystemencoding() and sys.getfilesystemencodeerrors(). @@ -295,30 +297,6 @@ typedef struct { If set to -1 (default), inherit Py_FrozenFlag value. */ int _frozen; - /* C locale coercion (PEP 538). - - The option is enabled by the PYTHONCOERCECLOCALE environment - variable. The option is also enabled if the LC_CTYPE locale is "C" - and a target locale (ex: "C.UTF-8") is supported by the platform. - - Py_Initialize() and Py_Main() must not enable C locale coercion: it is - always disabled. The option can only be enabled by the Python program - ("python3). - - See also the _coerce_c_locale_warn option. */ - int _coerce_c_locale; - - /* C locale coercion warning (PEP 538). - - Enabled by the PYTHONCOERCECLOCALE=warn environment variable. - - Py_Initialize() and Py_Main() must not enable C locale coercion warning: - it is always disabled. The warning can only be enabled by the Python - program ("python3). - - See also the _coerce_c_locale option. */ - int _coerce_c_locale_warn; - } _PyCoreConfig; #ifdef MS_WINDOWS @@ -336,8 +314,7 @@ typedef struct { .use_hash_seed = -1, \ .faulthandler = -1, \ .tracemalloc = -1, \ - ._coerce_c_locale = 0, \ - ._coerce_c_locale_warn = 0, \ + .coerce_c_locale = -1, \ .utf8_mode = -1, \ .argc = -1, \ .nmodule_search_path = -1, \ diff --git a/Include/pylifecycle.h b/Include/pylifecycle.h index f64bae3f6170..04e672e96e17 100644 --- a/Include/pylifecycle.h +++ b/Include/pylifecycle.h @@ -83,11 +83,7 @@ PyAPI_FUNC(int) Py_FdIsInteractive(FILE *, const char *); /* Bootstrap __main__ (defined in Modules/main.c) */ PyAPI_FUNC(int) Py_Main(int argc, wchar_t **argv); #ifdef Py_BUILD_CORE -# ifdef MS_WINDOWS -PyAPI_FUNC(int) _Py_WindowsMain(int argc, wchar_t **argv); -# else PyAPI_FUNC(int) _Py_UnixMain(int argc, char **argv); -# endif #endif /* In getpath.c */ diff --git a/Lib/test/test_c_locale_coercion.py b/Lib/test/test_c_locale_coercion.py index f62208ab2006..1db293b9c373 100644 --- a/Lib/test/test_c_locale_coercion.py +++ b/Lib/test/test_c_locale_coercion.py @@ -139,7 +139,7 @@ def _handle_output_variations(data): return data @classmethod - def get_child_details(cls, env_vars, xoption=None): + def get_child_details(cls, env_vars): """Retrieves fsencoding and standard stream details from a child process Returns (encoding_details, stderr_lines): @@ -150,11 +150,10 @@ def get_child_details(cls, env_vars, xoption=None): The child is run in isolated mode if the current interpreter supports that. """ - args = [] - if xoption: - args.extend(("-X", f"coerce_c_locale={xoption}")) - args.extend(("-X", "utf8=0", "-c", cls.CHILD_PROCESS_SCRIPT)) - result, py_cmd = run_python_until_end(*args, **env_vars) + result, py_cmd = run_python_until_end( + "-X", "utf8=0", "-c", cls.CHILD_PROCESS_SCRIPT, + **env_vars + ) if not result.rc == 0: result.fail(py_cmd) # All subprocess outputs in this test case should be pure ASCII @@ -213,8 +212,7 @@ def _check_child_encoding_details(self, expected_fs_encoding, expected_stream_encoding, expected_warnings, - coercion_expected, - xoption=None): + coercion_expected): """Check the C locale handling for the given process environment Parameters: @@ -222,7 +220,7 @@ def _check_child_encoding_details(self, expected_stream_encoding: expected encoding for standard streams expected_warning: stderr output to expect (if any) """ - result = EncodingDetails.get_child_details(env_vars, xoption) + result = EncodingDetails.get_child_details(env_vars) encoding_details, stderr_lines = result expected_details = EncodingDetails.get_expected_details( coercion_expected, @@ -292,7 +290,6 @@ def _check_c_locale_coercion(self, coerce_c_locale, expected_warnings=None, coercion_expected=True, - use_xoption=False, **extra_vars): """Check the C locale handling for various configurations @@ -322,12 +319,8 @@ def _check_c_locale_coercion(self, "PYTHONCOERCECLOCALE": "", } base_var_dict.update(extra_vars) - xoption = None if coerce_c_locale is not None: - if use_xoption: - xoption = coerce_c_locale - else: - base_var_dict["PYTHONCOERCECLOCALE"] = coerce_c_locale + base_var_dict["PYTHONCOERCECLOCALE"] = coerce_c_locale # Check behaviour for the default locale with self.subTest(default_locale=True, @@ -349,8 +342,7 @@ def _check_c_locale_coercion(self, fs_encoding, stream_encoding, _expected_warnings, - _coercion_expected, - xoption=xoption) + _coercion_expected) # Check behaviour for explicitly configured locales for locale_to_set in EXPECTED_C_LOCALE_EQUIVALENTS: @@ -365,8 +357,7 @@ def _check_c_locale_coercion(self, fs_encoding, stream_encoding, expected_warnings, - coercion_expected, - xoption=xoption) + coercion_expected) def test_PYTHONCOERCECLOCALE_not_set(self): # This should coerce to the first available target locale by default @@ -413,32 +404,6 @@ def test_LC_ALL_set_to_C(self): expected_warnings=[LEGACY_LOCALE_WARNING], coercion_expected=False) - def test_xoption_set_to_1(self): - self._check_c_locale_coercion("utf-8", "utf-8", coerce_c_locale="1", - use_xoption=True) - - def test_xoption_set_to_zero(self): - # The setting "0" should result in the locale coercion being disabled - self._check_c_locale_coercion(EXPECTED_C_LOCALE_FS_ENCODING, - EXPECTED_C_LOCALE_STREAM_ENCODING, - coerce_c_locale="0", - coercion_expected=False, - use_xoption=True) - # Setting LC_ALL=C shouldn't make any difference to the behaviour - self._check_c_locale_coercion(EXPECTED_C_LOCALE_FS_ENCODING, - EXPECTED_C_LOCALE_STREAM_ENCODING, - coerce_c_locale="0", - LC_ALL="C", - coercion_expected=False, - use_xoption=True) - - def test_xoption_set_to_warn(self): - # -X coerce_c_locale=warn enables runtime warnings for legacy locales - self._check_c_locale_coercion("utf-8", "utf-8", - coerce_c_locale="warn", - expected_warnings=[CLI_COERCION_WARNING], - use_xoption=True) - def test_main(): test.support.run_unittest( LocaleConfigurationTests, diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index 7e967b20ab88..21511b896cad 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -159,16 +159,13 @@ def test_undecodable_code(self): env = os.environ.copy() # Use C locale to get ascii for the locale encoding env['LC_ALL'] = 'C' + env['PYTHONCOERCECLOCALE'] = '0' code = ( b'import locale; ' b'print(ascii("' + undecodable + b'"), ' b'locale.getpreferredencoding())') p = subprocess.Popen( - [sys.executable, - # Disable C locale coercion and UTF-8 Mode to not use UTF-8 - "-X", "coerce_c_locale=0", - "-X", "utf8=0", - "-c", code], + [sys.executable, "-c", code], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=env) stdout, stderr = p.communicate() diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index e531fd49d5a3..80233a54b0b0 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -277,6 +277,8 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): 'filesystem_errors': None, 'utf8_mode': 0, + 'coerce_c_locale': 0, + 'coerce_c_locale_warn': 0, 'pycache_prefix': NULL_STR, 'program_name': './_testembed', @@ -304,8 +306,6 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): '_install_importlib': 1, '_check_hash_pycs_mode': 'default', '_frozen': 0, - '_coerce_c_locale': 0, - '_coerce_c_locale_warn': 0, } def get_stdio_encoding(self, env): @@ -324,6 +324,10 @@ def get_filesystem_encoding(self, isolated, env): 'print(sys.getfilesystemencoding(), ' 'sys.getfilesystemencodeerrors())') args = (sys.executable, '-c', code) + env = dict(env) + if not isolated: + env['PYTHONCOERCECLOCALE'] = '0' + env['PYTHONUTF8'] = '0' proc = subprocess.run(args, text=True, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE) diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index a7f292827130..b90366d81445 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -656,8 +656,9 @@ def test_getfilesystemencoding(self): def c_locale_get_error_handler(self, locale, isolated=False, encoding=None): # Force the POSIX locale - env = dict(os.environ) + env = os.environ.copy() env["LC_ALL"] = locale + env["PYTHONCOERCECLOCALE"] = "0" code = '\n'.join(( 'import sys', 'def dump(name):', @@ -667,10 +668,7 @@ def c_locale_get_error_handler(self, locale, isolated=False, encoding=None): 'dump("stdout")', 'dump("stderr")', )) - args = [sys.executable, - "-X", "utf8=0", - "-X", "coerce_c_locale=0", - "-c", code] + args = [sys.executable, "-X", "utf8=0", "-c", code] if isolated: args.append("-I") if encoding is not None: diff --git a/Lib/test/test_utf8_mode.py b/Lib/test/test_utf8_mode.py index c3cbb49060e7..7280ce77ef82 100644 --- a/Lib/test/test_utf8_mode.py +++ b/Lib/test/test_utf8_mode.py @@ -27,8 +27,6 @@ def posix_locale(self): return (loc in POSIX_LOCALES) def get_output(self, *args, failure=False, **kw): - # Always disable the C locale coercion (PEP 538) - args = ('-X', 'coerce_c_locale=0', *args) kw = dict(self.DEFAULT_ENV, **kw) if failure: out = assert_python_failure(*args, **kw) @@ -118,6 +116,7 @@ def test_filesystemencoding(self): # PYTHONLEGACYWINDOWSFSENCODING disables the UTF-8 mode # and has the priority over -X utf8 and PYTHONUTF8 out = self.get_output('-X', 'utf8', '-c', code, + PYTHONUTF8='strict', PYTHONLEGACYWINDOWSFSENCODING='1') self.assertEqual(out, 'mbcs/replace') diff --git a/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst b/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst deleted file mode 100644 index 27b6a6e0017f..000000000000 --- a/Misc/NEWS.d/next/C API/2018-09-18-00-09-31.bpo-34589.C7bUpq.rst +++ /dev/null @@ -1,3 +0,0 @@ -Py_Initialize() and Py_Main() cannot enable the C locale coercion (PEP 538) -anymore: it is always disabled. It can now only be enabled by the Python -program ("python3). diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst deleted file mode 100644 index 618092d192c4..000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2018-09-18-01-41-33.bpo-34589.lLVTYc.rst +++ /dev/null @@ -1,2 +0,0 @@ -Add a new :option:`-X` ``coerce_c_locale`` command line option to control C -locale coercion (:pep:`538`). diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index c21a6e305836..add642f223af 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -4701,6 +4701,10 @@ get_coreconfig(PyObject *self, PyObject *Py_UNUSED(args)) PyLong_FromLong(config->dump_refs)); SET_ITEM("malloc_stats", PyLong_FromLong(config->malloc_stats)); + SET_ITEM("coerce_c_locale", + PyLong_FromLong(config->coerce_c_locale)); + SET_ITEM("coerce_c_locale_warn", + PyLong_FromLong(config->coerce_c_locale_warn)); SET_ITEM("filesystem_encoding", FROM_STRING(config->filesystem_encoding)); SET_ITEM("filesystem_errors", @@ -4779,10 +4783,6 @@ get_coreconfig(PyObject *self, PyObject *Py_UNUSED(args)) FROM_STRING(config->_check_hash_pycs_mode)); SET_ITEM("_frozen", PyLong_FromLong(config->_frozen)); - SET_ITEM("_coerce_c_locale", - PyLong_FromLong(config->_coerce_c_locale)); - SET_ITEM("_coerce_c_locale_warn", - PyLong_FromLong(config->_coerce_c_locale_warn)); return dict; diff --git a/Modules/main.c b/Modules/main.c index d4ae4a0ae305..455178af8bab 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -1342,9 +1342,9 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, * See the documentation of the PYTHONCOERCECLOCALE setting for more * details. */ - if (config->_coerce_c_locale && !locale_coerced) { + if (config->coerce_c_locale && !locale_coerced) { locale_coerced = 1; - _Py_CoerceLegacyLocale(config->_coerce_c_locale_warn); + _Py_CoerceLegacyLocale(config->coerce_c_locale_warn); encoding_changed = 1; } @@ -1367,7 +1367,7 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, /* Reset the configuration before reading again the configuration, just keep UTF-8 Mode value. */ int new_utf8_mode = config->utf8_mode; - int new_coerce_c_locale = config->_coerce_c_locale; + int new_coerce_c_locale = config->coerce_c_locale; if (_PyCoreConfig_Copy(config, &save_config) < 0) { pymain->err = _Py_INIT_NO_MEMORY(); goto done; @@ -1375,7 +1375,7 @@ pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, pymain_clear_cmdline(pymain, cmdline); memset(cmdline, 0, sizeof(*cmdline)); config->utf8_mode = new_utf8_mode; - config->_coerce_c_locale = new_coerce_c_locale; + config->coerce_c_locale = new_coerce_c_locale; /* The encoding changed: read again the configuration with the new encoding */ @@ -1700,8 +1700,7 @@ pymain_cmdline(_PyMain *pymain, _PyCoreConfig *config) static int -pymain_init(_PyMain *pymain, PyInterpreterState **interp_p, - int use_c_locale_coercion) +pymain_init(_PyMain *pymain, PyInterpreterState **interp_p) { /* 754 requires that FP exceptions run in "no stop" mode by default, * and until C vendors implement C99's ways to control FP exceptions, @@ -1714,11 +1713,6 @@ pymain_init(_PyMain *pymain, PyInterpreterState **interp_p, _PyCoreConfig local_config = _PyCoreConfig_INIT; _PyCoreConfig *config = &local_config; - if (use_c_locale_coercion) { - /* set to -1 to be able to enable the feature */ - config->_coerce_c_locale = -1; - config->_coerce_c_locale_warn = -1; - } _PyCoreConfig_GetGlobalConfig(config); @@ -1753,10 +1747,10 @@ pymain_init(_PyMain *pymain, PyInterpreterState **interp_p, static int -pymain_main(_PyMain *pymain, int use_c_locale_coercion) +pymain_main(_PyMain *pymain) { PyInterpreterState *interp; - int res = pymain_init(pymain, &interp, use_c_locale_coercion); + int res = pymain_init(pymain, &interp); if (res != 1) { if (pymain_run_python(pymain, interp) < 0) { _Py_FatalInitError(pymain->err); @@ -1783,22 +1777,10 @@ Py_Main(int argc, wchar_t **argv) pymain.argc = argc; pymain.wchar_argv = argv; - return pymain_main(&pymain, 0); + return pymain_main(&pymain); } -#ifdef MS_WINDOWS -int -_Py_WindowsMain(int argc, wchar_t **argv) -{ - _PyMain pymain = _PyMain_INIT; - pymain.use_bytes_argv = 0; - pymain.argc = argc; - pymain.wchar_argv = argv; - - return pymain_main(&pymain, 1); -} -#else int _Py_UnixMain(int argc, char **argv) { @@ -1807,9 +1789,8 @@ _Py_UnixMain(int argc, char **argv) pymain.argc = argc; pymain.bytes_argv = argv; - return pymain_main(&pymain, 1); + return pymain_main(&pymain); } -#endif /* this is gonna seem *real weird*, but if you put some other code between diff --git a/Programs/_testembed.c b/Programs/_testembed.c index b9394690c3fc..99772eacbdc4 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -330,6 +330,8 @@ dump_config(void) printf("filesystem_encoding = %s\n", config->filesystem_encoding); printf("filesystem_errors = %s\n", config->filesystem_errors); + printf("coerce_c_locale = %i\n", config->coerce_c_locale); + printf("coerce_c_locale_warn = %i\n", config->coerce_c_locale_warn); printf("utf8_mode = %i\n", config->utf8_mode); printf("pycache_prefix = %ls\n", config->pycache_prefix); @@ -383,8 +385,6 @@ dump_config(void) printf("_install_importlib = %i\n", config->_install_importlib); printf("_check_hash_pycs_mode = %s\n", config->_check_hash_pycs_mode); printf("_frozen = %i\n", config->_frozen); - printf("_coerce_c_locale = %i\n", config->_coerce_c_locale); - printf("_coerce_c_locale_warn = %i\n", config->_coerce_c_locale_warn); #undef ASSERT_EQUAL #undef ASSERT_STR_EQUAL @@ -482,6 +482,8 @@ static int test_init_from_config(void) putenv("PYTHONMALLOCSTATS=0"); config.malloc_stats = 1; + /* FIXME: test coerce_c_locale and coerce_c_locale_warn */ + putenv("PYTHONUTF8=0"); Py_UTF8Mode = 0; config.utf8_mode = 1; @@ -604,7 +606,8 @@ static int test_init_isolated(void) /* Test _PyCoreConfig.isolated=1 */ _PyCoreConfig config = _PyCoreConfig_INIT; - /* Set utf8_mode to not depend on the locale */ + /* Set coerce_c_locale and utf8_mode to not depend on the locale */ + config.coerce_c_locale = 0; config.utf8_mode = 0; /* Use path starting with "./" avoids a search along the PATH */ config.program_name = L"./_testembed"; diff --git a/Programs/python.c b/Programs/python.c index c7697facbe3b..78e48f800c95 100644 --- a/Programs/python.c +++ b/Programs/python.c @@ -6,7 +6,7 @@ int wmain(int argc, wchar_t **argv) { - return _Py_WindowsMain(argc, argv); + return Py_Main(argc, argv); } #else int diff --git a/Python/coreconfig.c b/Python/coreconfig.c index b2459dca57b0..fae32e533aa9 100644 --- a/Python/coreconfig.c +++ b/Python/coreconfig.c @@ -303,8 +303,8 @@ _PyCoreConfig_Copy(_PyCoreConfig *config, const _PyCoreConfig *config2) COPY_ATTR(dump_refs); COPY_ATTR(malloc_stats); - COPY_ATTR(_coerce_c_locale); - COPY_ATTR(_coerce_c_locale_warn); + COPY_ATTR(coerce_c_locale); + COPY_ATTR(coerce_c_locale_warn); COPY_ATTR(utf8_mode); COPY_WSTR_ATTR(pycache_prefix); @@ -705,17 +705,6 @@ config_init_utf8_mode(_PyCoreConfig *config) return _Py_INIT_OK(); } -#ifndef MS_WINDOWS - /* The C locale and the POSIX locale enable the UTF-8 Mode (PEP 540) */ - const char *ctype_loc = setlocale(LC_CTYPE, NULL); - if (ctype_loc != NULL - && (strcmp(ctype_loc, "C") == 0 || strcmp(ctype_loc, "POSIX") == 0)) - { - config->utf8_mode = 1; - return _Py_INIT_OK(); - } -#endif - return _Py_INIT_OK(); } @@ -819,6 +808,23 @@ config_read_env_vars(_PyCoreConfig *config) config->malloc_stats = 1; } + const char *env = _PyCoreConfig_GetEnv(config, "PYTHONCOERCECLOCALE"); + if (env) { + if (strcmp(env, "0") == 0) { + if (config->coerce_c_locale < 0) { + config->coerce_c_locale = 0; + } + } + else if (strcmp(env, "warn") == 0) { + config->coerce_c_locale_warn = 1; + } + else { + if (config->coerce_c_locale < 0) { + config->coerce_c_locale = 1; + } + } + } + wchar_t *path; int res = _PyCoreConfig_GetEnvDup(config, &path, L"PYTHONPATH", "PYTHONPATH"); @@ -958,76 +964,28 @@ config_read_complex_options(_PyCoreConfig *config) } -static _PyInitError -config_init_coerce_c_locale(_PyCoreConfig *config) +static void +config_init_locale(_PyCoreConfig *config) { - const wchar_t *xopt = config_get_xoption(config, L"coerce_c_locale"); - if (xopt) { - wchar_t *sep = wcschr(xopt, L'='); - if (sep) { - xopt = sep + 1; - if (wcscmp(xopt, L"1") == 0) { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 1; - } - } - else if (wcscmp(xopt, L"0") == 0) { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 0; - } - } - else if (wcscmp(xopt, L"warn") == 0) { - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 1; - } - } - else { - return _Py_INIT_USER_ERR("invalid -X coerce_c_locale option value"); - } - } - else { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 1; - } - } - - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 0; - } - } - - const char *env = _PyCoreConfig_GetEnv(config, "PYTHONCOERCECLOCALE"); - if (env) { - if (strcmp(env, "0") == 0) { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 0; - } - } - else if (strcmp(env, "warn") == 0) { - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 1; - } - } - else { - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 1; - } - } - - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 0; - } - } - - if (config->_coerce_c_locale < 0) { + if (config->coerce_c_locale < 0) { /* The C locale enables the C locale coercion (PEP 538) */ if (_Py_LegacyLocaleDetected()) { - config->_coerce_c_locale = 1; - return _Py_INIT_OK(); + config->coerce_c_locale = 1; } } - return _Py_INIT_OK(); +#ifndef MS_WINDOWS + if (config->utf8_mode < 0) { + /* The C locale and the POSIX locale enable the UTF-8 Mode (PEP 540) */ + const char *ctype_loc = setlocale(LC_CTYPE, NULL); + if (ctype_loc != NULL + && (strcmp(ctype_loc, "C") == 0 + || strcmp(ctype_loc, "POSIX") == 0)) + { + config->utf8_mode = 1; + } + } +#endif } @@ -1333,11 +1291,8 @@ _PyCoreConfig_Read(_PyCoreConfig *config) } } - if (config->_coerce_c_locale < 0 || config->_coerce_c_locale_warn < 0) { - err = config_init_coerce_c_locale(config); - if (_Py_INIT_FAILED(err)) { - return err; - } + if (config->utf8_mode < 0 || config->coerce_c_locale < 0) { + config_init_locale(config); } if (config->_install_importlib) { @@ -1366,11 +1321,8 @@ _PyCoreConfig_Read(_PyCoreConfig *config) if (config->tracemalloc < 0) { config->tracemalloc = 0; } - if (config->_coerce_c_locale < 0) { - config->_coerce_c_locale = 0; - } - if (config->_coerce_c_locale_warn < 0) { - config->_coerce_c_locale_warn = 0; + if (config->coerce_c_locale < 0) { + config->coerce_c_locale = 0; } if (config->utf8_mode < 0) { config->utf8_mode = 0; @@ -1391,8 +1343,7 @@ _PyCoreConfig_Read(_PyCoreConfig *config) return err; } - assert(config->_coerce_c_locale >= 0); - assert(config->_coerce_c_locale_warn >= 0); + assert(config->coerce_c_locale >= 0); assert(config->use_environment >= 0); assert(config->filesystem_encoding != NULL); assert(config->filesystem_errors != NULL); diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 06ea6b4dcd7d..ec39500d4cd3 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -295,7 +295,7 @@ static const char *_C_LOCALE_WARNING = static void _emit_stderr_warning_for_legacy_locale(const _PyCoreConfig *core_config) { - if (core_config->_coerce_c_locale_warn && _Py_LegacyLocaleDetected()) { + if (core_config->coerce_c_locale_warn && _Py_LegacyLocaleDetected()) { PySys_FormatStderr("%s", _C_LOCALE_WARNING); } } From webhook-mailer at python.org Wed Sep 19 19:25:29 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 19 Sep 2018 23:25:29 -0000 Subject: [Python-checkins] Enables test result collection for CI builds (GH-9433) Message-ID: https://github.com/python/cpython/commit/fd54a45f7987d5b3343c74cb49a12bd39bdad5bc commit: fd54a45f7987d5b3343c74cb49a12bd39bdad5bc branch: master author: Steve Dower committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-19T16:25:25-07:00 summary: Enables test result collection for CI builds (GH-9433) files: M .vsts/linux-buildbot.yml M .vsts/macos-buildbot.yml M .vsts/windows-buildbot.yml diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml index 517040048937..417125ae3a46 100644 --- a/.vsts/linux-buildbot.yml +++ b/.vsts/linux-buildbot.yml @@ -46,5 +46,14 @@ steps: - script: make pythoninfo displayName: 'Display build info' -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(build.sourceBranchName)-linux' + platform: linux + condition: succeededOrFailed() diff --git a/.vsts/macos-buildbot.yml b/.vsts/macos-buildbot.yml index f58ea1626144..d9c190c46131 100644 --- a/.vsts/macos-buildbot.yml +++ b/.vsts/macos-buildbot.yml @@ -33,5 +33,14 @@ steps: - script: make pythoninfo displayName: 'Display build info' -- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(build.sourceBranchName)-macOS' + platform: macOS + condition: succeededOrFailed() diff --git a/.vsts/windows-buildbot.yml b/.vsts/windows-buildbot.yml index 5ec4522796ce..15aebeda4077 100644 --- a/.vsts/windows-buildbot.yml +++ b/.vsts/windows-buildbot.yml @@ -43,7 +43,16 @@ steps: - script: python.bat -m test.pythoninfo displayName: 'Display build info' -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" displayName: 'Tests' env: PREFIX: $(Py_OutDir)\$(outDirSuffix) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' + mergeTestResults: true + testRunTitle: '$(Build.SourceBranchName)-$(outDirSuffix)' + platform: $(outDirSuffix) + condition: succeededOrFailed() From webhook-mailer at python.org Wed Sep 19 19:41:14 2018 From: webhook-mailer at python.org (Steve Dower) Date: Wed, 19 Sep 2018 23:41:14 -0000 Subject: [Python-checkins] Enables test result collection for CI builds (GH-9433) Message-ID: https://github.com/python/cpython/commit/9804af27eae24d11c8e284cf936420cd220ba878 commit: 9804af27eae24d11c8e284cf936420cd220ba878 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Steve Dower date: 2018-09-19T16:41:11-07:00 summary: Enables test result collection for CI builds (GH-9433) (cherry picked from commit fd54a45f7987d5b3343c74cb49a12bd39bdad5bc) Co-authored-by: Steve Dower files: M .vsts/linux-buildbot.yml M .vsts/macos-buildbot.yml M .vsts/windows-buildbot.yml diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml index 517040048937..417125ae3a46 100644 --- a/.vsts/linux-buildbot.yml +++ b/.vsts/linux-buildbot.yml @@ -46,5 +46,14 @@ steps: - script: make pythoninfo displayName: 'Display build info' -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(build.sourceBranchName)-linux' + platform: linux + condition: succeededOrFailed() diff --git a/.vsts/macos-buildbot.yml b/.vsts/macos-buildbot.yml index f58ea1626144..d9c190c46131 100644 --- a/.vsts/macos-buildbot.yml +++ b/.vsts/macos-buildbot.yml @@ -33,5 +33,14 @@ steps: - script: make pythoninfo displayName: 'Display build info' -- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(build.sourceBranchName)-macOS' + platform: macOS + condition: succeededOrFailed() diff --git a/.vsts/windows-buildbot.yml b/.vsts/windows-buildbot.yml index 5ec4522796ce..15aebeda4077 100644 --- a/.vsts/windows-buildbot.yml +++ b/.vsts/windows-buildbot.yml @@ -43,7 +43,16 @@ steps: - script: python.bat -m test.pythoninfo displayName: 'Display build info' -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" displayName: 'Tests' env: PREFIX: $(Py_OutDir)\$(outDirSuffix) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' + mergeTestResults: true + testRunTitle: '$(Build.SourceBranchName)-$(outDirSuffix)' + platform: $(outDirSuffix) + condition: succeededOrFailed() From webhook-mailer at python.org Wed Sep 19 19:48:54 2018 From: webhook-mailer at python.org (Steve Dower) Date: Wed, 19 Sep 2018 23:48:54 -0000 Subject: [Python-checkins] Enables test result collection for CI builds (GH-9433) Message-ID: https://github.com/python/cpython/commit/1e7d77544057bbe102f9998bbe99f42105d529d5 commit: 1e7d77544057bbe102f9998bbe99f42105d529d5 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Steve Dower date: 2018-09-19T16:48:51-07:00 summary: Enables test result collection for CI builds (GH-9433) (cherry picked from commit fd54a45f7987d5b3343c74cb49a12bd39bdad5bc) Co-authored-by: Steve Dower files: M .vsts/linux-buildbot.yml M .vsts/macos-buildbot.yml M .vsts/windows-buildbot.yml diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml index 09084e07df77..bc5530cbaedc 100644 --- a/.vsts/linux-buildbot.yml +++ b/.vsts/linux-buildbot.yml @@ -37,5 +37,14 @@ steps: - script: make pythoninfo displayName: 'Display build info' -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(build.sourceBranchName)-linux' + platform: linux + condition: succeededOrFailed() diff --git a/.vsts/macos-buildbot.yml b/.vsts/macos-buildbot.yml index f58ea1626144..d9c190c46131 100644 --- a/.vsts/macos-buildbot.yml +++ b/.vsts/macos-buildbot.yml @@ -33,5 +33,14 @@ steps: - script: make pythoninfo displayName: 'Display build info' -- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" displayName: 'Tests' + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: '$(build.sourceBranchName)-macOS' + platform: macOS + condition: succeededOrFailed() diff --git a/.vsts/windows-buildbot.yml b/.vsts/windows-buildbot.yml index 5ec4522796ce..15aebeda4077 100644 --- a/.vsts/windows-buildbot.yml +++ b/.vsts/windows-buildbot.yml @@ -43,7 +43,16 @@ steps: - script: python.bat -m test.pythoninfo displayName: 'Display build info' -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" displayName: 'Tests' env: PREFIX: $(Py_OutDir)\$(outDirSuffix) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' + mergeTestResults: true + testRunTitle: '$(Build.SourceBranchName)-$(outDirSuffix)' + platform: $(outDirSuffix) + condition: succeededOrFailed() From webhook-mailer at python.org Thu Sep 20 01:17:14 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Thu, 20 Sep 2018 05:17:14 -0000 Subject: [Python-checkins] bpo-34746: Fix stop -> close (GH-9437) Message-ID: https://github.com/python/cpython/commit/ffef50f1f5907e5f0f175b12088d3e509011f126 commit: ffef50f1f5907e5f0f175b12088d3e509011f126 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-20T01:17:09-04:00 summary: bpo-34746: Fix stop -> close (GH-9437) files: M Doc/library/asyncio-llapi-index.rst diff --git a/Doc/library/asyncio-llapi-index.rst b/Doc/library/asyncio-llapi-index.rst index 7fb1e60f50de..0ab322af6dc7 100644 --- a/Doc/library/asyncio-llapi-index.rst +++ b/Doc/library/asyncio-llapi-index.rst @@ -53,7 +53,7 @@ See also the main documentation section about the * - :meth:`loop.stop` - Stop the event loop. - * - :meth:`loop.stop` + * - :meth:`loop.close` - Close the event loop. * - :meth:`loop.is_running()` From webhook-mailer at python.org Thu Sep 20 01:21:11 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 20 Sep 2018 05:21:11 -0000 Subject: [Python-checkins] bpo-34746: Fix stop -> close (GH-9437) Message-ID: https://github.com/python/cpython/commit/4fe8dc68577f9e22aaf24db08fb6647277c42d4c commit: 4fe8dc68577f9e22aaf24db08fb6647277c42d4c branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-19T22:21:08-07:00 summary: bpo-34746: Fix stop -> close (GH-9437) (cherry picked from commit ffef50f1f5907e5f0f175b12088d3e509011f126) Co-authored-by: Yury Selivanov files: M Doc/library/asyncio-llapi-index.rst diff --git a/Doc/library/asyncio-llapi-index.rst b/Doc/library/asyncio-llapi-index.rst index 7fb1e60f50de..0ab322af6dc7 100644 --- a/Doc/library/asyncio-llapi-index.rst +++ b/Doc/library/asyncio-llapi-index.rst @@ -53,7 +53,7 @@ See also the main documentation section about the * - :meth:`loop.stop` - Stop the event loop. - * - :meth:`loop.stop` + * - :meth:`loop.close` - Close the event loop. * - :meth:`loop.is_running()` From solipsis at pitrou.net Thu Sep 20 05:12:10 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 20 Sep 2018 09:12:10 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=5 Message-ID: <20180920091210.1.17DA325DB430A461@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [-7, 1, 7] memory blocks, sum=1 test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogiZYDRy', '--timeout', '7200'] From webhook-mailer at python.org Thu Sep 20 06:33:34 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 20 Sep 2018 10:33:34 -0000 Subject: [Python-checkins] [3.6] bpo-34542: Update test certs and keys (GH-8997) (GH-9396) Message-ID: https://github.com/python/cpython/commit/11485102cb7b3c57a1bc6d04c4ff4b1e25c53530 commit: 11485102cb7b3c57a1bc6d04c4ff4b1e25c53530 branch: 3.6 author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-20T03:33:28-07:00 summary: [3.6] bpo-34542: Update test certs and keys (GH-8997) (GH-9396) Update all test certs and keys to use future proof crypto settings: * 3072 bit RSA keys * SHA-256 signature Signed-off-by: Christian Heimes . (cherry picked from commit e6dac0077996b1e1f886f036d6f2606237fa4c85) https://bugs.python.org/issue34542 files: A Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst M Lib/test/allsans.pem M Lib/test/capath/b1930218.0 M Lib/test/capath/ceff1710.0 M Lib/test/keycert.passwd.pem M Lib/test/keycert.pem M Lib/test/keycert2.pem M Lib/test/keycert3.pem M Lib/test/keycert4.pem M Lib/test/make_ssl_certs.py M Lib/test/pycacert.pem M Lib/test/pycakey.pem M Lib/test/revocation.crl M Lib/test/ssl_cert.pem M Lib/test/ssl_key.passwd.pem M Lib/test/ssl_key.pem M Lib/test/test_asyncio/test_events.py M Lib/test/test_ssl.py diff --git a/Lib/test/allsans.pem b/Lib/test/allsans.pem index bf59f30abaad..6eebde7a57f1 100644 --- a/Lib/test/allsans.pem +++ b/Lib/test/allsans.pem @@ -1,64 +1,81 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQD59JyhPgYe7nhZ -Z2IGhaklNgtRkD+5BVs7lEWovYRBlXpPA6PuaHat25rI8EGYHmlufPherg2Qu6sC -GmZZKo7TgjlmDcwVS4hkebtFH7OZy5Il7Y2ZIdiK7Xp9Z0EPoqwacYowB0a8WhZY -I2Vm4EzCNKl6/htkwjgn2JXGizxvGt/1kNqP/GBAX+vjgeahOsn8jVh96KpFHJbS -g83cX4t8M7FJv7yNoDLvORHnvKCOXbQmr6ZMGcZN8PwS8awQ31khZTpEx+hCe+Pi -GzeOlxpZimXWDAGWA4tZ58Ka/QvO7VQbD5Ci166ODvvs+tEXfBUExtPcS+02IBJV -tzhBna9VAgMBAAECggEAPar9DccIqY76QEyCYcuOPLEFv9zP6+0HYj6lpQkE3U1s -vJvQURyS0zgQCy1Dca1nI6xPdsSIckHq4fzzbWJTlJlXYfdbd5GIGAn0iwxUOkiA -ST0/px0zmKsYgmH8KkhfH7MNfeX9rLCpPJuXA/eo2G03tzGEPqqwQhxsb2ygv2Qs -M7OqJz6RJu87K1Y+psWIv9+VhNVja0kvsg52QMK9mtp8layb54qLI5R5e09sIudq -RHegtnSOBo9kt32H9vWUFaF5PpYt4yks4KYI4ulKGWJGXHMDW4uHUaE/tjNQuYAX -DuDvjN+ECSJvigiUbu2k0xB2KYIb1fpcxlz/YBdADQKBgQD/Z2VtBUjOFnJKz00f -xN0akp7XPgd1yCb1/wZq9PQiGvzIAMDIplioTvjOjhOzPJaWD0GICNeypzQ48+0P -UsPIKbazpIZN6bZncr65plSpg0KANq46hbkPHOo8PHDa7yoxBUSPr8F7P1OCRkn6 -+QdgcnrAly7yfqO2ahAWOX7iCwKBgQD6ifXSCKfRF1GUb3Ws7S1rLxeBasWq+EmC -sUnck0S+AyaMkN+kZ5zejbN+NDuUMQ7+3wUIheTclUhzR0LP3+r5jjHsimJuvOml -wuV37F+Om5lD/Xx27NfbtRKn/bK6o0zDL8JB2eFB0N7Fh7hRYoUMdrpQs5sU91IC -pNYlAcLwHwKBgGvLK9eTf2LbvmksjRR3dgodD8UwfN2NGESC2iaSM+ehFEclajhF -XO3MRt6GwHHJhJTY44OSl9bjEvtmmAr7l34HfQDc04JWvZFzsGOSe/D/YTXT3jz8 -61ohjgrWR5tfjaMa4hDy0Oo/k/NLzzWJnT9rkbtvE3VtVZNLuHZo1dB5AoGBAMHO -wStV6MO1nzUNN+Gqo8zbY/qIJxsH8I26KaIJBk9azpJEa8yZHl+HDEffjgsoHCqL -STB7qzv7+0y53nRCClo8ZmBN+LEjUDcbWjl3z7/YnCpdR9ATjTP3kdQETCNWucXw -Bvy72CX6tqnlQG8soDGxEpXlKl2AqJ9E9icwgqUPAoGAL6xTDdgcYTbk9wxCd41l -NhHTSvLrGXLAzv61PCnlOJEJbuuezb2VW0ibsud5CA4Mi0tf9ET790XSOFd5nCjQ -6rr06AkjQsoFvjL1dO9EzVFPW0JrZ3C9y8ZOjdeAfPEmFL2T6VqmQ+IcCUNhSr39 -NBdKrboEFfnKanfbstekhAs= +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQCg/pM6dP7BTFNc +qe6wIJIBB7HjwL42bp0vjcCVl4Z3MRWFswYpfxy+o+8+PguMp4K6zndA5fwNkK/H +3HmtanncUfPqnV0usN0NHQGh/f9xRoNmB1q2L7kTuO99o0KLQgvonRT2snf8rq9n +tPRzhHUGYhog7zzNxetYV309PHpPr19BcKepDtM5RMk2aBnoN5vtItorjXiDosFm +6o5wQHrcupcVydszba6P75BEbc1XIWvq2Fv8muaw4pCe81QYINyLqgcPNO/nF3Os +5EI4HKjCNRSCOhOcWqYctXLXN9lBdMBBvQc3zDmYzh1eIZewzZXPVEQT33xPkhxz +HNmhcIctpWX4LTRF6FulkcbeuZDga3gkZYJf/M6IpU1WYXr6q8sNxbgmRRX/NuHo +V9oDwBzLG07rKUiqRHfjGqoCRmmVeVYpryvXUNjHGH0nlVzz/8lTUxAnJorO3Fdc +I+6zKLUPICdAlvz51AH6yopgPFhrdgA0pVzPO6L5G8SRQCxKhAUCAwEAAQKCAYAa +2jtOTcNMFGH3G7TfFZ+kolbuaPCQ/aQkEV2k1dAswzgWw8RsWXI+7fLyi8C7Zhks +9VD4tyNyU8at7D0zSoYm1Fh9sl+fcQp9rG/gSBA6IYu7EdD0gEM7YeY4K2nm9k4s +Lz8W4q+WqsBA6PK47cfjF6vKAH1AyRk28+jEtPiln9egf5zHWtyqOanh9D0V+Wh9 +hgmjqAYI1rWxZ7/4Qxj7Bfg7Px7blhi+kzOZ5kKQnNd2JT46hM+jgzah/G3zVE+R +FFW6ksmJgZ+dCuSbE7HEJmKms1CWq/1Cll0A3uy4JTDZOrK4KcZQ9UjjWJWvlXQm +uNXSSAp1k287DLVUm9c22SDeXpb9PyKmzyvJvVmMqqBx6QzHZ/L7WPzpUWAoLcU+ +ZHT7vggDymkIO+fcRbUzv8s5R7RnLbcBga51/5OCUvAWDoJXNw0qwYZOIbfTnQgs +8xbCmbMzllyYM/dK3GxQAwfn8Hzk+DbS/NObMjHLCWLfYeUvutXJSNly6Ny+ZcEC +gcEAzo5Y1UFOfBX4MZLIZ69LfgaXj9URobMwqlEwKil8pWQMa951ga3moLt91nOe +SAQz3meFTBX/VAb2ZHLeIf3FoNkiIx48PkxsR/hhLHpvl26zEg3yXs3tv0IFBx2R +EEnLNpQaAQFR9S1yDOaG2rsb17ZDKyp9isDpAENHAmEnT/XJn+Dc0SOH1EVDjUeM +JqToAF/fjIx/RF4oUJCAgOPBMlRy5ywLQk8uDi6ft0NCzzCi0eCuk1Ty3KzWFGwx +7cYRAoHBAMeIPCzHG3No4JGUFunslVwo5TuC7maO6qYKbq0OyvwWfL4b7gjrMBR9 +d5WyZlp/Vf40O463dg8x8qPNOFWp49f3hxTvvfnt2/m3+CQuDOLfqBbHufZApP1J +U9MubUNnDFHHeJ9l0tg2nhiLw24GHeMARZhA/BimMQPY0OpZPpLVxAUArM2EB7hI +glQpYCtdXhqwl1pl0u3TZ08y3BXYNg9BycdpGRMWSsAwsApJRgNuI/dfDKu0uMYF +/pUhXVPatQKBwGgLpAun3dT7bA3sli5ESo6s22OEPGFrVbQ1OUHDrBnTj742TJKJ ++oY0a2q+ypgUJdx94NM2sWquJybqBaKxpf8j4OI3tLjc3h5SqwAwnE13YZRSmifP +K1cP9mBjMFM4GLjhWUfwVkxeG/kLlhpP7fJ2yNbRjHN8QOH1AavdLGRGts1mA1UF +xMHUMfbUd3Bv2L13ja/KhcD2fPA4GcLS9tpXV5nCwdkg8V4LdkBmDR04rotx1f44 +6Czokt2usmfHQQKBwFkufxbUd2SB/72Rnxw27hse/DY5My0Lu70y9HzNG9TIiEDA +YwgBdp/x5D04W58fQuQ3nFcRkOcBwB2OYBuJr5ibvfiRnyvSMHvQykwBeSj+Jjbo +VinGgvfiimDdY2C48jyrFzLHZBHXd5oo/dRzT3Bicri2cvbhcQ7zHY1hDiK7AL3r +q1DALmMjpXzQcXdwZ9suCrgQwtIhpw8zAEOTO7ZeBT3nr5lkYUy9djFixrRJyjGK +fjNQtzVrAHrPStNr8QKBwQDCC0zhsCnTv4sAJmW7LL6Ayd5rbWhUZ6px1xY0yHMA +hehj+xbaiC6cfVr5Rg0ncvaa8AExu4kXpVsupTyNwvC4NgzLHtfBw6WUdOnd1awE +kSrDtDReBt2wByAcQwttQsrJ1/Pt6zcNJJI4Z9s8G4NTcQWJwUhU20N55JQKR//l +OQJqhq9NVhte/ctDjVwOHs/OhDNvxsAWxdjnf/O2up0os+M2bFkmHuaVW0vQbqTQ +mw7Vbzk2Ff5oT6E3kbC8Ur4= -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIGMDCCBRigAwIBAgIJAJYf8T95ptq5MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV +MIIHMDCCBZigAwIBAgIJALVVA6v9zJS5MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTgwMTE5 -MTkwOTA3WhcNMjgwMTE3MTkwOTA3WjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO +IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTgwODI5 +MTQyMzE3WhcNMjgwODI2MTQyMzE3WjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO Q2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3VuZGF0 -aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB -CgKCAQEA+fScoT4GHu54WWdiBoWpJTYLUZA/uQVbO5RFqL2EQZV6TwOj7mh2rdua -yPBBmB5pbnz4Xq4NkLurAhpmWSqO04I5Zg3MFUuIZHm7RR+zmcuSJe2NmSHYiu16 -fWdBD6KsGnGKMAdGvFoWWCNlZuBMwjSpev4bZMI4J9iVxos8bxrf9ZDaj/xgQF/r -44HmoTrJ/I1YfeiqRRyW0oPN3F+LfDOxSb+8jaAy7zkR57ygjl20Jq+mTBnGTfD8 -EvGsEN9ZIWU6RMfoQnvj4hs3jpcaWYpl1gwBlgOLWefCmv0Lzu1UGw+Qoteujg77 -7PrRF3wVBMbT3EvtNiASVbc4QZ2vVQIDAQABo4IC8TCCAu0wggEwBgNVHREEggEn -MIIBI4IHYWxsc2Fuc6AeBgMqAwSgFwwVc29tZSBvdGhlciBpZGVudGlmaWVyoDUG -BisGAQUCAqArMCmgEBsOS0VSQkVST1MuUkVBTE2hFTAToAMCAQGhDDAKGwh1c2Vy -bmFtZYEQdXNlckBleGFtcGxlLm9yZ4IPd3d3LmV4YW1wbGUub3JnpGcwZTELMAkG -A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo -b24gU29mdHdhcmUgRm91bmRhdGlvbjEYMBYGA1UEAwwPZGlybmFtZSBleGFtcGxl -hhdodHRwczovL3d3dy5weXRob24ub3JnL4cEfwAAAYcQAAAAAAAAAAAAAAAAAAAA -AYgEKgMEBTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG -AQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFH9ye3+WhBnHqNhtFu059bzY -SWM8MIGPBgNVHSMEgYcwgYSAFH9ye3+WhBnHqNhtFu059bzYSWM8oWGkXzBdMQsw -CQYDVQQGEwJYWTEXMBUGA1UEBwwOQ2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5 -dGhvbiBTb2Z0d2FyZSBGb3VuZGF0aW9uMRAwDgYDVQQDDAdhbGxzYW5zggkAlh/x -P3mm2rkwgYMGCCsGAQUFBwEBBHcwdTA8BggrBgEFBQcwAoYwaHR0cDovL3Rlc3Rj -YS5weXRob250ZXN0Lm5ldC90ZXN0Y2EvcHljYWNlcnQuY2VyMDUGCCsGAQUFBzAB -hilodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3RjYS9vY3NwLzBDBgNV -HR8EPDA6MDigNqA0hjJodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3Rj -YS9yZXZvY2F0aW9uLmNybDANBgkqhkiG9w0BAQsFAAOCAQEAYwYJcerUPvnsP7e2 -HGp/It0OZ8Cvpt8Qf7A+NSPvJqkyKakl8zK/50iq/qQKH09CnfEae4rfXLdlYsvV -2PZYK0LDWnyTcHSJWAVJjlSFIFt3ig9FdHv9GYtSWWod66cZ0sEZOoF2IHZUGby+ -Qa+JQpmv5jEuGIZzjcsh6hSOou8ph7LsCsRdVlQqk8rM97vB7DAgh01vedlbolsq -JxsuPRydNFV/eWq3AgAWgZL3LdYYIAgaVOTnnd3xARw8DlT1q6+Lzc71GBXrRZYh -qgd+xC/K1812gMPImTX02bxpkhCuIdVd7cztWi8sdQmSgDEFdYMXo4NzlFTK8dlC -Y4wa3Q== +aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIB +igKCAYEAoP6TOnT+wUxTXKnusCCSAQex48C+Nm6dL43AlZeGdzEVhbMGKX8cvqPv +Pj4LjKeCus53QOX8DZCvx9x5rWp53FHz6p1dLrDdDR0Bof3/cUaDZgdati+5E7jv +faNCi0IL6J0U9rJ3/K6vZ7T0c4R1BmIaIO88zcXrWFd9PTx6T69fQXCnqQ7TOUTJ +NmgZ6Deb7SLaK414g6LBZuqOcEB63LqXFcnbM22uj++QRG3NVyFr6thb/JrmsOKQ +nvNUGCDci6oHDzTv5xdzrORCOByowjUUgjoTnFqmHLVy1zfZQXTAQb0HN8w5mM4d +XiGXsM2Vz1REE998T5IccxzZoXCHLaVl+C00RehbpZHG3rmQ4Gt4JGWCX/zOiKVN +VmF6+qvLDcW4JkUV/zbh6FfaA8AcyxtO6ylIqkR34xqqAkZplXlWKa8r11DYxxh9 +J5Vc8//JU1MQJyaKztxXXCPusyi1DyAnQJb8+dQB+sqKYDxYa3YANKVczzui+RvE +kUAsSoQFAgMBAAGjggLxMIIC7TCCATAGA1UdEQSCAScwggEjggdhbGxzYW5zoB4G +AyoDBKAXDBVzb21lIG90aGVyIGlkZW50aWZpZXKgNQYGKwYBBQICoCswKaAQGw5L +RVJCRVJPUy5SRUFMTaEVMBOgAwIBAaEMMAobCHVzZXJuYW1lgRB1c2VyQGV4YW1w +bGUub3Jngg93d3cuZXhhbXBsZS5vcmekZzBlMQswCQYDVQQGEwJYWTEXMBUGA1UE +BwwOQ2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3Vu +ZGF0aW9uMRgwFgYDVQQDDA9kaXJuYW1lIGV4YW1wbGWGF2h0dHBzOi8vd3d3LnB5 +dGhvbi5vcmcvhwR/AAABhxAAAAAAAAAAAAAAAAAAAAABiAQqAwQFMA4GA1UdDwEB +/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/ +BAIwADAdBgNVHQ4EFgQUoLHAHNTWrHkSCUYkhn5NH0S40CAwgY8GA1UdIwSBhzCB +hIAUoLHAHNTWrHkSCUYkhn5NH0S40CChYaRfMF0xCzAJBgNVBAYTAlhZMRcwFQYD +VQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZv +dW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnOCCQC1VQOr/cyUuTCBgwYIKwYBBQUH +AQEEdzB1MDwGCCsGAQUFBzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0 +L3Rlc3RjYS9weWNhY2VydC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2Eu +cHl0aG9udGVzdC5uZXQvdGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0 +dHA6Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3Js +MA0GCSqGSIb3DQEBCwUAA4IBgQAeKJKycO2DES98gyR2e/GzPYEw87cCS0cEpiiP +3CEUgzfEbF0X89GDKEey4H3Irvosbvt2hEcf2RNpahLUL/fUv53bDmHNmL8qJg5E +UJVMOHvOpSOjqoqeRuSyG0GnnAuUwcxdrZY6UzLdslhuq9F8UjgHr6KSMx56G9uK +LmTy5njMab0in2xL/YRX/0nogK3BHqpUHrfCdEYZkciRxtAa+OPpWn4dcZi+Fpf7 +ZYSgPLNt+djtFDMIAk5Bo+XDaQdW3dhF0w44enrGAOV0xPE+/jOuenNhKBafjuNb +lkeSr45+QZsi1rd18ny8z3uuaGqIAziFgmllZOH2D8giTn6+5jZcCNZCoGKUkPI9 +l/GMWwxg4HQYYlZcsZzTCem9Rb2XcrasAbmhFapMtR+QAwSed5vKE7ZdtQhj74kB +7Q0E7Lkgpp6BaObb2As8/f0K/UlSVSvrYk+i3JT9wK/qqkRGxsTFEF7N9t0rKu8y +4JdQDtZCI552MsFvYW6m+IOYgxg= -----END CERTIFICATE----- diff --git a/Lib/test/capath/b1930218.0 b/Lib/test/capath/b1930218.0 index 07556ff9071a..730e7fd911a5 100644 --- a/Lib/test/capath/b1930218.0 +++ b/Lib/test/capath/b1930218.0 @@ -1,21 +1,26 @@ -----BEGIN CERTIFICATE----- -MIIDbTCCAlWgAwIBAgIJAILtv0HIgJGbMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIEbTCCAtWgAwIBAgIJAMstgJlaaVJbMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yODAxMTcx -OTA5MDZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg -Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCASIwDQYJKoZI -hvcNAQEBBQADggEPADCCAQoCggEBAMMYaWvJRymYjrFWwi76Dl68I4CzB2Ik0kJb -8Uq/qcghdcjj5iwfhzxufBvtOTKVt0CyYEjDmhYI/m1niDQ7d3dwHHBa0R9fBCFU -uQzjQYUdWO4v7fMO79gjofpz+0wo4OXmTQsCUkmGx75+veZWdotwjgqPBjMgHXtb -qtDFG6ubzFQJPL/kQGbx+9b3Fp3EGdTD8v8HvG9aniUbAkql7EKWOnDSbJkrzr7o -0gHv1bqwz5Q+gtAB1ktxgAMKEkWGeYHYS9LotbcsbJpMihAQ5OT1386EkcrRRuCE -cxdm22lDeICDvhRN8T4a1mz13kXzOa+R1T1URL9BzHNoGvwk25ECAwEAAaNQME4w -HQYDVR0OBBYEFJrPz27rcT3bPPGuiGtWcgPLCKdIMB8GA1UdIwQYMBaAFJrPz27r -cT3bPPGuiGtWcgPLCKdIMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEB -ABAlyNwMVVzLg25573fsDY4MBsFLDNb3dVIhuBdKOIids3jEQvu4fBQ4EPus2hEA -W0KHXkWfbU5CpJoYBjkPRaaWiTLWWbPTjuOVtsSiS3QvZ8H7u/lybzdK5/RIM3Hf -uPXmQT/V1S8mCfgOkv9w6varWPuQBNZDLo+x+waradDcqPhbB/LUZh9j+F3BnkFE -u8nofeBG5KfIMl8xYuUcXIndt6JPng0TuF+xhFNMH84Z4QEAXr9BVZSppRPb8lnz -1k65nZ25CtmyGG18sfeWqr32+ZUPSm48fEZb39R47Jrc4uMB5oh3OZOcuipj+SVL -T6wIeTnGe98HNbrAwlC/WoE= +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA4MjYx +NDIzMTZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg +Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCAaIwDQYJKoZI +hvcNAQEBBQADggGPADCCAYoCggGBAJftVUG6NheV23Ec0+FhrFhz48aWzysfuAj1 +nUtLxzD2uAuzUnKgu8lNO47fIo4BV4HJknPMAMbscLA6F0DB3/KMNkzEp4HntiRo +4qB+NQcvoFv5RUb3HvBGEf7KGjxQ8SapX5winPhB4d9PEpUZL1yQARdufj59z+kJ +ryX4+EJ3LW1fNvJ4Hn1Kh2hjbAZxG436Jf7U0/WlF7Hv6hfLVMgnmYDLPEXxLFIc +3R9RRSBQHl6rV3MbQXiW3oSket2PMIU2WHl2oNJhyBuplJljxu74FL+0UlYxl/rr +rFOelc5MxFpKt8oDJ1s1V84C3OzKafiKWjnLFiAVAyRhbPR6/LZI5VkQXEnQI5/7 +cV466WifNHKAJ7Y/TLHZ22N/Z2hKbhH46MD0WhY5Uwto3nf6Ref4kc14zSiUl3FU ++8/wN97JJsXcG56JbQmsyERxy23xlzHVTCAzv3VKoODcaRHtKrRkEBEwiw6wpxDY +isWqG8gmiiXnZp+lahovfF+DxnhPHwIDAQABo1AwTjAdBgNVHQ4EFgQU3b/K2ubR +NLo3dSHKb5oIKPI1tkgwHwYDVR0jBBgwFoAU3b/K2ubRNLo3dSHKb5oIKPI1tkgw +DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAYEAM2pU02vA1wFfnfQFwZNm +kFDQtxjpsB5KoLbadpOvhNutFVQxFRPk3n5ODNUJHDQ1tuVM1m9lfTJf6/ypawf3 +SYLlgX4HgJpj+CzDQLyP1CraPtHuCLdNp4TK9D+hmEW+sQVp59/Xmasb7oswzPf8 +59QLF66Xv+R7/Q+ntIV541niFoe/HylFLCOTdr7Ahx3e7CtCauW7yPQKSggKjFzY +fU3RuL/V9yntktGUBOg1Bld/LCOXh6U1jSbTGkfyFtfZxtQfI0PTJpk5yiD0cSNv +DEp2dvd2H7P+v0ew/CpWgeHS3e4I2PT/WtwlYYqRArmGHPJQc3YlNfy2JSYVy+vE +K2EMHOfuLxeb7PDUoYTn0q/e5BskFKcBh+OrKVhGoNnACuCN11nTG/hUID54paXI +T4sDxJaf7PtHz3YtjWU0J7/6rgEFivOSCt2JbJehx+dgUees60t9LLhlyf5dakhV +juTH+WpA4bhkRem1WSmlX899WH1keeWkCawedmU9lMRo -----END CERTIFICATE----- diff --git a/Lib/test/capath/ceff1710.0 b/Lib/test/capath/ceff1710.0 index 07556ff9071a..730e7fd911a5 100644 --- a/Lib/test/capath/ceff1710.0 +++ b/Lib/test/capath/ceff1710.0 @@ -1,21 +1,26 @@ -----BEGIN CERTIFICATE----- -MIIDbTCCAlWgAwIBAgIJAILtv0HIgJGbMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIEbTCCAtWgAwIBAgIJAMstgJlaaVJbMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yODAxMTcx -OTA5MDZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg -Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCASIwDQYJKoZI -hvcNAQEBBQADggEPADCCAQoCggEBAMMYaWvJRymYjrFWwi76Dl68I4CzB2Ik0kJb -8Uq/qcghdcjj5iwfhzxufBvtOTKVt0CyYEjDmhYI/m1niDQ7d3dwHHBa0R9fBCFU -uQzjQYUdWO4v7fMO79gjofpz+0wo4OXmTQsCUkmGx75+veZWdotwjgqPBjMgHXtb -qtDFG6ubzFQJPL/kQGbx+9b3Fp3EGdTD8v8HvG9aniUbAkql7EKWOnDSbJkrzr7o -0gHv1bqwz5Q+gtAB1ktxgAMKEkWGeYHYS9LotbcsbJpMihAQ5OT1386EkcrRRuCE -cxdm22lDeICDvhRN8T4a1mz13kXzOa+R1T1URL9BzHNoGvwk25ECAwEAAaNQME4w -HQYDVR0OBBYEFJrPz27rcT3bPPGuiGtWcgPLCKdIMB8GA1UdIwQYMBaAFJrPz27r -cT3bPPGuiGtWcgPLCKdIMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEB -ABAlyNwMVVzLg25573fsDY4MBsFLDNb3dVIhuBdKOIids3jEQvu4fBQ4EPus2hEA -W0KHXkWfbU5CpJoYBjkPRaaWiTLWWbPTjuOVtsSiS3QvZ8H7u/lybzdK5/RIM3Hf -uPXmQT/V1S8mCfgOkv9w6varWPuQBNZDLo+x+waradDcqPhbB/LUZh9j+F3BnkFE -u8nofeBG5KfIMl8xYuUcXIndt6JPng0TuF+xhFNMH84Z4QEAXr9BVZSppRPb8lnz -1k65nZ25CtmyGG18sfeWqr32+ZUPSm48fEZb39R47Jrc4uMB5oh3OZOcuipj+SVL -T6wIeTnGe98HNbrAwlC/WoE= +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA4MjYx +NDIzMTZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg +Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCAaIwDQYJKoZI +hvcNAQEBBQADggGPADCCAYoCggGBAJftVUG6NheV23Ec0+FhrFhz48aWzysfuAj1 +nUtLxzD2uAuzUnKgu8lNO47fIo4BV4HJknPMAMbscLA6F0DB3/KMNkzEp4HntiRo +4qB+NQcvoFv5RUb3HvBGEf7KGjxQ8SapX5winPhB4d9PEpUZL1yQARdufj59z+kJ +ryX4+EJ3LW1fNvJ4Hn1Kh2hjbAZxG436Jf7U0/WlF7Hv6hfLVMgnmYDLPEXxLFIc +3R9RRSBQHl6rV3MbQXiW3oSket2PMIU2WHl2oNJhyBuplJljxu74FL+0UlYxl/rr +rFOelc5MxFpKt8oDJ1s1V84C3OzKafiKWjnLFiAVAyRhbPR6/LZI5VkQXEnQI5/7 +cV466WifNHKAJ7Y/TLHZ22N/Z2hKbhH46MD0WhY5Uwto3nf6Ref4kc14zSiUl3FU ++8/wN97JJsXcG56JbQmsyERxy23xlzHVTCAzv3VKoODcaRHtKrRkEBEwiw6wpxDY +isWqG8gmiiXnZp+lahovfF+DxnhPHwIDAQABo1AwTjAdBgNVHQ4EFgQU3b/K2ubR +NLo3dSHKb5oIKPI1tkgwHwYDVR0jBBgwFoAU3b/K2ubRNLo3dSHKb5oIKPI1tkgw +DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAYEAM2pU02vA1wFfnfQFwZNm +kFDQtxjpsB5KoLbadpOvhNutFVQxFRPk3n5ODNUJHDQ1tuVM1m9lfTJf6/ypawf3 +SYLlgX4HgJpj+CzDQLyP1CraPtHuCLdNp4TK9D+hmEW+sQVp59/Xmasb7oswzPf8 +59QLF66Xv+R7/Q+ntIV541niFoe/HylFLCOTdr7Ahx3e7CtCauW7yPQKSggKjFzY +fU3RuL/V9yntktGUBOg1Bld/LCOXh6U1jSbTGkfyFtfZxtQfI0PTJpk5yiD0cSNv +DEp2dvd2H7P+v0ew/CpWgeHS3e4I2PT/WtwlYYqRArmGHPJQc3YlNfy2JSYVy+vE +K2EMHOfuLxeb7PDUoYTn0q/e5BskFKcBh+OrKVhGoNnACuCN11nTG/hUID54paXI +T4sDxJaf7PtHz3YtjWU0J7/6rgEFivOSCt2JbJehx+dgUees60t9LLhlyf5dakhV +juTH+WpA4bhkRem1WSmlX899WH1keeWkCawedmU9lMRo -----END CERTIFICATE----- diff --git a/Lib/test/keycert.passwd.pem b/Lib/test/keycert.passwd.pem index 0ad696055190..cbb3c3bccd2c 100644 --- a/Lib/test/keycert.passwd.pem +++ b/Lib/test/keycert.passwd.pem @@ -1,50 +1,68 @@ -----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED -DEK-Info: DES-EDE3-CBC,E74528136B90D2DD +DEK-Info: DES-EDE3-CBC,D134E931C96D9DEC -WRHVD2PJXPqjFSHg92HURIsUzvsTE4a9oi0SC5yMBFKNWA5Z933gK3XTifp6jul5 -zpNYi8jBXZ2EqJJBxCuVcefmXSxL0q7CMej25TdIC4BVAFJVveeprHPUFkNB0IM1 -go5Lg4YofYqTCg3OE3k7WvfR3Zg1cRYxksDKO+WNZgWyKBex5X4vjOiyUqDl3GKt -kQXnkg1VgPV2Vrx93S9XNdELNRTguwf+XG0fkhtYhp/zCto8uKTgy5elK2P/ulGp -7fe6uj7h/uN9L7EOC6CjRkitywfeBUER739mOcGT4imSFJ9G27TCqPzj2ea3uuaf -/v1xhkQ4M6lNY/gcRfgVpCXhW43aAQV8XXQRMJTqLmz5Y5hYTKn+Ugq5vJ/ngyRM -lu1gUJnYYaemBTb4hbm6aBvnYK9mORa891Pmf+vxU9rYuQIdVAhvvXh4KBreSEBI -1AFy6dFKXl8ZKs6Wrq5wPefmFFkRmZ8OBiiq0fp2ApCRGZw6LsjCgwrRM38JiY7d -3OdsJpKvRYufgUyuuzUE0xA+E4yMvD48M9pPq2fC8O5giuGL1uEekQWXJuq+6ZRI -XYKIeSkuQALbX3RAzCPXTUEMtCYXKm/gxrrwJ+Bet4ob2amf3MX0uvWwOuAq++Fk -J0HFSBxrwvIWOhyQXOPuJdAN8PXA7dWOXfOgOMF0hQYqZCl3T4TiVZJbwVQtg1sN -dO7oAD5ZPHiKzvveZuB6k1FlBG8j0TyAC+44ChxkPDD3jF4dd6zGe62sDf85p4/d -W80gxJeD3xnDxG0ePPns+GuKUpUaWS7WvHtDpeFW1JEhvOqf8p1Li9a7RzWVo8ML -mGTdQgBIYIf6/fk69pFKl0nKtBU75KaunZz4nAmd9bNED4naDurMBg44u5TvODbJ -vgYIYXIYjNvONbskJatVrrTS8zch2NwVIjCi8L/hecwBXbIXzo1pECpc6BU7sQT8 -+i9sDKBeJcRipzfKZNHvnO19mUZaPCY8+a/f9c21DgKXz+bgLcJbohpSaeGM8Gfc -aZd3Vp9n3OJ3g2zQR1++HO9v1vR/wLELu6MeydkvMduHLmOPCn54gZ9z51ZNPAwa -qfFIsH+mLh9ks0H74ssF59uIlstkgB9zmZHv/Q0dK9ZfG/VEH6rSgdETWhZxhoMQ -Z92jXBEFT0zhI3rrIPNY+XS7eJCQIc1wc84Ea3cRk7SP+S1og3JtAxX56ykUwtkM -LQ/Dwwa6h1aqD0l2d5x1/BSdavtTuSegISRWQ4iOmSvEdlFP7H4g6RZk/okbLzMD -Evq5gNc7vlXhVawoQU8JCanJ5ZbbWnIRZfiXxBQS4lpYPKvJt4ML9z/x+82XxcXv -Z93N2Wep7wWW5OwS2LcQcOgZRDSIPompwo/0pMFGOS+5oort0ZDRHdmmGLjvBcCb -1KQmKQ4+8brI/3rjRzts6uDLjTGNxSCieNsnqhwHUv9Mg9WDSWupcGa+x27L89x3 -rObf6+3umcFLSjIzU8wuv1hx/e/y98Kv7BDBNYpAr6kVMrLnzYjAfJbBmqlxkzkQ -IgQzgrk2QZoTdgwR+S374NAMO0AE5IlO+/qa6qp2SORGTDX64I3UNw== +nuGFEej7vIjkYWSMz5OJeVTNntDRQi6ZM4DBm3g8T7i/0odr3WFqGMMKZcIhLYQf +rgRq7RSKtrJ1y5taVucMV+EuCjyfzDo0TsYt+ZrXv/D08eZhjRmkhoHnGVF0TqQm +nQEXM/ERT4J2RM78dnG+homMkI76qOqxgGbRqQqJo6AiVRcAZ45y8s96bru2TAB8 ++pWjO/v0Je7AFVdwSU52N8OOY6uoSAygW+0UY1WVxbVGJF2XfRsNpPX+YQHYl6e+ +3xM5XBVCgr6kmdAyub5qUJ38X3TpdVGoR0i+CVS9GTr2pSRib1zURAeeHnlqiUZM +4m0Gn9s72nJevU1wxED8pwOhR8fnHEmMKGD2HPhKoOCbzDhwwBZO27TNa1uWeM3f +M5oixKDi2PqMn3y2cDx1NjJtP661688EcJ5a2Ih9BgO9xpnhSyzBWEKcAn0tJB0H +/56M0FW6cdOOIzMveGGL7sHW5E+iOdI1n5e7C6KJUzew78Y9qJnhS53EdI6qTz9R +wsIsj1i070Fk6RbPo6zpLlF6w7Zj8GlZaZA7OZZv9wo5VEV/0ST8gmiiBOBc4C6Y +u9hyLIIu4dFEBKyQHRvBnQSLNpKx6or1OGFDVBay2In9Yh2BHh1+vOj/OIz/wq48 +EHOIV27fRJxLu4jeK5LIGDhuPnMJ8AJYQ0bQOUP6fd7p+TxWkAQZPB/Dx/cs3hxr +nFEdzx+eO+IAsObx/b1EGZyEJyETBslu4GwYX7/KK3HsJhDJ1bdZ//28jOCaoir6 +ZOMT72GRwmVoQTJ0XpccfjHfKJDRLT7C1xvzo4Eibth0hpTZkA75IUYUp6qK/PuJ +kH/qdiC7QIkRKtsrawW4vEDna3YtxIYhQqz9+KwO6u/0gzooZtv1RU4U3ifMDB5u +5P5GAzACRqlY8QYBkM869lvWqzQPHvybC4ak9Yx6/heMO9ddjdIW9BaK8BLxvN/6 +UCD936Y4fWltt09jHZIoxWFykouBwmd7bXooNYXmDRNmjTdVhKJuOEOQw8hDzx7e +pWFJ9Z/V4Qm1tvXbCD7QFqMCDoY3qFvVG8DBqXpmxe1yPfz21FWrT7IuqDXAD3ns +vxfN/2a+Cy04U9FBNVCvWqWIs5AgNpdCMJC2FlXKTy+H3/7rIjNyFyvbX0vxIXtK +liOVNXiyVM++KZXqktqMUDlsJENmIHV9B046luqbgW018fHkyEYlL3iRZGbYegwr +XO9VVIKVPw1BEvJ8VNdGFGuZGepd8qX2ezfYADrNR+4t85HDm8inbjTobSjWuljs +ftUNkOeCHqAvWCFQTLCfdykvV08EJfVY79y7yFPtfRV2gxYokXFifjo3su9sVQr1 +UiIS5ZAsIC1hBXWeXoBN7QVTkFi7Yto6E1q2k10LiT3obpUUUQ/oclhrJOCJVjrS +oRcj2QBy8OT4T9slJr5maTWdgd7Lt6+I6cGQXPaDvjGOJl0eBYM14vhx4rRQWytJ +k07hhHFO4+9CGCuHS8AAy2gR6acYFWt2ZiiNZ0z/iPIHNK4YEyy9aLf6uZH/KQjE +jmHToo7XD6QvCAEC5qTHby3o3LfHIhyZi/4L+AhS4FKUHF6M0peeyYt4z3HaK2d2 +N6mHLPdjwNjra7GOmcns4gzcrdfoF+R293KpPal4PjknvR3dZL4kKP/ougTAM5zv +qDIvRbkHzjP8ChTpoLcJsNVXykNcNkjcSi0GHtIpYjh6QX6P2uvR/S4+Bbb9p9rn +hIy/ovu9tWN2hiPxGPe6torF6BulAxsTYlDercC204AyzsrdA0pr6HBgJH9C6ML1 +TchwodbFJqn9rSv91i1liusAGoOvE81AGBdrXY7LxfSNhYY1IK6yR/POJPTd53sA +uX2/j6Rtoksd/2BHPM6AUnI/2B9slhuzWX2aCtWLeuwvXDS6rYuTigaQmLkzTRfM +dlMI3s9KLXxgi5YVumUZleJWXwBNP7KiKajd+VTSD+7WAhyhM5FIG5wVOaxmy4G2 +TyqZ/Ax9d2VEjTQHWvQlLPQ4Mp0EIz0aEl94K/S8CK8bJRH6+PRkar+dJi1xqlL+ +BYb42At9mEJ8odLlFikvNi1+t7jqXk5jRi5C0xFKx3nTtzoH2zNUeuA3R6vSocVK +45jnze9IkKmxMlJ4loR5sgszdpDCD3kXqjtCcbMTmcrGyzJek3HSOTpiEORoTFOe +Rhg6jH5lm+QcC263oipojS0qEQcnsWJP2CylNYMYHR9O/9NQxT3o2lsRHqZTMELV +uQa/SFH+paQNbZOj8MRwPSqqiIxJFuLswKte1R+W7LKn1yBSM7Pp39lNbzGvJD2E +YRfnCwFpJ54voVAuQ4jXJvigCW2qeCjXlxeD6K2j4eGJEEOmIjIW1wjubyBY6OI3 -----END RSA PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIDWTCCAkGgAwIBAgIJAPm6B21bar2bMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV +MIIEWTCCAsGgAwIBAgIJAJinz4jHSjLtMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODAx -MTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH +IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODA4 +MjkxNDIzMTVaFw0yODA4MjYxNDIzMTVaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH DA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5k -YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEP -ADCCAQoCggEBAKvvsX2gEti4shve3iYMc+jE4Se7WHs1Bol2f21H8qNboDOFdeb1 -RKHjmq3exHpajywOUEgne9nKHJY/3f2phR4Y5klqG6liLgiSpVyRlcBGbeT2qEAj -9oLiLFUXLGfGDds2mTwivQDLJBWi51j7ff5k2Pr58fN5ugYMn24T9FNyn0moT+qj -SFoBNm58l9jrdkJSlgWfqPlbiMa+mqDn/SFtrwLF2Trbfzu42Sd9UdIzMaSSrzbN -sGm53pNhCh8KndWUQ8GPP2IsLPoUU4qAtmZuTxCx2S1cXrN9EkmT69tlOH84YfSn -96Ih9bWRc7M5y5bfVdEVM+fKQl3hBRf05qMCAwEAAaMYMBYwFAYDVR0RBA0wC4IJ -bG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQAtQ8f37cCEk7/rAcbYR53ce3iK -Vpihb0U2ni1QjG9Tg9UIExkIGkwTiCm7kwQL+GEStBu9AG/QVrOjeTriRiddhWkk -ze8kRaI3AC/63t6Vh9Q1x6PESgeE4OtAO9JpJCf4GILglA789Y/b/GF8zJZQxR13 -qpB4ZwWw7gCBhdEW59u6CFeBmfDa58hM8lWvuVoRrTi7bjUeC6PAn5HVMzZSykhu -4HaUfBp6bKFjuym2+h/VvM1n8C3chjVSmutsLb6ELdD8IK0vPV/yf5+LN256zSsS -dyUZYd8XwQaioEMKdbhLvnehyzHiWfQIUR3BdhONxoIJhHv/EAo8eCkHHYIF +YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGP +ADCCAYoCggGBALKUqUtopT6E68kN+uJNEt34i2EbmG/bwjcD8IaMsgJPSsMO2Bpd +3S6qWgkCeOyCfmAwBxK2kNbxGb63ouysEv7l8GCTJTWv3hG/HQcejJpnAEGi6K1U +fDbyE/db6yZ12SoHVTGkadN4vYGCPd1Wj9ZO1F877SHQ8rDWX3xgTWkxN2ojBw44 +T8RHSDiG8D/CvG4uEy+VUszL+Uvny5y2poNSqvI3J56sptWSrh8nIIbkPZPBdUne +LYMOHTFK3ZjXSmhlXgziTxK71nnzM3Y9K9gxPnRqoXbvu/wFo55hQCkETiRkYgmm +jXcBMZ0TClQVnQWuLjMthRnWFZs4Lfmwqjs7FZD/61581R2BYehvpWbLvvuOJhwv +DFzexL2sXcAl7SsxbzeQKRHqGbIDfbnQTXfs3/VC6Ye5P82P2ucj+XC32N9piRmO +gCBP8L3ub+YzzdxikZN2gZXXE2jsb3QyE/R2LkWdWyshpKe+RsZP1SBRbHShUyOh +yJ90baoiEwj2mwIDAQABoxgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZI +hvcNAQELBQADggGBAHRUO/UIHl3jXQENewYayHxkIx8t7nu40iO2DXbicSijz5bo +5//xAB6RxhBAlsDBehgQP1uoZg+WJW+nHu3CIVOU3qZNZRaozxiCl2UFKcNqLOmx +R3NKpo1jYf4REQIeG8Yw9+hSWLRbshNteP6bKUUf+vanhg9+axyOEOH/iOQvgk/m +b8wA8wNa4ujWljPbTQnj7ry8RqhTM0GcAN5LSdSvcKcpzLcs3aYwh+Z8e30sQWna +F40sa5u7izgBTOrwpcDm/w5kC46vpRQ5fnbshVw6pne2by0mdMECASid/p25N103 +jMqTFlmO7kpf/jpCSmamp3/JSEE1BJKHwQ6Ql4nzRA2N1mnvWH7Zxcv043gkHeAu +0x8evpvwuhdIyproejNFlBpKmW8OX7yKTCPPMC/VkX8Q1rVkxU0DQ6hmvwZlhoKa +9Wc2uXpw9xF8itV4Uvcdr3dwqByvIqn7iI/gB+4l41e0u8OmH2MKOx4Nxlly5TNW +HcVKQHyOeyvnINuBAQ== -----END CERTIFICATE----- diff --git a/Lib/test/keycert.pem b/Lib/test/keycert.pem index 9545dcf4b94f..0d398633739a 100644 --- a/Lib/test/keycert.pem +++ b/Lib/test/keycert.pem @@ -1,48 +1,66 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCr77F9oBLYuLIb -3t4mDHPoxOEnu1h7NQaJdn9tR/KjW6AzhXXm9USh45qt3sR6Wo8sDlBIJ3vZyhyW -P939qYUeGOZJahupYi4IkqVckZXARm3k9qhAI/aC4ixVFyxnxg3bNpk8Ir0AyyQV -oudY+33+ZNj6+fHzeboGDJ9uE/RTcp9JqE/qo0haATZufJfY63ZCUpYFn6j5W4jG -vpqg5/0hba8Cxdk62387uNknfVHSMzGkkq82zbBpud6TYQofCp3VlEPBjz9iLCz6 -FFOKgLZmbk8QsdktXF6zfRJJk+vbZTh/OGH0p/eiIfW1kXOzOcuW31XRFTPnykJd -4QUX9OajAgMBAAECggEAHppmXDbuw9Z0FVPg9KLIysioTtsgz6VLiZIm8juZK4x2 -glUh/D7xvWL2uDXrgN+3lh7iGUW13LkFx5SMncbbo9TIwI57Z/XKvcnkVwquve+L -RfLFVc1Q5lD9lROv2rS86KTaN4LzYz3FKXi6dvMkpPAsUtfEQhMLkmISypQQq/1z -EJaqo7r85OjN7e0wKazlKZpOzJEa5FQLMVRjTRFhLFNbHXX/tAet2jw+umATKbw8 -hYgiuZ44TwSEd9JeIV/oSYWfI/3HetuYW0ru3caiztRF2NySNu8lcsWgNC7fIku9 -mcHjtSNzs91QN1Qlu7GQvvhpt6OWDirNDCW+49WGaQKBgQDg9SDhfF0jRYslgYbH -cqO4ggaFdHjrAAYpwnAgvanhFZL/zEqm5G1E7l/e2fCkJ9VOSFO0A208chvwMcr+ -dCjHE2tVdE81aQ2v/Eo83VdS1RcOV4Y75yPH48rMhxPaHvxWD/FFDbf0/P2mtPB7 -SZ3kIeZMkE1wxdaO3AKUbQoozwKBgQDDqYgg7kVtygyICE1mB8Hwp6nUxFTczG7y -4XcsDqMIrKmw+PbQluvkoHoStxeVrsTloDhkTjIrpmYLyAiazg+PUJdkd6xrfLSj -VV6X93W0S/1egEb1F1CGFxtk8v/PWH4K76EPL2vxXdxjywz3GWlrL9yDYaB2szzS -DqgwVMqx7QKBgDCD7UF0Bsoyl13RX3XoPXLvZ+SkR+e2q52Z94C4JskKVBeiwX7Y -yNAS8M4pBoMArDoj0xmBm69rlKbqtjLGbnzwrTdSzDpim7cWnBQgUFLm7gAD1Elb -AhZ8BCK0Bw4FnLoa2hfga4oEfdfUMgEE0W5/+SEOBgWKRUmuHUhRc911AoGAY2EN -YmSDYSM5wDIvVb5k9B3EtevOiqNPSw/XnsoEZtiEC/44JnQxdltIBY93bDBrk5IQ -cmoBM4h91kgQjshQwOMXMhFSwvmBKmCm/hrTbvMVytTutXfVD3ZXFKwT4DW7N0TF -ElhsxBh/YzRz7mG62JVjtFt2zDN3ld2Z8YpvtXUCgYEA4EJ4ObS5YyvcXAKHJFo6 -Fxmavyrf8LSm3MFA65uSnFvWukMVqqRMReQc5jvpxHKCis+XvnHzyOfL0gW9ZTi7 -tWGGbBi0TRJCa8BkvgngUZxOxUlMfg/7cVxOIB0TPoUSgxFd/+qVz4GZMvr0dPu7 -eAF7J/8ECVvb0wSPTUI1N3c= +MIIG/wIBADANBgkqhkiG9w0BAQEFAASCBukwggblAgEAAoIBgQCylKlLaKU+hOvJ +DfriTRLd+IthG5hv28I3A/CGjLICT0rDDtgaXd0uqloJAnjsgn5gMAcStpDW8Rm+ +t6LsrBL+5fBgkyU1r94Rvx0HHoyaZwBBouitVHw28hP3W+smddkqB1UxpGnTeL2B +gj3dVo/WTtRfO+0h0PKw1l98YE1pMTdqIwcOOE/ER0g4hvA/wrxuLhMvlVLMy/lL +58uctqaDUqryNyeerKbVkq4fJyCG5D2TwXVJ3i2DDh0xSt2Y10poZV4M4k8Su9Z5 +8zN2PSvYMT50aqF277v8BaOeYUApBE4kZGIJpo13ATGdEwpUFZ0Fri4zLYUZ1hWb +OC35sKo7OxWQ/+tefNUdgWHob6Vmy777jiYcLwxc3sS9rF3AJe0rMW83kCkR6hmy +A3250E137N/1QumHuT/Nj9rnI/lwt9jfaYkZjoAgT/C97m/mM83cYpGTdoGV1xNo +7G90MhP0di5FnVsrIaSnvkbGT9UgUWx0oVMjocifdG2qIhMI9psCAwEAAQKCAYBT +sHmaPmNaZj59jZCqp0YVQlpHWwBYQ5vD3pPE6oCttm0p9nXt/VkfenQRTthOtmT1 +POzDp00/feP7zeGLmqSYUjgRekPw4gdnN7Ip2PY5kdW77NWwDSzdLxuOS8Rq1MW9 +/Yu+ZPe3RBlDbT8C0IM+Atlh/BqIQ3zIxN4g0pzUlF0M33d6AYfYSzOcUhibOO7H +j84r+YXBNkIRgYKZYbutRXuZYaGuqejRpBj3voVu0d3Ntdb6lCWuClpB9HzfGN0c +RTv8g6UYO4sK3qyFn90ibIR/1GB9watvtoWVZqggiWeBzSWVWRsGEf9O+Cx4oJw1 +IphglhmhbgNksbj7bD24on/icldSOiVkoUemUOFmHWhCm4PnB1GmbD8YMfEdSbks +qDr1Ps1zg4mGOinVD/4cY7vuPFO/HCH07wfeaUGzRt4g0/yLr+XjVofOA3oowyxv +JAzr+niHA3lg5ecj4r7M68efwzN1OCyjMrVJw2RAzwvGxE+rm5NiT08SWlKQZnkC +gcEA4wvyLpIur/UB84nV3XVJ89UMNBLm++aTFzld047BLJtMaOhvNqx6Cl5c8VuW +l261KHjiVzpfNM3/A2LBQJcYkhX7avkqEXlj57cl+dCWAVwUzKmLJTPjfaTTZnYJ +xeN3dMYjJz2z2WtgvfvDoJLukVwIMmhTY8wtqqYyQBJ/l06pBsfw5TNvmVIOQHds +8ASOiFt+WRLk2bl9xrGGayqt3VV93KVRzF27cpjOgEcG74F3c0ZW9snERN7vIYwB +JfrlAoHBAMlahPwMP2TYylG8OzHe7EiehTekSO26LGh0Cq3wTGXYsK/q8hQCzL14 +kWW638vpwXL6L9ntvrd7hjzWRO3vX/VxnYEA6f0bpqHq1tZi6lzix5CTUN5McpDg +QnjenSJNrNjS1zEF8WeY9iLEuDI/M/iUW4y9R6s3WpgQhPDXpSvd2g3gMGRUYhxQ +Xna8auiJeYFq0oNaOxvJj+VeOfJ3ZMJttd+Y7gTOYZcbg3SdRb/kdxYki0RMD2hF +4ZvjJ6CTfwKBwQDiMqiZFTJGQwYqp4vWEmAW+I4r4xkUpWatoI2Fk5eI5T9+1PLX +uYXsho56NxEU1UrOg4Cb/p+TcBc8PErkGqR0BkpxDMOInTOXSrQe6lxIBoECVXc3 +HTbrmiay0a5y5GfCgxPKqIJhfcToAceoVjovv0y7S4yoxGZKuUEe7E8JY2iqRNAO +yOvKCCICv/hcN235E44RF+2/rDlOltagNej5tY6rIFkaDdgOF4bD7f9O5eEni1Bg +litfoesDtQP/3rECgcEAkQfvQ7D6tIPmbqsbJBfCr6fmoqZllT4FIJN84b50+OL0 +mTGsfjdqC4tdhx3sdu7/VPbaIqm5NmX10bowWgWSY7MbVME4yQPyqSwC5NbIonEC +d6N0mzoLR0kQ+Ai4u+2g82gicgAq2oj1uSNi3WZi48jQjHYFulCbo246o1NgeFFK +77WshYe2R1ioQfQDOU1URKCR0uTaMHClgfu112yiGd12JAD+aF3TM0kxDXz+sXI5 +SKy311DFxECZeXRLpcC3AoHBAJkNMJWTyPYbeVu+CTQkec8Uun233EkXa2kUNZc/ +5DuXDaK+A3DMgYRufTKSPpDHGaCZ1SYPInX1Uoe2dgVjWssRL2uitR4ENabDoAOA +ICVYXYYNagqQu5wwirF0QeaMXo1fjhuuHQh8GsMdXZvYEaAITZ9/NG5x/oY08+8H +kr78SMBOPy3XQn964uKG+e3JwpOG14GKABdAlrHKFXNWchu/6dgcYXB87mrC/GhO +zNwzC+QhFTZoOomFoqMgFWujng== -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIDWTCCAkGgAwIBAgIJAPm6B21bar2bMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV +MIIEWTCCAsGgAwIBAgIJAJinz4jHSjLtMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODAx -MTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH +IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODA4 +MjkxNDIzMTVaFw0yODA4MjYxNDIzMTVaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH DA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5k -YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEP -ADCCAQoCggEBAKvvsX2gEti4shve3iYMc+jE4Se7WHs1Bol2f21H8qNboDOFdeb1 -RKHjmq3exHpajywOUEgne9nKHJY/3f2phR4Y5klqG6liLgiSpVyRlcBGbeT2qEAj -9oLiLFUXLGfGDds2mTwivQDLJBWi51j7ff5k2Pr58fN5ugYMn24T9FNyn0moT+qj -SFoBNm58l9jrdkJSlgWfqPlbiMa+mqDn/SFtrwLF2Trbfzu42Sd9UdIzMaSSrzbN -sGm53pNhCh8KndWUQ8GPP2IsLPoUU4qAtmZuTxCx2S1cXrN9EkmT69tlOH84YfSn -96Ih9bWRc7M5y5bfVdEVM+fKQl3hBRf05qMCAwEAAaMYMBYwFAYDVR0RBA0wC4IJ -bG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQAtQ8f37cCEk7/rAcbYR53ce3iK -Vpihb0U2ni1QjG9Tg9UIExkIGkwTiCm7kwQL+GEStBu9AG/QVrOjeTriRiddhWkk -ze8kRaI3AC/63t6Vh9Q1x6PESgeE4OtAO9JpJCf4GILglA789Y/b/GF8zJZQxR13 -qpB4ZwWw7gCBhdEW59u6CFeBmfDa58hM8lWvuVoRrTi7bjUeC6PAn5HVMzZSykhu -4HaUfBp6bKFjuym2+h/VvM1n8C3chjVSmutsLb6ELdD8IK0vPV/yf5+LN256zSsS -dyUZYd8XwQaioEMKdbhLvnehyzHiWfQIUR3BdhONxoIJhHv/EAo8eCkHHYIF +YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGP +ADCCAYoCggGBALKUqUtopT6E68kN+uJNEt34i2EbmG/bwjcD8IaMsgJPSsMO2Bpd +3S6qWgkCeOyCfmAwBxK2kNbxGb63ouysEv7l8GCTJTWv3hG/HQcejJpnAEGi6K1U +fDbyE/db6yZ12SoHVTGkadN4vYGCPd1Wj9ZO1F877SHQ8rDWX3xgTWkxN2ojBw44 +T8RHSDiG8D/CvG4uEy+VUszL+Uvny5y2poNSqvI3J56sptWSrh8nIIbkPZPBdUne +LYMOHTFK3ZjXSmhlXgziTxK71nnzM3Y9K9gxPnRqoXbvu/wFo55hQCkETiRkYgmm +jXcBMZ0TClQVnQWuLjMthRnWFZs4Lfmwqjs7FZD/61581R2BYehvpWbLvvuOJhwv +DFzexL2sXcAl7SsxbzeQKRHqGbIDfbnQTXfs3/VC6Ye5P82P2ucj+XC32N9piRmO +gCBP8L3ub+YzzdxikZN2gZXXE2jsb3QyE/R2LkWdWyshpKe+RsZP1SBRbHShUyOh +yJ90baoiEwj2mwIDAQABoxgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZI +hvcNAQELBQADggGBAHRUO/UIHl3jXQENewYayHxkIx8t7nu40iO2DXbicSijz5bo +5//xAB6RxhBAlsDBehgQP1uoZg+WJW+nHu3CIVOU3qZNZRaozxiCl2UFKcNqLOmx +R3NKpo1jYf4REQIeG8Yw9+hSWLRbshNteP6bKUUf+vanhg9+axyOEOH/iOQvgk/m +b8wA8wNa4ujWljPbTQnj7ry8RqhTM0GcAN5LSdSvcKcpzLcs3aYwh+Z8e30sQWna +F40sa5u7izgBTOrwpcDm/w5kC46vpRQ5fnbshVw6pne2by0mdMECASid/p25N103 +jMqTFlmO7kpf/jpCSmamp3/JSEE1BJKHwQ6Ql4nzRA2N1mnvWH7Zxcv043gkHeAu +0x8evpvwuhdIyproejNFlBpKmW8OX7yKTCPPMC/VkX8Q1rVkxU0DQ6hmvwZlhoKa +9Wc2uXpw9xF8itV4Uvcdr3dwqByvIqn7iI/gB+4l41e0u8OmH2MKOx4Nxlly5TNW +HcVKQHyOeyvnINuBAQ== -----END CERTIFICATE----- diff --git a/Lib/test/keycert2.pem b/Lib/test/keycert2.pem index bb5fa65a8aca..ed6ae85a4649 100644 --- a/Lib/test/keycert2.pem +++ b/Lib/test/keycert2.pem @@ -1,49 +1,66 @@ -----BEGIN PRIVATE KEY----- -MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQC3ulRNfhbOAey/ -B+wIVYx+d5az7EV4riR6yi/qE6G+bxbTvay2pqySHtDweuaYSh2cVmcasBKKIFJm -rCD1zR8UmLb5i2XFIina1t3eePCuBZMrvZZwkzlQUSM1AZtjGOO/W0I3FwO6y645 -9xA5PduKI7SMYkH/VL3zE5W1JwMovv6bvNiT+GU5l6mB9ylCTgLpmUqoQhRqz/35 -zCzVyoh+ppDvVcpWYfvXywsXsgQwbAF0QJm8SSFi0TZm5ykv4WE16afQp08yuZS0 -3U4K3MJCa4rxO58edcxBopWYfQ29K3iINM8enRfr5q+u5mAAbALAEEvyFjgLWl/u -7arxn7bJAgMBAAECggEBAJfMt8KfHzBunrDnVrk8FayYGkfmOzAOkc1yKEx6k/TH -zFB+Mqlm5MaF95P5t3S0J+r36JBAUdEWC38RUNpF9BwMYYGlDxzlsTdCuGYL/q+J -o6NMLXQt7/jQUQqGnWAvPFzqhbcGqOo5R2ZVH25sEWv9PDuRI35XAepIkDTwWsfa -P6UcJJoP+4v9B++fb3sSL4zNwp1BqS4wxR8YTR0t1zQqOxJ5BGPw1J8aBMs1sq5t -qyosAQAT63kLrdqWotHaM26QxjqEQUMlh12XMWb5GdBXUxbvyGtEabsqskGa/f8B -RdHE437J8D8l+jxb2mZLzrlaH3dq2tbFGCe1rT8qLRECgYEA5CWIvoD/YnQydLGA -OlEhCSocqURuqcotg9Ev0nt/C60jkr/NHFLGppz9lhqjIDjixt3sIMGZMFzxRtwM -pSYal3XiR7rZuHau9iM35yDhpuytEiGbYy1ADakJRzY5jq/Qa8RfPP9Atua5xAeP -q6DiSnq9vhHv9G+O4MxzHBmrw9sCgYEAziiJWFthcwvuXn3Jv9xFYKEb/06puZAx -EgQCz/3rPzv5fmGD/sKVo1U/K4z/eA82DNeKG8QRTFJCxT8TCNRxOmGV7HdCYo/B -4BTNNvbKcdi3l0j75kKoADg+nt5CD5lz6gLG0GrUEnVO1y5HVfCTb3BEAfa36C85 -9i0sfQGiwysCgYEAuus9k8cgdct5oz3iLuVVSark/JGCkT2B+OOkaLChsDFUWeEm -7TOsaclpwldkmvvAYOplkZjMJ2GelE2pVo1XcAw3LkmaI5WpVyQXoxe/iQGT8qzy -IFlsh0Scw2lb0tmcyw6CcPk4TiHOxRrkzNrtS9QwLM+JZx0XVHptPPKTVc0CgYAu -j/VFYY5G/8Dc0qhIjyWUR48dQNUQtkJ/ASzpcT46z/7vznKTjbtiYpSb74KbyUO5 -7sygrM4DYOj3x+Eys1jHiNbly6HQxQtS4x/edCsRP5NntfI+9XsgYZOzKhvdjhki -F3J0DEzNxnUCIM+311hVaRPTJbgv1srOkTFlIoNydQKBgQC6/OHGaC/OewQqRlRK -Mg5KZm01/pk4iKrpA5nG7OTAeoa70NzXNtG8J3WnaJ4mWanNwNUOyRMAMrsUAy9q -EeGqHM5mMFpY4TeVuNLL21lu/x3KYw6mKL3Ctinn+JLAoYoqEy8deZnEA5/tjYlz -YhFBchnUicjoUN1chdpM6SpV2Q== +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQDKjrjWZlfOs1Ch +qt1RoyLfqyXbHVXIAW0fTzAxfJnxvFOiWqAAKgC2qVQM8Y080kRUuRaXP/w9ywXT ++MzX6tByy5VbTYJYyTjHOH46EWLNdcqEJs4+FCVqOIYrQPQ6pGAhCXmgBy4Vb42J +ABLwb+Kt+y2Dk15tggVcAHP2Khri+lRXWvda+kZAe2F1IojmuWyCTy3FEYHic5qN +BsXcf6u1oyFV8MybOuz1zGj3vd2C+dEKO4Ohw9rRwnvHSatjM+CfwiXf8kTXzDBF +Z/8W3+6yA49pHxRbG7FE3K1TAPhkrp+BVTIUOcdI74wEA6UEkWFF5sQcmmAth59M +EQrl2CXorftEPhsKZE59dUP1+nYAPvR/mTySNCSw7/rvdf+csRSZ5ollMu/lxsht +ulJYJI03+IiDTn47FI5D+IF25REK7d4LzGIo6T73ktsT+qpSXHuTWC+IABm8AMF9 +7ljxHSwMRU/z+O5uiONRItDAgKH/OItFG54PtY2vAhaO0YiZrZcCAwEAAQKCAYB2 +hTo8IVghlySH5B1p5kXCkDcvVaPaypLaLhCp9Blzq9lX9yUF043lU4Ddrf0RaIsY +88/3IjZqxb+cP0lE0Z20fdDfwqORZfQ2BaU+PuwMAm9EEhy9kDYwR/ChoHkHUyT4 +T7392BWr70Dmt8ddLmp5mK4R/gnTk6+lHJK9p/dhdk4haxWvAyBWHJty2Yk3T6nh +OYkzdUIFidUVza+6jG2hc1lPGv3tmnYKgNeulkblm10oWphz79C6ycx5WG7TNgef +CQ3z7//Nn89YTiaUBjLvoLvxRTMwO96r7E/FaslSl/fWnF3HP3lut26Z/mNfhiwj +qn7AhUwpSNPV0qcxFWXr/rXUjdk745wv8wOODK8atjjE/vt/MRBK0rAOIPSm3ecx +37PKNtR4i+sNeDEcY1IyTHE6wFvJSy5y8AFpn5y8tbqYfhlEVWZ4pcnlrKxhWm7j +oBkB/4GBjKQgbQ7ttym9eNG1wIbZ8v9N06+yeLs/NCc4bFZEgcWjFqBH1bLtAYEC +gcEA8tt8iYNqbsDH2ognjEmbbBxrDBmyYcEKRpg1i1SUopcZl8i93IHpG7EgJIaj +l7aWSbASAxjnK02t0VZ3nNS60acibzRwY/+e8OrSqlQdMXlAB2ggBA86drDJpfBl +WGJG8TJVY9bc1TU2uuwtZR1LAMSsRHVp+3IvKLpHrne5exPd3x6KGYcuaM+Uk/rE +u6tLsFNwaCdh+iBFFDT2bnYIw7jAsokJUkwxMVxSC0/21s2blhO/q5LsN1gFC1kN +TbpXAoHBANWE7TmG2szPvujPwrK18v6iJlHCA2n50AgAQXrsetj2JcF3HYHYdHnq +z36MQ6FpBKOiQumozWvb32WTjEwdG2kix7GEfam4DAUBdqYuCHzPcR12K5Tc8hsX +NG7JXUAeS8ZJEiOdu95X59JHyBxUQtNfte5rcbaV17SVw6K6bsWVJnj60YjtJrpa +xHvv1ZRnT2WEzJGpA+ii1h3I52N7ipGBiw172qcW+bKJukMi8eHxx5CC9e5tBpnu +C+Ou/eYewQKBwHxNa0jXQrq9YY2w8s0TP8HuKbxfyrXOIHxRm9ZczFcMD8VosgUT +WUUbO+B2KXWVtwawYAfFz0ySzcy//SkAmT6F1VIl/QCx7aBSENGti+Ous98WpIxv +XvUxN4T/rl+2raj2ok4fw5g9TG4QRIvkmmciQyonDr/sicbG0bmy/fTJDl8NOpIm +ZtKurNWxHNERtAPkMTyeK7/ilHjrQtb3AzVqcvbuvR6qcONa5YN0wlrfkisWoJwo +707EdpCAXBbUsQKBwQCnpzcpu2Sj+t9ZKIElF87T93gFLETH+ppJHgJMRdDz+NqO +fTwTD2XtsNz57aLQ44f8AFVv6NZbQYq41FEOFrDGLcQE9BZDpDrz10FVnMGXVr7n +tjjkK1SCxwapkr0AsoknCYsPojO4kud46loLPHI4TGeq7HyeNCvqJMo3RRHjXIiX +58GNNUD6hHjRI/FdFH14Jf0GxmJGUU20l2Jwb7nPJJuNm9mE53pqoNA7FP4+Pj1H +kD0Q2FSdmxeE0IuWHEECgcBgw6ogJ/FRRGLcym+aApqP9BChK+W8FDfDc9Mi4p/J +g+XmetWNFGCGTlOefGqUDIkwSG+QVOEN3hxziXbsjnvfpGApqoaulAI5oRvrwIcj +QIvD2mt0PB52k5ZL9QL2K9sgBa43BJDyCKooMAlTy2XMM+NyXVxQKmzf3r3jQ5sl +Rptk7ro38a9G8Rs99RFDyOmP1haOM0KXZvPksN4nsXuTlE01cnwnI29XKAlEZaoA +pQPLXD8W/KK4mwDbmokYXmo= -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIDYjCCAkqgAwIBAgIJALJXRr8qF6oIMA0GCSqGSIb3DQEBCwUAMGIxCzAJBgNV +MIIEYjCCAsqgAwIBAgIJAJm2YulYpr+6MA0GCSqGSIb3DQEBCwUAMGIxCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u IFNvZnR3YXJlIEZvdW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTAeFw0x -ODAxMTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMGIxCzAJBgNVBAYTAlhZMRcwFQYD +ODA4MjkxNDIzMTZaFw0yODA4MjYxNDIzMTZaMGIxCzAJBgNVBAYTAlhZMRcwFQYD VQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZv -dW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTCCASIwDQYJKoZIhvcNAQEB -BQADggEPADCCAQoCggEBALe6VE1+Fs4B7L8H7AhVjH53lrPsRXiuJHrKL+oTob5v -FtO9rLamrJIe0PB65phKHZxWZxqwEoogUmasIPXNHxSYtvmLZcUiKdrW3d548K4F -kyu9lnCTOVBRIzUBm2MY479bQjcXA7rLrjn3EDk924ojtIxiQf9UvfMTlbUnAyi+ -/pu82JP4ZTmXqYH3KUJOAumZSqhCFGrP/fnMLNXKiH6mkO9VylZh+9fLCxeyBDBs -AXRAmbxJIWLRNmbnKS/hYTXpp9CnTzK5lLTdTgrcwkJrivE7nx51zEGilZh9Db0r -eIg0zx6dF+vmr67mYABsAsAQS/IWOAtaX+7tqvGftskCAwEAAaMbMBkwFwYDVR0R -BBAwDoIMZmFrZWhvc3RuYW1lMA0GCSqGSIb3DQEBCwUAA4IBAQCZhHhGItpkqhEq -ntMRd6Hv0GoOJixNvgeMwK4NJSRT/no3OirtUTzccn46h+SWibSa2eVssAV+pAVJ -HbzkN/DH27A1mMx1zJL1ekcOKA1AF6MXhUnrUGXMqW36YNtzHfXJLrwvpLJ13OQg -/Kxo4Nw68bGzM+PyRtKU/mpgYyfcvwR+ZSeIDh1fvUZK/IEVCf8ub42GPVs5wPfv -M+k5aHxWTxeif3K1byTRzxHupYNG2yWO4XEdnBGOuOwzzN4/iQyNcsuQKeuKHGrt -YvIlG/ri04CQ7xISZCj74yjTZ+/A2bXre2mQXAHqKPumHL7cl34+erzbUaxYxbTE -u5FcOmLQ +dW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTCCAaIwDQYJKoZIhvcNAQEB +BQADggGPADCCAYoCggGBAMqOuNZmV86zUKGq3VGjIt+rJdsdVcgBbR9PMDF8mfG8 +U6JaoAAqALapVAzxjTzSRFS5Fpc//D3LBdP4zNfq0HLLlVtNgljJOMc4fjoRYs11 +yoQmzj4UJWo4hitA9DqkYCEJeaAHLhVvjYkAEvBv4q37LYOTXm2CBVwAc/YqGuL6 +VFda91r6RkB7YXUiiOa5bIJPLcURgeJzmo0Gxdx/q7WjIVXwzJs67PXMaPe93YL5 +0Qo7g6HD2tHCe8dJq2Mz4J/CJd/yRNfMMEVn/xbf7rIDj2kfFFsbsUTcrVMA+GSu +n4FVMhQ5x0jvjAQDpQSRYUXmxByaYC2Hn0wRCuXYJeit+0Q+GwpkTn11Q/X6dgA+ +9H+ZPJI0JLDv+u91/5yxFJnmiWUy7+XGyG26UlgkjTf4iINOfjsUjkP4gXblEQrt +3gvMYijpPveS2xP6qlJce5NYL4gAGbwAwX3uWPEdLAxFT/P47m6I41Ei0MCAof84 +i0Ubng+1ja8CFo7RiJmtlwIDAQABoxswGTAXBgNVHREEEDAOggxmYWtlaG9zdG5h +bWUwDQYJKoZIhvcNAQELBQADggGBAMIVLp6e6saH2NQSg8iFg8Ewg/K/etI++jHo +gCJ697AY02wtfrBox1XtljlmI2xpJtVAYZWHhrNqwrEG43aB7YEV6RqTcG6QUVqa +NbD8iNCnMKm7fP89hZizmqA1l4aHnieI3ucOqpgooM7FQwLX6qk+rSue6lD5N/5f +avsublnj8rNKyDfHpQ3AWduLoj8QqctpzI3CqoDZNLNzaDnzVWpxT1SKDQ88q7VI +W5zb+lndpdQlCu3v5HM4w5UpwL/k1htl/z6PnPseS2UdlXv6A8KITnCLg5PLP4tz +2oTAg9gjOtRP/0uwkhvicwoFzFJNVT813lzTLE1jlobMPiZhsS1mjaJGPD9GQZDK +ny3j8ogrIRGjnI4xpOMNNDVphcvwtV8fRbvURSHCj9Y4kCLpD5ODuoyEyLYicJIv +GZP456GP0iSCK5GKO0ij/YzGCkPWD5zA+mYFpMMGZPTwajenMw7TVaPXcc9CZBtr +oOjwwiLEqdkpxUj13mJYTlt5wsS/Kw== -----END CERTIFICATE----- diff --git a/Lib/test/keycert3.pem b/Lib/test/keycert3.pem index 621eb08bb0ce..e0a8205a660e 100644 --- a/Lib/test/keycert3.pem +++ b/Lib/test/keycert3.pem @@ -1,64 +1,84 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDgV4G+Zzf2DT5n -oAisIGFhn/bz7Vn5WiXUqbDsxROJOh/7BtOlduZka0pPhFylGbnxS8l1kEWHRI2T -6hOoWzumB6ItKiN+T5J30lAvSyo7iwdFoAQ/S5nPXQfhNARQe/NEOhRtpcuNdyx4 -BWdPdPuJQA1ASNJCLwcLOoRxaLbKLvb2V5T5FCAkeNPtRvPuT4gKQItMmiHfAhoV -C8MZWF/GC0RukHINys5MwqeFexam8CznmQPMYrLdhmKTj3DTivCPoh97EDIFGlgZ -SCaaYDVQA+aqlo/q2pi52PFwC1KzhNEA7EeOqSwC1NQjwjHuhcnf9WbxrgTq2zh3 -rv5YEW2ZAgMBAAECggEAPfSMtTumPcJskIumuXp7yk02EyliZrWZqwBuBwVqHsS5 -nkbFXnXWrLbgn9MrDsFrE5NdgKUmPnQVMVs8sIr5jyGejSCNCs4I4iRn1pfIgwcj -K/xEEALd6GGF0pDd/CgvB5GOoLVf4KKf2kmLvWrOKJpSzoUN5A8+v8AaYYOMr4sC -czbvfGomzEIewEG+Rw9zOVUDlmwyEKPQZ47E7PQ+EEA7oeFdR+1Zj6eT9ndegf8B -54frySYCLRUCk/sHCpWhaJBtBrcpht7Y8CfY7hiH/7x866fvuLnYPz4YALtUb0wN -7zUCNS9ol3n4LbjFFKfZtiRjKaCBRzMjK0rz6ydFcQKBgQDyLI3oGbnW73vqcDe/ -6eR0w++fiCAVhfMs3AO/gOaJi2la2JHlJ5u+cIHQIOFwEhn6Zq0AtdmnFx1TS5IQ -C0MdXI0XoQQw7rEF8EJcvfe85Z0QxENVhzydtdb8QpJfnQGfBfLyQlaaRYzRRHB6 -VdYUHF3EIPVIhbjbghal+Qep/QKBgQDtJlRPHkWwTMevu0J0fYbWN1ywtVTFUR// -k7VyORSf8yuuSnaQRop4cbcqONxmDKH6Or1fl3NYBsAxtXkkOK1E2OZNo2sfQdRa -wpA7o7mPHRhztQFpT5vflp+8P6+PEFat8D04eBOhNwrwwfhiPjD4gv5KvN4XutRW -VWv/2pnmzQKBgHPvHGg2mJ7quvm6ixXW1MWJX1eSBToIjCe3lBvDi5nhIaiZ8Q4w -7gA3QA3xD7tlDwauzLeAVxgEmsdbcCs6GQEfY3QiYy1Bt4FOSZa4YrcNfSmfq1Rw -j3Y4rRjKjeQz96i3YlzToT3tecJc7zPBj+DEy6au2H3Fdn+vQURneWHJAoGBANG7 -XES8mRVaUh/wlM1BVsaNH8SIGfiHzqzRjV7/bGYpQTBbWpAuUrhCmaMVtpXqBjav -TFwGLVRkZAWSYRjPpy2ERenT5SE3rv61o6mbGrifGsj6A82HQmtzYsGx8SmtYXtj -REF0sKebbmmOooUAS379GrguYJzL9o6D7YfRZNrhAoGAVfb/tiFU4S67DSpYpQey -ULhgfsFpDByICY6Potsg67gVFf9jIaB83NPTx3u/r6sHFgxFw7lQsuZcgSuWMu7t -glzOXVIP11Y5sl5CJ5OsfeK1/0umMZF5MWPyAQCx/qrPlZL86vXjt24Y/VaOxsAi -CZYdyJsjgOrJrWoMbo5ta54= +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQCfKC83Qe9/ZGMW +YhbpARRiKco6mJI9CNNeaf7A89TE+w5Y3GSwS8uzqp5C6QebZzPNueg8HYoTwN85 +Z3xM036/Qw9KhQVth+XDAqM+19e5KHkYcxg3d3ZI1HgY170eakaLBvMDN5ULoFOw +Is2PtwM2o9cjd5mfSuWttI6+fCqop8/l8cerG9iX2GH39p3iWwWoTZuYndAA9qYv +07YWajuQ1ESWKPjHYGTnMvu4xIzibC1mXd2M6u/IjNO6g426SKFaRDWQkx01gIV/ +CyKs9DgZoeMHkKZuPqZVOxOK+A/NrmrqHFsPIsrs5wk7QAVju5/X1skpn/UGQlmM +RwBaQULOs1FagA+54RXU6qUPW0YmhJ4xOB4gHHD1vjAKEsRZ7/6zcxMyOm+M1DbK +RTH4NWjVWpnY8XaVGdRhtTpH9MjycpKhF+D2Zdy2tQXtqu2GdcMnUedt13fn9xDu +P4PophE0ip/IMgn+kb4m9e+S+K9lldQl0B+4BcGWAqHelh2KuU0CAwEAAQKCAYEA +lKiWIYjmyRjdLKUGPTES9vWNvNmRjozV0RQ0LcoSbMMLDZkeO0UwyWqOVHUQ8+ib +jIcfEjeNJxI57oZopeHOO5vJhpNlFH+g7ltiW2qERqA1K88lSXm99Bzw6FNqhCRE +K8ub5N9fyfJA+P4o/xm0WK8EXk5yIUV17p/9zJJxzgKgv2jsVTi3QG2OZGvn4Oug +ByomMZEGHkBDzdxz8c/cP1Tlk1RFuwSgews178k2xq7AYSM/s0YmHi7b/RSvptX6 +1v8P8kXNUe4AwTaNyrlvF2lwIadZ8h1hA7tCE2n44b7a7KfhAkwcbr1T59ioYh6P +zxsyPT678uD51dbtD/DXJCcoeeFOb8uzkR2KNcrnQzZpCJnRq4Gp5ybxwsxxuzpr +gz0gbNlhuWtE7EoSzmIK9t+WTS7IM2CvZymd6/OAh1Fuw6AQhSp64XRp3OfMMAAC +Ie2EPtKj4islWGT8VoUjuRYGmdRh4duAH1dkiAXOWA3R7y5a1/y/iE8KE8BtxocB +AoHBAM8aiURgpu1Fs0Oqz6izec7KSLL3l8hmW+MKUOfk/Ybng6FrTFsL5YtzR+Ap +wW4wwWnnIKEc1JLiZ7g8agRETK8hr5PwFXUn/GSWC0SMsazLJToySQS5LOV0tLzK +kJ3jtNU7tnlDGNkCHTHSoVL2T/8t+IkZI/h5Z6wjlYPvU2Iu0nVIXtiG+alv4A6M +Hrh9l5or4mjB6rGnVXeYohLkCm6s/W97ahVxLMcEdbsBo1prm2JqGnSoiR/tEFC/ +QHQnbQKBwQDEu7kW0Yg9sZ89QtYtVQ1YpixFZORaUeRIRLnpEs1w7L1mCbOZ2Lj9 +JHxsH05cYAc7HJfPwwxv3+3aGAIC/dfu4VSwEFtatAzUpzlhzKS5+HQCWB4JUNNU +MQ3+FwK2xQX4Ph8t+OzrFiYcK2g0An5UxWMa2HWIAWUOhnTOydAVsoH6yP31cVm4 +0hxoABCwflaNLNGjRUyfBpLTAcNu/YtcE+KREy7YAAgXXrhRSO4XpLsSXwLnLT7/ +YOkoBWDcTWECgcBPWnSUDZCIQ3efithMZJBciqd2Y2X19Dpq8O31HImD4jtOY0V7 +cUB/wSkeHAGwjd/eCyA2e0x8B2IEdqmMfvr+86JJxekC3dJYXCFvH5WIhsH53YCa +3bT1KlWCLP9ib/g+58VQC0R/Cc9T4sfLePNH7D5ZkZd1wlbV30CPr+i8KwKay6MD +xhvtLx+jk07GE+E9wmjbCMo7TclyrLoVEOlqZMAqshgApT+p9eyCPetwXuDHwa3n +WxhHclcZCV7R4rUCgcAkdGSnxcvpIrDPOUNWwxvmAWTStw9ZbTNP8OxCNCm9cyDl +d4bAS1h8D/a+Uk7C70hnu7Sl2w7C7Eu2zhwRUdhhe3+l4GINPK/j99i6NqGPlGpq +xMlMEJ4YS768BqeKFpg0l85PRoEgTsphDeoROSUPsEPdBZ9BxIBlYKTkbKESZDGR +twzYHljx1n1NCDYPflmrb1KpXn4EOcObNghw2KqqNUUWfOeBPwBA1FxzM4BrAStp +DBINpGS4Dc0mjViVegECgcA3hTtm82XdxQXj9LQmb/E3lKx/7H87XIOeNMmvjYuZ +iS9wKrkF+u42vyoDxcKMCnxP5056wpdST4p56r+SBwVTHcc3lGBSGcMTIfwRXrj3 +thOA2our2n4ouNIsYyTlcsQSzifwmpRmVMRPxl9fYVdEWUgB83FgHT0D9avvZnF9 +t9OccnGJXShAIZIBADhVj/JwG4FbaX42NijD5PNpVLk1Y17OV0I576T9SfaQoBjJ +aH1M/zC4aVaS0DYB/Gxq7v8= -----END PRIVATE KEY----- Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9c - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5c + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Nov 28 19:09:06 2027 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Jul 7 14:23:16 2028 GMT Subject: C=XY, L=Castle Anthrax, O=Python Software Foundation, CN=localhost Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:e0:57:81:be:67:37:f6:0d:3e:67:a0:08:ac:20: - 61:61:9f:f6:f3:ed:59:f9:5a:25:d4:a9:b0:ec:c5: - 13:89:3a:1f:fb:06:d3:a5:76:e6:64:6b:4a:4f:84: - 5c:a5:19:b9:f1:4b:c9:75:90:45:87:44:8d:93:ea: - 13:a8:5b:3b:a6:07:a2:2d:2a:23:7e:4f:92:77:d2: - 50:2f:4b:2a:3b:8b:07:45:a0:04:3f:4b:99:cf:5d: - 07:e1:34:04:50:7b:f3:44:3a:14:6d:a5:cb:8d:77: - 2c:78:05:67:4f:74:fb:89:40:0d:40:48:d2:42:2f: - 07:0b:3a:84:71:68:b6:ca:2e:f6:f6:57:94:f9:14: - 20:24:78:d3:ed:46:f3:ee:4f:88:0a:40:8b:4c:9a: - 21:df:02:1a:15:0b:c3:19:58:5f:c6:0b:44:6e:90: - 72:0d:ca:ce:4c:c2:a7:85:7b:16:a6:f0:2c:e7:99: - 03:cc:62:b2:dd:86:62:93:8f:70:d3:8a:f0:8f:a2: - 1f:7b:10:32:05:1a:58:19:48:26:9a:60:35:50:03: - e6:aa:96:8f:ea:da:98:b9:d8:f1:70:0b:52:b3:84: - d1:00:ec:47:8e:a9:2c:02:d4:d4:23:c2:31:ee:85: - c9:df:f5:66:f1:ae:04:ea:db:38:77:ae:fe:58:11: - 6d:99 + 00:9f:28:2f:37:41:ef:7f:64:63:16:62:16:e9:01: + 14:62:29:ca:3a:98:92:3d:08:d3:5e:69:fe:c0:f3: + d4:c4:fb:0e:58:dc:64:b0:4b:cb:b3:aa:9e:42:e9: + 07:9b:67:33:cd:b9:e8:3c:1d:8a:13:c0:df:39:67: + 7c:4c:d3:7e:bf:43:0f:4a:85:05:6d:87:e5:c3:02: + a3:3e:d7:d7:b9:28:79:18:73:18:37:77:76:48:d4: + 78:18:d7:bd:1e:6a:46:8b:06:f3:03:37:95:0b:a0: + 53:b0:22:cd:8f:b7:03:36:a3:d7:23:77:99:9f:4a: + e5:ad:b4:8e:be:7c:2a:a8:a7:cf:e5:f1:c7:ab:1b: + d8:97:d8:61:f7:f6:9d:e2:5b:05:a8:4d:9b:98:9d: + d0:00:f6:a6:2f:d3:b6:16:6a:3b:90:d4:44:96:28: + f8:c7:60:64:e7:32:fb:b8:c4:8c:e2:6c:2d:66:5d: + dd:8c:ea:ef:c8:8c:d3:ba:83:8d:ba:48:a1:5a:44: + 35:90:93:1d:35:80:85:7f:0b:22:ac:f4:38:19:a1: + e3:07:90:a6:6e:3e:a6:55:3b:13:8a:f8:0f:cd:ae: + 6a:ea:1c:5b:0f:22:ca:ec:e7:09:3b:40:05:63:bb: + 9f:d7:d6:c9:29:9f:f5:06:42:59:8c:47:00:5a:41: + 42:ce:b3:51:5a:80:0f:b9:e1:15:d4:ea:a5:0f:5b: + 46:26:84:9e:31:38:1e:20:1c:70:f5:be:30:0a:12: + c4:59:ef:fe:b3:73:13:32:3a:6f:8c:d4:36:ca:45: + 31:f8:35:68:d5:5a:99:d8:f1:76:95:19:d4:61:b5: + 3a:47:f4:c8:f2:72:92:a1:17:e0:f6:65:dc:b6:b5: + 05:ed:aa:ed:86:75:c3:27:51:e7:6d:d7:77:e7:f7: + 10:ee:3f:83:e8:a6:11:34:8a:9f:c8:32:09:fe:91: + be:26:f5:ef:92:f8:af:65:95:d4:25:d0:1f:b8:05: + c1:96:02:a1:de:96:1d:8a:b9:4d Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Alternative Name: @@ -70,11 +90,11 @@ Certificate: X509v3 Basic Constraints: critical CA:FALSE X509v3 Subject Key Identifier: - 85:11:BE:16:47:04:D1:30:EE:86:8A:18:70:BE:A8:28:6F:82:3D:CE + 8F:EA:1D:E3:33:5C:00:16:B3:8B:6F:6B:6F:D3:4C:CB:B5:CB:7C:55 X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 DirName:/C=XY/O=Python Software Foundation CA/CN=our-ca-server - serial:82:ED:BF:41:C8:80:91:9B + serial:CB:2D:80:99:5A:69:52:5B Authority Information Access: CA Issuers - URI:http://testca.pythontest.net/testca/pycacert.cer @@ -85,48 +105,60 @@ Certificate: Full Name: URI:http://testca.pythontest.net/testca/revocation.crl - Signature Algorithm: sha1WithRSAEncryption - 7f:a1:7e:3e:68:01:b0:32:b8:57:b8:03:68:13:13:b3:e3:f4: - 70:2f:15:e5:0f:87:b9:fd:e0:12:e3:16:f2:91:53:c7:4e:25: - af:ca:cb:a7:d9:9d:57:4d:bf:a2:80:d4:78:aa:04:31:fd:6d: - cc:6d:82:43:e9:62:16:0d:0e:26:8b:e7:f1:3d:57:5c:68:02: - 9c:2b:b6:c9:fd:62:2f:10:85:88:cc:44:a5:e7:a2:3e:89:f2: - 1f:02:6a:3f:d0:3c:6c:24:2d:bc:51:62:7a:ec:25:c5:86:87: - 77:35:8f:f9:7e:d0:17:3d:77:56:bf:1a:0c:be:09:78:ee:ea: - 73:97:65:60:94:91:35:b3:5c:46:8a:5e:6d:94:52:de:48:b7: - 1f:6c:28:79:7f:ff:08:8d:e4:7d:d0:b9:0b:7c:ae:c4:1d:2a: - a1:b3:50:11:82:03:5e:6c:e7:26:fa:05:32:39:07:83:49:b9: - a2:fa:04:da:0d:e5:ff:4c:db:97:d0:c3:a7:43:37:4c:16:de: - 3c:b5:e9:7e:82:d4:b3:10:df:d1:c1:66:72:9c:15:67:19:3b: - 7b:91:0a:82:07:67:c5:06:03:5f:80:54:08:81:8a:b1:5c:7c: - 4c:d2:07:38:92:eb:12:f5:71:ae:de:05:15:c8:e1:33:f0:e4: - 96:0f:0f:1e + Signature Algorithm: sha256WithRSAEncryption + 27:f5:8c:59:10:f4:c6:e7:28:00:bf:ba:8d:7b:13:03:f1:1c: + a6:5f:b3:06:55:a4:22:b9:db:b2:d5:46:bd:f7:0c:dd:43:6e: + b4:79:65:67:21:0c:2a:55:ee:40:8e:85:9f:9f:47:bb:0a:2a: + 4d:b6:64:74:98:a0:7f:ae:dc:f1:2e:db:42:77:18:e0:75:8b: + 26:35:68:c3:41:ed:6b:c8:77:72:6f:6a:9a:5d:55:69:02:fd: + 5a:54:c8:57:cb:b0:65:03:16:e2:0f:00:39:99:66:a0:9b:88: + 93:17:e2:5a:2d:79:35:5f:97:57:78:c4:af:f5:99:5e:86:ab: + d3:11:ad:1a:a2:0d:fa:52:10:b9:fe:bf:9d:ce:33:d9:86:b2: + 9c:16:f8:d6:75:08:8a:db:0a:e5:b4:2b:16:7f:b4:f9:2a:9f: + c3:d2:77:d7:cd:65:1e:f4:6c:1e:eb:59:b9:f0:ae:5f:a4:1f: + cc:4a:c4:b9:7a:a9:d9:6b:32:68:3b:e1:65:b0:84:b7:90:c4: + ae:fe:f4:37:4f:21:a0:de:9f:3a:b1:e5:cc:16:04:66:3f:0b: + 41:dc:42:3d:20:3e:ec:b7:95:2b:35:57:fa:be:7f:b6:3a:ba: + ca:4f:58:fe:75:3e:08:89:2c:8c:b0:5d:2e:f9:89:10:2b:f9: + 41:46:4f:3c:00:b7:27:d3:65:24:28:17:23:26:31:42:ea:7e: + 4e:93:e4:7b:68:54:ca:9f:46:f3:ef:2b:e9:85:0c:b5:84:b2: + d5:35:34:80:75:2b:f0:91:23:b8:08:01:8e:b9:0a:54:d4:fb: + 34:52:fe:d9:45:f0:80:3b:b6:c1:6f:82:d1:1f:f2:3b:08:f6: + 46:a6:96:27:61:4b:58:32:7a:0e:1d:59:c5:44:ad:5e:1a:79: + 33:c1:d4:05:2f:4a:d3:d8:42:42:8d:33:e3:63:ca:d5:87:97: + 9b:4d:b8:1a:03:34:bb:1c:d2:02:3f:59:23:e2:23:80:88:63: + c2:f0:a2:63:a8:8b -----BEGIN CERTIFICATE----- -MIIE8TCCA9mgAwIBAgIJAILtv0HIgJGcMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIF8TCCBFmgAwIBAgIJAMstgJlaaVJcMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yNzExMjgx -OTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA3MDcx +NDIzMTZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj MCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxv -Y2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOBXgb5nN/YN -PmegCKwgYWGf9vPtWflaJdSpsOzFE4k6H/sG06V25mRrSk+EXKUZufFLyXWQRYdE -jZPqE6hbO6YHoi0qI35PknfSUC9LKjuLB0WgBD9Lmc9dB+E0BFB780Q6FG2ly413 -LHgFZ090+4lADUBI0kIvBws6hHFotsou9vZXlPkUICR40+1G8+5PiApAi0yaId8C -GhULwxlYX8YLRG6Qcg3KzkzCp4V7FqbwLOeZA8xist2GYpOPcNOK8I+iH3sQMgUa -WBlIJppgNVAD5qqWj+ramLnY8XALUrOE0QDsR46pLALU1CPCMe6Fyd/1ZvGuBOrb -OHeu/lgRbZkCAwEAAaOCAcAwggG8MBQGA1UdEQQNMAuCCWxvY2FsaG9zdDAOBgNV -HQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1Ud -EwEB/wQCMAAwHQYDVR0OBBYEFIURvhZHBNEw7oaKGHC+qChvgj3OMH0GA1UdIwR2 -MHSAFJrPz27rcT3bPPGuiGtWcgPLCKdIoVGkTzBNMQswCQYDVQQGEwJYWTEmMCQG -A1UECgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0ExFjAUBgNVBAMMDW91 -ci1jYS1zZXJ2ZXKCCQCC7b9ByICRmzCBgwYIKwYBBQUHAQEEdzB1MDwGCCsGAQUF -BzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3RjYS9weWNhY2Vy -dC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQv -dGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly90ZXN0Y2EucHl0 -aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3JsMA0GCSqGSIb3DQEBBQUA -A4IBAQB/oX4+aAGwMrhXuANoExOz4/RwLxXlD4e5/eAS4xbykVPHTiWvysun2Z1X -Tb+igNR4qgQx/W3MbYJD6WIWDQ4mi+fxPVdcaAKcK7bJ/WIvEIWIzESl56I+ifIf -Amo/0DxsJC28UWJ67CXFhod3NY/5ftAXPXdWvxoMvgl47upzl2VglJE1s1xGil5t -lFLeSLcfbCh5f/8IjeR90LkLfK7EHSqhs1ARggNebOcm+gUyOQeDSbmi+gTaDeX/ -TNuX0MOnQzdMFt48tel+gtSzEN/RwWZynBVnGTt7kQqCB2fFBgNfgFQIgYqxXHxM -0gc4kusS9XGu3gUVyOEz8OSWDw8e +Y2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAJ8oLzdB739k +YxZiFukBFGIpyjqYkj0I015p/sDz1MT7DljcZLBLy7OqnkLpB5tnM8256DwdihPA +3zlnfEzTfr9DD0qFBW2H5cMCoz7X17koeRhzGDd3dkjUeBjXvR5qRosG8wM3lQug +U7AizY+3Azaj1yN3mZ9K5a20jr58Kqinz+Xxx6sb2JfYYff2neJbBahNm5id0AD2 +pi/TthZqO5DURJYo+MdgZOcy+7jEjOJsLWZd3Yzq78iM07qDjbpIoVpENZCTHTWA +hX8LIqz0OBmh4weQpm4+plU7E4r4D82uauocWw8iyuznCTtABWO7n9fWySmf9QZC +WYxHAFpBQs6zUVqAD7nhFdTqpQ9bRiaEnjE4HiAccPW+MAoSxFnv/rNzEzI6b4zU +NspFMfg1aNVamdjxdpUZ1GG1Okf0yPJykqEX4PZl3La1Be2q7YZ1wydR523Xd+f3 +EO4/g+imETSKn8gyCf6Rvib175L4r2WV1CXQH7gFwZYCod6WHYq5TQIDAQABo4IB +wDCCAbwwFAYDVR0RBA0wC4IJbG9jYWxob3N0MA4GA1UdDwEB/wQEAwIFoDAdBgNV +HSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4E +FgQUj+od4zNcABazi29rb9NMy7XLfFUwfQYDVR0jBHYwdIAU3b/K2ubRNLo3dSHK +b5oIKPI1tkihUaRPME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29m +dHdhcmUgRm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcoIJAMst +gJlaaVJbMIGDBggrBgEFBQcBAQR3MHUwPAYIKwYBBQUHMAKGMGh0dHA6Ly90ZXN0 +Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3B5Y2FjZXJ0LmNlcjA1BggrBgEFBQcw +AYYpaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0Y2Evb2NzcC8wQwYD +VR0fBDwwOjA4oDagNIYyaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0 +Y2EvcmV2b2NhdGlvbi5jcmwwDQYJKoZIhvcNAQELBQADggGBACf1jFkQ9MbnKAC/ +uo17EwPxHKZfswZVpCK527LVRr33DN1DbrR5ZWchDCpV7kCOhZ+fR7sKKk22ZHSY +oH+u3PEu20J3GOB1iyY1aMNB7WvId3JvappdVWkC/VpUyFfLsGUDFuIPADmZZqCb +iJMX4loteTVfl1d4xK/1mV6Gq9MRrRqiDfpSELn+v53OM9mGspwW+NZ1CIrbCuW0 +KxZ/tPkqn8PSd9fNZR70bB7rWbnwrl+kH8xKxLl6qdlrMmg74WWwhLeQxK7+9DdP +IaDenzqx5cwWBGY/C0HcQj0gPuy3lSs1V/q+f7Y6uspPWP51PgiJLIywXS75iRAr ++UFGTzwAtyfTZSQoFyMmMULqfk6T5HtoVMqfRvPvK+mFDLWEstU1NIB1K/CRI7gI +AY65ClTU+zRS/tlF8IA7tsFvgtEf8jsI9kamlidhS1gyeg4dWcVErV4aeTPB1AUv +StPYQkKNM+NjytWHl5tNuBoDNLsc0gI/WSPiI4CIY8LwomOoiw== -----END CERTIFICATE----- diff --git a/Lib/test/keycert4.pem b/Lib/test/keycert4.pem index b7df7f3f2c71..d1ebb82486de 100644 --- a/Lib/test/keycert4.pem +++ b/Lib/test/keycert4.pem @@ -1,64 +1,84 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDH/76hZAZH4cSV -CmVZa5HEqKCjCKrcPwBECs9BS+3ibwN4x9NnFNP+tCeFGgJXl7WGFoeXgg3oK+1p -FsOWpsRHuF3BdqkCnShSydmT8bLaGHwKeL0cPxJP5T/uW7ezPKW2VWXGMwmwRaRJ -9dj2VCUu20vDZWSGFr9zjnjoJczBtH3RsVUgpK7euEHuQ5pIM9QSOaCo+5FPR7s7 -1nU7YqbFWtd+NhC8Og1G497B31DQlHciF6BRm6/cNGAmHaAErKUGBFdkGtFPHBn4 -vktoEg9fwxJAZLvGpoTZWrB4HRsRwVTmFdGvK+JXK225xF23AXRXp/snhSuSFeLj -E5cpyJJ7AgMBAAECggEAQOv527X2e/sDr0XSpHZQuT/r9UBpBlnFIlFH+fBF5k0X -GWv0ae/O6U1dzs0kmX57xG0n0ry6+vTXeleTYiH8cTOd66EzN9AAOO+hG29IGZf9 -HAEZkkO/FARc/mjzdtFnEYsjIHWM3ZWdwQx3Q28JKu6w51rQiN51g3NqOCGdF/uF -rE5XPKsKndn+nLHvsNuApFgUYZEwdrozgUueEgRaPTUCNhzotcA9eWoBdA24XNhk -x8Cm/bZWabXm7gBO75zl3Cu2F21ay+EuwyOZTsx6lZi6YX9/zo1mkO81Zi3tQk50 -NMEI0feLNwsdxTbmOcVJadjOgd+QVghlFyr5HGBWMQKBgQD3AH3rhnAo6tOyNkGN -+IzIU1MhUS452O7IavykUYO9sM24BVChpRtlI9Dpev4yE/q3BAO3+oWT3cJrN7/3 -iyo1dzAkpGvI65XWfElXFM4nLjEiZzx4W9fiPN91Oucpr0ED6+BZXTtz4gVm0TP/ -TUc2xvTB6EKvIyWmKOYEi0snxQKBgQDPSOjbz9jWOrC9XY7PmtLB6QJDDz7XSGVK -wzD+gDAPpAwhk58BEokdOhBx2Lwl8zMJi0CRHgH2vNvkRyhvUQ4UFzisrqann/Tw -klp5sw3iWC6ERC8z9zL7GfHs7sK3mOVeAdK6ffowPM3JrZ2vPusVBdr0MN3oZwki -CtNXqbY1PwKBgGheQNbAW6wubX0kB9chavtKmhm937Z5v4vYCSC1gOEqUAKt3EAx -L74wwBmn6rjmUE382EVpCgBM99WuHONQXmlxD1qsTw763LlgkuzE0cckcYaD8L06 -saHa7uDuHrcyYlpx1L5t8q0ol/e19i6uTKUMtGcq6OJwC3yGU4sgAIWxAoGBAMVq -qiQXm2vFL+jafxYoXUvDMJ1PmskMsTP4HOR2j8+FrOwZnVk3HxGP6HOVOPRn4JbZ -YiAT1Uj6a+7I+rCyINdvmlGUcTK6fFzW9oZryvBkjcD483/pkktmVWwTpa2YV/Ml -h16IdsyUTGYlDUYHhXtbPUJOfDpIT4F1j/0wrFGfAoGAO82BcUsehEUQE0xvQLIn -7QaFtUI5z19WW730jVuEobiYlh9Ka4DPbKMvka8MwyOxEwhk39gZQavmfG6+wZm+ -kjERU23LhHziJGWS2Um4yIhC7myKbWaLzjHEq72dszLpQku4BzE5fT60fxI7cURD -WGm/Z3Q2weS3ZGIoMj1RNPI= +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQDGjpiHzq7ghxhM +ZzrnRsGBC/cmw8EREIdbqlrz/l8BFaWeipvO5Hb/MyU8xs2zLUrqIr2JNf+Eii8Y +m4bYmZclFra4jomaiSlxTZOe3dMV8m4vAq4eT2mSfZZC1+XAutqdz7WhHxhMVEm3 +AyTWvTC3qCbnlbX5VIoQUwFrsSWqDiHyaGdK3rrOTKFUKM8YPiq/BZkm6A4eiFci +5wd/SPD+w0pIscZbQW1MUr5bs54uylWaUmtfI8KJt6BDZQ/uA06c6i863sSCEI6L +gq+wyikeJGNMxZMfgu3dzfv4BiZBQX0ZhiRvqseDSdPcuVa2Ifb6CFlg298neweY +4EAIE1O+uqo5h8FF1aUOMZpQEZuzsp9R/TAMBHX1YmVjG/kRdBeaHe3whzB1Pfue +PIX2ZTMmLNYbYbfnmxhk1nn8aAvoT98pNw8y3/2k2KNsu24n9uSkkxAoqJ19WKwm +mL8MpJKAzLv45tRvhN+QLtnRdu+LJ9m29npQHFmYLbdqRfmidnMCAwEAAQKCAYBd +w1C8MRnb5W/QBJ+IP515NxFLOP2e9VM2MkgpGGH8vSAssf/Jv5GCCcD35lmU1zqd +PjKK7PjwueBrmmYfOshpN0Sp+oV4eHUdkCi5yL65inYFtRpMLewIxU2D2zgfvx0l +kMSQhYKP6O22gsGOtmCfGcTlb4kzaHyaINh25nyGxY26TxsX+/3zFbTJbUv+grzk +39vmx4aDXJbpYHfl36gOZmJZ2bl1tnvKovhJjZSRO/MYoPsbPmPLbO89ZCgVmXFc +GVkb5Cram6i3iyutSDjxWN7Fb8uy8pFLPGAXZgF7pQoXPSEHZe8GEWBnWSC9KaDa +uM9Ir847/Muy1ceCmxKcI2WrSjoH2AhPcmHgvbPE9Mynr6+uzReSP3q7Wh9PHm23 +oFx3DwdCfmjysnpAMBawNmJdWyxVDbZ6eyrhp17ADpsMaDTynZ+fJjgMr+MmWtbU +YSRD0wWtl/DrzsaePZsOjCpKYJyulC+rh9/Zz1aiwrGWPbgEAzDrD6Q1Zp0mUCEC +gcEA+XskmGIB9rRPy+YQmRgzQ555PsjLWsnQsNktP6KBhlQjFKJZXRZ0DxDTS7h8 +NrJrUDBmwfsgzggVbeO55VP5FGwD6DNeO/Bz++Fdevh8uKQFHDfk4sbIUPS91qw4 +s7OW7PR7C7Jf7Dnjmsn42o2lO4FsbcEn2F+PHOvoLl/OrSx73lS/RkdOEItW8d8/ +ExRohylnba/I2vCE9bNZd4DGjMW87j/THKPadDZWEqWggcrjY8x6ibSQGm2n2Tka +8B+vAoHBAMu+zl8kqFlYDG24dGfVpMaOYj5Osj0cV5f7O2pZ15sCevjiqoKGHH7G +O8EiI5pRBZ893+Fsx6YWmcKue88lfGvaoQjV0LUbfMfX/FoD39w/ZLx31yKEiFuc +KvMiRV5mO3kQiHBVX9vamzr5NeaErccXY9LnhaKOMX9blgiDQZH7fc3RhodcFWrC +9yfX6ryfidpPnRvK7Ops7hVnFKyyS4FaAarnzH1B2WcVcD4lYYxhMc8VXeU3eKOh +j1fI/F5ifQKBwQDpCjB670HqUzAexL9IYqSwSz3yedoK6m24ZIWx5XicI8fJJIXZ +QHoVAKB/IMtWxH8dnri+Bnj0O/TYe1pQb4pBm0xjAGjMEKYm6LNLhQXr67qiS0vQ +0eKYTKVv+9vTcLRQj2bI3Exh+wkys+tzK9DmrtS8CSvRICIs3+g4OWJzvRPP8NXj +LgQrzBzhPqpKhkvFxdVJTmSOrxFj+a5exLmzEZqT6qanIB+VYpQwQuqVkxGpTX5B +V5ssNLYPYRpapx0CgcByCtQixzcAA1u5knR9pkT76ris3YnA0Ptqk3I3XiBjoGjK +pL0CICUVBMpvmTdKai12a8DDwgqiOaZJJTchxH63NAHNGzkeFkuq5IdYrzB/bHBr +WbzukjZs6KXVv4oKg7ioVAu6rN7iBaO7x8BWzk8i0EHMzFCto1+rRM1e6HEsUBOj +v7LIU0+dmZGUGLRIbhhQPR3Yb6ZatSwyiKc23vmKZqHmUqbQOaqBm6te7beDRugF +XJVY9sqs9IJyhYpVHlUCgcAPoslwYKeAXagsxdQrH3D9VJDXVOHWKMBqQZDio5dB +Q80uWpuxtt6nhZkQO1JIWnYb6v+zbDbcgjByBIDuxCdBW9d+QQnanKmVyrXguK91 +C3OcHHOmSduFdWC3/zYW1mW97Tz1sXyam2hly1u3L5kW+GnE1hr9VVPjQNrO9+Ge +qW0coaJqKY78q3Rm2dtyZeJSFFI1o/DQ3blyItsFpg/QrR+a5XrS6Nw2ZLIL4Azo +J1CTgMwjhwlMNCI4t4dkHd0= -----END PRIVATE KEY----- Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9d - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5d + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Nov 28 19:09:06 2027 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Jul 7 14:23:16 2028 GMT Subject: C=XY, L=Castle Anthrax, O=Python Software Foundation, CN=fakehostname Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:c7:ff:be:a1:64:06:47:e1:c4:95:0a:65:59:6b: - 91:c4:a8:a0:a3:08:aa:dc:3f:00:44:0a:cf:41:4b: - ed:e2:6f:03:78:c7:d3:67:14:d3:fe:b4:27:85:1a: - 02:57:97:b5:86:16:87:97:82:0d:e8:2b:ed:69:16: - c3:96:a6:c4:47:b8:5d:c1:76:a9:02:9d:28:52:c9: - d9:93:f1:b2:da:18:7c:0a:78:bd:1c:3f:12:4f:e5: - 3f:ee:5b:b7:b3:3c:a5:b6:55:65:c6:33:09:b0:45: - a4:49:f5:d8:f6:54:25:2e:db:4b:c3:65:64:86:16: - bf:73:8e:78:e8:25:cc:c1:b4:7d:d1:b1:55:20:a4: - ae:de:b8:41:ee:43:9a:48:33:d4:12:39:a0:a8:fb: - 91:4f:47:bb:3b:d6:75:3b:62:a6:c5:5a:d7:7e:36: - 10:bc:3a:0d:46:e3:de:c1:df:50:d0:94:77:22:17: - a0:51:9b:af:dc:34:60:26:1d:a0:04:ac:a5:06:04: - 57:64:1a:d1:4f:1c:19:f8:be:4b:68:12:0f:5f:c3: - 12:40:64:bb:c6:a6:84:d9:5a:b0:78:1d:1b:11:c1: - 54:e6:15:d1:af:2b:e2:57:2b:6d:b9:c4:5d:b7:01: - 74:57:a7:fb:27:85:2b:92:15:e2:e3:13:97:29:c8: - 92:7b + 00:c6:8e:98:87:ce:ae:e0:87:18:4c:67:3a:e7:46: + c1:81:0b:f7:26:c3:c1:11:10:87:5b:aa:5a:f3:fe: + 5f:01:15:a5:9e:8a:9b:ce:e4:76:ff:33:25:3c:c6: + cd:b3:2d:4a:ea:22:bd:89:35:ff:84:8a:2f:18:9b: + 86:d8:99:97:25:16:b6:b8:8e:89:9a:89:29:71:4d: + 93:9e:dd:d3:15:f2:6e:2f:02:ae:1e:4f:69:92:7d: + 96:42:d7:e5:c0:ba:da:9d:cf:b5:a1:1f:18:4c:54: + 49:b7:03:24:d6:bd:30:b7:a8:26:e7:95:b5:f9:54: + 8a:10:53:01:6b:b1:25:aa:0e:21:f2:68:67:4a:de: + ba:ce:4c:a1:54:28:cf:18:3e:2a:bf:05:99:26:e8: + 0e:1e:88:57:22:e7:07:7f:48:f0:fe:c3:4a:48:b1: + c6:5b:41:6d:4c:52:be:5b:b3:9e:2e:ca:55:9a:52: + 6b:5f:23:c2:89:b7:a0:43:65:0f:ee:03:4e:9c:ea: + 2f:3a:de:c4:82:10:8e:8b:82:af:b0:ca:29:1e:24: + 63:4c:c5:93:1f:82:ed:dd:cd:fb:f8:06:26:41:41: + 7d:19:86:24:6f:aa:c7:83:49:d3:dc:b9:56:b6:21: + f6:fa:08:59:60:db:df:27:7b:07:98:e0:40:08:13: + 53:be:ba:aa:39:87:c1:45:d5:a5:0e:31:9a:50:11: + 9b:b3:b2:9f:51:fd:30:0c:04:75:f5:62:65:63:1b: + f9:11:74:17:9a:1d:ed:f0:87:30:75:3d:fb:9e:3c: + 85:f6:65:33:26:2c:d6:1b:61:b7:e7:9b:18:64:d6: + 79:fc:68:0b:e8:4f:df:29:37:0f:32:df:fd:a4:d8: + a3:6c:bb:6e:27:f6:e4:a4:93:10:28:a8:9d:7d:58: + ac:26:98:bf:0c:a4:92:80:cc:bb:f8:e6:d4:6f:84: + df:90:2e:d9:d1:76:ef:8b:27:d9:b6:f6:7a:50:1c: + 59:98:2d:b7:6a:45:f9:a2:76:73 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Alternative Name: @@ -70,11 +90,11 @@ Certificate: X509v3 Basic Constraints: critical CA:FALSE X509v3 Subject Key Identifier: - F8:76:79:CB:11:85:F0:46:E5:95:E6:7E:69:CB:12:5E:4E:AA:EC:4D + 52:E0:93:AA:52:55:B7:BB:E7:A8:E0:8C:DE:41:2E:F4:07:F0:36:FB X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 DirName:/C=XY/O=Python Software Foundation CA/CN=our-ca-server - serial:82:ED:BF:41:C8:80:91:9B + serial:CB:2D:80:99:5A:69:52:5B Authority Information Access: CA Issuers - URI:http://testca.pythontest.net/testca/pycacert.cer @@ -85,48 +105,60 @@ Certificate: Full Name: URI:http://testca.pythontest.net/testca/revocation.crl - Signature Algorithm: sha1WithRSAEncryption - 6d:50:8d:fb:ee:4e:93:8b:eb:47:56:ba:38:cc:80:e1:9d:c7: - e1:9e:1f:9c:22:0c:d2:08:9b:ed:bf:31:d9:00:ee:af:8c:56: - 78:92:d1:7c:ba:4e:81:7f:82:1f:f4:68:99:86:91:c6:cb:57: - d3:b9:41:12:fa:75:53:fd:22:32:21:50:af:6b:4c:b1:34:36: - d1:a8:25:0a:d0:f0:f8:81:7d:69:58:6e:af:e3:d2:c4:32:87: - 79:d7:cd:ad:0c:56:f3:15:27:10:0c:f9:57:59:53:00:ed:af: - 5d:4d:07:86:7a:e5:f3:97:88:bc:86:b4:f1:17:46:33:55:28: - 66:7b:70:d3:a5:12:b9:4f:c7:ed:e6:13:20:2d:f0:9e:ec:17: - 64:cf:fd:13:14:1b:76:ba:64:ac:c5:51:b6:cd:13:0a:93:b1: - fd:43:09:a0:0b:44:6c:77:45:43:0b:e5:ed:70:b2:76:dc:08: - 4a:5b:73:5f:c1:fc:7f:63:70:f8:b9:ca:3c:98:06:5f:fd:98: - d1:e4:e6:61:5f:09:8f:6c:18:86:98:9c:cb:3f:73:7b:3f:38: - f5:a7:09:20:ee:a5:63:1c:ff:8b:a6:d1:8c:e8:f4:84:3d:99: - 38:0f:cc:e0:52:03:f9:18:05:23:76:39:de:52:ce:8e:fb:a6: - 6e:f5:4f:c3 + Signature Algorithm: sha256WithRSAEncryption + 29:d2:3f:82:3f:c1:38:35:a6:bd:81:10:fe:64:ec:ff:7e:e1: + c6:6f:7f:86:65:f9:31:6f:fb:ef:32:4e:2f:87:c8:42:de:6c: + 8d:b8:06:08:8f:37:70:95:7d:e1:40:d4:82:2b:8d:b3:4a:fd: + 34:c5:9e:df:ff:01:53:4a:4f:08:f4:58:e1:74:fc:78:e3:3e: + 71:a7:5e:66:07:ea:d2:04:31:e2:75:a8:4c:80:17:86:92:20: + d2:32:a7:9a:65:8b:1a:5f:f1:4c:c8:50:6d:00:fc:99:bf:69: + b3:48:f3:45:5a:ee:35:50:14:b8:f3:92:92:c6:9f:0e:5d:eb: + 0d:e8:ec:f2:a4:09:6b:dc:66:2b:fc:df:4c:fc:65:a1:ae:d3: + b5:88:6a:a4:e7:08:1c:94:49:e0:b8:c1:04:8c:21:09:6c:55: + 4b:2c:97:10:f1:8c:6c:d0:bb:ba:8d:93:e8:47:8b:4d:8e:7d: + 7d:85:53:18:c8:f8:f4:8f:67:3a:b1:aa:3e:18:34:6c:3a:e6: + a6:c7:2f:be:83:38:f5:d5:e5:d2:17:28:61:6c:b6:49:99:21: + 69:a4:a8:b6:94:76:fd:18:ad:35:52:45:64:fb:f1:5d:8e:bb: + c0:21:2e:59:55:24:af:bb:8f:b2:0a:7b:17:f0:34:97:8e:68: + a9:f2:d0:3e:f6:73:82:f8:7c:4e:9a:70:7d:d6:b3:8c:cc:85: + 04:5c:02:8f:74:da:88:3a:20:a8:7e:c2:9e:b0:dd:56:1f:ce: + cd:42:16:c6:14:91:ad:30:e0:dc:76:f2:2c:56:ea:38:45:d8: + c0:3e:b8:90:fa:f3:38:99:ec:44:07:35:8f:69:62:0c:f9:ef: + b7:9d:e5:15:42:6e:fb:fe:4c:ff:e8:94:5a:1a:b0:80:b2:0e: + 17:3d:e1:87:a8:08:84:93:74:68:8d:29:df:ca:0b:6a:44:32: + 8a:51:3b:d6:38:db:bd:e3:2a:1b:5e:20:48:81:82:19:91:c6: + 87:8c:0f:cd:51:ea -----BEGIN CERTIFICATE----- -MIIE9zCCA9+gAwIBAgIJAILtv0HIgJGdMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIF9zCCBF+gAwIBAgIJAMstgJlaaVJdMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yNzExMjgx -OTA5MDZaMGIxCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA3MDcx +NDIzMTZaMGIxCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj MCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24xFTATBgNVBAMMDGZh -a2Vob3N0bmFtZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMf/vqFk -BkfhxJUKZVlrkcSooKMIqtw/AEQKz0FL7eJvA3jH02cU0/60J4UaAleXtYYWh5eC -Degr7WkWw5amxEe4XcF2qQKdKFLJ2ZPxstoYfAp4vRw/Ek/lP+5bt7M8pbZVZcYz -CbBFpEn12PZUJS7bS8NlZIYWv3OOeOglzMG0fdGxVSCkrt64Qe5Dmkgz1BI5oKj7 -kU9HuzvWdTtipsVa1342ELw6DUbj3sHfUNCUdyIXoFGbr9w0YCYdoASspQYEV2Qa -0U8cGfi+S2gSD1/DEkBku8amhNlasHgdGxHBVOYV0a8r4lcrbbnEXbcBdFen+yeF -K5IV4uMTlynIknsCAwEAAaOCAcMwggG/MBcGA1UdEQQQMA6CDGZha2Vob3N0bmFt -ZTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC -MAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFPh2ecsRhfBG5ZXmfmnLEl5OquxNMH0G -A1UdIwR2MHSAFJrPz27rcT3bPPGuiGtWcgPLCKdIoVGkTzBNMQswCQYDVQQGEwJY -WTEmMCQGA1UECgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0ExFjAUBgNV -BAMMDW91ci1jYS1zZXJ2ZXKCCQCC7b9ByICRmzCBgwYIKwYBBQUHAQEEdzB1MDwG -CCsGAQUFBzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3RjYS9w -eWNhY2VydC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2EucHl0aG9udGVz -dC5uZXQvdGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly90ZXN0 -Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3JsMA0GCSqGSIb3 -DQEBBQUAA4IBAQBtUI377k6Ti+tHVro4zIDhncfhnh+cIgzSCJvtvzHZAO6vjFZ4 -ktF8uk6Bf4If9GiZhpHGy1fTuUES+nVT/SIyIVCva0yxNDbRqCUK0PD4gX1pWG6v -49LEMod5182tDFbzFScQDPlXWVMA7a9dTQeGeuXzl4i8hrTxF0YzVShme3DTpRK5 -T8ft5hMgLfCe7Bdkz/0TFBt2umSsxVG2zRMKk7H9QwmgC0Rsd0VDC+XtcLJ23AhK -W3Nfwfx/Y3D4uco8mAZf/ZjR5OZhXwmPbBiGmJzLP3N7Pzj1pwkg7qVjHP+LptGM -6PSEPZk4D8zgUgP5GAUjdjneUs6O+6Zu9U/D +a2Vob3N0bmFtZTCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAMaOmIfO +ruCHGExnOudGwYEL9ybDwREQh1uqWvP+XwEVpZ6Km87kdv8zJTzGzbMtSuoivYk1 +/4SKLxibhtiZlyUWtriOiZqJKXFNk57d0xXybi8Crh5PaZJ9lkLX5cC62p3PtaEf +GExUSbcDJNa9MLeoJueVtflUihBTAWuxJaoOIfJoZ0reus5MoVQozxg+Kr8FmSbo +Dh6IVyLnB39I8P7DSkixxltBbUxSvluzni7KVZpSa18jwom3oENlD+4DTpzqLzre +xIIQjouCr7DKKR4kY0zFkx+C7d3N+/gGJkFBfRmGJG+qx4NJ09y5VrYh9voIWWDb +3yd7B5jgQAgTU766qjmHwUXVpQ4xmlARm7Oyn1H9MAwEdfViZWMb+RF0F5od7fCH +MHU9+548hfZlMyYs1htht+ebGGTWefxoC+hP3yk3DzLf/aTYo2y7bif25KSTECio +nX1YrCaYvwykkoDMu/jm1G+E35Au2dF274sn2bb2elAcWZgtt2pF+aJ2cwIDAQAB +o4IBwzCCAb8wFwYDVR0RBBAwDoIMZmFrZWhvc3RuYW1lMA4GA1UdDwEB/wQEAwIF +oDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAd +BgNVHQ4EFgQUUuCTqlJVt7vnqOCM3kEu9AfwNvswfQYDVR0jBHYwdIAU3b/K2ubR +NLo3dSHKb5oIKPI1tkihUaRPME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZl +coIJAMstgJlaaVJbMIGDBggrBgEFBQcBAQR3MHUwPAYIKwYBBQUHMAKGMGh0dHA6 +Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3B5Y2FjZXJ0LmNlcjA1Bggr +BgEFBQcwAYYpaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0Y2Evb2Nz +cC8wQwYDVR0fBDwwOjA4oDagNIYyaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5l +dC90ZXN0Y2EvcmV2b2NhdGlvbi5jcmwwDQYJKoZIhvcNAQELBQADggGBACnSP4I/ +wTg1pr2BEP5k7P9+4cZvf4Zl+TFv++8yTi+HyELebI24BgiPN3CVfeFA1IIrjbNK +/TTFnt//AVNKTwj0WOF0/HjjPnGnXmYH6tIEMeJ1qEyAF4aSINIyp5plixpf8UzI +UG0A/Jm/abNI80Va7jVQFLjzkpLGnw5d6w3o7PKkCWvcZiv830z8ZaGu07WIaqTn +CByUSeC4wQSMIQlsVUsslxDxjGzQu7qNk+hHi02OfX2FUxjI+PSPZzqxqj4YNGw6 +5qbHL76DOPXV5dIXKGFstkmZIWmkqLaUdv0YrTVSRWT78V2Ou8AhLllVJK+7j7IK +exfwNJeOaKny0D72c4L4fE6acH3Ws4zMhQRcAo902og6IKh+wp6w3VYfzs1CFsYU +ka0w4Nx28ixW6jhF2MA+uJD68ziZ7EQHNY9pYgz577ed5RVCbvv+TP/olFoasICy +Dhc94YeoCISTdGiNKd/KC2pEMopRO9Y4273jKhteIEiBghmRxoeMD81R6g== -----END CERTIFICATE----- diff --git a/Lib/test/make_ssl_certs.py b/Lib/test/make_ssl_certs.py index b908c40c11ef..362276583f50 100644 --- a/Lib/test/make_ssl_certs.py +++ b/Lib/test/make_ssl_certs.py @@ -69,7 +69,7 @@ dir = cadir database = $dir/index.txt crlnumber = $dir/crl.txt - default_md = sha1 + default_md = sha256 default_days = 3600 default_crl_days = 3600 certificate = pycacert.pem @@ -108,7 +108,7 @@ def make_cert_key(hostname, sign=False, extra_san='', - ext='req_x509_extensions_full', key='rsa:2048'): + ext='req_x509_extensions_full', key='rsa:3072'): print("creating cert for " + hostname) tempnames = [] for i in range(3): @@ -174,7 +174,7 @@ def make_ca(): t.flush() with tempfile.NamedTemporaryFile() as f: args = ['req', '-new', '-days', '3650', '-extensions', 'v3_ca', '-nodes', - '-newkey', 'rsa:2048', '-keyout', 'pycakey.pem', + '-newkey', 'rsa:3072', '-keyout', 'pycakey.pem', '-out', f.name, '-subj', '/C=XY/L=Castle Anthrax/O=Python Software Foundation CA/CN=our-ca-server'] check_call(['openssl'] + args) diff --git a/Lib/test/pycacert.pem b/Lib/test/pycacert.pem index 850fa32aef7e..73150c960f35 100644 --- a/Lib/test/pycacert.pem +++ b/Lib/test/pycacert.pem @@ -2,78 +2,98 @@ Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9b - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5b + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Jan 17 19:09:06 2028 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Aug 26 14:23:16 2028 GMT Subject: C=XY, O=Python Software Foundation CA, CN=our-ca-server Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:c3:18:69:6b:c9:47:29:98:8e:b1:56:c2:2e:fa: - 0e:5e:bc:23:80:b3:07:62:24:d2:42:5b:f1:4a:bf: - a9:c8:21:75:c8:e3:e6:2c:1f:87:3c:6e:7c:1b:ed: - 39:32:95:b7:40:b2:60:48:c3:9a:16:08:fe:6d:67: - 88:34:3b:77:77:70:1c:70:5a:d1:1f:5f:04:21:54: - b9:0c:e3:41:85:1d:58:ee:2f:ed:f3:0e:ef:d8:23: - a1:fa:73:fb:4c:28:e0:e5:e6:4d:0b:02:52:49:86: - c7:be:7e:bd:e6:56:76:8b:70:8e:0a:8f:06:33:20: - 1d:7b:5b:aa:d0:c5:1b:ab:9b:cc:54:09:3c:bf:e4: - 40:66:f1:fb:d6:f7:16:9d:c4:19:d4:c3:f2:ff:07: - bc:6f:5a:9e:25:1b:02:4a:a5:ec:42:96:3a:70:d2: - 6c:99:2b:ce:be:e8:d2:01:ef:d5:ba:b0:cf:94:3e: - 82:d0:01:d6:4b:71:80:03:0a:12:45:86:79:81:d8: - 4b:d2:e8:b5:b7:2c:6c:9a:4c:8a:10:10:e4:e4:f5: - df:ce:84:91:ca:d1:46:e0:84:73:17:66:db:69:43: - 78:80:83:be:14:4d:f1:3e:1a:d6:6c:f5:de:45:f3: - 39:af:91:d5:3d:54:44:bf:41:cc:73:68:1a:fc:24: - db:91 + 00:97:ed:55:41:ba:36:17:95:db:71:1c:d3:e1:61: + ac:58:73:e3:c6:96:cf:2b:1f:b8:08:f5:9d:4b:4b: + c7:30:f6:b8:0b:b3:52:72:a0:bb:c9:4d:3b:8e:df: + 22:8e:01:57:81:c9:92:73:cc:00:c6:ec:70:b0:3a: + 17:40:c1:df:f2:8c:36:4c:c4:a7:81:e7:b6:24:68: + e2:a0:7e:35:07:2f:a0:5b:f9:45:46:f7:1e:f0:46: + 11:fe:ca:1a:3c:50:f1:26:a9:5f:9c:22:9c:f8:41: + e1:df:4f:12:95:19:2f:5c:90:01:17:6e:7e:3e:7d: + cf:e9:09:af:25:f8:f8:42:77:2d:6d:5f:36:f2:78: + 1e:7d:4a:87:68:63:6c:06:71:1b:8d:fa:25:fe:d4: + d3:f5:a5:17:b1:ef:ea:17:cb:54:c8:27:99:80:cb: + 3c:45:f1:2c:52:1c:dd:1f:51:45:20:50:1e:5e:ab: + 57:73:1b:41:78:96:de:84:a4:7a:dd:8f:30:85:36: + 58:79:76:a0:d2:61:c8:1b:a9:94:99:63:c6:ee:f8: + 14:bf:b4:52:56:31:97:fa:eb:ac:53:9e:95:ce:4c: + c4:5a:4a:b7:ca:03:27:5b:35:57:ce:02:dc:ec:ca: + 69:f8:8a:5a:39:cb:16:20:15:03:24:61:6c:f4:7a: + fc:b6:48:e5:59:10:5c:49:d0:23:9f:fb:71:5e:3a: + e9:68:9f:34:72:80:27:b6:3f:4c:b1:d9:db:63:7f: + 67:68:4a:6e:11:f8:e8:c0:f4:5a:16:39:53:0b:68: + de:77:fa:45:e7:f8:91:cd:78:cd:28:94:97:71:54: + fb:cf:f0:37:de:c9:26:c5:dc:1b:9e:89:6d:09:ac: + c8:44:71:cb:6d:f1:97:31:d5:4c:20:33:bf:75:4a: + a0:e0:dc:69:11:ed:2a:b4:64:10:11:30:8b:0e:b0: + a7:10:d8:8a:c5:aa:1b:c8:26:8a:25:e7:66:9f:a5: + 6a:1a:2f:7c:5f:83:c6:78:4f:1f Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Key Identifier: - 9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 X509v3 Basic Constraints: CA:TRUE - Signature Algorithm: sha1WithRSAEncryption - 10:25:c8:dc:0c:55:5c:cb:83:6e:79:ef:77:ec:0d:8e:0c:06: - c1:4b:0c:d6:f7:75:52:21:b8:17:4a:38:88:9d:b3:78:c4:42: - fb:b8:7c:14:38:10:fb:ac:da:11:00:5b:42:87:5e:45:9f:6d: - 4e:42:a4:9a:18:06:39:0f:45:a6:96:89:32:d6:59:b3:d3:8e: - e3:95:b6:c4:a2:4b:74:2f:67:c1:fb:bb:f9:72:6f:37:4a:e7: - f4:48:33:71:df:b8:f5:e6:41:3f:d5:d5:2f:26:09:f8:0e:92: - ff:70:ea:f6:ab:58:fb:90:04:d6:43:2e:8f:b1:fb:06:ab:69: - d0:dc:a8:f8:5b:07:f2:d4:66:1f:63:f8:5d:c1:9e:41:44:bb: - c9:e8:7d:e0:46:e4:a7:c8:32:5f:31:62:e5:1c:5c:89:dd:b7: - a2:4f:9e:0d:13:b8:5f:b1:84:53:4c:1f:ce:19:e1:01:00:5e: - bf:41:55:94:a9:a5:13:db:f2:59:f3:d6:4e:b9:9d:9d:b9:0a: - d9:b2:18:6d:7c:b1:f7:96:aa:bd:f6:f9:95:0f:4a:6e:3c:7c: - 46:5b:df:d4:78:ec:9a:dc:e2:e3:01:e6:88:77:39:93:9c:ba: - 2a:63:f9:25:4b:4f:ac:08:79:39:c6:7b:df:07:35:ba:c0:c2: - 50:bf:5a:81 + Signature Algorithm: sha256WithRSAEncryption + 33:6a:54:d3:6b:c0:d7:01:5f:9d:f4:05:c1:93:66:90:50:d0: + b7:18:e9:b0:1e:4a:a0:b6:da:76:93:af:84:db:ad:15:54:31: + 15:13:e4:de:7e:4e:0c:d5:09:1c:34:35:b6:e5:4c:d6:6f:65: + 7d:32:5f:eb:fc:a9:6b:07:f7:49:82:e5:81:7e:07:80:9a:63: + f8:2c:c3:40:bc:8f:d4:2a:da:3e:d1:ee:08:b7:4d:a7:84:ca: + f4:3f:a1:98:45:be:b1:05:69:e7:df:d7:99:ab:1b:ee:8b:30: + cc:f7:fc:e7:d4:0b:17:ae:97:bf:e4:7b:fd:0f:a7:b4:85:79: + e3:59:e2:16:87:bf:1f:29:45:2c:23:93:76:be:c0:87:1d:de: + ec:2b:42:6a:e5:bb:c8:f4:0a:4a:08:0a:8c:5c:d8:7d:4d:d1: + b8:bf:d5:f7:29:ed:92:d1:94:04:e8:35:06:57:7f:2c:23:97: + 87:a5:35:8d:26:d3:1a:47:f2:16:d7:d9:c6:d4:1f:23:43:d3: + 26:99:39:ca:20:f4:71:23:6f:0c:4a:76:76:f7:76:1f:b3:fe: + bf:47:b0:fc:2a:56:81:e1:d2:dd:ee:08:d8:f4:ff:5a:dc:25: + 61:8a:91:02:b9:86:1c:f2:50:73:76:25:35:fc:b6:25:26:15: + cb:eb:c4:2b:61:0c:1c:e7:ee:2f:17:9b:ec:f0:d4:a1:84:e7: + d2:af:de:e4:1b:24:14:a7:01:87:e3:ab:29:58:46:a0:d9:c0: + 0a:e0:8d:d7:59:d3:1b:f8:54:20:3e:78:a5:a5:c8:4f:8b:03: + c4:96:9f:ec:fb:47:cf:76:2d:8d:65:34:27:bf:fa:ae:01:05: + 8a:f3:92:0a:dd:89:6c:97:a1:c7:e7:60:51:e7:ac:eb:4b:7d: + 2c:b8:65:c9:fe:5d:6a:48:55:8e:e4:c7:f9:6a:40:e1:b8:64: + 45:e9:b5:59:29:a5:5f:cf:7d:58:7d:64:79:e5:a4:09:ac:1e: + 76:65:3d:94:c4:68 -----BEGIN CERTIFICATE----- -MIIDbTCCAlWgAwIBAgIJAILtv0HIgJGbMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIEbTCCAtWgAwIBAgIJAMstgJlaaVJbMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yODAxMTcx -OTA5MDZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg -Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCASIwDQYJKoZI -hvcNAQEBBQADggEPADCCAQoCggEBAMMYaWvJRymYjrFWwi76Dl68I4CzB2Ik0kJb -8Uq/qcghdcjj5iwfhzxufBvtOTKVt0CyYEjDmhYI/m1niDQ7d3dwHHBa0R9fBCFU -uQzjQYUdWO4v7fMO79gjofpz+0wo4OXmTQsCUkmGx75+veZWdotwjgqPBjMgHXtb -qtDFG6ubzFQJPL/kQGbx+9b3Fp3EGdTD8v8HvG9aniUbAkql7EKWOnDSbJkrzr7o -0gHv1bqwz5Q+gtAB1ktxgAMKEkWGeYHYS9LotbcsbJpMihAQ5OT1386EkcrRRuCE -cxdm22lDeICDvhRN8T4a1mz13kXzOa+R1T1URL9BzHNoGvwk25ECAwEAAaNQME4w -HQYDVR0OBBYEFJrPz27rcT3bPPGuiGtWcgPLCKdIMB8GA1UdIwQYMBaAFJrPz27r -cT3bPPGuiGtWcgPLCKdIMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEB -ABAlyNwMVVzLg25573fsDY4MBsFLDNb3dVIhuBdKOIids3jEQvu4fBQ4EPus2hEA -W0KHXkWfbU5CpJoYBjkPRaaWiTLWWbPTjuOVtsSiS3QvZ8H7u/lybzdK5/RIM3Hf -uPXmQT/V1S8mCfgOkv9w6varWPuQBNZDLo+x+waradDcqPhbB/LUZh9j+F3BnkFE -u8nofeBG5KfIMl8xYuUcXIndt6JPng0TuF+xhFNMH84Z4QEAXr9BVZSppRPb8lnz -1k65nZ25CtmyGG18sfeWqr32+ZUPSm48fEZb39R47Jrc4uMB5oh3OZOcuipj+SVL -T6wIeTnGe98HNbrAwlC/WoE= +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA4MjYx +NDIzMTZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg +Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCAaIwDQYJKoZI +hvcNAQEBBQADggGPADCCAYoCggGBAJftVUG6NheV23Ec0+FhrFhz48aWzysfuAj1 +nUtLxzD2uAuzUnKgu8lNO47fIo4BV4HJknPMAMbscLA6F0DB3/KMNkzEp4HntiRo +4qB+NQcvoFv5RUb3HvBGEf7KGjxQ8SapX5winPhB4d9PEpUZL1yQARdufj59z+kJ +ryX4+EJ3LW1fNvJ4Hn1Kh2hjbAZxG436Jf7U0/WlF7Hv6hfLVMgnmYDLPEXxLFIc +3R9RRSBQHl6rV3MbQXiW3oSket2PMIU2WHl2oNJhyBuplJljxu74FL+0UlYxl/rr +rFOelc5MxFpKt8oDJ1s1V84C3OzKafiKWjnLFiAVAyRhbPR6/LZI5VkQXEnQI5/7 +cV466WifNHKAJ7Y/TLHZ22N/Z2hKbhH46MD0WhY5Uwto3nf6Ref4kc14zSiUl3FU ++8/wN97JJsXcG56JbQmsyERxy23xlzHVTCAzv3VKoODcaRHtKrRkEBEwiw6wpxDY +isWqG8gmiiXnZp+lahovfF+DxnhPHwIDAQABo1AwTjAdBgNVHQ4EFgQU3b/K2ubR +NLo3dSHKb5oIKPI1tkgwHwYDVR0jBBgwFoAU3b/K2ubRNLo3dSHKb5oIKPI1tkgw +DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAYEAM2pU02vA1wFfnfQFwZNm +kFDQtxjpsB5KoLbadpOvhNutFVQxFRPk3n5ODNUJHDQ1tuVM1m9lfTJf6/ypawf3 +SYLlgX4HgJpj+CzDQLyP1CraPtHuCLdNp4TK9D+hmEW+sQVp59/Xmasb7oswzPf8 +59QLF66Xv+R7/Q+ntIV541niFoe/HylFLCOTdr7Ahx3e7CtCauW7yPQKSggKjFzY +fU3RuL/V9yntktGUBOg1Bld/LCOXh6U1jSbTGkfyFtfZxtQfI0PTJpk5yiD0cSNv +DEp2dvd2H7P+v0ew/CpWgeHS3e4I2PT/WtwlYYqRArmGHPJQc3YlNfy2JSYVy+vE +K2EMHOfuLxeb7PDUoYTn0q/e5BskFKcBh+OrKVhGoNnACuCN11nTG/hUID54paXI +T4sDxJaf7PtHz3YtjWU0J7/6rgEFivOSCt2JbJehx+dgUees60t9LLhlyf5dakhV +juTH+WpA4bhkRem1WSmlX899WH1keeWkCawedmU9lMRo -----END CERTIFICATE----- diff --git a/Lib/test/pycakey.pem b/Lib/test/pycakey.pem index 16b75879d09b..c283f8909868 100644 --- a/Lib/test/pycakey.pem +++ b/Lib/test/pycakey.pem @@ -1,28 +1,40 @@ -----BEGIN PRIVATE KEY----- -MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDDGGlryUcpmI6x -VsIu+g5evCOAswdiJNJCW/FKv6nIIXXI4+YsH4c8bnwb7TkylbdAsmBIw5oWCP5t -Z4g0O3d3cBxwWtEfXwQhVLkM40GFHVjuL+3zDu/YI6H6c/tMKODl5k0LAlJJhse+ -fr3mVnaLcI4KjwYzIB17W6rQxRurm8xUCTy/5EBm8fvW9xadxBnUw/L/B7xvWp4l -GwJKpexCljpw0myZK86+6NIB79W6sM+UPoLQAdZLcYADChJFhnmB2EvS6LW3LGya -TIoQEOTk9d/OhJHK0UbghHMXZttpQ3iAg74UTfE+GtZs9d5F8zmvkdU9VES/Qcxz -aBr8JNuRAgMBAAECggEAZHgv4hg3k45C/cSmH7caq2LMDb0kskAwH4hlzI7DipLg -q2Hh6Rsbc92aAG+8IvbC9ohl2VMSCQL8s667j9qH/XQ40QuT4kn2QIv2+FIYLcsd -Pxxjt+YbUf2XrvkHkwMCPqLJTkAVzFOijdGLThF83vZJz9oz4SRKyno8j2LSix68 -WEfnjdyWqYb0eS0luKrLHw+IL7bD5vfc/P0q6u31zJ9h8zEyN5EBCj5OxM/hD0VO -nObrp6r9Bs+xx+yRx+8J5Db6LPXggl5nBqsqrDKVDe6uTysYVgstqkfaDv1L78Vu -3BNdKPAdJ+ucPJrQufzFHBDIIN+Xwckf/09gdQagGQKBgQDnvFaOjZfqc6wL/kNK -tszQtedbdwP20L+EWdNEVsVWK1TOw36Pmkrp2AYLXMd7W1QQu0KukM89EFb84wKo -s4C9V/ch162mUhEAveaLioi7bMwMPIib2V6pHmYGG8nQVRvgkZVYx6ZtPEvWye1v -wmCzzxxK0gC6PQGxp8MSv9yXDwKBgQDXhe57ufc52pgJ+Agyl4PLkllIbG2DKQHG -LwY06v73jllirTpWBOBvN0NvEsI2Pj4aK/BXRNYN1PS7xi/3C6MVWxnOpBtbq3H5 -DwFb5mpfgJmhV6DZ6jMw7h3Yvy35ViKoiI9UK3eTmhkerH3DsILEje7jE9dGmIOJ -4oLa50JjXwKBgQDdTfyveMNasIrejTzATmC89Or0a22KuQIdKBddjSw5xXnhV8s2 -4temCJqFIV6UDLz0mZDt2vc+zqr0KOtyJrLMoAQv+qQoUPlR5wkTvAImU5luGiUw -CN+gzJoMPV93KMBNr1qcBVaHvWyDvCWXdF8beLABOBpfwUEr4xWlgzrruwKBgCvf -cr2zDJW1Xu/gkuKhn02ofA5XLC/gACF03yGUmNSSILYKp25tTba2HD8XJXvfTcsM -GL/bHmvwZuV2obr7nnYxdl5vX7ZYfzoBCPjJPew1BJEognD50PPr9R1zRYuVMjb2 -nZ63vn7IhsaMvIlCfExAzFljZ5ZSY6yE9LhVDVmnAoGBALOwMwpkm1drx5UNSJO7 -70Q8kYzg0oQhCo/7b6DWbAglDPSWQS5IA4rHYOwL3sE+69G2Exe+1454rVDisojW -XdSyA3svI/YQeom8R2LIM/ayCPxCc3/Dxy9+aQQT4lW3F0XQIxod/QsQJxpZIOnF -jOSPclypgV2X6dDOwDkd2Tgh +MIIG/AIBADANBgkqhkiG9w0BAQEFAASCBuYwggbiAgEAAoIBgQCX7VVBujYXldtx +HNPhYaxYc+PGls8rH7gI9Z1LS8cw9rgLs1JyoLvJTTuO3yKOAVeByZJzzADG7HCw +OhdAwd/yjDZMxKeB57YkaOKgfjUHL6Bb+UVG9x7wRhH+yho8UPEmqV+cIpz4QeHf +TxKVGS9ckAEXbn4+fc/pCa8l+PhCdy1tXzbyeB59SodoY2wGcRuN+iX+1NP1pRex +7+oXy1TIJ5mAyzxF8SxSHN0fUUUgUB5eq1dzG0F4lt6EpHrdjzCFNlh5dqDSYcgb +qZSZY8bu+BS/tFJWMZf666xTnpXOTMRaSrfKAydbNVfOAtzsymn4ilo5yxYgFQMk +YWz0evy2SOVZEFxJ0COf+3FeOulonzRygCe2P0yx2dtjf2doSm4R+OjA9FoWOVML +aN53+kXn+JHNeM0olJdxVPvP8DfeySbF3BueiW0JrMhEcctt8Zcx1UwgM791SqDg +3GkR7Sq0ZBARMIsOsKcQ2IrFqhvIJool52afpWoaL3xfg8Z4Tx8CAwEAAQKCAYB6 +1g1kwyYRE70FS4WUaOBr8+dqHW0LNO4bcFrpEi/PSuurqiUzQPoT3DoXXhoWLseN +zGh476yBKZJDKfS7CwYCmZMdprK4uZvu/E6f7Or7EGrbckOtCQkew8iw9L8ZnWgd +FjyThPjdUIdLgidIHcDJWjVHuLKh3B9KD+ZpEU/IjYtRLvbCPJSKQMQShrBE1Rau +SF6IF5P4vK7X0162NlQqMLpQBAKLml93VJcERzVY1u53JJnkG1loIrNvE32zvZ0C +ZnGVpWwamixVrO9K66F+Ml3Y3bkguF8aPUitc+l+yPmUSXmcDcKmhw9DZA0k5t39 +FxVYGn1uJlvHll8QvV9aZtzuTSkAN8VWNufGwivLLsoRlRb1LtGWqHY583FlUWtz +99ABCBehZ2EpAP+MrRj+pyKuMrkQH61bbOhjqifBM8MhHdn9uNmMpvnKyGPMIdRY +cLH4i2S5aQVvmsQbOa98DLOFGXdf+z5HuwdxHtG1S3J7jkT+FkIyYDehJA3X8UEC +gcEAyCpD8rMFfR5qLwrajhy8vbV7TIkNfFHEkm9tCWDBHwuOJqA0DFuMDAKl7cMv +Uo7Z6R2Fqe2OyWvxYkOi/CSjvtT+PTiA0ux1tXUZcxcRSIsLqQZV+elsKcv+QJPy +vf02vNvHjaMaRwl+NYtqpfr1s/3EdJnWCNC3nV1dD+mWVJoO3kGAK5grLAPM1/uP +stARN5Tnh3Doh8e1Yl4V4UKcVqyVqDykX1OLSmPqNH86T4Um0B4h+jf4UBBdDBz1 +rD3JAoHBAMJOZ3M7LqX+F2haSrF/hnG1y9qAqDWGsvy+8YgjFwPFWu7LvqLuXLuz +S3+5GGhplMuM0itqA9PyPotlgtG5O9hAU8SyMitrx1uTW+Q2U3iYPZQ9O327l1cO +F2jKljq0aJrXp+5iWUq8t/FG6DAqYYUCY/X1SheqEXCgCh4ldRhXig3TBYbVZNs9 +7azN0lk408AO/Hx7WYreFQVS7A/EJhk/M1yyIqnJESuxkDefjV4hTVkRXhh+MrCe +vF/jHqh5pwKBwHxXPQRbzvINXbrBTEjxcxGJ1gESNg1fIfQxQZOMxgrJ+9DkvdBb +YiDn2DldgV0Qni8ghrKrfoKDClyXVXy6KfnWh+Rx4BymhOxmxJto3fSpY2HpLKll +JirErLli7my1CjbBdDH4+s7cB8mtRF+9CLp5znr8QSgSt60KnU/QM/F0Df5kxADQ +syjRZ4NXoslaVQeo+TZ6nggSuAtWFNNstH9nEESE/zq0RBe+/3MDAa76MMUhosuz +zw21TIfEyZvoeQKBwDpszNpvPzWWU3+DNtZsXAaw/Vz0Np/xorwwxfuDYZY2r4MC +LI5dUfD2losPIvGyXZVfAIshU4lVW80adt2M7xu1K/sHAeLgg49bndPfKfYnAM0k +JFFIKNd6WzudPtLkEFgO5GXfmK3KVRztjz98XtpZv6jjWqYG8zuEQ8aQyMbK+63w +d8b1P2BVHLRLJybA2Zr0ZqMfi+sfn/570pNjDXml8VG8FoQq+0jCGXVAOofFR7ay +bDK9L4zADjBe4IcUHQKBwFwj8TEVlWxtG+IWO5d+vyuP0OPjSYOmYq4dCMyZ2+Xy +Y+XDYEhlgGTVxafRMTwt57VV3hJTtRxUZziDA++atr8+gPio+QHBYg1JgCKsqKYL +TGoSVrM1jTfdl1orwkpgQmq2q5j7ExpNA3Spsm5kyCaJ1S/8Ivusqaod8S4t7UhW +BRec3gQ+UYv/V3Pc9hS1Zdzf5+G+PDOYoNmveY16hcu0DKc/PlqGtJuBoQjjH7ir +1YVK9GAgLk0VqJvePnF57A== -----END PRIVATE KEY----- diff --git a/Lib/test/revocation.crl b/Lib/test/revocation.crl index 53cb4b3721d9..c05461ca7f93 100644 --- a/Lib/test/revocation.crl +++ b/Lib/test/revocation.crl @@ -1,11 +1,14 @@ -----BEGIN X509 CRL----- -MIIBpjCBjwIBATANBgkqhkiG9w0BAQUFADBNMQswCQYDVQQGEwJYWTEmMCQGA1UE +MIICJjCBjwIBATANBgkqhkiG9w0BAQsFADBNMQswCQYDVQQGEwJYWTEmMCQGA1UE CgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0ExFjAUBgNVBAMMDW91ci1j -YS1zZXJ2ZXIXDTE4MDExOTE5MDkwNloXDTI3MTEyODE5MDkwNlqgDjAMMAoGA1Ud -FAQDAgEAMA0GCSqGSIb3DQEBBQUAA4IBAQBYVzH8n2LdyJJ/t8CpWjz652hZJ0sY -DeNYcwuTPzR9CbSwEwNbf0kY9bgWXGfoRD2SnnCnuNNJXO2MuXtxf6qYx2ZjhJm8 -qgxXs0Bz4agRswNMbumjHCmqIv1t88vbrO0+ItEZDK7RJVIMBtVJ0XYOHvD/IG/i -zqa1Fl3uCTvQbTJ2TrqzJeP/Vl40hOD+VdBBZK3j0r4AkCKU3tAiHYTGmHKhPxy1 -f8Yet+4SRMGp1BdDezTI1bICpSZhRJ4geW0UzuCZnXPW8IZzioUmdUBAmAMHPWFr -B0sTTc/ntD4jHG1/T5b0oiDMbXIbh5Iz9iQNcY0IbotkCw39h+K90wY6 +YS1zZXJ2ZXIXDTE4MDgyOTE0MjMxNloXDTI4MDcwNzE0MjMxNlqgDjAMMAoGA1Ud +FAQDAgEAMA0GCSqGSIb3DQEBCwUAA4IBgQCPhrtGSbuvxPAI3YWQFDB4iOWdBnVk +ugW1lsifmCsE86FfID0EwUut1SRHlksltMtcoULMEIdu8yMLWci++4ve22EEuMKT +HUc3T/wBIuQUhA7U4deFG8CZPAxRpNoK470y7dkD4OVf0Gxa6WYDl9z8mXKmWCB9 +hvzqVfLWNSLTAVPsHtkD5PXdi5yRkQr6wYD7poWaIvkpsn7EKCY6Tw5V3rsbRuZq +AGVCq5TH3mctcmwLloCJ4Xr/1q0DsRrYxeeLYxE+UpvvCbVBKgtjBK7zINS7AbcJ +CYCYKUwGWv1fYKJ+KQQHf75mT3jQ9lWuzOj/YWK4k1EBnYmVGuKKt73lLFxC6h3y +MUnaBZc1KZSyJj0IxfHg/o6qx8NgKOl9XRIQ5g5B30cwpPOskGhEhodbTTY3bPtm +RQ36JvQZngzmkhyhr+MDEV5yUTOShfUiclzQOx26CmLmLHWxOZgXtFZob/oKrvbm +Gen/+7K7YTw6hfY52U7J2FuQRGOyzBXfBYQ= -----END X509 CRL----- diff --git a/Lib/test/ssl_cert.pem b/Lib/test/ssl_cert.pem index b1dd3f387f7c..de596717bd85 100644 --- a/Lib/test/ssl_cert.pem +++ b/Lib/test/ssl_cert.pem @@ -1,20 +1,26 @@ -----BEGIN CERTIFICATE----- -MIIDWTCCAkGgAwIBAgIJAPm6B21bar2bMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV +MIIEWTCCAsGgAwIBAgIJAJinz4jHSjLtMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODAx -MTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH +IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODA4 +MjkxNDIzMTVaFw0yODA4MjYxNDIzMTVaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH DA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5k -YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEP -ADCCAQoCggEBAKvvsX2gEti4shve3iYMc+jE4Se7WHs1Bol2f21H8qNboDOFdeb1 -RKHjmq3exHpajywOUEgne9nKHJY/3f2phR4Y5klqG6liLgiSpVyRlcBGbeT2qEAj -9oLiLFUXLGfGDds2mTwivQDLJBWi51j7ff5k2Pr58fN5ugYMn24T9FNyn0moT+qj -SFoBNm58l9jrdkJSlgWfqPlbiMa+mqDn/SFtrwLF2Trbfzu42Sd9UdIzMaSSrzbN -sGm53pNhCh8KndWUQ8GPP2IsLPoUU4qAtmZuTxCx2S1cXrN9EkmT69tlOH84YfSn -96Ih9bWRc7M5y5bfVdEVM+fKQl3hBRf05qMCAwEAAaMYMBYwFAYDVR0RBA0wC4IJ -bG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQAtQ8f37cCEk7/rAcbYR53ce3iK -Vpihb0U2ni1QjG9Tg9UIExkIGkwTiCm7kwQL+GEStBu9AG/QVrOjeTriRiddhWkk -ze8kRaI3AC/63t6Vh9Q1x6PESgeE4OtAO9JpJCf4GILglA789Y/b/GF8zJZQxR13 -qpB4ZwWw7gCBhdEW59u6CFeBmfDa58hM8lWvuVoRrTi7bjUeC6PAn5HVMzZSykhu -4HaUfBp6bKFjuym2+h/VvM1n8C3chjVSmutsLb6ELdD8IK0vPV/yf5+LN256zSsS -dyUZYd8XwQaioEMKdbhLvnehyzHiWfQIUR3BdhONxoIJhHv/EAo8eCkHHYIF +YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGP +ADCCAYoCggGBALKUqUtopT6E68kN+uJNEt34i2EbmG/bwjcD8IaMsgJPSsMO2Bpd +3S6qWgkCeOyCfmAwBxK2kNbxGb63ouysEv7l8GCTJTWv3hG/HQcejJpnAEGi6K1U +fDbyE/db6yZ12SoHVTGkadN4vYGCPd1Wj9ZO1F877SHQ8rDWX3xgTWkxN2ojBw44 +T8RHSDiG8D/CvG4uEy+VUszL+Uvny5y2poNSqvI3J56sptWSrh8nIIbkPZPBdUne +LYMOHTFK3ZjXSmhlXgziTxK71nnzM3Y9K9gxPnRqoXbvu/wFo55hQCkETiRkYgmm +jXcBMZ0TClQVnQWuLjMthRnWFZs4Lfmwqjs7FZD/61581R2BYehvpWbLvvuOJhwv +DFzexL2sXcAl7SsxbzeQKRHqGbIDfbnQTXfs3/VC6Ye5P82P2ucj+XC32N9piRmO +gCBP8L3ub+YzzdxikZN2gZXXE2jsb3QyE/R2LkWdWyshpKe+RsZP1SBRbHShUyOh +yJ90baoiEwj2mwIDAQABoxgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZI +hvcNAQELBQADggGBAHRUO/UIHl3jXQENewYayHxkIx8t7nu40iO2DXbicSijz5bo +5//xAB6RxhBAlsDBehgQP1uoZg+WJW+nHu3CIVOU3qZNZRaozxiCl2UFKcNqLOmx +R3NKpo1jYf4REQIeG8Yw9+hSWLRbshNteP6bKUUf+vanhg9+axyOEOH/iOQvgk/m +b8wA8wNa4ujWljPbTQnj7ry8RqhTM0GcAN5LSdSvcKcpzLcs3aYwh+Z8e30sQWna +F40sa5u7izgBTOrwpcDm/w5kC46vpRQ5fnbshVw6pne2by0mdMECASid/p25N103 +jMqTFlmO7kpf/jpCSmamp3/JSEE1BJKHwQ6Ql4nzRA2N1mnvWH7Zxcv043gkHeAu +0x8evpvwuhdIyproejNFlBpKmW8OX7yKTCPPMC/VkX8Q1rVkxU0DQ6hmvwZlhoKa +9Wc2uXpw9xF8itV4Uvcdr3dwqByvIqn7iI/gB+4l41e0u8OmH2MKOx4Nxlly5TNW +HcVKQHyOeyvnINuBAQ== -----END CERTIFICATE----- diff --git a/Lib/test/ssl_key.passwd.pem b/Lib/test/ssl_key.passwd.pem index 669c7ce3c815..e4f1370ab270 100644 --- a/Lib/test/ssl_key.passwd.pem +++ b/Lib/test/ssl_key.passwd.pem @@ -1,30 +1,42 @@ -----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED -DEK-Info: DES-EDE3-CBC,2D5DD30B9D440DBB +DEK-Info: DES-EDE3-CBC,8064BE1494B24B13 -01gIwpy3XxPGsY0PoK59vAxdLhkVj3odO0Z1ULamUzIte6ThKL1HqnZiUlXpYKfK -XqHVVeQ1xouxiDRNFLJ4CqBG4HbRtqTkl+sfaNTVveL18lOOMAZy6W3dCGAnWOTZ -Z0RJyZlQxxjNQLuko4tIvFkrShXIgdiVFjwAhRU0KTUb7UQ2xfFA9R0Kfde30pzz -zSjb/OmYqAIhkdvafGXvJxzZAorQkU9akDh+uJ6cht5B/RGZsbKACYDSv2WSV5yW -r+fKVYcTup33r0Jj8hAD6fVY15K8BJknpkF9HfSlZnmmr2WDaffLokOOnCV/I1ie -WD7ENA7K//48km5D3Ogh2b2/0Iwuzjq8Mvd8aR39N9nINbGR+HNT85pawoo1S0W9 -pQTU4XTmxfXjtR2287C6XZyQ/tBwvNDMFPVhlxsGOdLYwoV5e/L1t1qIfkTlbuvd -JaMzOhSSLjiC156IFoH7PTPe+g75hw2b32XJURFGlaYknHF7P4BmCiwXOQYo5CCo -MQGGlw5qBCqODrIsc03wpL2jUzgvyPqLyaw395ITuSoGX+WO7vUQaGW0Tz/sOoTs -3pK+bTi2QmqZMe7xBOj07CYMMOo4QPrM6NpbObt+Jja2UXaxvKa9BwqCEQzA4pQZ -8ZHHfEWIaDffKTGkAlqm+S8qCtsrEZJhIn3aI/ikzK8v+YkWw6w+8t/tR1V8ET/s -CoYGIR7I8WhdfKAwgx2QT5bt1jkYKJyKPm4Iacp2mNh9gNFVq+JSKF318e7BrR3+ -wyqMkDxRYnov3ybtf6kiICxPREDqa6UG1xRq3SbWz6NnIF/1hoHs79YlSYbMfXNU -ffIxBaXNCcH6jM9duP2YRnO29jLwfLM/mmokTBBjyOBaKZia9GPa4naXoATXW3z+ -Xx4EKIUkKdb53kiV6NtEKMPialAnkeoHTEjyLPgaV8mCHLvGQbnxbYwvpPJH0e2f -CWgiw6ci4ROOzcZ7HJHIDUprwK0xRKn43hoI44fivlSHOFX6da6o3wIqhEUqMKwL -JQDS1GORRk1ndRXP+7Ub1dO+Vo/DqO1VcTr2o5RwZ1LWPnzLqbCG50mvTLH4djB+ -+hf6vlmnFC30N3yUFXWE5vS10nJHYP88dD9CB2RsaWzpxD9Zxl+PKcRsppen6HyO -u3b71a/TBOkJcI+lkOatEFvbuqzBAqhMceMctO+Dl55RFsbxfIw/IXZjdP0PYZ0C -t20DrIdBsvl9F/mfYpmkV4DF7yci78DqnRBcxylVNF2vwX7o+2fq/TsEwsHn3KnT -kvcF5Cq8Vr5C8ugWX8JfveNym0BjLu6Lr58qS4a6qCNGEGPFKyB+xkm4KEScbarQ -aLbEbfulMM7q9//sEOOLexIx7mNoLd29Xzn5hsLCAZLWX6wMq6JVJ/zbBOAHDbBT -yhi03yd5Kvw3swSt4QZj+uR3qTFwxkXUFiVvrSfxRZoyKsxsLr9Z7D8aoH9Rkb2L -6KjZ31nt9Drh7NJfh6ReANBW6INdDW0Y2mbzoDozLszAYjVfuUUEE76iJqXY0N4W -kNr0OQQTUtDpVk0AZZZvy17xV+rkqGgwlOqTvHbwFYEQvgwVz4EKUw== +KJrffOMbo8M0I3PzcYxRZGMpKD1yB3Ii4+bT5XoanxjIJ+4fdx6LfZ0Rsx+riyzs +tymsQu/iYY9j+4rCvN9+eetsL1X6iZpiimKsLexcid9M3fb0vxED5Sgw0dvunCUA +xhqjLIKR92MKbODHf6KrDKCpsiPbjq4gZ7P+uCGXAMHL3MXIJSC0hW9rK7Ce6oyO +CjpIcgB8x+GUWZZZhAFdlzIHMZrteNP2P5HK6QcaT71P034Dz1hhqoj4Q0t+Fta2 +4tfsM/bnTR/l6hwlhPa1e3Uj322tDTDWBScgWANn5+sEWldLmozMaWhZsn22pfk2 +KjRMGXG024JVheV882nbdOBvG7oq+lxkZ/ZP+vvqJqnvYtf7WtM8UivzYpe5Hz5b +kVvWzPjBLUSZ9whM9rDLqSSqMPyPvDTuEmLkuq+xm7pYJmsLqIMP2klZLqRxLX6K +uqwplb8UG440qauxgnQ905PId1l2fJEnRtV+7vXprA0L0QotgXLVHBhLmTFM+3PH +9H3onf31dionUAPrn3nfVE36HhvVgRyvDBnBzJSIMighgq21Qx/d1dk0DRYi1hUI +nCHl0YJPXheVcXR7JiSF2XQCAaFuS1Mr7NCXfWZOZQC/0dkvmHnl9DUAhuqq9BNZ +1cKhZXcKHadg2/r0Zup/oDzmHPUEfTAXT0xbqoWlhkdwbF2veWQ96A/ncx3ISTb4 +PkXBlX9rdia8nmtyQDQRn4NuvchbaGkj4WKFC8pF8Hn7naHqwjpHaDUimBc0CoQW +edNJqruKWwtSVLuwKHCC2gZFX9AXSKJXJz/QRSUlhFGOhuF/J6yKaXj6n5lxWNiQ +54J+OP/hz2aS95CD2+Zf1SKpxdWiLZSIQqESpmmUrXROixNJZ/Z7gI74Dd9dSJOH +W+3AU03vrrFZVrJVZhjcINHoH1Skh6JKscH18L6x4U868nSr4SrRLX8BhHllOQyD +bmU+PZAjF8ZBIaCtTGulDXD29F73MeAZeTSsgQjFu0iKLj1wPiphbx8i/SUtR4YP +X6PVA04g66r1NBw+3RQASVorZ3g1MSFvITHXcbKkBDeJH2z1+c6t/VVyTONnQhM5 +lLgRSk6HCbetvT9PKxWrWutA12pdBYEHdZhMHVf2+xclky7l09w8hg2/qqcdGRGe +oAOZ72t0l5ObNyaruDKUS6f4AjOyWq/Xj5xuFtf1n3tQHyslSyCTPcAbQhDfTHUx +vixb/V9qvYPt7OCn8py7v1M69NH42QVFAvwveDIFjZdqfIKBoJK2V4qPoevJI6uj +Q5ByMt8OXOjSXNpHXpYQWUiWeCwOEBXJX8rzCHdMtg37jJ0zCmeErR1NTdg+EujM +TWYgd06jlT67tURST0aB2kg4ijKgUJefD313LW1zC6gVsTbjSZxYyRbPfSP6flQB +yCi1C19E2OsgleqbkBVC5GlYUzaJT7SGjCRmGx1eqtbrALu+LVH24Wceexlpjydl ++s2nf/DZlKun/tlPh6YioifPCJjByZMQOCEfIox6BkemZETz8uYA4TTWimG13Z03 +gyDGC2jdpEW414J2qcQDvrdUgJ+HlhrAAHaWpMQDbXYxBGoZ+3+ORvQV4kAsCwL8 +k3EIrVpePdik+1xgOWsyLj6QxFXlTMvL6Wc5pnArFPORsgHEolJvxSPTf9aAHNPn +V2WBvxiLBtYpGrujAUM40Syx/aN2RPtcXYPAusHUBw+S8/p+/8Kg8GZmnIXG3F89 +45Eepl2quZYIrou7a1fwIpIIZ0hFiBQ1mlHVMFtxwVHS1bQb3SU2GeO+JcGjdVXc +04qeGuQ5M164eQ5C0T7ZQ1ULiUlFWKD30m+cjqmZzt3d7Q0mKpMKuESIuZJo/wpD +Nas432aLKUhcNx/pOYLkKJRpGZKOupQoD5iUj/j44o8JoFkDK33v2S57XB5QGz28 +9Zuhx49b3W8mbM6EBanlQKLWJGCxXqc/jhYhFWn+b0MhidynFgA0oeWvf6ZDyt6H +Yi5Etxsar09xp0Do3NxtQXLuSUu0ji2pQzSIKuoqQWKqldm6VrpwojiqJhy4WQBQ +aVVyFeWBC7G3Zj76dO+yp2sfJ0itJUQ8AIB9Cg0f34rEZu+r9luPmqBoUeL95Tk7 +YvCOU3Jl8Iqysv8aNpVXT8sa8rrSbruWCByEePZ37RIdHLMVBwVY0eVaFQjrjU7E +mXmM9eaoYLfXOllsQ+M2+qPFUITr/GU3Qig13DhK/+yC1R6V2a0l0WRhMltIPYKW +Ztvvr4hK5LcYCeS113BLiMbDIMMZZYGDZGMdC8DnnVbT2loF0Rfmp80Af31KmMQ4 +6XvMatW9UDjBoY5a/YMpdm7SRwm+MgV2KNPpc2kST87/yi9oprGAb8qiarHiHTM0 -----END RSA PRIVATE KEY----- diff --git a/Lib/test/ssl_key.pem b/Lib/test/ssl_key.pem index b63f38bc5cf2..1ea4578d81ec 100644 --- a/Lib/test/ssl_key.pem +++ b/Lib/test/ssl_key.pem @@ -1,28 +1,40 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCr77F9oBLYuLIb -3t4mDHPoxOEnu1h7NQaJdn9tR/KjW6AzhXXm9USh45qt3sR6Wo8sDlBIJ3vZyhyW -P939qYUeGOZJahupYi4IkqVckZXARm3k9qhAI/aC4ixVFyxnxg3bNpk8Ir0AyyQV -oudY+33+ZNj6+fHzeboGDJ9uE/RTcp9JqE/qo0haATZufJfY63ZCUpYFn6j5W4jG -vpqg5/0hba8Cxdk62387uNknfVHSMzGkkq82zbBpud6TYQofCp3VlEPBjz9iLCz6 -FFOKgLZmbk8QsdktXF6zfRJJk+vbZTh/OGH0p/eiIfW1kXOzOcuW31XRFTPnykJd -4QUX9OajAgMBAAECggEAHppmXDbuw9Z0FVPg9KLIysioTtsgz6VLiZIm8juZK4x2 -glUh/D7xvWL2uDXrgN+3lh7iGUW13LkFx5SMncbbo9TIwI57Z/XKvcnkVwquve+L -RfLFVc1Q5lD9lROv2rS86KTaN4LzYz3FKXi6dvMkpPAsUtfEQhMLkmISypQQq/1z -EJaqo7r85OjN7e0wKazlKZpOzJEa5FQLMVRjTRFhLFNbHXX/tAet2jw+umATKbw8 -hYgiuZ44TwSEd9JeIV/oSYWfI/3HetuYW0ru3caiztRF2NySNu8lcsWgNC7fIku9 -mcHjtSNzs91QN1Qlu7GQvvhpt6OWDirNDCW+49WGaQKBgQDg9SDhfF0jRYslgYbH -cqO4ggaFdHjrAAYpwnAgvanhFZL/zEqm5G1E7l/e2fCkJ9VOSFO0A208chvwMcr+ -dCjHE2tVdE81aQ2v/Eo83VdS1RcOV4Y75yPH48rMhxPaHvxWD/FFDbf0/P2mtPB7 -SZ3kIeZMkE1wxdaO3AKUbQoozwKBgQDDqYgg7kVtygyICE1mB8Hwp6nUxFTczG7y -4XcsDqMIrKmw+PbQluvkoHoStxeVrsTloDhkTjIrpmYLyAiazg+PUJdkd6xrfLSj -VV6X93W0S/1egEb1F1CGFxtk8v/PWH4K76EPL2vxXdxjywz3GWlrL9yDYaB2szzS -DqgwVMqx7QKBgDCD7UF0Bsoyl13RX3XoPXLvZ+SkR+e2q52Z94C4JskKVBeiwX7Y -yNAS8M4pBoMArDoj0xmBm69rlKbqtjLGbnzwrTdSzDpim7cWnBQgUFLm7gAD1Elb -AhZ8BCK0Bw4FnLoa2hfga4oEfdfUMgEE0W5/+SEOBgWKRUmuHUhRc911AoGAY2EN -YmSDYSM5wDIvVb5k9B3EtevOiqNPSw/XnsoEZtiEC/44JnQxdltIBY93bDBrk5IQ -cmoBM4h91kgQjshQwOMXMhFSwvmBKmCm/hrTbvMVytTutXfVD3ZXFKwT4DW7N0TF -ElhsxBh/YzRz7mG62JVjtFt2zDN3ld2Z8YpvtXUCgYEA4EJ4ObS5YyvcXAKHJFo6 -Fxmavyrf8LSm3MFA65uSnFvWukMVqqRMReQc5jvpxHKCis+XvnHzyOfL0gW9ZTi7 -tWGGbBi0TRJCa8BkvgngUZxOxUlMfg/7cVxOIB0TPoUSgxFd/+qVz4GZMvr0dPu7 -eAF7J/8ECVvb0wSPTUI1N3c= +MIIG/wIBADANBgkqhkiG9w0BAQEFAASCBukwggblAgEAAoIBgQCylKlLaKU+hOvJ +DfriTRLd+IthG5hv28I3A/CGjLICT0rDDtgaXd0uqloJAnjsgn5gMAcStpDW8Rm+ +t6LsrBL+5fBgkyU1r94Rvx0HHoyaZwBBouitVHw28hP3W+smddkqB1UxpGnTeL2B +gj3dVo/WTtRfO+0h0PKw1l98YE1pMTdqIwcOOE/ER0g4hvA/wrxuLhMvlVLMy/lL +58uctqaDUqryNyeerKbVkq4fJyCG5D2TwXVJ3i2DDh0xSt2Y10poZV4M4k8Su9Z5 +8zN2PSvYMT50aqF277v8BaOeYUApBE4kZGIJpo13ATGdEwpUFZ0Fri4zLYUZ1hWb +OC35sKo7OxWQ/+tefNUdgWHob6Vmy777jiYcLwxc3sS9rF3AJe0rMW83kCkR6hmy +A3250E137N/1QumHuT/Nj9rnI/lwt9jfaYkZjoAgT/C97m/mM83cYpGTdoGV1xNo +7G90MhP0di5FnVsrIaSnvkbGT9UgUWx0oVMjocifdG2qIhMI9psCAwEAAQKCAYBT +sHmaPmNaZj59jZCqp0YVQlpHWwBYQ5vD3pPE6oCttm0p9nXt/VkfenQRTthOtmT1 +POzDp00/feP7zeGLmqSYUjgRekPw4gdnN7Ip2PY5kdW77NWwDSzdLxuOS8Rq1MW9 +/Yu+ZPe3RBlDbT8C0IM+Atlh/BqIQ3zIxN4g0pzUlF0M33d6AYfYSzOcUhibOO7H +j84r+YXBNkIRgYKZYbutRXuZYaGuqejRpBj3voVu0d3Ntdb6lCWuClpB9HzfGN0c +RTv8g6UYO4sK3qyFn90ibIR/1GB9watvtoWVZqggiWeBzSWVWRsGEf9O+Cx4oJw1 +IphglhmhbgNksbj7bD24on/icldSOiVkoUemUOFmHWhCm4PnB1GmbD8YMfEdSbks +qDr1Ps1zg4mGOinVD/4cY7vuPFO/HCH07wfeaUGzRt4g0/yLr+XjVofOA3oowyxv +JAzr+niHA3lg5ecj4r7M68efwzN1OCyjMrVJw2RAzwvGxE+rm5NiT08SWlKQZnkC +gcEA4wvyLpIur/UB84nV3XVJ89UMNBLm++aTFzld047BLJtMaOhvNqx6Cl5c8VuW +l261KHjiVzpfNM3/A2LBQJcYkhX7avkqEXlj57cl+dCWAVwUzKmLJTPjfaTTZnYJ +xeN3dMYjJz2z2WtgvfvDoJLukVwIMmhTY8wtqqYyQBJ/l06pBsfw5TNvmVIOQHds +8ASOiFt+WRLk2bl9xrGGayqt3VV93KVRzF27cpjOgEcG74F3c0ZW9snERN7vIYwB +JfrlAoHBAMlahPwMP2TYylG8OzHe7EiehTekSO26LGh0Cq3wTGXYsK/q8hQCzL14 +kWW638vpwXL6L9ntvrd7hjzWRO3vX/VxnYEA6f0bpqHq1tZi6lzix5CTUN5McpDg +QnjenSJNrNjS1zEF8WeY9iLEuDI/M/iUW4y9R6s3WpgQhPDXpSvd2g3gMGRUYhxQ +Xna8auiJeYFq0oNaOxvJj+VeOfJ3ZMJttd+Y7gTOYZcbg3SdRb/kdxYki0RMD2hF +4ZvjJ6CTfwKBwQDiMqiZFTJGQwYqp4vWEmAW+I4r4xkUpWatoI2Fk5eI5T9+1PLX +uYXsho56NxEU1UrOg4Cb/p+TcBc8PErkGqR0BkpxDMOInTOXSrQe6lxIBoECVXc3 +HTbrmiay0a5y5GfCgxPKqIJhfcToAceoVjovv0y7S4yoxGZKuUEe7E8JY2iqRNAO +yOvKCCICv/hcN235E44RF+2/rDlOltagNej5tY6rIFkaDdgOF4bD7f9O5eEni1Bg +litfoesDtQP/3rECgcEAkQfvQ7D6tIPmbqsbJBfCr6fmoqZllT4FIJN84b50+OL0 +mTGsfjdqC4tdhx3sdu7/VPbaIqm5NmX10bowWgWSY7MbVME4yQPyqSwC5NbIonEC +d6N0mzoLR0kQ+Ai4u+2g82gicgAq2oj1uSNi3WZi48jQjHYFulCbo246o1NgeFFK +77WshYe2R1ioQfQDOU1URKCR0uTaMHClgfu112yiGd12JAD+aF3TM0kxDXz+sXI5 +SKy311DFxECZeXRLpcC3AoHBAJkNMJWTyPYbeVu+CTQkec8Uun233EkXa2kUNZc/ +5DuXDaK+A3DMgYRufTKSPpDHGaCZ1SYPInX1Uoe2dgVjWssRL2uitR4ENabDoAOA +ICVYXYYNagqQu5wwirF0QeaMXo1fjhuuHQh8GsMdXZvYEaAITZ9/NG5x/oY08+8H +kr78SMBOPy3XQn964uKG+e3JwpOG14GKABdAlrHKFXNWchu/6dgcYXB87mrC/GhO +zNwzC+QhFTZoOomFoqMgFWujng== -----END PRIVATE KEY----- diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py index e832ee6e41b2..b024ef4ca007 100644 --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -78,9 +78,9 @@ def _test_get_event_loop_new_process__sub_proc(): 'issuer': ((('countryName', 'XY'),), (('organizationName', 'Python Software Foundation CA'),), (('commonName', 'our-ca-server'),)), - 'notAfter': 'Nov 28 19:09:06 2027 GMT', - 'notBefore': 'Jan 19 19:09:06 2018 GMT', - 'serialNumber': '82EDBF41C880919C', + 'notAfter': 'Jul 7 14:23:16 2028 GMT', + 'notBefore': 'Aug 29 14:23:16 2018 GMT', + 'serialNumber': 'CB2D80995A69525C', 'subject': ((('countryName', 'XY'),), (('localityName', 'Castle Anthrax'),), (('organizationName', 'Python Software Foundation'),), diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index cf2b7514056c..4ba988fd57ee 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -251,6 +251,8 @@ def test_random_fork(self): self.assertNotEqual(child_random, parent_random) + maxDiff = None + def test_parse_cert(self): # note that this uses an 'unofficial' function in _ssl.c, # provided solely for this test, to exercise the certificate @@ -265,9 +267,9 @@ def test_parse_cert(self): (('commonName', 'localhost'),)) ) # Note the next three asserts will fail if the keys are regenerated - self.assertEqual(p['notAfter'], asn1time('Jan 17 19:09:06 2028 GMT')) - self.assertEqual(p['notBefore'], asn1time('Jan 19 19:09:06 2018 GMT')) - self.assertEqual(p['serialNumber'], 'F9BA076D5B6ABD9B') + self.assertEqual(p['notAfter'], asn1time('Aug 26 14:23:15 2028 GMT')) + self.assertEqual(p['notBefore'], asn1time('Aug 29 14:23:15 2018 GMT')) + self.assertEqual(p['serialNumber'], '98A7CF88C74A32ED') self.assertEqual(p['subject'], ((('countryName', 'XY'),), (('localityName', 'Castle Anthrax'),), diff --git a/Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst b/Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst new file mode 100644 index 000000000000..1ca3c7d7996c --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst @@ -0,0 +1 @@ +Use 3072 RSA keys and SHA-256 signature for test certs and keys. From webhook-mailer at python.org Thu Sep 20 06:33:57 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 20 Sep 2018 10:33:57 -0000 Subject: [Python-checkins] [2.7] bpo-34542: Update test certs and keys (GH-8997) (GH-9397) Message-ID: https://github.com/python/cpython/commit/49d65958e13db03b9a4240d8bdaff1a4be69a1d7 commit: 49d65958e13db03b9a4240d8bdaff1a4be69a1d7 branch: 2.7 author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-20T03:33:53-07:00 summary: [2.7] bpo-34542: Update test certs and keys (GH-8997) (GH-9397) Update all test certs and keys to use future proof crypto settings: * 3072 bit RSA keys * SHA-256 signature Signed-off-by: Christian Heimes . (cherry picked from commit e6dac0077996b1e1f886f036d6f2606237fa4c85) https://bugs.python.org/issue34542 files: A Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst M Lib/test/allsans.pem M Lib/test/keycert.passwd.pem M Lib/test/keycert.pem M Lib/test/keycert2.pem M Lib/test/keycert3.pem M Lib/test/keycert4.pem M Lib/test/make_ssl_certs.py M Lib/test/pycacert.pem M Lib/test/revocation.crl M Lib/test/ssl_cert.pem M Lib/test/ssl_key.passwd.pem M Lib/test/ssl_key.pem M Lib/test/test_ssl.py diff --git a/Lib/test/allsans.pem b/Lib/test/allsans.pem index bf59f30abaad..6eebde7a57f1 100644 --- a/Lib/test/allsans.pem +++ b/Lib/test/allsans.pem @@ -1,64 +1,81 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQD59JyhPgYe7nhZ -Z2IGhaklNgtRkD+5BVs7lEWovYRBlXpPA6PuaHat25rI8EGYHmlufPherg2Qu6sC -GmZZKo7TgjlmDcwVS4hkebtFH7OZy5Il7Y2ZIdiK7Xp9Z0EPoqwacYowB0a8WhZY -I2Vm4EzCNKl6/htkwjgn2JXGizxvGt/1kNqP/GBAX+vjgeahOsn8jVh96KpFHJbS -g83cX4t8M7FJv7yNoDLvORHnvKCOXbQmr6ZMGcZN8PwS8awQ31khZTpEx+hCe+Pi -GzeOlxpZimXWDAGWA4tZ58Ka/QvO7VQbD5Ci166ODvvs+tEXfBUExtPcS+02IBJV -tzhBna9VAgMBAAECggEAPar9DccIqY76QEyCYcuOPLEFv9zP6+0HYj6lpQkE3U1s -vJvQURyS0zgQCy1Dca1nI6xPdsSIckHq4fzzbWJTlJlXYfdbd5GIGAn0iwxUOkiA -ST0/px0zmKsYgmH8KkhfH7MNfeX9rLCpPJuXA/eo2G03tzGEPqqwQhxsb2ygv2Qs -M7OqJz6RJu87K1Y+psWIv9+VhNVja0kvsg52QMK9mtp8layb54qLI5R5e09sIudq -RHegtnSOBo9kt32H9vWUFaF5PpYt4yks4KYI4ulKGWJGXHMDW4uHUaE/tjNQuYAX -DuDvjN+ECSJvigiUbu2k0xB2KYIb1fpcxlz/YBdADQKBgQD/Z2VtBUjOFnJKz00f -xN0akp7XPgd1yCb1/wZq9PQiGvzIAMDIplioTvjOjhOzPJaWD0GICNeypzQ48+0P -UsPIKbazpIZN6bZncr65plSpg0KANq46hbkPHOo8PHDa7yoxBUSPr8F7P1OCRkn6 -+QdgcnrAly7yfqO2ahAWOX7iCwKBgQD6ifXSCKfRF1GUb3Ws7S1rLxeBasWq+EmC -sUnck0S+AyaMkN+kZ5zejbN+NDuUMQ7+3wUIheTclUhzR0LP3+r5jjHsimJuvOml -wuV37F+Om5lD/Xx27NfbtRKn/bK6o0zDL8JB2eFB0N7Fh7hRYoUMdrpQs5sU91IC -pNYlAcLwHwKBgGvLK9eTf2LbvmksjRR3dgodD8UwfN2NGESC2iaSM+ehFEclajhF -XO3MRt6GwHHJhJTY44OSl9bjEvtmmAr7l34HfQDc04JWvZFzsGOSe/D/YTXT3jz8 -61ohjgrWR5tfjaMa4hDy0Oo/k/NLzzWJnT9rkbtvE3VtVZNLuHZo1dB5AoGBAMHO -wStV6MO1nzUNN+Gqo8zbY/qIJxsH8I26KaIJBk9azpJEa8yZHl+HDEffjgsoHCqL -STB7qzv7+0y53nRCClo8ZmBN+LEjUDcbWjl3z7/YnCpdR9ATjTP3kdQETCNWucXw -Bvy72CX6tqnlQG8soDGxEpXlKl2AqJ9E9icwgqUPAoGAL6xTDdgcYTbk9wxCd41l -NhHTSvLrGXLAzv61PCnlOJEJbuuezb2VW0ibsud5CA4Mi0tf9ET790XSOFd5nCjQ -6rr06AkjQsoFvjL1dO9EzVFPW0JrZ3C9y8ZOjdeAfPEmFL2T6VqmQ+IcCUNhSr39 -NBdKrboEFfnKanfbstekhAs= +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQCg/pM6dP7BTFNc +qe6wIJIBB7HjwL42bp0vjcCVl4Z3MRWFswYpfxy+o+8+PguMp4K6zndA5fwNkK/H +3HmtanncUfPqnV0usN0NHQGh/f9xRoNmB1q2L7kTuO99o0KLQgvonRT2snf8rq9n +tPRzhHUGYhog7zzNxetYV309PHpPr19BcKepDtM5RMk2aBnoN5vtItorjXiDosFm +6o5wQHrcupcVydszba6P75BEbc1XIWvq2Fv8muaw4pCe81QYINyLqgcPNO/nF3Os +5EI4HKjCNRSCOhOcWqYctXLXN9lBdMBBvQc3zDmYzh1eIZewzZXPVEQT33xPkhxz +HNmhcIctpWX4LTRF6FulkcbeuZDga3gkZYJf/M6IpU1WYXr6q8sNxbgmRRX/NuHo +V9oDwBzLG07rKUiqRHfjGqoCRmmVeVYpryvXUNjHGH0nlVzz/8lTUxAnJorO3Fdc +I+6zKLUPICdAlvz51AH6yopgPFhrdgA0pVzPO6L5G8SRQCxKhAUCAwEAAQKCAYAa +2jtOTcNMFGH3G7TfFZ+kolbuaPCQ/aQkEV2k1dAswzgWw8RsWXI+7fLyi8C7Zhks +9VD4tyNyU8at7D0zSoYm1Fh9sl+fcQp9rG/gSBA6IYu7EdD0gEM7YeY4K2nm9k4s +Lz8W4q+WqsBA6PK47cfjF6vKAH1AyRk28+jEtPiln9egf5zHWtyqOanh9D0V+Wh9 +hgmjqAYI1rWxZ7/4Qxj7Bfg7Px7blhi+kzOZ5kKQnNd2JT46hM+jgzah/G3zVE+R +FFW6ksmJgZ+dCuSbE7HEJmKms1CWq/1Cll0A3uy4JTDZOrK4KcZQ9UjjWJWvlXQm +uNXSSAp1k287DLVUm9c22SDeXpb9PyKmzyvJvVmMqqBx6QzHZ/L7WPzpUWAoLcU+ +ZHT7vggDymkIO+fcRbUzv8s5R7RnLbcBga51/5OCUvAWDoJXNw0qwYZOIbfTnQgs +8xbCmbMzllyYM/dK3GxQAwfn8Hzk+DbS/NObMjHLCWLfYeUvutXJSNly6Ny+ZcEC +gcEAzo5Y1UFOfBX4MZLIZ69LfgaXj9URobMwqlEwKil8pWQMa951ga3moLt91nOe +SAQz3meFTBX/VAb2ZHLeIf3FoNkiIx48PkxsR/hhLHpvl26zEg3yXs3tv0IFBx2R +EEnLNpQaAQFR9S1yDOaG2rsb17ZDKyp9isDpAENHAmEnT/XJn+Dc0SOH1EVDjUeM +JqToAF/fjIx/RF4oUJCAgOPBMlRy5ywLQk8uDi6ft0NCzzCi0eCuk1Ty3KzWFGwx +7cYRAoHBAMeIPCzHG3No4JGUFunslVwo5TuC7maO6qYKbq0OyvwWfL4b7gjrMBR9 +d5WyZlp/Vf40O463dg8x8qPNOFWp49f3hxTvvfnt2/m3+CQuDOLfqBbHufZApP1J +U9MubUNnDFHHeJ9l0tg2nhiLw24GHeMARZhA/BimMQPY0OpZPpLVxAUArM2EB7hI +glQpYCtdXhqwl1pl0u3TZ08y3BXYNg9BycdpGRMWSsAwsApJRgNuI/dfDKu0uMYF +/pUhXVPatQKBwGgLpAun3dT7bA3sli5ESo6s22OEPGFrVbQ1OUHDrBnTj742TJKJ ++oY0a2q+ypgUJdx94NM2sWquJybqBaKxpf8j4OI3tLjc3h5SqwAwnE13YZRSmifP +K1cP9mBjMFM4GLjhWUfwVkxeG/kLlhpP7fJ2yNbRjHN8QOH1AavdLGRGts1mA1UF +xMHUMfbUd3Bv2L13ja/KhcD2fPA4GcLS9tpXV5nCwdkg8V4LdkBmDR04rotx1f44 +6Czokt2usmfHQQKBwFkufxbUd2SB/72Rnxw27hse/DY5My0Lu70y9HzNG9TIiEDA +YwgBdp/x5D04W58fQuQ3nFcRkOcBwB2OYBuJr5ibvfiRnyvSMHvQykwBeSj+Jjbo +VinGgvfiimDdY2C48jyrFzLHZBHXd5oo/dRzT3Bicri2cvbhcQ7zHY1hDiK7AL3r +q1DALmMjpXzQcXdwZ9suCrgQwtIhpw8zAEOTO7ZeBT3nr5lkYUy9djFixrRJyjGK +fjNQtzVrAHrPStNr8QKBwQDCC0zhsCnTv4sAJmW7LL6Ayd5rbWhUZ6px1xY0yHMA +hehj+xbaiC6cfVr5Rg0ncvaa8AExu4kXpVsupTyNwvC4NgzLHtfBw6WUdOnd1awE +kSrDtDReBt2wByAcQwttQsrJ1/Pt6zcNJJI4Z9s8G4NTcQWJwUhU20N55JQKR//l +OQJqhq9NVhte/ctDjVwOHs/OhDNvxsAWxdjnf/O2up0os+M2bFkmHuaVW0vQbqTQ +mw7Vbzk2Ff5oT6E3kbC8Ur4= -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIGMDCCBRigAwIBAgIJAJYf8T95ptq5MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV +MIIHMDCCBZigAwIBAgIJALVVA6v9zJS5MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTgwMTE5 -MTkwOTA3WhcNMjgwMTE3MTkwOTA3WjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO +IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTgwODI5 +MTQyMzE3WhcNMjgwODI2MTQyMzE3WjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO Q2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3VuZGF0 -aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB -CgKCAQEA+fScoT4GHu54WWdiBoWpJTYLUZA/uQVbO5RFqL2EQZV6TwOj7mh2rdua -yPBBmB5pbnz4Xq4NkLurAhpmWSqO04I5Zg3MFUuIZHm7RR+zmcuSJe2NmSHYiu16 -fWdBD6KsGnGKMAdGvFoWWCNlZuBMwjSpev4bZMI4J9iVxos8bxrf9ZDaj/xgQF/r -44HmoTrJ/I1YfeiqRRyW0oPN3F+LfDOxSb+8jaAy7zkR57ygjl20Jq+mTBnGTfD8 -EvGsEN9ZIWU6RMfoQnvj4hs3jpcaWYpl1gwBlgOLWefCmv0Lzu1UGw+Qoteujg77 -7PrRF3wVBMbT3EvtNiASVbc4QZ2vVQIDAQABo4IC8TCCAu0wggEwBgNVHREEggEn -MIIBI4IHYWxsc2Fuc6AeBgMqAwSgFwwVc29tZSBvdGhlciBpZGVudGlmaWVyoDUG -BisGAQUCAqArMCmgEBsOS0VSQkVST1MuUkVBTE2hFTAToAMCAQGhDDAKGwh1c2Vy -bmFtZYEQdXNlckBleGFtcGxlLm9yZ4IPd3d3LmV4YW1wbGUub3JnpGcwZTELMAkG -A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo -b24gU29mdHdhcmUgRm91bmRhdGlvbjEYMBYGA1UEAwwPZGlybmFtZSBleGFtcGxl -hhdodHRwczovL3d3dy5weXRob24ub3JnL4cEfwAAAYcQAAAAAAAAAAAAAAAAAAAA -AYgEKgMEBTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG -AQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFH9ye3+WhBnHqNhtFu059bzY -SWM8MIGPBgNVHSMEgYcwgYSAFH9ye3+WhBnHqNhtFu059bzYSWM8oWGkXzBdMQsw -CQYDVQQGEwJYWTEXMBUGA1UEBwwOQ2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5 -dGhvbiBTb2Z0d2FyZSBGb3VuZGF0aW9uMRAwDgYDVQQDDAdhbGxzYW5zggkAlh/x -P3mm2rkwgYMGCCsGAQUFBwEBBHcwdTA8BggrBgEFBQcwAoYwaHR0cDovL3Rlc3Rj -YS5weXRob250ZXN0Lm5ldC90ZXN0Y2EvcHljYWNlcnQuY2VyMDUGCCsGAQUFBzAB -hilodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3RjYS9vY3NwLzBDBgNV -HR8EPDA6MDigNqA0hjJodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3Rj -YS9yZXZvY2F0aW9uLmNybDANBgkqhkiG9w0BAQsFAAOCAQEAYwYJcerUPvnsP7e2 -HGp/It0OZ8Cvpt8Qf7A+NSPvJqkyKakl8zK/50iq/qQKH09CnfEae4rfXLdlYsvV -2PZYK0LDWnyTcHSJWAVJjlSFIFt3ig9FdHv9GYtSWWod66cZ0sEZOoF2IHZUGby+ -Qa+JQpmv5jEuGIZzjcsh6hSOou8ph7LsCsRdVlQqk8rM97vB7DAgh01vedlbolsq -JxsuPRydNFV/eWq3AgAWgZL3LdYYIAgaVOTnnd3xARw8DlT1q6+Lzc71GBXrRZYh -qgd+xC/K1812gMPImTX02bxpkhCuIdVd7cztWi8sdQmSgDEFdYMXo4NzlFTK8dlC -Y4wa3Q== +aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIB +igKCAYEAoP6TOnT+wUxTXKnusCCSAQex48C+Nm6dL43AlZeGdzEVhbMGKX8cvqPv +Pj4LjKeCus53QOX8DZCvx9x5rWp53FHz6p1dLrDdDR0Bof3/cUaDZgdati+5E7jv +faNCi0IL6J0U9rJ3/K6vZ7T0c4R1BmIaIO88zcXrWFd9PTx6T69fQXCnqQ7TOUTJ +NmgZ6Deb7SLaK414g6LBZuqOcEB63LqXFcnbM22uj++QRG3NVyFr6thb/JrmsOKQ +nvNUGCDci6oHDzTv5xdzrORCOByowjUUgjoTnFqmHLVy1zfZQXTAQb0HN8w5mM4d +XiGXsM2Vz1REE998T5IccxzZoXCHLaVl+C00RehbpZHG3rmQ4Gt4JGWCX/zOiKVN +VmF6+qvLDcW4JkUV/zbh6FfaA8AcyxtO6ylIqkR34xqqAkZplXlWKa8r11DYxxh9 +J5Vc8//JU1MQJyaKztxXXCPusyi1DyAnQJb8+dQB+sqKYDxYa3YANKVczzui+RvE +kUAsSoQFAgMBAAGjggLxMIIC7TCCATAGA1UdEQSCAScwggEjggdhbGxzYW5zoB4G +AyoDBKAXDBVzb21lIG90aGVyIGlkZW50aWZpZXKgNQYGKwYBBQICoCswKaAQGw5L +RVJCRVJPUy5SRUFMTaEVMBOgAwIBAaEMMAobCHVzZXJuYW1lgRB1c2VyQGV4YW1w +bGUub3Jngg93d3cuZXhhbXBsZS5vcmekZzBlMQswCQYDVQQGEwJYWTEXMBUGA1UE +BwwOQ2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3Vu +ZGF0aW9uMRgwFgYDVQQDDA9kaXJuYW1lIGV4YW1wbGWGF2h0dHBzOi8vd3d3LnB5 +dGhvbi5vcmcvhwR/AAABhxAAAAAAAAAAAAAAAAAAAAABiAQqAwQFMA4GA1UdDwEB +/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/ +BAIwADAdBgNVHQ4EFgQUoLHAHNTWrHkSCUYkhn5NH0S40CAwgY8GA1UdIwSBhzCB +hIAUoLHAHNTWrHkSCUYkhn5NH0S40CChYaRfMF0xCzAJBgNVBAYTAlhZMRcwFQYD +VQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZv +dW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnOCCQC1VQOr/cyUuTCBgwYIKwYBBQUH +AQEEdzB1MDwGCCsGAQUFBzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0 +L3Rlc3RjYS9weWNhY2VydC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2Eu +cHl0aG9udGVzdC5uZXQvdGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0 +dHA6Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3Js +MA0GCSqGSIb3DQEBCwUAA4IBgQAeKJKycO2DES98gyR2e/GzPYEw87cCS0cEpiiP +3CEUgzfEbF0X89GDKEey4H3Irvosbvt2hEcf2RNpahLUL/fUv53bDmHNmL8qJg5E +UJVMOHvOpSOjqoqeRuSyG0GnnAuUwcxdrZY6UzLdslhuq9F8UjgHr6KSMx56G9uK +LmTy5njMab0in2xL/YRX/0nogK3BHqpUHrfCdEYZkciRxtAa+OPpWn4dcZi+Fpf7 +ZYSgPLNt+djtFDMIAk5Bo+XDaQdW3dhF0w44enrGAOV0xPE+/jOuenNhKBafjuNb +lkeSr45+QZsi1rd18ny8z3uuaGqIAziFgmllZOH2D8giTn6+5jZcCNZCoGKUkPI9 +l/GMWwxg4HQYYlZcsZzTCem9Rb2XcrasAbmhFapMtR+QAwSed5vKE7ZdtQhj74kB +7Q0E7Lkgpp6BaObb2As8/f0K/UlSVSvrYk+i3JT9wK/qqkRGxsTFEF7N9t0rKu8y +4JdQDtZCI552MsFvYW6m+IOYgxg= -----END CERTIFICATE----- diff --git a/Lib/test/keycert.passwd.pem b/Lib/test/keycert.passwd.pem index 0ad696055190..cbb3c3bccd2c 100644 --- a/Lib/test/keycert.passwd.pem +++ b/Lib/test/keycert.passwd.pem @@ -1,50 +1,68 @@ -----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED -DEK-Info: DES-EDE3-CBC,E74528136B90D2DD +DEK-Info: DES-EDE3-CBC,D134E931C96D9DEC -WRHVD2PJXPqjFSHg92HURIsUzvsTE4a9oi0SC5yMBFKNWA5Z933gK3XTifp6jul5 -zpNYi8jBXZ2EqJJBxCuVcefmXSxL0q7CMej25TdIC4BVAFJVveeprHPUFkNB0IM1 -go5Lg4YofYqTCg3OE3k7WvfR3Zg1cRYxksDKO+WNZgWyKBex5X4vjOiyUqDl3GKt -kQXnkg1VgPV2Vrx93S9XNdELNRTguwf+XG0fkhtYhp/zCto8uKTgy5elK2P/ulGp -7fe6uj7h/uN9L7EOC6CjRkitywfeBUER739mOcGT4imSFJ9G27TCqPzj2ea3uuaf -/v1xhkQ4M6lNY/gcRfgVpCXhW43aAQV8XXQRMJTqLmz5Y5hYTKn+Ugq5vJ/ngyRM -lu1gUJnYYaemBTb4hbm6aBvnYK9mORa891Pmf+vxU9rYuQIdVAhvvXh4KBreSEBI -1AFy6dFKXl8ZKs6Wrq5wPefmFFkRmZ8OBiiq0fp2ApCRGZw6LsjCgwrRM38JiY7d -3OdsJpKvRYufgUyuuzUE0xA+E4yMvD48M9pPq2fC8O5giuGL1uEekQWXJuq+6ZRI -XYKIeSkuQALbX3RAzCPXTUEMtCYXKm/gxrrwJ+Bet4ob2amf3MX0uvWwOuAq++Fk -J0HFSBxrwvIWOhyQXOPuJdAN8PXA7dWOXfOgOMF0hQYqZCl3T4TiVZJbwVQtg1sN -dO7oAD5ZPHiKzvveZuB6k1FlBG8j0TyAC+44ChxkPDD3jF4dd6zGe62sDf85p4/d -W80gxJeD3xnDxG0ePPns+GuKUpUaWS7WvHtDpeFW1JEhvOqf8p1Li9a7RzWVo8ML -mGTdQgBIYIf6/fk69pFKl0nKtBU75KaunZz4nAmd9bNED4naDurMBg44u5TvODbJ -vgYIYXIYjNvONbskJatVrrTS8zch2NwVIjCi8L/hecwBXbIXzo1pECpc6BU7sQT8 -+i9sDKBeJcRipzfKZNHvnO19mUZaPCY8+a/f9c21DgKXz+bgLcJbohpSaeGM8Gfc -aZd3Vp9n3OJ3g2zQR1++HO9v1vR/wLELu6MeydkvMduHLmOPCn54gZ9z51ZNPAwa -qfFIsH+mLh9ks0H74ssF59uIlstkgB9zmZHv/Q0dK9ZfG/VEH6rSgdETWhZxhoMQ -Z92jXBEFT0zhI3rrIPNY+XS7eJCQIc1wc84Ea3cRk7SP+S1og3JtAxX56ykUwtkM -LQ/Dwwa6h1aqD0l2d5x1/BSdavtTuSegISRWQ4iOmSvEdlFP7H4g6RZk/okbLzMD -Evq5gNc7vlXhVawoQU8JCanJ5ZbbWnIRZfiXxBQS4lpYPKvJt4ML9z/x+82XxcXv -Z93N2Wep7wWW5OwS2LcQcOgZRDSIPompwo/0pMFGOS+5oort0ZDRHdmmGLjvBcCb -1KQmKQ4+8brI/3rjRzts6uDLjTGNxSCieNsnqhwHUv9Mg9WDSWupcGa+x27L89x3 -rObf6+3umcFLSjIzU8wuv1hx/e/y98Kv7BDBNYpAr6kVMrLnzYjAfJbBmqlxkzkQ -IgQzgrk2QZoTdgwR+S374NAMO0AE5IlO+/qa6qp2SORGTDX64I3UNw== +nuGFEej7vIjkYWSMz5OJeVTNntDRQi6ZM4DBm3g8T7i/0odr3WFqGMMKZcIhLYQf +rgRq7RSKtrJ1y5taVucMV+EuCjyfzDo0TsYt+ZrXv/D08eZhjRmkhoHnGVF0TqQm +nQEXM/ERT4J2RM78dnG+homMkI76qOqxgGbRqQqJo6AiVRcAZ45y8s96bru2TAB8 ++pWjO/v0Je7AFVdwSU52N8OOY6uoSAygW+0UY1WVxbVGJF2XfRsNpPX+YQHYl6e+ +3xM5XBVCgr6kmdAyub5qUJ38X3TpdVGoR0i+CVS9GTr2pSRib1zURAeeHnlqiUZM +4m0Gn9s72nJevU1wxED8pwOhR8fnHEmMKGD2HPhKoOCbzDhwwBZO27TNa1uWeM3f +M5oixKDi2PqMn3y2cDx1NjJtP661688EcJ5a2Ih9BgO9xpnhSyzBWEKcAn0tJB0H +/56M0FW6cdOOIzMveGGL7sHW5E+iOdI1n5e7C6KJUzew78Y9qJnhS53EdI6qTz9R +wsIsj1i070Fk6RbPo6zpLlF6w7Zj8GlZaZA7OZZv9wo5VEV/0ST8gmiiBOBc4C6Y +u9hyLIIu4dFEBKyQHRvBnQSLNpKx6or1OGFDVBay2In9Yh2BHh1+vOj/OIz/wq48 +EHOIV27fRJxLu4jeK5LIGDhuPnMJ8AJYQ0bQOUP6fd7p+TxWkAQZPB/Dx/cs3hxr +nFEdzx+eO+IAsObx/b1EGZyEJyETBslu4GwYX7/KK3HsJhDJ1bdZ//28jOCaoir6 +ZOMT72GRwmVoQTJ0XpccfjHfKJDRLT7C1xvzo4Eibth0hpTZkA75IUYUp6qK/PuJ +kH/qdiC7QIkRKtsrawW4vEDna3YtxIYhQqz9+KwO6u/0gzooZtv1RU4U3ifMDB5u +5P5GAzACRqlY8QYBkM869lvWqzQPHvybC4ak9Yx6/heMO9ddjdIW9BaK8BLxvN/6 +UCD936Y4fWltt09jHZIoxWFykouBwmd7bXooNYXmDRNmjTdVhKJuOEOQw8hDzx7e +pWFJ9Z/V4Qm1tvXbCD7QFqMCDoY3qFvVG8DBqXpmxe1yPfz21FWrT7IuqDXAD3ns +vxfN/2a+Cy04U9FBNVCvWqWIs5AgNpdCMJC2FlXKTy+H3/7rIjNyFyvbX0vxIXtK +liOVNXiyVM++KZXqktqMUDlsJENmIHV9B046luqbgW018fHkyEYlL3iRZGbYegwr +XO9VVIKVPw1BEvJ8VNdGFGuZGepd8qX2ezfYADrNR+4t85HDm8inbjTobSjWuljs +ftUNkOeCHqAvWCFQTLCfdykvV08EJfVY79y7yFPtfRV2gxYokXFifjo3su9sVQr1 +UiIS5ZAsIC1hBXWeXoBN7QVTkFi7Yto6E1q2k10LiT3obpUUUQ/oclhrJOCJVjrS +oRcj2QBy8OT4T9slJr5maTWdgd7Lt6+I6cGQXPaDvjGOJl0eBYM14vhx4rRQWytJ +k07hhHFO4+9CGCuHS8AAy2gR6acYFWt2ZiiNZ0z/iPIHNK4YEyy9aLf6uZH/KQjE +jmHToo7XD6QvCAEC5qTHby3o3LfHIhyZi/4L+AhS4FKUHF6M0peeyYt4z3HaK2d2 +N6mHLPdjwNjra7GOmcns4gzcrdfoF+R293KpPal4PjknvR3dZL4kKP/ougTAM5zv +qDIvRbkHzjP8ChTpoLcJsNVXykNcNkjcSi0GHtIpYjh6QX6P2uvR/S4+Bbb9p9rn +hIy/ovu9tWN2hiPxGPe6torF6BulAxsTYlDercC204AyzsrdA0pr6HBgJH9C6ML1 +TchwodbFJqn9rSv91i1liusAGoOvE81AGBdrXY7LxfSNhYY1IK6yR/POJPTd53sA +uX2/j6Rtoksd/2BHPM6AUnI/2B9slhuzWX2aCtWLeuwvXDS6rYuTigaQmLkzTRfM +dlMI3s9KLXxgi5YVumUZleJWXwBNP7KiKajd+VTSD+7WAhyhM5FIG5wVOaxmy4G2 +TyqZ/Ax9d2VEjTQHWvQlLPQ4Mp0EIz0aEl94K/S8CK8bJRH6+PRkar+dJi1xqlL+ +BYb42At9mEJ8odLlFikvNi1+t7jqXk5jRi5C0xFKx3nTtzoH2zNUeuA3R6vSocVK +45jnze9IkKmxMlJ4loR5sgszdpDCD3kXqjtCcbMTmcrGyzJek3HSOTpiEORoTFOe +Rhg6jH5lm+QcC263oipojS0qEQcnsWJP2CylNYMYHR9O/9NQxT3o2lsRHqZTMELV +uQa/SFH+paQNbZOj8MRwPSqqiIxJFuLswKte1R+W7LKn1yBSM7Pp39lNbzGvJD2E +YRfnCwFpJ54voVAuQ4jXJvigCW2qeCjXlxeD6K2j4eGJEEOmIjIW1wjubyBY6OI3 -----END RSA PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIDWTCCAkGgAwIBAgIJAPm6B21bar2bMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV +MIIEWTCCAsGgAwIBAgIJAJinz4jHSjLtMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODAx -MTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH +IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODA4 +MjkxNDIzMTVaFw0yODA4MjYxNDIzMTVaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH DA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5k -YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEP -ADCCAQoCggEBAKvvsX2gEti4shve3iYMc+jE4Se7WHs1Bol2f21H8qNboDOFdeb1 -RKHjmq3exHpajywOUEgne9nKHJY/3f2phR4Y5klqG6liLgiSpVyRlcBGbeT2qEAj -9oLiLFUXLGfGDds2mTwivQDLJBWi51j7ff5k2Pr58fN5ugYMn24T9FNyn0moT+qj -SFoBNm58l9jrdkJSlgWfqPlbiMa+mqDn/SFtrwLF2Trbfzu42Sd9UdIzMaSSrzbN -sGm53pNhCh8KndWUQ8GPP2IsLPoUU4qAtmZuTxCx2S1cXrN9EkmT69tlOH84YfSn -96Ih9bWRc7M5y5bfVdEVM+fKQl3hBRf05qMCAwEAAaMYMBYwFAYDVR0RBA0wC4IJ -bG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQAtQ8f37cCEk7/rAcbYR53ce3iK -Vpihb0U2ni1QjG9Tg9UIExkIGkwTiCm7kwQL+GEStBu9AG/QVrOjeTriRiddhWkk -ze8kRaI3AC/63t6Vh9Q1x6PESgeE4OtAO9JpJCf4GILglA789Y/b/GF8zJZQxR13 -qpB4ZwWw7gCBhdEW59u6CFeBmfDa58hM8lWvuVoRrTi7bjUeC6PAn5HVMzZSykhu -4HaUfBp6bKFjuym2+h/VvM1n8C3chjVSmutsLb6ELdD8IK0vPV/yf5+LN256zSsS -dyUZYd8XwQaioEMKdbhLvnehyzHiWfQIUR3BdhONxoIJhHv/EAo8eCkHHYIF +YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGP +ADCCAYoCggGBALKUqUtopT6E68kN+uJNEt34i2EbmG/bwjcD8IaMsgJPSsMO2Bpd +3S6qWgkCeOyCfmAwBxK2kNbxGb63ouysEv7l8GCTJTWv3hG/HQcejJpnAEGi6K1U +fDbyE/db6yZ12SoHVTGkadN4vYGCPd1Wj9ZO1F877SHQ8rDWX3xgTWkxN2ojBw44 +T8RHSDiG8D/CvG4uEy+VUszL+Uvny5y2poNSqvI3J56sptWSrh8nIIbkPZPBdUne +LYMOHTFK3ZjXSmhlXgziTxK71nnzM3Y9K9gxPnRqoXbvu/wFo55hQCkETiRkYgmm +jXcBMZ0TClQVnQWuLjMthRnWFZs4Lfmwqjs7FZD/61581R2BYehvpWbLvvuOJhwv +DFzexL2sXcAl7SsxbzeQKRHqGbIDfbnQTXfs3/VC6Ye5P82P2ucj+XC32N9piRmO +gCBP8L3ub+YzzdxikZN2gZXXE2jsb3QyE/R2LkWdWyshpKe+RsZP1SBRbHShUyOh +yJ90baoiEwj2mwIDAQABoxgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZI +hvcNAQELBQADggGBAHRUO/UIHl3jXQENewYayHxkIx8t7nu40iO2DXbicSijz5bo +5//xAB6RxhBAlsDBehgQP1uoZg+WJW+nHu3CIVOU3qZNZRaozxiCl2UFKcNqLOmx +R3NKpo1jYf4REQIeG8Yw9+hSWLRbshNteP6bKUUf+vanhg9+axyOEOH/iOQvgk/m +b8wA8wNa4ujWljPbTQnj7ry8RqhTM0GcAN5LSdSvcKcpzLcs3aYwh+Z8e30sQWna +F40sa5u7izgBTOrwpcDm/w5kC46vpRQ5fnbshVw6pne2by0mdMECASid/p25N103 +jMqTFlmO7kpf/jpCSmamp3/JSEE1BJKHwQ6Ql4nzRA2N1mnvWH7Zxcv043gkHeAu +0x8evpvwuhdIyproejNFlBpKmW8OX7yKTCPPMC/VkX8Q1rVkxU0DQ6hmvwZlhoKa +9Wc2uXpw9xF8itV4Uvcdr3dwqByvIqn7iI/gB+4l41e0u8OmH2MKOx4Nxlly5TNW +HcVKQHyOeyvnINuBAQ== -----END CERTIFICATE----- diff --git a/Lib/test/keycert.pem b/Lib/test/keycert.pem index 9545dcf4b94f..0d398633739a 100644 --- a/Lib/test/keycert.pem +++ b/Lib/test/keycert.pem @@ -1,48 +1,66 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCr77F9oBLYuLIb -3t4mDHPoxOEnu1h7NQaJdn9tR/KjW6AzhXXm9USh45qt3sR6Wo8sDlBIJ3vZyhyW -P939qYUeGOZJahupYi4IkqVckZXARm3k9qhAI/aC4ixVFyxnxg3bNpk8Ir0AyyQV -oudY+33+ZNj6+fHzeboGDJ9uE/RTcp9JqE/qo0haATZufJfY63ZCUpYFn6j5W4jG -vpqg5/0hba8Cxdk62387uNknfVHSMzGkkq82zbBpud6TYQofCp3VlEPBjz9iLCz6 -FFOKgLZmbk8QsdktXF6zfRJJk+vbZTh/OGH0p/eiIfW1kXOzOcuW31XRFTPnykJd -4QUX9OajAgMBAAECggEAHppmXDbuw9Z0FVPg9KLIysioTtsgz6VLiZIm8juZK4x2 -glUh/D7xvWL2uDXrgN+3lh7iGUW13LkFx5SMncbbo9TIwI57Z/XKvcnkVwquve+L -RfLFVc1Q5lD9lROv2rS86KTaN4LzYz3FKXi6dvMkpPAsUtfEQhMLkmISypQQq/1z -EJaqo7r85OjN7e0wKazlKZpOzJEa5FQLMVRjTRFhLFNbHXX/tAet2jw+umATKbw8 -hYgiuZ44TwSEd9JeIV/oSYWfI/3HetuYW0ru3caiztRF2NySNu8lcsWgNC7fIku9 -mcHjtSNzs91QN1Qlu7GQvvhpt6OWDirNDCW+49WGaQKBgQDg9SDhfF0jRYslgYbH -cqO4ggaFdHjrAAYpwnAgvanhFZL/zEqm5G1E7l/e2fCkJ9VOSFO0A208chvwMcr+ -dCjHE2tVdE81aQ2v/Eo83VdS1RcOV4Y75yPH48rMhxPaHvxWD/FFDbf0/P2mtPB7 -SZ3kIeZMkE1wxdaO3AKUbQoozwKBgQDDqYgg7kVtygyICE1mB8Hwp6nUxFTczG7y -4XcsDqMIrKmw+PbQluvkoHoStxeVrsTloDhkTjIrpmYLyAiazg+PUJdkd6xrfLSj -VV6X93W0S/1egEb1F1CGFxtk8v/PWH4K76EPL2vxXdxjywz3GWlrL9yDYaB2szzS -DqgwVMqx7QKBgDCD7UF0Bsoyl13RX3XoPXLvZ+SkR+e2q52Z94C4JskKVBeiwX7Y -yNAS8M4pBoMArDoj0xmBm69rlKbqtjLGbnzwrTdSzDpim7cWnBQgUFLm7gAD1Elb -AhZ8BCK0Bw4FnLoa2hfga4oEfdfUMgEE0W5/+SEOBgWKRUmuHUhRc911AoGAY2EN -YmSDYSM5wDIvVb5k9B3EtevOiqNPSw/XnsoEZtiEC/44JnQxdltIBY93bDBrk5IQ -cmoBM4h91kgQjshQwOMXMhFSwvmBKmCm/hrTbvMVytTutXfVD3ZXFKwT4DW7N0TF -ElhsxBh/YzRz7mG62JVjtFt2zDN3ld2Z8YpvtXUCgYEA4EJ4ObS5YyvcXAKHJFo6 -Fxmavyrf8LSm3MFA65uSnFvWukMVqqRMReQc5jvpxHKCis+XvnHzyOfL0gW9ZTi7 -tWGGbBi0TRJCa8BkvgngUZxOxUlMfg/7cVxOIB0TPoUSgxFd/+qVz4GZMvr0dPu7 -eAF7J/8ECVvb0wSPTUI1N3c= +MIIG/wIBADANBgkqhkiG9w0BAQEFAASCBukwggblAgEAAoIBgQCylKlLaKU+hOvJ +DfriTRLd+IthG5hv28I3A/CGjLICT0rDDtgaXd0uqloJAnjsgn5gMAcStpDW8Rm+ +t6LsrBL+5fBgkyU1r94Rvx0HHoyaZwBBouitVHw28hP3W+smddkqB1UxpGnTeL2B +gj3dVo/WTtRfO+0h0PKw1l98YE1pMTdqIwcOOE/ER0g4hvA/wrxuLhMvlVLMy/lL +58uctqaDUqryNyeerKbVkq4fJyCG5D2TwXVJ3i2DDh0xSt2Y10poZV4M4k8Su9Z5 +8zN2PSvYMT50aqF277v8BaOeYUApBE4kZGIJpo13ATGdEwpUFZ0Fri4zLYUZ1hWb +OC35sKo7OxWQ/+tefNUdgWHob6Vmy777jiYcLwxc3sS9rF3AJe0rMW83kCkR6hmy +A3250E137N/1QumHuT/Nj9rnI/lwt9jfaYkZjoAgT/C97m/mM83cYpGTdoGV1xNo +7G90MhP0di5FnVsrIaSnvkbGT9UgUWx0oVMjocifdG2qIhMI9psCAwEAAQKCAYBT +sHmaPmNaZj59jZCqp0YVQlpHWwBYQ5vD3pPE6oCttm0p9nXt/VkfenQRTthOtmT1 +POzDp00/feP7zeGLmqSYUjgRekPw4gdnN7Ip2PY5kdW77NWwDSzdLxuOS8Rq1MW9 +/Yu+ZPe3RBlDbT8C0IM+Atlh/BqIQ3zIxN4g0pzUlF0M33d6AYfYSzOcUhibOO7H +j84r+YXBNkIRgYKZYbutRXuZYaGuqejRpBj3voVu0d3Ntdb6lCWuClpB9HzfGN0c +RTv8g6UYO4sK3qyFn90ibIR/1GB9watvtoWVZqggiWeBzSWVWRsGEf9O+Cx4oJw1 +IphglhmhbgNksbj7bD24on/icldSOiVkoUemUOFmHWhCm4PnB1GmbD8YMfEdSbks +qDr1Ps1zg4mGOinVD/4cY7vuPFO/HCH07wfeaUGzRt4g0/yLr+XjVofOA3oowyxv +JAzr+niHA3lg5ecj4r7M68efwzN1OCyjMrVJw2RAzwvGxE+rm5NiT08SWlKQZnkC +gcEA4wvyLpIur/UB84nV3XVJ89UMNBLm++aTFzld047BLJtMaOhvNqx6Cl5c8VuW +l261KHjiVzpfNM3/A2LBQJcYkhX7avkqEXlj57cl+dCWAVwUzKmLJTPjfaTTZnYJ +xeN3dMYjJz2z2WtgvfvDoJLukVwIMmhTY8wtqqYyQBJ/l06pBsfw5TNvmVIOQHds +8ASOiFt+WRLk2bl9xrGGayqt3VV93KVRzF27cpjOgEcG74F3c0ZW9snERN7vIYwB +JfrlAoHBAMlahPwMP2TYylG8OzHe7EiehTekSO26LGh0Cq3wTGXYsK/q8hQCzL14 +kWW638vpwXL6L9ntvrd7hjzWRO3vX/VxnYEA6f0bpqHq1tZi6lzix5CTUN5McpDg +QnjenSJNrNjS1zEF8WeY9iLEuDI/M/iUW4y9R6s3WpgQhPDXpSvd2g3gMGRUYhxQ +Xna8auiJeYFq0oNaOxvJj+VeOfJ3ZMJttd+Y7gTOYZcbg3SdRb/kdxYki0RMD2hF +4ZvjJ6CTfwKBwQDiMqiZFTJGQwYqp4vWEmAW+I4r4xkUpWatoI2Fk5eI5T9+1PLX +uYXsho56NxEU1UrOg4Cb/p+TcBc8PErkGqR0BkpxDMOInTOXSrQe6lxIBoECVXc3 +HTbrmiay0a5y5GfCgxPKqIJhfcToAceoVjovv0y7S4yoxGZKuUEe7E8JY2iqRNAO +yOvKCCICv/hcN235E44RF+2/rDlOltagNej5tY6rIFkaDdgOF4bD7f9O5eEni1Bg +litfoesDtQP/3rECgcEAkQfvQ7D6tIPmbqsbJBfCr6fmoqZllT4FIJN84b50+OL0 +mTGsfjdqC4tdhx3sdu7/VPbaIqm5NmX10bowWgWSY7MbVME4yQPyqSwC5NbIonEC +d6N0mzoLR0kQ+Ai4u+2g82gicgAq2oj1uSNi3WZi48jQjHYFulCbo246o1NgeFFK +77WshYe2R1ioQfQDOU1URKCR0uTaMHClgfu112yiGd12JAD+aF3TM0kxDXz+sXI5 +SKy311DFxECZeXRLpcC3AoHBAJkNMJWTyPYbeVu+CTQkec8Uun233EkXa2kUNZc/ +5DuXDaK+A3DMgYRufTKSPpDHGaCZ1SYPInX1Uoe2dgVjWssRL2uitR4ENabDoAOA +ICVYXYYNagqQu5wwirF0QeaMXo1fjhuuHQh8GsMdXZvYEaAITZ9/NG5x/oY08+8H +kr78SMBOPy3XQn964uKG+e3JwpOG14GKABdAlrHKFXNWchu/6dgcYXB87mrC/GhO +zNwzC+QhFTZoOomFoqMgFWujng== -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIDWTCCAkGgAwIBAgIJAPm6B21bar2bMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV +MIIEWTCCAsGgAwIBAgIJAJinz4jHSjLtMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODAx -MTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH +IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODA4 +MjkxNDIzMTVaFw0yODA4MjYxNDIzMTVaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH DA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5k -YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEP -ADCCAQoCggEBAKvvsX2gEti4shve3iYMc+jE4Se7WHs1Bol2f21H8qNboDOFdeb1 -RKHjmq3exHpajywOUEgne9nKHJY/3f2phR4Y5klqG6liLgiSpVyRlcBGbeT2qEAj -9oLiLFUXLGfGDds2mTwivQDLJBWi51j7ff5k2Pr58fN5ugYMn24T9FNyn0moT+qj -SFoBNm58l9jrdkJSlgWfqPlbiMa+mqDn/SFtrwLF2Trbfzu42Sd9UdIzMaSSrzbN -sGm53pNhCh8KndWUQ8GPP2IsLPoUU4qAtmZuTxCx2S1cXrN9EkmT69tlOH84YfSn -96Ih9bWRc7M5y5bfVdEVM+fKQl3hBRf05qMCAwEAAaMYMBYwFAYDVR0RBA0wC4IJ -bG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQAtQ8f37cCEk7/rAcbYR53ce3iK -Vpihb0U2ni1QjG9Tg9UIExkIGkwTiCm7kwQL+GEStBu9AG/QVrOjeTriRiddhWkk -ze8kRaI3AC/63t6Vh9Q1x6PESgeE4OtAO9JpJCf4GILglA789Y/b/GF8zJZQxR13 -qpB4ZwWw7gCBhdEW59u6CFeBmfDa58hM8lWvuVoRrTi7bjUeC6PAn5HVMzZSykhu -4HaUfBp6bKFjuym2+h/VvM1n8C3chjVSmutsLb6ELdD8IK0vPV/yf5+LN256zSsS -dyUZYd8XwQaioEMKdbhLvnehyzHiWfQIUR3BdhONxoIJhHv/EAo8eCkHHYIF +YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGP +ADCCAYoCggGBALKUqUtopT6E68kN+uJNEt34i2EbmG/bwjcD8IaMsgJPSsMO2Bpd +3S6qWgkCeOyCfmAwBxK2kNbxGb63ouysEv7l8GCTJTWv3hG/HQcejJpnAEGi6K1U +fDbyE/db6yZ12SoHVTGkadN4vYGCPd1Wj9ZO1F877SHQ8rDWX3xgTWkxN2ojBw44 +T8RHSDiG8D/CvG4uEy+VUszL+Uvny5y2poNSqvI3J56sptWSrh8nIIbkPZPBdUne +LYMOHTFK3ZjXSmhlXgziTxK71nnzM3Y9K9gxPnRqoXbvu/wFo55hQCkETiRkYgmm +jXcBMZ0TClQVnQWuLjMthRnWFZs4Lfmwqjs7FZD/61581R2BYehvpWbLvvuOJhwv +DFzexL2sXcAl7SsxbzeQKRHqGbIDfbnQTXfs3/VC6Ye5P82P2ucj+XC32N9piRmO +gCBP8L3ub+YzzdxikZN2gZXXE2jsb3QyE/R2LkWdWyshpKe+RsZP1SBRbHShUyOh +yJ90baoiEwj2mwIDAQABoxgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZI +hvcNAQELBQADggGBAHRUO/UIHl3jXQENewYayHxkIx8t7nu40iO2DXbicSijz5bo +5//xAB6RxhBAlsDBehgQP1uoZg+WJW+nHu3CIVOU3qZNZRaozxiCl2UFKcNqLOmx +R3NKpo1jYf4REQIeG8Yw9+hSWLRbshNteP6bKUUf+vanhg9+axyOEOH/iOQvgk/m +b8wA8wNa4ujWljPbTQnj7ry8RqhTM0GcAN5LSdSvcKcpzLcs3aYwh+Z8e30sQWna +F40sa5u7izgBTOrwpcDm/w5kC46vpRQ5fnbshVw6pne2by0mdMECASid/p25N103 +jMqTFlmO7kpf/jpCSmamp3/JSEE1BJKHwQ6Ql4nzRA2N1mnvWH7Zxcv043gkHeAu +0x8evpvwuhdIyproejNFlBpKmW8OX7yKTCPPMC/VkX8Q1rVkxU0DQ6hmvwZlhoKa +9Wc2uXpw9xF8itV4Uvcdr3dwqByvIqn7iI/gB+4l41e0u8OmH2MKOx4Nxlly5TNW +HcVKQHyOeyvnINuBAQ== -----END CERTIFICATE----- diff --git a/Lib/test/keycert2.pem b/Lib/test/keycert2.pem index bb5fa65a8aca..ed6ae85a4649 100644 --- a/Lib/test/keycert2.pem +++ b/Lib/test/keycert2.pem @@ -1,49 +1,66 @@ -----BEGIN PRIVATE KEY----- -MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQC3ulRNfhbOAey/ -B+wIVYx+d5az7EV4riR6yi/qE6G+bxbTvay2pqySHtDweuaYSh2cVmcasBKKIFJm -rCD1zR8UmLb5i2XFIina1t3eePCuBZMrvZZwkzlQUSM1AZtjGOO/W0I3FwO6y645 -9xA5PduKI7SMYkH/VL3zE5W1JwMovv6bvNiT+GU5l6mB9ylCTgLpmUqoQhRqz/35 -zCzVyoh+ppDvVcpWYfvXywsXsgQwbAF0QJm8SSFi0TZm5ykv4WE16afQp08yuZS0 -3U4K3MJCa4rxO58edcxBopWYfQ29K3iINM8enRfr5q+u5mAAbALAEEvyFjgLWl/u -7arxn7bJAgMBAAECggEBAJfMt8KfHzBunrDnVrk8FayYGkfmOzAOkc1yKEx6k/TH -zFB+Mqlm5MaF95P5t3S0J+r36JBAUdEWC38RUNpF9BwMYYGlDxzlsTdCuGYL/q+J -o6NMLXQt7/jQUQqGnWAvPFzqhbcGqOo5R2ZVH25sEWv9PDuRI35XAepIkDTwWsfa -P6UcJJoP+4v9B++fb3sSL4zNwp1BqS4wxR8YTR0t1zQqOxJ5BGPw1J8aBMs1sq5t -qyosAQAT63kLrdqWotHaM26QxjqEQUMlh12XMWb5GdBXUxbvyGtEabsqskGa/f8B -RdHE437J8D8l+jxb2mZLzrlaH3dq2tbFGCe1rT8qLRECgYEA5CWIvoD/YnQydLGA -OlEhCSocqURuqcotg9Ev0nt/C60jkr/NHFLGppz9lhqjIDjixt3sIMGZMFzxRtwM -pSYal3XiR7rZuHau9iM35yDhpuytEiGbYy1ADakJRzY5jq/Qa8RfPP9Atua5xAeP -q6DiSnq9vhHv9G+O4MxzHBmrw9sCgYEAziiJWFthcwvuXn3Jv9xFYKEb/06puZAx -EgQCz/3rPzv5fmGD/sKVo1U/K4z/eA82DNeKG8QRTFJCxT8TCNRxOmGV7HdCYo/B -4BTNNvbKcdi3l0j75kKoADg+nt5CD5lz6gLG0GrUEnVO1y5HVfCTb3BEAfa36C85 -9i0sfQGiwysCgYEAuus9k8cgdct5oz3iLuVVSark/JGCkT2B+OOkaLChsDFUWeEm -7TOsaclpwldkmvvAYOplkZjMJ2GelE2pVo1XcAw3LkmaI5WpVyQXoxe/iQGT8qzy -IFlsh0Scw2lb0tmcyw6CcPk4TiHOxRrkzNrtS9QwLM+JZx0XVHptPPKTVc0CgYAu -j/VFYY5G/8Dc0qhIjyWUR48dQNUQtkJ/ASzpcT46z/7vznKTjbtiYpSb74KbyUO5 -7sygrM4DYOj3x+Eys1jHiNbly6HQxQtS4x/edCsRP5NntfI+9XsgYZOzKhvdjhki -F3J0DEzNxnUCIM+311hVaRPTJbgv1srOkTFlIoNydQKBgQC6/OHGaC/OewQqRlRK -Mg5KZm01/pk4iKrpA5nG7OTAeoa70NzXNtG8J3WnaJ4mWanNwNUOyRMAMrsUAy9q -EeGqHM5mMFpY4TeVuNLL21lu/x3KYw6mKL3Ctinn+JLAoYoqEy8deZnEA5/tjYlz -YhFBchnUicjoUN1chdpM6SpV2Q== +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQDKjrjWZlfOs1Ch +qt1RoyLfqyXbHVXIAW0fTzAxfJnxvFOiWqAAKgC2qVQM8Y080kRUuRaXP/w9ywXT ++MzX6tByy5VbTYJYyTjHOH46EWLNdcqEJs4+FCVqOIYrQPQ6pGAhCXmgBy4Vb42J +ABLwb+Kt+y2Dk15tggVcAHP2Khri+lRXWvda+kZAe2F1IojmuWyCTy3FEYHic5qN +BsXcf6u1oyFV8MybOuz1zGj3vd2C+dEKO4Ohw9rRwnvHSatjM+CfwiXf8kTXzDBF +Z/8W3+6yA49pHxRbG7FE3K1TAPhkrp+BVTIUOcdI74wEA6UEkWFF5sQcmmAth59M +EQrl2CXorftEPhsKZE59dUP1+nYAPvR/mTySNCSw7/rvdf+csRSZ5ollMu/lxsht +ulJYJI03+IiDTn47FI5D+IF25REK7d4LzGIo6T73ktsT+qpSXHuTWC+IABm8AMF9 +7ljxHSwMRU/z+O5uiONRItDAgKH/OItFG54PtY2vAhaO0YiZrZcCAwEAAQKCAYB2 +hTo8IVghlySH5B1p5kXCkDcvVaPaypLaLhCp9Blzq9lX9yUF043lU4Ddrf0RaIsY +88/3IjZqxb+cP0lE0Z20fdDfwqORZfQ2BaU+PuwMAm9EEhy9kDYwR/ChoHkHUyT4 +T7392BWr70Dmt8ddLmp5mK4R/gnTk6+lHJK9p/dhdk4haxWvAyBWHJty2Yk3T6nh +OYkzdUIFidUVza+6jG2hc1lPGv3tmnYKgNeulkblm10oWphz79C6ycx5WG7TNgef +CQ3z7//Nn89YTiaUBjLvoLvxRTMwO96r7E/FaslSl/fWnF3HP3lut26Z/mNfhiwj +qn7AhUwpSNPV0qcxFWXr/rXUjdk745wv8wOODK8atjjE/vt/MRBK0rAOIPSm3ecx +37PKNtR4i+sNeDEcY1IyTHE6wFvJSy5y8AFpn5y8tbqYfhlEVWZ4pcnlrKxhWm7j +oBkB/4GBjKQgbQ7ttym9eNG1wIbZ8v9N06+yeLs/NCc4bFZEgcWjFqBH1bLtAYEC +gcEA8tt8iYNqbsDH2ognjEmbbBxrDBmyYcEKRpg1i1SUopcZl8i93IHpG7EgJIaj +l7aWSbASAxjnK02t0VZ3nNS60acibzRwY/+e8OrSqlQdMXlAB2ggBA86drDJpfBl +WGJG8TJVY9bc1TU2uuwtZR1LAMSsRHVp+3IvKLpHrne5exPd3x6KGYcuaM+Uk/rE +u6tLsFNwaCdh+iBFFDT2bnYIw7jAsokJUkwxMVxSC0/21s2blhO/q5LsN1gFC1kN +TbpXAoHBANWE7TmG2szPvujPwrK18v6iJlHCA2n50AgAQXrsetj2JcF3HYHYdHnq +z36MQ6FpBKOiQumozWvb32WTjEwdG2kix7GEfam4DAUBdqYuCHzPcR12K5Tc8hsX +NG7JXUAeS8ZJEiOdu95X59JHyBxUQtNfte5rcbaV17SVw6K6bsWVJnj60YjtJrpa +xHvv1ZRnT2WEzJGpA+ii1h3I52N7ipGBiw172qcW+bKJukMi8eHxx5CC9e5tBpnu +C+Ou/eYewQKBwHxNa0jXQrq9YY2w8s0TP8HuKbxfyrXOIHxRm9ZczFcMD8VosgUT +WUUbO+B2KXWVtwawYAfFz0ySzcy//SkAmT6F1VIl/QCx7aBSENGti+Ous98WpIxv +XvUxN4T/rl+2raj2ok4fw5g9TG4QRIvkmmciQyonDr/sicbG0bmy/fTJDl8NOpIm +ZtKurNWxHNERtAPkMTyeK7/ilHjrQtb3AzVqcvbuvR6qcONa5YN0wlrfkisWoJwo +707EdpCAXBbUsQKBwQCnpzcpu2Sj+t9ZKIElF87T93gFLETH+ppJHgJMRdDz+NqO +fTwTD2XtsNz57aLQ44f8AFVv6NZbQYq41FEOFrDGLcQE9BZDpDrz10FVnMGXVr7n +tjjkK1SCxwapkr0AsoknCYsPojO4kud46loLPHI4TGeq7HyeNCvqJMo3RRHjXIiX +58GNNUD6hHjRI/FdFH14Jf0GxmJGUU20l2Jwb7nPJJuNm9mE53pqoNA7FP4+Pj1H +kD0Q2FSdmxeE0IuWHEECgcBgw6ogJ/FRRGLcym+aApqP9BChK+W8FDfDc9Mi4p/J +g+XmetWNFGCGTlOefGqUDIkwSG+QVOEN3hxziXbsjnvfpGApqoaulAI5oRvrwIcj +QIvD2mt0PB52k5ZL9QL2K9sgBa43BJDyCKooMAlTy2XMM+NyXVxQKmzf3r3jQ5sl +Rptk7ro38a9G8Rs99RFDyOmP1haOM0KXZvPksN4nsXuTlE01cnwnI29XKAlEZaoA +pQPLXD8W/KK4mwDbmokYXmo= -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- -MIIDYjCCAkqgAwIBAgIJALJXRr8qF6oIMA0GCSqGSIb3DQEBCwUAMGIxCzAJBgNV +MIIEYjCCAsqgAwIBAgIJAJm2YulYpr+6MA0GCSqGSIb3DQEBCwUAMGIxCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u IFNvZnR3YXJlIEZvdW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTAeFw0x -ODAxMTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMGIxCzAJBgNVBAYTAlhZMRcwFQYD +ODA4MjkxNDIzMTZaFw0yODA4MjYxNDIzMTZaMGIxCzAJBgNVBAYTAlhZMRcwFQYD VQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZv -dW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTCCASIwDQYJKoZIhvcNAQEB -BQADggEPADCCAQoCggEBALe6VE1+Fs4B7L8H7AhVjH53lrPsRXiuJHrKL+oTob5v -FtO9rLamrJIe0PB65phKHZxWZxqwEoogUmasIPXNHxSYtvmLZcUiKdrW3d548K4F -kyu9lnCTOVBRIzUBm2MY479bQjcXA7rLrjn3EDk924ojtIxiQf9UvfMTlbUnAyi+ -/pu82JP4ZTmXqYH3KUJOAumZSqhCFGrP/fnMLNXKiH6mkO9VylZh+9fLCxeyBDBs -AXRAmbxJIWLRNmbnKS/hYTXpp9CnTzK5lLTdTgrcwkJrivE7nx51zEGilZh9Db0r -eIg0zx6dF+vmr67mYABsAsAQS/IWOAtaX+7tqvGftskCAwEAAaMbMBkwFwYDVR0R -BBAwDoIMZmFrZWhvc3RuYW1lMA0GCSqGSIb3DQEBCwUAA4IBAQCZhHhGItpkqhEq -ntMRd6Hv0GoOJixNvgeMwK4NJSRT/no3OirtUTzccn46h+SWibSa2eVssAV+pAVJ -HbzkN/DH27A1mMx1zJL1ekcOKA1AF6MXhUnrUGXMqW36YNtzHfXJLrwvpLJ13OQg -/Kxo4Nw68bGzM+PyRtKU/mpgYyfcvwR+ZSeIDh1fvUZK/IEVCf8ub42GPVs5wPfv -M+k5aHxWTxeif3K1byTRzxHupYNG2yWO4XEdnBGOuOwzzN4/iQyNcsuQKeuKHGrt -YvIlG/ri04CQ7xISZCj74yjTZ+/A2bXre2mQXAHqKPumHL7cl34+erzbUaxYxbTE -u5FcOmLQ +dW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTCCAaIwDQYJKoZIhvcNAQEB +BQADggGPADCCAYoCggGBAMqOuNZmV86zUKGq3VGjIt+rJdsdVcgBbR9PMDF8mfG8 +U6JaoAAqALapVAzxjTzSRFS5Fpc//D3LBdP4zNfq0HLLlVtNgljJOMc4fjoRYs11 +yoQmzj4UJWo4hitA9DqkYCEJeaAHLhVvjYkAEvBv4q37LYOTXm2CBVwAc/YqGuL6 +VFda91r6RkB7YXUiiOa5bIJPLcURgeJzmo0Gxdx/q7WjIVXwzJs67PXMaPe93YL5 +0Qo7g6HD2tHCe8dJq2Mz4J/CJd/yRNfMMEVn/xbf7rIDj2kfFFsbsUTcrVMA+GSu +n4FVMhQ5x0jvjAQDpQSRYUXmxByaYC2Hn0wRCuXYJeit+0Q+GwpkTn11Q/X6dgA+ +9H+ZPJI0JLDv+u91/5yxFJnmiWUy7+XGyG26UlgkjTf4iINOfjsUjkP4gXblEQrt +3gvMYijpPveS2xP6qlJce5NYL4gAGbwAwX3uWPEdLAxFT/P47m6I41Ei0MCAof84 +i0Ubng+1ja8CFo7RiJmtlwIDAQABoxswGTAXBgNVHREEEDAOggxmYWtlaG9zdG5h +bWUwDQYJKoZIhvcNAQELBQADggGBAMIVLp6e6saH2NQSg8iFg8Ewg/K/etI++jHo +gCJ697AY02wtfrBox1XtljlmI2xpJtVAYZWHhrNqwrEG43aB7YEV6RqTcG6QUVqa +NbD8iNCnMKm7fP89hZizmqA1l4aHnieI3ucOqpgooM7FQwLX6qk+rSue6lD5N/5f +avsublnj8rNKyDfHpQ3AWduLoj8QqctpzI3CqoDZNLNzaDnzVWpxT1SKDQ88q7VI +W5zb+lndpdQlCu3v5HM4w5UpwL/k1htl/z6PnPseS2UdlXv6A8KITnCLg5PLP4tz +2oTAg9gjOtRP/0uwkhvicwoFzFJNVT813lzTLE1jlobMPiZhsS1mjaJGPD9GQZDK +ny3j8ogrIRGjnI4xpOMNNDVphcvwtV8fRbvURSHCj9Y4kCLpD5ODuoyEyLYicJIv +GZP456GP0iSCK5GKO0ij/YzGCkPWD5zA+mYFpMMGZPTwajenMw7TVaPXcc9CZBtr +oOjwwiLEqdkpxUj13mJYTlt5wsS/Kw== -----END CERTIFICATE----- diff --git a/Lib/test/keycert3.pem b/Lib/test/keycert3.pem index 621eb08bb0ce..e0a8205a660e 100644 --- a/Lib/test/keycert3.pem +++ b/Lib/test/keycert3.pem @@ -1,64 +1,84 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDgV4G+Zzf2DT5n -oAisIGFhn/bz7Vn5WiXUqbDsxROJOh/7BtOlduZka0pPhFylGbnxS8l1kEWHRI2T -6hOoWzumB6ItKiN+T5J30lAvSyo7iwdFoAQ/S5nPXQfhNARQe/NEOhRtpcuNdyx4 -BWdPdPuJQA1ASNJCLwcLOoRxaLbKLvb2V5T5FCAkeNPtRvPuT4gKQItMmiHfAhoV -C8MZWF/GC0RukHINys5MwqeFexam8CznmQPMYrLdhmKTj3DTivCPoh97EDIFGlgZ -SCaaYDVQA+aqlo/q2pi52PFwC1KzhNEA7EeOqSwC1NQjwjHuhcnf9WbxrgTq2zh3 -rv5YEW2ZAgMBAAECggEAPfSMtTumPcJskIumuXp7yk02EyliZrWZqwBuBwVqHsS5 -nkbFXnXWrLbgn9MrDsFrE5NdgKUmPnQVMVs8sIr5jyGejSCNCs4I4iRn1pfIgwcj -K/xEEALd6GGF0pDd/CgvB5GOoLVf4KKf2kmLvWrOKJpSzoUN5A8+v8AaYYOMr4sC -czbvfGomzEIewEG+Rw9zOVUDlmwyEKPQZ47E7PQ+EEA7oeFdR+1Zj6eT9ndegf8B -54frySYCLRUCk/sHCpWhaJBtBrcpht7Y8CfY7hiH/7x866fvuLnYPz4YALtUb0wN -7zUCNS9ol3n4LbjFFKfZtiRjKaCBRzMjK0rz6ydFcQKBgQDyLI3oGbnW73vqcDe/ -6eR0w++fiCAVhfMs3AO/gOaJi2la2JHlJ5u+cIHQIOFwEhn6Zq0AtdmnFx1TS5IQ -C0MdXI0XoQQw7rEF8EJcvfe85Z0QxENVhzydtdb8QpJfnQGfBfLyQlaaRYzRRHB6 -VdYUHF3EIPVIhbjbghal+Qep/QKBgQDtJlRPHkWwTMevu0J0fYbWN1ywtVTFUR// -k7VyORSf8yuuSnaQRop4cbcqONxmDKH6Or1fl3NYBsAxtXkkOK1E2OZNo2sfQdRa -wpA7o7mPHRhztQFpT5vflp+8P6+PEFat8D04eBOhNwrwwfhiPjD4gv5KvN4XutRW -VWv/2pnmzQKBgHPvHGg2mJ7quvm6ixXW1MWJX1eSBToIjCe3lBvDi5nhIaiZ8Q4w -7gA3QA3xD7tlDwauzLeAVxgEmsdbcCs6GQEfY3QiYy1Bt4FOSZa4YrcNfSmfq1Rw -j3Y4rRjKjeQz96i3YlzToT3tecJc7zPBj+DEy6au2H3Fdn+vQURneWHJAoGBANG7 -XES8mRVaUh/wlM1BVsaNH8SIGfiHzqzRjV7/bGYpQTBbWpAuUrhCmaMVtpXqBjav -TFwGLVRkZAWSYRjPpy2ERenT5SE3rv61o6mbGrifGsj6A82HQmtzYsGx8SmtYXtj -REF0sKebbmmOooUAS379GrguYJzL9o6D7YfRZNrhAoGAVfb/tiFU4S67DSpYpQey -ULhgfsFpDByICY6Potsg67gVFf9jIaB83NPTx3u/r6sHFgxFw7lQsuZcgSuWMu7t -glzOXVIP11Y5sl5CJ5OsfeK1/0umMZF5MWPyAQCx/qrPlZL86vXjt24Y/VaOxsAi -CZYdyJsjgOrJrWoMbo5ta54= +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQCfKC83Qe9/ZGMW +YhbpARRiKco6mJI9CNNeaf7A89TE+w5Y3GSwS8uzqp5C6QebZzPNueg8HYoTwN85 +Z3xM036/Qw9KhQVth+XDAqM+19e5KHkYcxg3d3ZI1HgY170eakaLBvMDN5ULoFOw +Is2PtwM2o9cjd5mfSuWttI6+fCqop8/l8cerG9iX2GH39p3iWwWoTZuYndAA9qYv +07YWajuQ1ESWKPjHYGTnMvu4xIzibC1mXd2M6u/IjNO6g426SKFaRDWQkx01gIV/ +CyKs9DgZoeMHkKZuPqZVOxOK+A/NrmrqHFsPIsrs5wk7QAVju5/X1skpn/UGQlmM +RwBaQULOs1FagA+54RXU6qUPW0YmhJ4xOB4gHHD1vjAKEsRZ7/6zcxMyOm+M1DbK +RTH4NWjVWpnY8XaVGdRhtTpH9MjycpKhF+D2Zdy2tQXtqu2GdcMnUedt13fn9xDu +P4PophE0ip/IMgn+kb4m9e+S+K9lldQl0B+4BcGWAqHelh2KuU0CAwEAAQKCAYEA +lKiWIYjmyRjdLKUGPTES9vWNvNmRjozV0RQ0LcoSbMMLDZkeO0UwyWqOVHUQ8+ib +jIcfEjeNJxI57oZopeHOO5vJhpNlFH+g7ltiW2qERqA1K88lSXm99Bzw6FNqhCRE +K8ub5N9fyfJA+P4o/xm0WK8EXk5yIUV17p/9zJJxzgKgv2jsVTi3QG2OZGvn4Oug +ByomMZEGHkBDzdxz8c/cP1Tlk1RFuwSgews178k2xq7AYSM/s0YmHi7b/RSvptX6 +1v8P8kXNUe4AwTaNyrlvF2lwIadZ8h1hA7tCE2n44b7a7KfhAkwcbr1T59ioYh6P +zxsyPT678uD51dbtD/DXJCcoeeFOb8uzkR2KNcrnQzZpCJnRq4Gp5ybxwsxxuzpr +gz0gbNlhuWtE7EoSzmIK9t+WTS7IM2CvZymd6/OAh1Fuw6AQhSp64XRp3OfMMAAC +Ie2EPtKj4islWGT8VoUjuRYGmdRh4duAH1dkiAXOWA3R7y5a1/y/iE8KE8BtxocB +AoHBAM8aiURgpu1Fs0Oqz6izec7KSLL3l8hmW+MKUOfk/Ybng6FrTFsL5YtzR+Ap +wW4wwWnnIKEc1JLiZ7g8agRETK8hr5PwFXUn/GSWC0SMsazLJToySQS5LOV0tLzK +kJ3jtNU7tnlDGNkCHTHSoVL2T/8t+IkZI/h5Z6wjlYPvU2Iu0nVIXtiG+alv4A6M +Hrh9l5or4mjB6rGnVXeYohLkCm6s/W97ahVxLMcEdbsBo1prm2JqGnSoiR/tEFC/ +QHQnbQKBwQDEu7kW0Yg9sZ89QtYtVQ1YpixFZORaUeRIRLnpEs1w7L1mCbOZ2Lj9 +JHxsH05cYAc7HJfPwwxv3+3aGAIC/dfu4VSwEFtatAzUpzlhzKS5+HQCWB4JUNNU +MQ3+FwK2xQX4Ph8t+OzrFiYcK2g0An5UxWMa2HWIAWUOhnTOydAVsoH6yP31cVm4 +0hxoABCwflaNLNGjRUyfBpLTAcNu/YtcE+KREy7YAAgXXrhRSO4XpLsSXwLnLT7/ +YOkoBWDcTWECgcBPWnSUDZCIQ3efithMZJBciqd2Y2X19Dpq8O31HImD4jtOY0V7 +cUB/wSkeHAGwjd/eCyA2e0x8B2IEdqmMfvr+86JJxekC3dJYXCFvH5WIhsH53YCa +3bT1KlWCLP9ib/g+58VQC0R/Cc9T4sfLePNH7D5ZkZd1wlbV30CPr+i8KwKay6MD +xhvtLx+jk07GE+E9wmjbCMo7TclyrLoVEOlqZMAqshgApT+p9eyCPetwXuDHwa3n +WxhHclcZCV7R4rUCgcAkdGSnxcvpIrDPOUNWwxvmAWTStw9ZbTNP8OxCNCm9cyDl +d4bAS1h8D/a+Uk7C70hnu7Sl2w7C7Eu2zhwRUdhhe3+l4GINPK/j99i6NqGPlGpq +xMlMEJ4YS768BqeKFpg0l85PRoEgTsphDeoROSUPsEPdBZ9BxIBlYKTkbKESZDGR +twzYHljx1n1NCDYPflmrb1KpXn4EOcObNghw2KqqNUUWfOeBPwBA1FxzM4BrAStp +DBINpGS4Dc0mjViVegECgcA3hTtm82XdxQXj9LQmb/E3lKx/7H87XIOeNMmvjYuZ +iS9wKrkF+u42vyoDxcKMCnxP5056wpdST4p56r+SBwVTHcc3lGBSGcMTIfwRXrj3 +thOA2our2n4ouNIsYyTlcsQSzifwmpRmVMRPxl9fYVdEWUgB83FgHT0D9avvZnF9 +t9OccnGJXShAIZIBADhVj/JwG4FbaX42NijD5PNpVLk1Y17OV0I576T9SfaQoBjJ +aH1M/zC4aVaS0DYB/Gxq7v8= -----END PRIVATE KEY----- Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9c - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5c + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Nov 28 19:09:06 2027 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Jul 7 14:23:16 2028 GMT Subject: C=XY, L=Castle Anthrax, O=Python Software Foundation, CN=localhost Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:e0:57:81:be:67:37:f6:0d:3e:67:a0:08:ac:20: - 61:61:9f:f6:f3:ed:59:f9:5a:25:d4:a9:b0:ec:c5: - 13:89:3a:1f:fb:06:d3:a5:76:e6:64:6b:4a:4f:84: - 5c:a5:19:b9:f1:4b:c9:75:90:45:87:44:8d:93:ea: - 13:a8:5b:3b:a6:07:a2:2d:2a:23:7e:4f:92:77:d2: - 50:2f:4b:2a:3b:8b:07:45:a0:04:3f:4b:99:cf:5d: - 07:e1:34:04:50:7b:f3:44:3a:14:6d:a5:cb:8d:77: - 2c:78:05:67:4f:74:fb:89:40:0d:40:48:d2:42:2f: - 07:0b:3a:84:71:68:b6:ca:2e:f6:f6:57:94:f9:14: - 20:24:78:d3:ed:46:f3:ee:4f:88:0a:40:8b:4c:9a: - 21:df:02:1a:15:0b:c3:19:58:5f:c6:0b:44:6e:90: - 72:0d:ca:ce:4c:c2:a7:85:7b:16:a6:f0:2c:e7:99: - 03:cc:62:b2:dd:86:62:93:8f:70:d3:8a:f0:8f:a2: - 1f:7b:10:32:05:1a:58:19:48:26:9a:60:35:50:03: - e6:aa:96:8f:ea:da:98:b9:d8:f1:70:0b:52:b3:84: - d1:00:ec:47:8e:a9:2c:02:d4:d4:23:c2:31:ee:85: - c9:df:f5:66:f1:ae:04:ea:db:38:77:ae:fe:58:11: - 6d:99 + 00:9f:28:2f:37:41:ef:7f:64:63:16:62:16:e9:01: + 14:62:29:ca:3a:98:92:3d:08:d3:5e:69:fe:c0:f3: + d4:c4:fb:0e:58:dc:64:b0:4b:cb:b3:aa:9e:42:e9: + 07:9b:67:33:cd:b9:e8:3c:1d:8a:13:c0:df:39:67: + 7c:4c:d3:7e:bf:43:0f:4a:85:05:6d:87:e5:c3:02: + a3:3e:d7:d7:b9:28:79:18:73:18:37:77:76:48:d4: + 78:18:d7:bd:1e:6a:46:8b:06:f3:03:37:95:0b:a0: + 53:b0:22:cd:8f:b7:03:36:a3:d7:23:77:99:9f:4a: + e5:ad:b4:8e:be:7c:2a:a8:a7:cf:e5:f1:c7:ab:1b: + d8:97:d8:61:f7:f6:9d:e2:5b:05:a8:4d:9b:98:9d: + d0:00:f6:a6:2f:d3:b6:16:6a:3b:90:d4:44:96:28: + f8:c7:60:64:e7:32:fb:b8:c4:8c:e2:6c:2d:66:5d: + dd:8c:ea:ef:c8:8c:d3:ba:83:8d:ba:48:a1:5a:44: + 35:90:93:1d:35:80:85:7f:0b:22:ac:f4:38:19:a1: + e3:07:90:a6:6e:3e:a6:55:3b:13:8a:f8:0f:cd:ae: + 6a:ea:1c:5b:0f:22:ca:ec:e7:09:3b:40:05:63:bb: + 9f:d7:d6:c9:29:9f:f5:06:42:59:8c:47:00:5a:41: + 42:ce:b3:51:5a:80:0f:b9:e1:15:d4:ea:a5:0f:5b: + 46:26:84:9e:31:38:1e:20:1c:70:f5:be:30:0a:12: + c4:59:ef:fe:b3:73:13:32:3a:6f:8c:d4:36:ca:45: + 31:f8:35:68:d5:5a:99:d8:f1:76:95:19:d4:61:b5: + 3a:47:f4:c8:f2:72:92:a1:17:e0:f6:65:dc:b6:b5: + 05:ed:aa:ed:86:75:c3:27:51:e7:6d:d7:77:e7:f7: + 10:ee:3f:83:e8:a6:11:34:8a:9f:c8:32:09:fe:91: + be:26:f5:ef:92:f8:af:65:95:d4:25:d0:1f:b8:05: + c1:96:02:a1:de:96:1d:8a:b9:4d Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Alternative Name: @@ -70,11 +90,11 @@ Certificate: X509v3 Basic Constraints: critical CA:FALSE X509v3 Subject Key Identifier: - 85:11:BE:16:47:04:D1:30:EE:86:8A:18:70:BE:A8:28:6F:82:3D:CE + 8F:EA:1D:E3:33:5C:00:16:B3:8B:6F:6B:6F:D3:4C:CB:B5:CB:7C:55 X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 DirName:/C=XY/O=Python Software Foundation CA/CN=our-ca-server - serial:82:ED:BF:41:C8:80:91:9B + serial:CB:2D:80:99:5A:69:52:5B Authority Information Access: CA Issuers - URI:http://testca.pythontest.net/testca/pycacert.cer @@ -85,48 +105,60 @@ Certificate: Full Name: URI:http://testca.pythontest.net/testca/revocation.crl - Signature Algorithm: sha1WithRSAEncryption - 7f:a1:7e:3e:68:01:b0:32:b8:57:b8:03:68:13:13:b3:e3:f4: - 70:2f:15:e5:0f:87:b9:fd:e0:12:e3:16:f2:91:53:c7:4e:25: - af:ca:cb:a7:d9:9d:57:4d:bf:a2:80:d4:78:aa:04:31:fd:6d: - cc:6d:82:43:e9:62:16:0d:0e:26:8b:e7:f1:3d:57:5c:68:02: - 9c:2b:b6:c9:fd:62:2f:10:85:88:cc:44:a5:e7:a2:3e:89:f2: - 1f:02:6a:3f:d0:3c:6c:24:2d:bc:51:62:7a:ec:25:c5:86:87: - 77:35:8f:f9:7e:d0:17:3d:77:56:bf:1a:0c:be:09:78:ee:ea: - 73:97:65:60:94:91:35:b3:5c:46:8a:5e:6d:94:52:de:48:b7: - 1f:6c:28:79:7f:ff:08:8d:e4:7d:d0:b9:0b:7c:ae:c4:1d:2a: - a1:b3:50:11:82:03:5e:6c:e7:26:fa:05:32:39:07:83:49:b9: - a2:fa:04:da:0d:e5:ff:4c:db:97:d0:c3:a7:43:37:4c:16:de: - 3c:b5:e9:7e:82:d4:b3:10:df:d1:c1:66:72:9c:15:67:19:3b: - 7b:91:0a:82:07:67:c5:06:03:5f:80:54:08:81:8a:b1:5c:7c: - 4c:d2:07:38:92:eb:12:f5:71:ae:de:05:15:c8:e1:33:f0:e4: - 96:0f:0f:1e + Signature Algorithm: sha256WithRSAEncryption + 27:f5:8c:59:10:f4:c6:e7:28:00:bf:ba:8d:7b:13:03:f1:1c: + a6:5f:b3:06:55:a4:22:b9:db:b2:d5:46:bd:f7:0c:dd:43:6e: + b4:79:65:67:21:0c:2a:55:ee:40:8e:85:9f:9f:47:bb:0a:2a: + 4d:b6:64:74:98:a0:7f:ae:dc:f1:2e:db:42:77:18:e0:75:8b: + 26:35:68:c3:41:ed:6b:c8:77:72:6f:6a:9a:5d:55:69:02:fd: + 5a:54:c8:57:cb:b0:65:03:16:e2:0f:00:39:99:66:a0:9b:88: + 93:17:e2:5a:2d:79:35:5f:97:57:78:c4:af:f5:99:5e:86:ab: + d3:11:ad:1a:a2:0d:fa:52:10:b9:fe:bf:9d:ce:33:d9:86:b2: + 9c:16:f8:d6:75:08:8a:db:0a:e5:b4:2b:16:7f:b4:f9:2a:9f: + c3:d2:77:d7:cd:65:1e:f4:6c:1e:eb:59:b9:f0:ae:5f:a4:1f: + cc:4a:c4:b9:7a:a9:d9:6b:32:68:3b:e1:65:b0:84:b7:90:c4: + ae:fe:f4:37:4f:21:a0:de:9f:3a:b1:e5:cc:16:04:66:3f:0b: + 41:dc:42:3d:20:3e:ec:b7:95:2b:35:57:fa:be:7f:b6:3a:ba: + ca:4f:58:fe:75:3e:08:89:2c:8c:b0:5d:2e:f9:89:10:2b:f9: + 41:46:4f:3c:00:b7:27:d3:65:24:28:17:23:26:31:42:ea:7e: + 4e:93:e4:7b:68:54:ca:9f:46:f3:ef:2b:e9:85:0c:b5:84:b2: + d5:35:34:80:75:2b:f0:91:23:b8:08:01:8e:b9:0a:54:d4:fb: + 34:52:fe:d9:45:f0:80:3b:b6:c1:6f:82:d1:1f:f2:3b:08:f6: + 46:a6:96:27:61:4b:58:32:7a:0e:1d:59:c5:44:ad:5e:1a:79: + 33:c1:d4:05:2f:4a:d3:d8:42:42:8d:33:e3:63:ca:d5:87:97: + 9b:4d:b8:1a:03:34:bb:1c:d2:02:3f:59:23:e2:23:80:88:63: + c2:f0:a2:63:a8:8b -----BEGIN CERTIFICATE----- -MIIE8TCCA9mgAwIBAgIJAILtv0HIgJGcMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIF8TCCBFmgAwIBAgIJAMstgJlaaVJcMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yNzExMjgx -OTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA3MDcx +NDIzMTZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj MCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxv -Y2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOBXgb5nN/YN -PmegCKwgYWGf9vPtWflaJdSpsOzFE4k6H/sG06V25mRrSk+EXKUZufFLyXWQRYdE -jZPqE6hbO6YHoi0qI35PknfSUC9LKjuLB0WgBD9Lmc9dB+E0BFB780Q6FG2ly413 -LHgFZ090+4lADUBI0kIvBws6hHFotsou9vZXlPkUICR40+1G8+5PiApAi0yaId8C -GhULwxlYX8YLRG6Qcg3KzkzCp4V7FqbwLOeZA8xist2GYpOPcNOK8I+iH3sQMgUa -WBlIJppgNVAD5qqWj+ramLnY8XALUrOE0QDsR46pLALU1CPCMe6Fyd/1ZvGuBOrb -OHeu/lgRbZkCAwEAAaOCAcAwggG8MBQGA1UdEQQNMAuCCWxvY2FsaG9zdDAOBgNV -HQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1Ud -EwEB/wQCMAAwHQYDVR0OBBYEFIURvhZHBNEw7oaKGHC+qChvgj3OMH0GA1UdIwR2 -MHSAFJrPz27rcT3bPPGuiGtWcgPLCKdIoVGkTzBNMQswCQYDVQQGEwJYWTEmMCQG -A1UECgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0ExFjAUBgNVBAMMDW91 -ci1jYS1zZXJ2ZXKCCQCC7b9ByICRmzCBgwYIKwYBBQUHAQEEdzB1MDwGCCsGAQUF -BzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3RjYS9weWNhY2Vy -dC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQv -dGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly90ZXN0Y2EucHl0 -aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3JsMA0GCSqGSIb3DQEBBQUA -A4IBAQB/oX4+aAGwMrhXuANoExOz4/RwLxXlD4e5/eAS4xbykVPHTiWvysun2Z1X -Tb+igNR4qgQx/W3MbYJD6WIWDQ4mi+fxPVdcaAKcK7bJ/WIvEIWIzESl56I+ifIf -Amo/0DxsJC28UWJ67CXFhod3NY/5ftAXPXdWvxoMvgl47upzl2VglJE1s1xGil5t -lFLeSLcfbCh5f/8IjeR90LkLfK7EHSqhs1ARggNebOcm+gUyOQeDSbmi+gTaDeX/ -TNuX0MOnQzdMFt48tel+gtSzEN/RwWZynBVnGTt7kQqCB2fFBgNfgFQIgYqxXHxM -0gc4kusS9XGu3gUVyOEz8OSWDw8e +Y2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAJ8oLzdB739k +YxZiFukBFGIpyjqYkj0I015p/sDz1MT7DljcZLBLy7OqnkLpB5tnM8256DwdihPA +3zlnfEzTfr9DD0qFBW2H5cMCoz7X17koeRhzGDd3dkjUeBjXvR5qRosG8wM3lQug +U7AizY+3Azaj1yN3mZ9K5a20jr58Kqinz+Xxx6sb2JfYYff2neJbBahNm5id0AD2 +pi/TthZqO5DURJYo+MdgZOcy+7jEjOJsLWZd3Yzq78iM07qDjbpIoVpENZCTHTWA +hX8LIqz0OBmh4weQpm4+plU7E4r4D82uauocWw8iyuznCTtABWO7n9fWySmf9QZC +WYxHAFpBQs6zUVqAD7nhFdTqpQ9bRiaEnjE4HiAccPW+MAoSxFnv/rNzEzI6b4zU +NspFMfg1aNVamdjxdpUZ1GG1Okf0yPJykqEX4PZl3La1Be2q7YZ1wydR523Xd+f3 +EO4/g+imETSKn8gyCf6Rvib175L4r2WV1CXQH7gFwZYCod6WHYq5TQIDAQABo4IB +wDCCAbwwFAYDVR0RBA0wC4IJbG9jYWxob3N0MA4GA1UdDwEB/wQEAwIFoDAdBgNV +HSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4E +FgQUj+od4zNcABazi29rb9NMy7XLfFUwfQYDVR0jBHYwdIAU3b/K2ubRNLo3dSHK +b5oIKPI1tkihUaRPME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29m +dHdhcmUgRm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcoIJAMst +gJlaaVJbMIGDBggrBgEFBQcBAQR3MHUwPAYIKwYBBQUHMAKGMGh0dHA6Ly90ZXN0 +Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3B5Y2FjZXJ0LmNlcjA1BggrBgEFBQcw +AYYpaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0Y2Evb2NzcC8wQwYD +VR0fBDwwOjA4oDagNIYyaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0 +Y2EvcmV2b2NhdGlvbi5jcmwwDQYJKoZIhvcNAQELBQADggGBACf1jFkQ9MbnKAC/ +uo17EwPxHKZfswZVpCK527LVRr33DN1DbrR5ZWchDCpV7kCOhZ+fR7sKKk22ZHSY +oH+u3PEu20J3GOB1iyY1aMNB7WvId3JvappdVWkC/VpUyFfLsGUDFuIPADmZZqCb +iJMX4loteTVfl1d4xK/1mV6Gq9MRrRqiDfpSELn+v53OM9mGspwW+NZ1CIrbCuW0 +KxZ/tPkqn8PSd9fNZR70bB7rWbnwrl+kH8xKxLl6qdlrMmg74WWwhLeQxK7+9DdP +IaDenzqx5cwWBGY/C0HcQj0gPuy3lSs1V/q+f7Y6uspPWP51PgiJLIywXS75iRAr ++UFGTzwAtyfTZSQoFyMmMULqfk6T5HtoVMqfRvPvK+mFDLWEstU1NIB1K/CRI7gI +AY65ClTU+zRS/tlF8IA7tsFvgtEf8jsI9kamlidhS1gyeg4dWcVErV4aeTPB1AUv +StPYQkKNM+NjytWHl5tNuBoDNLsc0gI/WSPiI4CIY8LwomOoiw== -----END CERTIFICATE----- diff --git a/Lib/test/keycert4.pem b/Lib/test/keycert4.pem index b7df7f3f2c71..d1ebb82486de 100644 --- a/Lib/test/keycert4.pem +++ b/Lib/test/keycert4.pem @@ -1,64 +1,84 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDH/76hZAZH4cSV -CmVZa5HEqKCjCKrcPwBECs9BS+3ibwN4x9NnFNP+tCeFGgJXl7WGFoeXgg3oK+1p -FsOWpsRHuF3BdqkCnShSydmT8bLaGHwKeL0cPxJP5T/uW7ezPKW2VWXGMwmwRaRJ -9dj2VCUu20vDZWSGFr9zjnjoJczBtH3RsVUgpK7euEHuQ5pIM9QSOaCo+5FPR7s7 -1nU7YqbFWtd+NhC8Og1G497B31DQlHciF6BRm6/cNGAmHaAErKUGBFdkGtFPHBn4 -vktoEg9fwxJAZLvGpoTZWrB4HRsRwVTmFdGvK+JXK225xF23AXRXp/snhSuSFeLj -E5cpyJJ7AgMBAAECggEAQOv527X2e/sDr0XSpHZQuT/r9UBpBlnFIlFH+fBF5k0X -GWv0ae/O6U1dzs0kmX57xG0n0ry6+vTXeleTYiH8cTOd66EzN9AAOO+hG29IGZf9 -HAEZkkO/FARc/mjzdtFnEYsjIHWM3ZWdwQx3Q28JKu6w51rQiN51g3NqOCGdF/uF -rE5XPKsKndn+nLHvsNuApFgUYZEwdrozgUueEgRaPTUCNhzotcA9eWoBdA24XNhk -x8Cm/bZWabXm7gBO75zl3Cu2F21ay+EuwyOZTsx6lZi6YX9/zo1mkO81Zi3tQk50 -NMEI0feLNwsdxTbmOcVJadjOgd+QVghlFyr5HGBWMQKBgQD3AH3rhnAo6tOyNkGN -+IzIU1MhUS452O7IavykUYO9sM24BVChpRtlI9Dpev4yE/q3BAO3+oWT3cJrN7/3 -iyo1dzAkpGvI65XWfElXFM4nLjEiZzx4W9fiPN91Oucpr0ED6+BZXTtz4gVm0TP/ -TUc2xvTB6EKvIyWmKOYEi0snxQKBgQDPSOjbz9jWOrC9XY7PmtLB6QJDDz7XSGVK -wzD+gDAPpAwhk58BEokdOhBx2Lwl8zMJi0CRHgH2vNvkRyhvUQ4UFzisrqann/Tw -klp5sw3iWC6ERC8z9zL7GfHs7sK3mOVeAdK6ffowPM3JrZ2vPusVBdr0MN3oZwki -CtNXqbY1PwKBgGheQNbAW6wubX0kB9chavtKmhm937Z5v4vYCSC1gOEqUAKt3EAx -L74wwBmn6rjmUE382EVpCgBM99WuHONQXmlxD1qsTw763LlgkuzE0cckcYaD8L06 -saHa7uDuHrcyYlpx1L5t8q0ol/e19i6uTKUMtGcq6OJwC3yGU4sgAIWxAoGBAMVq -qiQXm2vFL+jafxYoXUvDMJ1PmskMsTP4HOR2j8+FrOwZnVk3HxGP6HOVOPRn4JbZ -YiAT1Uj6a+7I+rCyINdvmlGUcTK6fFzW9oZryvBkjcD483/pkktmVWwTpa2YV/Ml -h16IdsyUTGYlDUYHhXtbPUJOfDpIT4F1j/0wrFGfAoGAO82BcUsehEUQE0xvQLIn -7QaFtUI5z19WW730jVuEobiYlh9Ka4DPbKMvka8MwyOxEwhk39gZQavmfG6+wZm+ -kjERU23LhHziJGWS2Um4yIhC7myKbWaLzjHEq72dszLpQku4BzE5fT60fxI7cURD -WGm/Z3Q2weS3ZGIoMj1RNPI= +MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQDGjpiHzq7ghxhM +ZzrnRsGBC/cmw8EREIdbqlrz/l8BFaWeipvO5Hb/MyU8xs2zLUrqIr2JNf+Eii8Y +m4bYmZclFra4jomaiSlxTZOe3dMV8m4vAq4eT2mSfZZC1+XAutqdz7WhHxhMVEm3 +AyTWvTC3qCbnlbX5VIoQUwFrsSWqDiHyaGdK3rrOTKFUKM8YPiq/BZkm6A4eiFci +5wd/SPD+w0pIscZbQW1MUr5bs54uylWaUmtfI8KJt6BDZQ/uA06c6i863sSCEI6L +gq+wyikeJGNMxZMfgu3dzfv4BiZBQX0ZhiRvqseDSdPcuVa2Ifb6CFlg298neweY +4EAIE1O+uqo5h8FF1aUOMZpQEZuzsp9R/TAMBHX1YmVjG/kRdBeaHe3whzB1Pfue +PIX2ZTMmLNYbYbfnmxhk1nn8aAvoT98pNw8y3/2k2KNsu24n9uSkkxAoqJ19WKwm +mL8MpJKAzLv45tRvhN+QLtnRdu+LJ9m29npQHFmYLbdqRfmidnMCAwEAAQKCAYBd +w1C8MRnb5W/QBJ+IP515NxFLOP2e9VM2MkgpGGH8vSAssf/Jv5GCCcD35lmU1zqd +PjKK7PjwueBrmmYfOshpN0Sp+oV4eHUdkCi5yL65inYFtRpMLewIxU2D2zgfvx0l +kMSQhYKP6O22gsGOtmCfGcTlb4kzaHyaINh25nyGxY26TxsX+/3zFbTJbUv+grzk +39vmx4aDXJbpYHfl36gOZmJZ2bl1tnvKovhJjZSRO/MYoPsbPmPLbO89ZCgVmXFc +GVkb5Cram6i3iyutSDjxWN7Fb8uy8pFLPGAXZgF7pQoXPSEHZe8GEWBnWSC9KaDa +uM9Ir847/Muy1ceCmxKcI2WrSjoH2AhPcmHgvbPE9Mynr6+uzReSP3q7Wh9PHm23 +oFx3DwdCfmjysnpAMBawNmJdWyxVDbZ6eyrhp17ADpsMaDTynZ+fJjgMr+MmWtbU +YSRD0wWtl/DrzsaePZsOjCpKYJyulC+rh9/Zz1aiwrGWPbgEAzDrD6Q1Zp0mUCEC +gcEA+XskmGIB9rRPy+YQmRgzQ555PsjLWsnQsNktP6KBhlQjFKJZXRZ0DxDTS7h8 +NrJrUDBmwfsgzggVbeO55VP5FGwD6DNeO/Bz++Fdevh8uKQFHDfk4sbIUPS91qw4 +s7OW7PR7C7Jf7Dnjmsn42o2lO4FsbcEn2F+PHOvoLl/OrSx73lS/RkdOEItW8d8/ +ExRohylnba/I2vCE9bNZd4DGjMW87j/THKPadDZWEqWggcrjY8x6ibSQGm2n2Tka +8B+vAoHBAMu+zl8kqFlYDG24dGfVpMaOYj5Osj0cV5f7O2pZ15sCevjiqoKGHH7G +O8EiI5pRBZ893+Fsx6YWmcKue88lfGvaoQjV0LUbfMfX/FoD39w/ZLx31yKEiFuc +KvMiRV5mO3kQiHBVX9vamzr5NeaErccXY9LnhaKOMX9blgiDQZH7fc3RhodcFWrC +9yfX6ryfidpPnRvK7Ops7hVnFKyyS4FaAarnzH1B2WcVcD4lYYxhMc8VXeU3eKOh +j1fI/F5ifQKBwQDpCjB670HqUzAexL9IYqSwSz3yedoK6m24ZIWx5XicI8fJJIXZ +QHoVAKB/IMtWxH8dnri+Bnj0O/TYe1pQb4pBm0xjAGjMEKYm6LNLhQXr67qiS0vQ +0eKYTKVv+9vTcLRQj2bI3Exh+wkys+tzK9DmrtS8CSvRICIs3+g4OWJzvRPP8NXj +LgQrzBzhPqpKhkvFxdVJTmSOrxFj+a5exLmzEZqT6qanIB+VYpQwQuqVkxGpTX5B +V5ssNLYPYRpapx0CgcByCtQixzcAA1u5knR9pkT76ris3YnA0Ptqk3I3XiBjoGjK +pL0CICUVBMpvmTdKai12a8DDwgqiOaZJJTchxH63NAHNGzkeFkuq5IdYrzB/bHBr +WbzukjZs6KXVv4oKg7ioVAu6rN7iBaO7x8BWzk8i0EHMzFCto1+rRM1e6HEsUBOj +v7LIU0+dmZGUGLRIbhhQPR3Yb6ZatSwyiKc23vmKZqHmUqbQOaqBm6te7beDRugF +XJVY9sqs9IJyhYpVHlUCgcAPoslwYKeAXagsxdQrH3D9VJDXVOHWKMBqQZDio5dB +Q80uWpuxtt6nhZkQO1JIWnYb6v+zbDbcgjByBIDuxCdBW9d+QQnanKmVyrXguK91 +C3OcHHOmSduFdWC3/zYW1mW97Tz1sXyam2hly1u3L5kW+GnE1hr9VVPjQNrO9+Ge +qW0coaJqKY78q3Rm2dtyZeJSFFI1o/DQ3blyItsFpg/QrR+a5XrS6Nw2ZLIL4Azo +J1CTgMwjhwlMNCI4t4dkHd0= -----END PRIVATE KEY----- Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9d - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5d + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Nov 28 19:09:06 2027 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Jul 7 14:23:16 2028 GMT Subject: C=XY, L=Castle Anthrax, O=Python Software Foundation, CN=fakehostname Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:c7:ff:be:a1:64:06:47:e1:c4:95:0a:65:59:6b: - 91:c4:a8:a0:a3:08:aa:dc:3f:00:44:0a:cf:41:4b: - ed:e2:6f:03:78:c7:d3:67:14:d3:fe:b4:27:85:1a: - 02:57:97:b5:86:16:87:97:82:0d:e8:2b:ed:69:16: - c3:96:a6:c4:47:b8:5d:c1:76:a9:02:9d:28:52:c9: - d9:93:f1:b2:da:18:7c:0a:78:bd:1c:3f:12:4f:e5: - 3f:ee:5b:b7:b3:3c:a5:b6:55:65:c6:33:09:b0:45: - a4:49:f5:d8:f6:54:25:2e:db:4b:c3:65:64:86:16: - bf:73:8e:78:e8:25:cc:c1:b4:7d:d1:b1:55:20:a4: - ae:de:b8:41:ee:43:9a:48:33:d4:12:39:a0:a8:fb: - 91:4f:47:bb:3b:d6:75:3b:62:a6:c5:5a:d7:7e:36: - 10:bc:3a:0d:46:e3:de:c1:df:50:d0:94:77:22:17: - a0:51:9b:af:dc:34:60:26:1d:a0:04:ac:a5:06:04: - 57:64:1a:d1:4f:1c:19:f8:be:4b:68:12:0f:5f:c3: - 12:40:64:bb:c6:a6:84:d9:5a:b0:78:1d:1b:11:c1: - 54:e6:15:d1:af:2b:e2:57:2b:6d:b9:c4:5d:b7:01: - 74:57:a7:fb:27:85:2b:92:15:e2:e3:13:97:29:c8: - 92:7b + 00:c6:8e:98:87:ce:ae:e0:87:18:4c:67:3a:e7:46: + c1:81:0b:f7:26:c3:c1:11:10:87:5b:aa:5a:f3:fe: + 5f:01:15:a5:9e:8a:9b:ce:e4:76:ff:33:25:3c:c6: + cd:b3:2d:4a:ea:22:bd:89:35:ff:84:8a:2f:18:9b: + 86:d8:99:97:25:16:b6:b8:8e:89:9a:89:29:71:4d: + 93:9e:dd:d3:15:f2:6e:2f:02:ae:1e:4f:69:92:7d: + 96:42:d7:e5:c0:ba:da:9d:cf:b5:a1:1f:18:4c:54: + 49:b7:03:24:d6:bd:30:b7:a8:26:e7:95:b5:f9:54: + 8a:10:53:01:6b:b1:25:aa:0e:21:f2:68:67:4a:de: + ba:ce:4c:a1:54:28:cf:18:3e:2a:bf:05:99:26:e8: + 0e:1e:88:57:22:e7:07:7f:48:f0:fe:c3:4a:48:b1: + c6:5b:41:6d:4c:52:be:5b:b3:9e:2e:ca:55:9a:52: + 6b:5f:23:c2:89:b7:a0:43:65:0f:ee:03:4e:9c:ea: + 2f:3a:de:c4:82:10:8e:8b:82:af:b0:ca:29:1e:24: + 63:4c:c5:93:1f:82:ed:dd:cd:fb:f8:06:26:41:41: + 7d:19:86:24:6f:aa:c7:83:49:d3:dc:b9:56:b6:21: + f6:fa:08:59:60:db:df:27:7b:07:98:e0:40:08:13: + 53:be:ba:aa:39:87:c1:45:d5:a5:0e:31:9a:50:11: + 9b:b3:b2:9f:51:fd:30:0c:04:75:f5:62:65:63:1b: + f9:11:74:17:9a:1d:ed:f0:87:30:75:3d:fb:9e:3c: + 85:f6:65:33:26:2c:d6:1b:61:b7:e7:9b:18:64:d6: + 79:fc:68:0b:e8:4f:df:29:37:0f:32:df:fd:a4:d8: + a3:6c:bb:6e:27:f6:e4:a4:93:10:28:a8:9d:7d:58: + ac:26:98:bf:0c:a4:92:80:cc:bb:f8:e6:d4:6f:84: + df:90:2e:d9:d1:76:ef:8b:27:d9:b6:f6:7a:50:1c: + 59:98:2d:b7:6a:45:f9:a2:76:73 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Alternative Name: @@ -70,11 +90,11 @@ Certificate: X509v3 Basic Constraints: critical CA:FALSE X509v3 Subject Key Identifier: - F8:76:79:CB:11:85:F0:46:E5:95:E6:7E:69:CB:12:5E:4E:AA:EC:4D + 52:E0:93:AA:52:55:B7:BB:E7:A8:E0:8C:DE:41:2E:F4:07:F0:36:FB X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 DirName:/C=XY/O=Python Software Foundation CA/CN=our-ca-server - serial:82:ED:BF:41:C8:80:91:9B + serial:CB:2D:80:99:5A:69:52:5B Authority Information Access: CA Issuers - URI:http://testca.pythontest.net/testca/pycacert.cer @@ -85,48 +105,60 @@ Certificate: Full Name: URI:http://testca.pythontest.net/testca/revocation.crl - Signature Algorithm: sha1WithRSAEncryption - 6d:50:8d:fb:ee:4e:93:8b:eb:47:56:ba:38:cc:80:e1:9d:c7: - e1:9e:1f:9c:22:0c:d2:08:9b:ed:bf:31:d9:00:ee:af:8c:56: - 78:92:d1:7c:ba:4e:81:7f:82:1f:f4:68:99:86:91:c6:cb:57: - d3:b9:41:12:fa:75:53:fd:22:32:21:50:af:6b:4c:b1:34:36: - d1:a8:25:0a:d0:f0:f8:81:7d:69:58:6e:af:e3:d2:c4:32:87: - 79:d7:cd:ad:0c:56:f3:15:27:10:0c:f9:57:59:53:00:ed:af: - 5d:4d:07:86:7a:e5:f3:97:88:bc:86:b4:f1:17:46:33:55:28: - 66:7b:70:d3:a5:12:b9:4f:c7:ed:e6:13:20:2d:f0:9e:ec:17: - 64:cf:fd:13:14:1b:76:ba:64:ac:c5:51:b6:cd:13:0a:93:b1: - fd:43:09:a0:0b:44:6c:77:45:43:0b:e5:ed:70:b2:76:dc:08: - 4a:5b:73:5f:c1:fc:7f:63:70:f8:b9:ca:3c:98:06:5f:fd:98: - d1:e4:e6:61:5f:09:8f:6c:18:86:98:9c:cb:3f:73:7b:3f:38: - f5:a7:09:20:ee:a5:63:1c:ff:8b:a6:d1:8c:e8:f4:84:3d:99: - 38:0f:cc:e0:52:03:f9:18:05:23:76:39:de:52:ce:8e:fb:a6: - 6e:f5:4f:c3 + Signature Algorithm: sha256WithRSAEncryption + 29:d2:3f:82:3f:c1:38:35:a6:bd:81:10:fe:64:ec:ff:7e:e1: + c6:6f:7f:86:65:f9:31:6f:fb:ef:32:4e:2f:87:c8:42:de:6c: + 8d:b8:06:08:8f:37:70:95:7d:e1:40:d4:82:2b:8d:b3:4a:fd: + 34:c5:9e:df:ff:01:53:4a:4f:08:f4:58:e1:74:fc:78:e3:3e: + 71:a7:5e:66:07:ea:d2:04:31:e2:75:a8:4c:80:17:86:92:20: + d2:32:a7:9a:65:8b:1a:5f:f1:4c:c8:50:6d:00:fc:99:bf:69: + b3:48:f3:45:5a:ee:35:50:14:b8:f3:92:92:c6:9f:0e:5d:eb: + 0d:e8:ec:f2:a4:09:6b:dc:66:2b:fc:df:4c:fc:65:a1:ae:d3: + b5:88:6a:a4:e7:08:1c:94:49:e0:b8:c1:04:8c:21:09:6c:55: + 4b:2c:97:10:f1:8c:6c:d0:bb:ba:8d:93:e8:47:8b:4d:8e:7d: + 7d:85:53:18:c8:f8:f4:8f:67:3a:b1:aa:3e:18:34:6c:3a:e6: + a6:c7:2f:be:83:38:f5:d5:e5:d2:17:28:61:6c:b6:49:99:21: + 69:a4:a8:b6:94:76:fd:18:ad:35:52:45:64:fb:f1:5d:8e:bb: + c0:21:2e:59:55:24:af:bb:8f:b2:0a:7b:17:f0:34:97:8e:68: + a9:f2:d0:3e:f6:73:82:f8:7c:4e:9a:70:7d:d6:b3:8c:cc:85: + 04:5c:02:8f:74:da:88:3a:20:a8:7e:c2:9e:b0:dd:56:1f:ce: + cd:42:16:c6:14:91:ad:30:e0:dc:76:f2:2c:56:ea:38:45:d8: + c0:3e:b8:90:fa:f3:38:99:ec:44:07:35:8f:69:62:0c:f9:ef: + b7:9d:e5:15:42:6e:fb:fe:4c:ff:e8:94:5a:1a:b0:80:b2:0e: + 17:3d:e1:87:a8:08:84:93:74:68:8d:29:df:ca:0b:6a:44:32: + 8a:51:3b:d6:38:db:bd:e3:2a:1b:5e:20:48:81:82:19:91:c6: + 87:8c:0f:cd:51:ea -----BEGIN CERTIFICATE----- -MIIE9zCCA9+gAwIBAgIJAILtv0HIgJGdMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIF9zCCBF+gAwIBAgIJAMstgJlaaVJdMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yNzExMjgx -OTA5MDZaMGIxCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA3MDcx +NDIzMTZaMGIxCzAJBgNVBAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEj MCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24xFTATBgNVBAMMDGZh -a2Vob3N0bmFtZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMf/vqFk -BkfhxJUKZVlrkcSooKMIqtw/AEQKz0FL7eJvA3jH02cU0/60J4UaAleXtYYWh5eC -Degr7WkWw5amxEe4XcF2qQKdKFLJ2ZPxstoYfAp4vRw/Ek/lP+5bt7M8pbZVZcYz -CbBFpEn12PZUJS7bS8NlZIYWv3OOeOglzMG0fdGxVSCkrt64Qe5Dmkgz1BI5oKj7 -kU9HuzvWdTtipsVa1342ELw6DUbj3sHfUNCUdyIXoFGbr9w0YCYdoASspQYEV2Qa -0U8cGfi+S2gSD1/DEkBku8amhNlasHgdGxHBVOYV0a8r4lcrbbnEXbcBdFen+yeF -K5IV4uMTlynIknsCAwEAAaOCAcMwggG/MBcGA1UdEQQQMA6CDGZha2Vob3N0bmFt -ZTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC -MAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFPh2ecsRhfBG5ZXmfmnLEl5OquxNMH0G -A1UdIwR2MHSAFJrPz27rcT3bPPGuiGtWcgPLCKdIoVGkTzBNMQswCQYDVQQGEwJY -WTEmMCQGA1UECgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0ExFjAUBgNV -BAMMDW91ci1jYS1zZXJ2ZXKCCQCC7b9ByICRmzCBgwYIKwYBBQUHAQEEdzB1MDwG -CCsGAQUFBzAChjBodHRwOi8vdGVzdGNhLnB5dGhvbnRlc3QubmV0L3Rlc3RjYS9w -eWNhY2VydC5jZXIwNQYIKwYBBQUHMAGGKWh0dHA6Ly90ZXN0Y2EucHl0aG9udGVz -dC5uZXQvdGVzdGNhL29jc3AvMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly90ZXN0 -Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3Jldm9jYXRpb24uY3JsMA0GCSqGSIb3 -DQEBBQUAA4IBAQBtUI377k6Ti+tHVro4zIDhncfhnh+cIgzSCJvtvzHZAO6vjFZ4 -ktF8uk6Bf4If9GiZhpHGy1fTuUES+nVT/SIyIVCva0yxNDbRqCUK0PD4gX1pWG6v -49LEMod5182tDFbzFScQDPlXWVMA7a9dTQeGeuXzl4i8hrTxF0YzVShme3DTpRK5 -T8ft5hMgLfCe7Bdkz/0TFBt2umSsxVG2zRMKk7H9QwmgC0Rsd0VDC+XtcLJ23AhK -W3Nfwfx/Y3D4uco8mAZf/ZjR5OZhXwmPbBiGmJzLP3N7Pzj1pwkg7qVjHP+LptGM -6PSEPZk4D8zgUgP5GAUjdjneUs6O+6Zu9U/D +a2Vob3N0bmFtZTCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAMaOmIfO +ruCHGExnOudGwYEL9ybDwREQh1uqWvP+XwEVpZ6Km87kdv8zJTzGzbMtSuoivYk1 +/4SKLxibhtiZlyUWtriOiZqJKXFNk57d0xXybi8Crh5PaZJ9lkLX5cC62p3PtaEf +GExUSbcDJNa9MLeoJueVtflUihBTAWuxJaoOIfJoZ0reus5MoVQozxg+Kr8FmSbo +Dh6IVyLnB39I8P7DSkixxltBbUxSvluzni7KVZpSa18jwom3oENlD+4DTpzqLzre +xIIQjouCr7DKKR4kY0zFkx+C7d3N+/gGJkFBfRmGJG+qx4NJ09y5VrYh9voIWWDb +3yd7B5jgQAgTU766qjmHwUXVpQ4xmlARm7Oyn1H9MAwEdfViZWMb+RF0F5od7fCH +MHU9+548hfZlMyYs1htht+ebGGTWefxoC+hP3yk3DzLf/aTYo2y7bif25KSTECio +nX1YrCaYvwykkoDMu/jm1G+E35Au2dF274sn2bb2elAcWZgtt2pF+aJ2cwIDAQAB +o4IBwzCCAb8wFwYDVR0RBBAwDoIMZmFrZWhvc3RuYW1lMA4GA1UdDwEB/wQEAwIF +oDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAd +BgNVHQ4EFgQUUuCTqlJVt7vnqOCM3kEu9AfwNvswfQYDVR0jBHYwdIAU3b/K2ubR +NLo3dSHKb5oIKPI1tkihUaRPME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZl +coIJAMstgJlaaVJbMIGDBggrBgEFBQcBAQR3MHUwPAYIKwYBBQUHMAKGMGh0dHA6 +Ly90ZXN0Y2EucHl0aG9udGVzdC5uZXQvdGVzdGNhL3B5Y2FjZXJ0LmNlcjA1Bggr +BgEFBQcwAYYpaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5ldC90ZXN0Y2Evb2Nz +cC8wQwYDVR0fBDwwOjA4oDagNIYyaHR0cDovL3Rlc3RjYS5weXRob250ZXN0Lm5l +dC90ZXN0Y2EvcmV2b2NhdGlvbi5jcmwwDQYJKoZIhvcNAQELBQADggGBACnSP4I/ +wTg1pr2BEP5k7P9+4cZvf4Zl+TFv++8yTi+HyELebI24BgiPN3CVfeFA1IIrjbNK +/TTFnt//AVNKTwj0WOF0/HjjPnGnXmYH6tIEMeJ1qEyAF4aSINIyp5plixpf8UzI +UG0A/Jm/abNI80Va7jVQFLjzkpLGnw5d6w3o7PKkCWvcZiv830z8ZaGu07WIaqTn +CByUSeC4wQSMIQlsVUsslxDxjGzQu7qNk+hHi02OfX2FUxjI+PSPZzqxqj4YNGw6 +5qbHL76DOPXV5dIXKGFstkmZIWmkqLaUdv0YrTVSRWT78V2Ou8AhLllVJK+7j7IK +exfwNJeOaKny0D72c4L4fE6acH3Ws4zMhQRcAo902og6IKh+wp6w3VYfzs1CFsYU +ka0w4Nx28ixW6jhF2MA+uJD68ziZ7EQHNY9pYgz577ed5RVCbvv+TP/olFoasICy +Dhc94YeoCISTdGiNKd/KC2pEMopRO9Y4273jKhteIEiBghmRxoeMD81R6g== -----END CERTIFICATE----- diff --git a/Lib/test/make_ssl_certs.py b/Lib/test/make_ssl_certs.py index a1f298de3497..ca2c12cada17 100644 --- a/Lib/test/make_ssl_certs.py +++ b/Lib/test/make_ssl_certs.py @@ -50,7 +50,7 @@ dir = cadir database = $dir/index.txt crlnumber = $dir/crl.txt - default_md = sha1 + default_md = sha256 default_days = 3600 default_crl_days = 3600 certificate = pycacert.pem @@ -88,7 +88,9 @@ here = os.path.abspath(os.path.dirname(__file__)) -def make_cert_key(hostname, sign=False, extra_san=''): + +def make_cert_key(hostname, sign=False, extra_san='', + ext='req_x509_extensions_full', key='rsa:3072'): print("creating cert for " + hostname) tempnames = [] for i in range(3): @@ -146,7 +148,7 @@ def make_ca(): t.flush() with tempfile.NamedTemporaryFile() as f: args = ['req', '-new', '-days', '3650', '-extensions', 'v3_ca', '-nodes', - '-newkey', 'rsa:2048', '-keyout', 'pycakey.pem', + '-newkey', 'rsa:3072', '-keyout', 'pycakey.pem', '-out', f.name, '-subj', '/C=XY/L=Castle Anthrax/O=Python Software Foundation CA/CN=our-ca-server'] check_call(['openssl'] + args) diff --git a/Lib/test/pycacert.pem b/Lib/test/pycacert.pem index 850fa32aef7e..73150c960f35 100644 --- a/Lib/test/pycacert.pem +++ b/Lib/test/pycacert.pem @@ -2,78 +2,98 @@ Certificate: Data: Version: 3 (0x2) Serial Number: - 82:ed:bf:41:c8:80:91:9b - Signature Algorithm: sha1WithRSAEncryption + cb:2d:80:99:5a:69:52:5b + Signature Algorithm: sha256WithRSAEncryption Issuer: C=XY, O=Python Software Foundation CA, CN=our-ca-server Validity - Not Before: Jan 19 19:09:06 2018 GMT - Not After : Jan 17 19:09:06 2028 GMT + Not Before: Aug 29 14:23:16 2018 GMT + Not After : Aug 26 14:23:16 2028 GMT Subject: C=XY, O=Python Software Foundation CA, CN=our-ca-server Subject Public Key Info: Public Key Algorithm: rsaEncryption - Public-Key: (2048 bit) + Public-Key: (3072 bit) Modulus: - 00:c3:18:69:6b:c9:47:29:98:8e:b1:56:c2:2e:fa: - 0e:5e:bc:23:80:b3:07:62:24:d2:42:5b:f1:4a:bf: - a9:c8:21:75:c8:e3:e6:2c:1f:87:3c:6e:7c:1b:ed: - 39:32:95:b7:40:b2:60:48:c3:9a:16:08:fe:6d:67: - 88:34:3b:77:77:70:1c:70:5a:d1:1f:5f:04:21:54: - b9:0c:e3:41:85:1d:58:ee:2f:ed:f3:0e:ef:d8:23: - a1:fa:73:fb:4c:28:e0:e5:e6:4d:0b:02:52:49:86: - c7:be:7e:bd:e6:56:76:8b:70:8e:0a:8f:06:33:20: - 1d:7b:5b:aa:d0:c5:1b:ab:9b:cc:54:09:3c:bf:e4: - 40:66:f1:fb:d6:f7:16:9d:c4:19:d4:c3:f2:ff:07: - bc:6f:5a:9e:25:1b:02:4a:a5:ec:42:96:3a:70:d2: - 6c:99:2b:ce:be:e8:d2:01:ef:d5:ba:b0:cf:94:3e: - 82:d0:01:d6:4b:71:80:03:0a:12:45:86:79:81:d8: - 4b:d2:e8:b5:b7:2c:6c:9a:4c:8a:10:10:e4:e4:f5: - df:ce:84:91:ca:d1:46:e0:84:73:17:66:db:69:43: - 78:80:83:be:14:4d:f1:3e:1a:d6:6c:f5:de:45:f3: - 39:af:91:d5:3d:54:44:bf:41:cc:73:68:1a:fc:24: - db:91 + 00:97:ed:55:41:ba:36:17:95:db:71:1c:d3:e1:61: + ac:58:73:e3:c6:96:cf:2b:1f:b8:08:f5:9d:4b:4b: + c7:30:f6:b8:0b:b3:52:72:a0:bb:c9:4d:3b:8e:df: + 22:8e:01:57:81:c9:92:73:cc:00:c6:ec:70:b0:3a: + 17:40:c1:df:f2:8c:36:4c:c4:a7:81:e7:b6:24:68: + e2:a0:7e:35:07:2f:a0:5b:f9:45:46:f7:1e:f0:46: + 11:fe:ca:1a:3c:50:f1:26:a9:5f:9c:22:9c:f8:41: + e1:df:4f:12:95:19:2f:5c:90:01:17:6e:7e:3e:7d: + cf:e9:09:af:25:f8:f8:42:77:2d:6d:5f:36:f2:78: + 1e:7d:4a:87:68:63:6c:06:71:1b:8d:fa:25:fe:d4: + d3:f5:a5:17:b1:ef:ea:17:cb:54:c8:27:99:80:cb: + 3c:45:f1:2c:52:1c:dd:1f:51:45:20:50:1e:5e:ab: + 57:73:1b:41:78:96:de:84:a4:7a:dd:8f:30:85:36: + 58:79:76:a0:d2:61:c8:1b:a9:94:99:63:c6:ee:f8: + 14:bf:b4:52:56:31:97:fa:eb:ac:53:9e:95:ce:4c: + c4:5a:4a:b7:ca:03:27:5b:35:57:ce:02:dc:ec:ca: + 69:f8:8a:5a:39:cb:16:20:15:03:24:61:6c:f4:7a: + fc:b6:48:e5:59:10:5c:49:d0:23:9f:fb:71:5e:3a: + e9:68:9f:34:72:80:27:b6:3f:4c:b1:d9:db:63:7f: + 67:68:4a:6e:11:f8:e8:c0:f4:5a:16:39:53:0b:68: + de:77:fa:45:e7:f8:91:cd:78:cd:28:94:97:71:54: + fb:cf:f0:37:de:c9:26:c5:dc:1b:9e:89:6d:09:ac: + c8:44:71:cb:6d:f1:97:31:d5:4c:20:33:bf:75:4a: + a0:e0:dc:69:11:ed:2a:b4:64:10:11:30:8b:0e:b0: + a7:10:d8:8a:c5:aa:1b:c8:26:8a:25:e7:66:9f:a5: + 6a:1a:2f:7c:5f:83:c6:78:4f:1f Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Key Identifier: - 9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 X509v3 Authority Key Identifier: - keyid:9A:CF:CF:6E:EB:71:3D:DB:3C:F1:AE:88:6B:56:72:03:CB:08:A7:48 + keyid:DD:BF:CA:DA:E6:D1:34:BA:37:75:21:CA:6F:9A:08:28:F2:35:B6:48 X509v3 Basic Constraints: CA:TRUE - Signature Algorithm: sha1WithRSAEncryption - 10:25:c8:dc:0c:55:5c:cb:83:6e:79:ef:77:ec:0d:8e:0c:06: - c1:4b:0c:d6:f7:75:52:21:b8:17:4a:38:88:9d:b3:78:c4:42: - fb:b8:7c:14:38:10:fb:ac:da:11:00:5b:42:87:5e:45:9f:6d: - 4e:42:a4:9a:18:06:39:0f:45:a6:96:89:32:d6:59:b3:d3:8e: - e3:95:b6:c4:a2:4b:74:2f:67:c1:fb:bb:f9:72:6f:37:4a:e7: - f4:48:33:71:df:b8:f5:e6:41:3f:d5:d5:2f:26:09:f8:0e:92: - ff:70:ea:f6:ab:58:fb:90:04:d6:43:2e:8f:b1:fb:06:ab:69: - d0:dc:a8:f8:5b:07:f2:d4:66:1f:63:f8:5d:c1:9e:41:44:bb: - c9:e8:7d:e0:46:e4:a7:c8:32:5f:31:62:e5:1c:5c:89:dd:b7: - a2:4f:9e:0d:13:b8:5f:b1:84:53:4c:1f:ce:19:e1:01:00:5e: - bf:41:55:94:a9:a5:13:db:f2:59:f3:d6:4e:b9:9d:9d:b9:0a: - d9:b2:18:6d:7c:b1:f7:96:aa:bd:f6:f9:95:0f:4a:6e:3c:7c: - 46:5b:df:d4:78:ec:9a:dc:e2:e3:01:e6:88:77:39:93:9c:ba: - 2a:63:f9:25:4b:4f:ac:08:79:39:c6:7b:df:07:35:ba:c0:c2: - 50:bf:5a:81 + Signature Algorithm: sha256WithRSAEncryption + 33:6a:54:d3:6b:c0:d7:01:5f:9d:f4:05:c1:93:66:90:50:d0: + b7:18:e9:b0:1e:4a:a0:b6:da:76:93:af:84:db:ad:15:54:31: + 15:13:e4:de:7e:4e:0c:d5:09:1c:34:35:b6:e5:4c:d6:6f:65: + 7d:32:5f:eb:fc:a9:6b:07:f7:49:82:e5:81:7e:07:80:9a:63: + f8:2c:c3:40:bc:8f:d4:2a:da:3e:d1:ee:08:b7:4d:a7:84:ca: + f4:3f:a1:98:45:be:b1:05:69:e7:df:d7:99:ab:1b:ee:8b:30: + cc:f7:fc:e7:d4:0b:17:ae:97:bf:e4:7b:fd:0f:a7:b4:85:79: + e3:59:e2:16:87:bf:1f:29:45:2c:23:93:76:be:c0:87:1d:de: + ec:2b:42:6a:e5:bb:c8:f4:0a:4a:08:0a:8c:5c:d8:7d:4d:d1: + b8:bf:d5:f7:29:ed:92:d1:94:04:e8:35:06:57:7f:2c:23:97: + 87:a5:35:8d:26:d3:1a:47:f2:16:d7:d9:c6:d4:1f:23:43:d3: + 26:99:39:ca:20:f4:71:23:6f:0c:4a:76:76:f7:76:1f:b3:fe: + bf:47:b0:fc:2a:56:81:e1:d2:dd:ee:08:d8:f4:ff:5a:dc:25: + 61:8a:91:02:b9:86:1c:f2:50:73:76:25:35:fc:b6:25:26:15: + cb:eb:c4:2b:61:0c:1c:e7:ee:2f:17:9b:ec:f0:d4:a1:84:e7: + d2:af:de:e4:1b:24:14:a7:01:87:e3:ab:29:58:46:a0:d9:c0: + 0a:e0:8d:d7:59:d3:1b:f8:54:20:3e:78:a5:a5:c8:4f:8b:03: + c4:96:9f:ec:fb:47:cf:76:2d:8d:65:34:27:bf:fa:ae:01:05: + 8a:f3:92:0a:dd:89:6c:97:a1:c7:e7:60:51:e7:ac:eb:4b:7d: + 2c:b8:65:c9:fe:5d:6a:48:55:8e:e4:c7:f9:6a:40:e1:b8:64: + 45:e9:b5:59:29:a5:5f:cf:7d:58:7d:64:79:e5:a4:09:ac:1e: + 76:65:3d:94:c4:68 -----BEGIN CERTIFICATE----- -MIIDbTCCAlWgAwIBAgIJAILtv0HIgJGbMA0GCSqGSIb3DQEBBQUAME0xCzAJBgNV +MIIEbTCCAtWgAwIBAgIJAMstgJlaaVJbMA0GCSqGSIb3DQEBCwUAME0xCzAJBgNV BAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUgRm91bmRhdGlvbiBDQTEW -MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODAxMTkxOTA5MDZaFw0yODAxMTcx -OTA5MDZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg -Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCASIwDQYJKoZI -hvcNAQEBBQADggEPADCCAQoCggEBAMMYaWvJRymYjrFWwi76Dl68I4CzB2Ik0kJb -8Uq/qcghdcjj5iwfhzxufBvtOTKVt0CyYEjDmhYI/m1niDQ7d3dwHHBa0R9fBCFU -uQzjQYUdWO4v7fMO79gjofpz+0wo4OXmTQsCUkmGx75+veZWdotwjgqPBjMgHXtb -qtDFG6ubzFQJPL/kQGbx+9b3Fp3EGdTD8v8HvG9aniUbAkql7EKWOnDSbJkrzr7o -0gHv1bqwz5Q+gtAB1ktxgAMKEkWGeYHYS9LotbcsbJpMihAQ5OT1386EkcrRRuCE -cxdm22lDeICDvhRN8T4a1mz13kXzOa+R1T1URL9BzHNoGvwk25ECAwEAAaNQME4w -HQYDVR0OBBYEFJrPz27rcT3bPPGuiGtWcgPLCKdIMB8GA1UdIwQYMBaAFJrPz27r -cT3bPPGuiGtWcgPLCKdIMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEB -ABAlyNwMVVzLg25573fsDY4MBsFLDNb3dVIhuBdKOIids3jEQvu4fBQ4EPus2hEA -W0KHXkWfbU5CpJoYBjkPRaaWiTLWWbPTjuOVtsSiS3QvZ8H7u/lybzdK5/RIM3Hf -uPXmQT/V1S8mCfgOkv9w6varWPuQBNZDLo+x+waradDcqPhbB/LUZh9j+F3BnkFE -u8nofeBG5KfIMl8xYuUcXIndt6JPng0TuF+xhFNMH84Z4QEAXr9BVZSppRPb8lnz -1k65nZ25CtmyGG18sfeWqr32+ZUPSm48fEZb39R47Jrc4uMB5oh3OZOcuipj+SVL -T6wIeTnGe98HNbrAwlC/WoE= +MBQGA1UEAwwNb3VyLWNhLXNlcnZlcjAeFw0xODA4MjkxNDIzMTZaFw0yODA4MjYx +NDIzMTZaME0xCzAJBgNVBAYTAlhZMSYwJAYDVQQKDB1QeXRob24gU29mdHdhcmUg +Rm91bmRhdGlvbiBDQTEWMBQGA1UEAwwNb3VyLWNhLXNlcnZlcjCCAaIwDQYJKoZI +hvcNAQEBBQADggGPADCCAYoCggGBAJftVUG6NheV23Ec0+FhrFhz48aWzysfuAj1 +nUtLxzD2uAuzUnKgu8lNO47fIo4BV4HJknPMAMbscLA6F0DB3/KMNkzEp4HntiRo +4qB+NQcvoFv5RUb3HvBGEf7KGjxQ8SapX5winPhB4d9PEpUZL1yQARdufj59z+kJ +ryX4+EJ3LW1fNvJ4Hn1Kh2hjbAZxG436Jf7U0/WlF7Hv6hfLVMgnmYDLPEXxLFIc +3R9RRSBQHl6rV3MbQXiW3oSket2PMIU2WHl2oNJhyBuplJljxu74FL+0UlYxl/rr +rFOelc5MxFpKt8oDJ1s1V84C3OzKafiKWjnLFiAVAyRhbPR6/LZI5VkQXEnQI5/7 +cV466WifNHKAJ7Y/TLHZ22N/Z2hKbhH46MD0WhY5Uwto3nf6Ref4kc14zSiUl3FU ++8/wN97JJsXcG56JbQmsyERxy23xlzHVTCAzv3VKoODcaRHtKrRkEBEwiw6wpxDY +isWqG8gmiiXnZp+lahovfF+DxnhPHwIDAQABo1AwTjAdBgNVHQ4EFgQU3b/K2ubR +NLo3dSHKb5oIKPI1tkgwHwYDVR0jBBgwFoAU3b/K2ubRNLo3dSHKb5oIKPI1tkgw +DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAYEAM2pU02vA1wFfnfQFwZNm +kFDQtxjpsB5KoLbadpOvhNutFVQxFRPk3n5ODNUJHDQ1tuVM1m9lfTJf6/ypawf3 +SYLlgX4HgJpj+CzDQLyP1CraPtHuCLdNp4TK9D+hmEW+sQVp59/Xmasb7oswzPf8 +59QLF66Xv+R7/Q+ntIV541niFoe/HylFLCOTdr7Ahx3e7CtCauW7yPQKSggKjFzY +fU3RuL/V9yntktGUBOg1Bld/LCOXh6U1jSbTGkfyFtfZxtQfI0PTJpk5yiD0cSNv +DEp2dvd2H7P+v0ew/CpWgeHS3e4I2PT/WtwlYYqRArmGHPJQc3YlNfy2JSYVy+vE +K2EMHOfuLxeb7PDUoYTn0q/e5BskFKcBh+OrKVhGoNnACuCN11nTG/hUID54paXI +T4sDxJaf7PtHz3YtjWU0J7/6rgEFivOSCt2JbJehx+dgUees60t9LLhlyf5dakhV +juTH+WpA4bhkRem1WSmlX899WH1keeWkCawedmU9lMRo -----END CERTIFICATE----- diff --git a/Lib/test/revocation.crl b/Lib/test/revocation.crl index 53cb4b3721d9..c05461ca7f93 100644 --- a/Lib/test/revocation.crl +++ b/Lib/test/revocation.crl @@ -1,11 +1,14 @@ -----BEGIN X509 CRL----- -MIIBpjCBjwIBATANBgkqhkiG9w0BAQUFADBNMQswCQYDVQQGEwJYWTEmMCQGA1UE +MIICJjCBjwIBATANBgkqhkiG9w0BAQsFADBNMQswCQYDVQQGEwJYWTEmMCQGA1UE CgwdUHl0aG9uIFNvZnR3YXJlIEZvdW5kYXRpb24gQ0ExFjAUBgNVBAMMDW91ci1j -YS1zZXJ2ZXIXDTE4MDExOTE5MDkwNloXDTI3MTEyODE5MDkwNlqgDjAMMAoGA1Ud -FAQDAgEAMA0GCSqGSIb3DQEBBQUAA4IBAQBYVzH8n2LdyJJ/t8CpWjz652hZJ0sY -DeNYcwuTPzR9CbSwEwNbf0kY9bgWXGfoRD2SnnCnuNNJXO2MuXtxf6qYx2ZjhJm8 -qgxXs0Bz4agRswNMbumjHCmqIv1t88vbrO0+ItEZDK7RJVIMBtVJ0XYOHvD/IG/i -zqa1Fl3uCTvQbTJ2TrqzJeP/Vl40hOD+VdBBZK3j0r4AkCKU3tAiHYTGmHKhPxy1 -f8Yet+4SRMGp1BdDezTI1bICpSZhRJ4geW0UzuCZnXPW8IZzioUmdUBAmAMHPWFr -B0sTTc/ntD4jHG1/T5b0oiDMbXIbh5Iz9iQNcY0IbotkCw39h+K90wY6 +YS1zZXJ2ZXIXDTE4MDgyOTE0MjMxNloXDTI4MDcwNzE0MjMxNlqgDjAMMAoGA1Ud +FAQDAgEAMA0GCSqGSIb3DQEBCwUAA4IBgQCPhrtGSbuvxPAI3YWQFDB4iOWdBnVk +ugW1lsifmCsE86FfID0EwUut1SRHlksltMtcoULMEIdu8yMLWci++4ve22EEuMKT +HUc3T/wBIuQUhA7U4deFG8CZPAxRpNoK470y7dkD4OVf0Gxa6WYDl9z8mXKmWCB9 +hvzqVfLWNSLTAVPsHtkD5PXdi5yRkQr6wYD7poWaIvkpsn7EKCY6Tw5V3rsbRuZq +AGVCq5TH3mctcmwLloCJ4Xr/1q0DsRrYxeeLYxE+UpvvCbVBKgtjBK7zINS7AbcJ +CYCYKUwGWv1fYKJ+KQQHf75mT3jQ9lWuzOj/YWK4k1EBnYmVGuKKt73lLFxC6h3y +MUnaBZc1KZSyJj0IxfHg/o6qx8NgKOl9XRIQ5g5B30cwpPOskGhEhodbTTY3bPtm +RQ36JvQZngzmkhyhr+MDEV5yUTOShfUiclzQOx26CmLmLHWxOZgXtFZob/oKrvbm +Gen/+7K7YTw6hfY52U7J2FuQRGOyzBXfBYQ= -----END X509 CRL----- diff --git a/Lib/test/ssl_cert.pem b/Lib/test/ssl_cert.pem index b1dd3f387f7c..de596717bd85 100644 --- a/Lib/test/ssl_cert.pem +++ b/Lib/test/ssl_cert.pem @@ -1,20 +1,26 @@ -----BEGIN CERTIFICATE----- -MIIDWTCCAkGgAwIBAgIJAPm6B21bar2bMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV +MIIEWTCCAsGgAwIBAgIJAJinz4jHSjLtMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u -IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODAx -MTkxOTA5MDZaFw0yODAxMTcxOTA5MDZaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH +IFNvZnR3YXJlIEZvdW5kYXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xODA4 +MjkxNDIzMTVaFw0yODA4MjYxNDIzMTVaMF8xCzAJBgNVBAYTAlhZMRcwFQYDVQQH DA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZvdW5k -YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEP -ADCCAQoCggEBAKvvsX2gEti4shve3iYMc+jE4Se7WHs1Bol2f21H8qNboDOFdeb1 -RKHjmq3exHpajywOUEgne9nKHJY/3f2phR4Y5klqG6liLgiSpVyRlcBGbeT2qEAj -9oLiLFUXLGfGDds2mTwivQDLJBWi51j7ff5k2Pr58fN5ugYMn24T9FNyn0moT+qj -SFoBNm58l9jrdkJSlgWfqPlbiMa+mqDn/SFtrwLF2Trbfzu42Sd9UdIzMaSSrzbN -sGm53pNhCh8KndWUQ8GPP2IsLPoUU4qAtmZuTxCx2S1cXrN9EkmT69tlOH84YfSn -96Ih9bWRc7M5y5bfVdEVM+fKQl3hBRf05qMCAwEAAaMYMBYwFAYDVR0RBA0wC4IJ -bG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQAtQ8f37cCEk7/rAcbYR53ce3iK -Vpihb0U2ni1QjG9Tg9UIExkIGkwTiCm7kwQL+GEStBu9AG/QVrOjeTriRiddhWkk -ze8kRaI3AC/63t6Vh9Q1x6PESgeE4OtAO9JpJCf4GILglA789Y/b/GF8zJZQxR13 -qpB4ZwWw7gCBhdEW59u6CFeBmfDa58hM8lWvuVoRrTi7bjUeC6PAn5HVMzZSykhu -4HaUfBp6bKFjuym2+h/VvM1n8C3chjVSmutsLb6ELdD8IK0vPV/yf5+LN256zSsS -dyUZYd8XwQaioEMKdbhLvnehyzHiWfQIUR3BdhONxoIJhHv/EAo8eCkHHYIF +YXRpb24xEjAQBgNVBAMMCWxvY2FsaG9zdDCCAaIwDQYJKoZIhvcNAQEBBQADggGP +ADCCAYoCggGBALKUqUtopT6E68kN+uJNEt34i2EbmG/bwjcD8IaMsgJPSsMO2Bpd +3S6qWgkCeOyCfmAwBxK2kNbxGb63ouysEv7l8GCTJTWv3hG/HQcejJpnAEGi6K1U +fDbyE/db6yZ12SoHVTGkadN4vYGCPd1Wj9ZO1F877SHQ8rDWX3xgTWkxN2ojBw44 +T8RHSDiG8D/CvG4uEy+VUszL+Uvny5y2poNSqvI3J56sptWSrh8nIIbkPZPBdUne +LYMOHTFK3ZjXSmhlXgziTxK71nnzM3Y9K9gxPnRqoXbvu/wFo55hQCkETiRkYgmm +jXcBMZ0TClQVnQWuLjMthRnWFZs4Lfmwqjs7FZD/61581R2BYehvpWbLvvuOJhwv +DFzexL2sXcAl7SsxbzeQKRHqGbIDfbnQTXfs3/VC6Ye5P82P2ucj+XC32N9piRmO +gCBP8L3ub+YzzdxikZN2gZXXE2jsb3QyE/R2LkWdWyshpKe+RsZP1SBRbHShUyOh +yJ90baoiEwj2mwIDAQABoxgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZI +hvcNAQELBQADggGBAHRUO/UIHl3jXQENewYayHxkIx8t7nu40iO2DXbicSijz5bo +5//xAB6RxhBAlsDBehgQP1uoZg+WJW+nHu3CIVOU3qZNZRaozxiCl2UFKcNqLOmx +R3NKpo1jYf4REQIeG8Yw9+hSWLRbshNteP6bKUUf+vanhg9+axyOEOH/iOQvgk/m +b8wA8wNa4ujWljPbTQnj7ry8RqhTM0GcAN5LSdSvcKcpzLcs3aYwh+Z8e30sQWna +F40sa5u7izgBTOrwpcDm/w5kC46vpRQ5fnbshVw6pne2by0mdMECASid/p25N103 +jMqTFlmO7kpf/jpCSmamp3/JSEE1BJKHwQ6Ql4nzRA2N1mnvWH7Zxcv043gkHeAu +0x8evpvwuhdIyproejNFlBpKmW8OX7yKTCPPMC/VkX8Q1rVkxU0DQ6hmvwZlhoKa +9Wc2uXpw9xF8itV4Uvcdr3dwqByvIqn7iI/gB+4l41e0u8OmH2MKOx4Nxlly5TNW +HcVKQHyOeyvnINuBAQ== -----END CERTIFICATE----- diff --git a/Lib/test/ssl_key.passwd.pem b/Lib/test/ssl_key.passwd.pem index 669c7ce3c815..e4f1370ab270 100644 --- a/Lib/test/ssl_key.passwd.pem +++ b/Lib/test/ssl_key.passwd.pem @@ -1,30 +1,42 @@ -----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED -DEK-Info: DES-EDE3-CBC,2D5DD30B9D440DBB +DEK-Info: DES-EDE3-CBC,8064BE1494B24B13 -01gIwpy3XxPGsY0PoK59vAxdLhkVj3odO0Z1ULamUzIte6ThKL1HqnZiUlXpYKfK -XqHVVeQ1xouxiDRNFLJ4CqBG4HbRtqTkl+sfaNTVveL18lOOMAZy6W3dCGAnWOTZ -Z0RJyZlQxxjNQLuko4tIvFkrShXIgdiVFjwAhRU0KTUb7UQ2xfFA9R0Kfde30pzz -zSjb/OmYqAIhkdvafGXvJxzZAorQkU9akDh+uJ6cht5B/RGZsbKACYDSv2WSV5yW -r+fKVYcTup33r0Jj8hAD6fVY15K8BJknpkF9HfSlZnmmr2WDaffLokOOnCV/I1ie -WD7ENA7K//48km5D3Ogh2b2/0Iwuzjq8Mvd8aR39N9nINbGR+HNT85pawoo1S0W9 -pQTU4XTmxfXjtR2287C6XZyQ/tBwvNDMFPVhlxsGOdLYwoV5e/L1t1qIfkTlbuvd -JaMzOhSSLjiC156IFoH7PTPe+g75hw2b32XJURFGlaYknHF7P4BmCiwXOQYo5CCo -MQGGlw5qBCqODrIsc03wpL2jUzgvyPqLyaw395ITuSoGX+WO7vUQaGW0Tz/sOoTs -3pK+bTi2QmqZMe7xBOj07CYMMOo4QPrM6NpbObt+Jja2UXaxvKa9BwqCEQzA4pQZ -8ZHHfEWIaDffKTGkAlqm+S8qCtsrEZJhIn3aI/ikzK8v+YkWw6w+8t/tR1V8ET/s -CoYGIR7I8WhdfKAwgx2QT5bt1jkYKJyKPm4Iacp2mNh9gNFVq+JSKF318e7BrR3+ -wyqMkDxRYnov3ybtf6kiICxPREDqa6UG1xRq3SbWz6NnIF/1hoHs79YlSYbMfXNU -ffIxBaXNCcH6jM9duP2YRnO29jLwfLM/mmokTBBjyOBaKZia9GPa4naXoATXW3z+ -Xx4EKIUkKdb53kiV6NtEKMPialAnkeoHTEjyLPgaV8mCHLvGQbnxbYwvpPJH0e2f -CWgiw6ci4ROOzcZ7HJHIDUprwK0xRKn43hoI44fivlSHOFX6da6o3wIqhEUqMKwL -JQDS1GORRk1ndRXP+7Ub1dO+Vo/DqO1VcTr2o5RwZ1LWPnzLqbCG50mvTLH4djB+ -+hf6vlmnFC30N3yUFXWE5vS10nJHYP88dD9CB2RsaWzpxD9Zxl+PKcRsppen6HyO -u3b71a/TBOkJcI+lkOatEFvbuqzBAqhMceMctO+Dl55RFsbxfIw/IXZjdP0PYZ0C -t20DrIdBsvl9F/mfYpmkV4DF7yci78DqnRBcxylVNF2vwX7o+2fq/TsEwsHn3KnT -kvcF5Cq8Vr5C8ugWX8JfveNym0BjLu6Lr58qS4a6qCNGEGPFKyB+xkm4KEScbarQ -aLbEbfulMM7q9//sEOOLexIx7mNoLd29Xzn5hsLCAZLWX6wMq6JVJ/zbBOAHDbBT -yhi03yd5Kvw3swSt4QZj+uR3qTFwxkXUFiVvrSfxRZoyKsxsLr9Z7D8aoH9Rkb2L -6KjZ31nt9Drh7NJfh6ReANBW6INdDW0Y2mbzoDozLszAYjVfuUUEE76iJqXY0N4W -kNr0OQQTUtDpVk0AZZZvy17xV+rkqGgwlOqTvHbwFYEQvgwVz4EKUw== +KJrffOMbo8M0I3PzcYxRZGMpKD1yB3Ii4+bT5XoanxjIJ+4fdx6LfZ0Rsx+riyzs +tymsQu/iYY9j+4rCvN9+eetsL1X6iZpiimKsLexcid9M3fb0vxED5Sgw0dvunCUA +xhqjLIKR92MKbODHf6KrDKCpsiPbjq4gZ7P+uCGXAMHL3MXIJSC0hW9rK7Ce6oyO +CjpIcgB8x+GUWZZZhAFdlzIHMZrteNP2P5HK6QcaT71P034Dz1hhqoj4Q0t+Fta2 +4tfsM/bnTR/l6hwlhPa1e3Uj322tDTDWBScgWANn5+sEWldLmozMaWhZsn22pfk2 +KjRMGXG024JVheV882nbdOBvG7oq+lxkZ/ZP+vvqJqnvYtf7WtM8UivzYpe5Hz5b +kVvWzPjBLUSZ9whM9rDLqSSqMPyPvDTuEmLkuq+xm7pYJmsLqIMP2klZLqRxLX6K +uqwplb8UG440qauxgnQ905PId1l2fJEnRtV+7vXprA0L0QotgXLVHBhLmTFM+3PH +9H3onf31dionUAPrn3nfVE36HhvVgRyvDBnBzJSIMighgq21Qx/d1dk0DRYi1hUI +nCHl0YJPXheVcXR7JiSF2XQCAaFuS1Mr7NCXfWZOZQC/0dkvmHnl9DUAhuqq9BNZ +1cKhZXcKHadg2/r0Zup/oDzmHPUEfTAXT0xbqoWlhkdwbF2veWQ96A/ncx3ISTb4 +PkXBlX9rdia8nmtyQDQRn4NuvchbaGkj4WKFC8pF8Hn7naHqwjpHaDUimBc0CoQW +edNJqruKWwtSVLuwKHCC2gZFX9AXSKJXJz/QRSUlhFGOhuF/J6yKaXj6n5lxWNiQ +54J+OP/hz2aS95CD2+Zf1SKpxdWiLZSIQqESpmmUrXROixNJZ/Z7gI74Dd9dSJOH +W+3AU03vrrFZVrJVZhjcINHoH1Skh6JKscH18L6x4U868nSr4SrRLX8BhHllOQyD +bmU+PZAjF8ZBIaCtTGulDXD29F73MeAZeTSsgQjFu0iKLj1wPiphbx8i/SUtR4YP +X6PVA04g66r1NBw+3RQASVorZ3g1MSFvITHXcbKkBDeJH2z1+c6t/VVyTONnQhM5 +lLgRSk6HCbetvT9PKxWrWutA12pdBYEHdZhMHVf2+xclky7l09w8hg2/qqcdGRGe +oAOZ72t0l5ObNyaruDKUS6f4AjOyWq/Xj5xuFtf1n3tQHyslSyCTPcAbQhDfTHUx +vixb/V9qvYPt7OCn8py7v1M69NH42QVFAvwveDIFjZdqfIKBoJK2V4qPoevJI6uj +Q5ByMt8OXOjSXNpHXpYQWUiWeCwOEBXJX8rzCHdMtg37jJ0zCmeErR1NTdg+EujM +TWYgd06jlT67tURST0aB2kg4ijKgUJefD313LW1zC6gVsTbjSZxYyRbPfSP6flQB +yCi1C19E2OsgleqbkBVC5GlYUzaJT7SGjCRmGx1eqtbrALu+LVH24Wceexlpjydl ++s2nf/DZlKun/tlPh6YioifPCJjByZMQOCEfIox6BkemZETz8uYA4TTWimG13Z03 +gyDGC2jdpEW414J2qcQDvrdUgJ+HlhrAAHaWpMQDbXYxBGoZ+3+ORvQV4kAsCwL8 +k3EIrVpePdik+1xgOWsyLj6QxFXlTMvL6Wc5pnArFPORsgHEolJvxSPTf9aAHNPn +V2WBvxiLBtYpGrujAUM40Syx/aN2RPtcXYPAusHUBw+S8/p+/8Kg8GZmnIXG3F89 +45Eepl2quZYIrou7a1fwIpIIZ0hFiBQ1mlHVMFtxwVHS1bQb3SU2GeO+JcGjdVXc +04qeGuQ5M164eQ5C0T7ZQ1ULiUlFWKD30m+cjqmZzt3d7Q0mKpMKuESIuZJo/wpD +Nas432aLKUhcNx/pOYLkKJRpGZKOupQoD5iUj/j44o8JoFkDK33v2S57XB5QGz28 +9Zuhx49b3W8mbM6EBanlQKLWJGCxXqc/jhYhFWn+b0MhidynFgA0oeWvf6ZDyt6H +Yi5Etxsar09xp0Do3NxtQXLuSUu0ji2pQzSIKuoqQWKqldm6VrpwojiqJhy4WQBQ +aVVyFeWBC7G3Zj76dO+yp2sfJ0itJUQ8AIB9Cg0f34rEZu+r9luPmqBoUeL95Tk7 +YvCOU3Jl8Iqysv8aNpVXT8sa8rrSbruWCByEePZ37RIdHLMVBwVY0eVaFQjrjU7E +mXmM9eaoYLfXOllsQ+M2+qPFUITr/GU3Qig13DhK/+yC1R6V2a0l0WRhMltIPYKW +Ztvvr4hK5LcYCeS113BLiMbDIMMZZYGDZGMdC8DnnVbT2loF0Rfmp80Af31KmMQ4 +6XvMatW9UDjBoY5a/YMpdm7SRwm+MgV2KNPpc2kST87/yi9oprGAb8qiarHiHTM0 -----END RSA PRIVATE KEY----- diff --git a/Lib/test/ssl_key.pem b/Lib/test/ssl_key.pem index b63f38bc5cf2..1ea4578d81ec 100644 --- a/Lib/test/ssl_key.pem +++ b/Lib/test/ssl_key.pem @@ -1,28 +1,40 @@ -----BEGIN PRIVATE KEY----- -MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCr77F9oBLYuLIb -3t4mDHPoxOEnu1h7NQaJdn9tR/KjW6AzhXXm9USh45qt3sR6Wo8sDlBIJ3vZyhyW -P939qYUeGOZJahupYi4IkqVckZXARm3k9qhAI/aC4ixVFyxnxg3bNpk8Ir0AyyQV -oudY+33+ZNj6+fHzeboGDJ9uE/RTcp9JqE/qo0haATZufJfY63ZCUpYFn6j5W4jG -vpqg5/0hba8Cxdk62387uNknfVHSMzGkkq82zbBpud6TYQofCp3VlEPBjz9iLCz6 -FFOKgLZmbk8QsdktXF6zfRJJk+vbZTh/OGH0p/eiIfW1kXOzOcuW31XRFTPnykJd -4QUX9OajAgMBAAECggEAHppmXDbuw9Z0FVPg9KLIysioTtsgz6VLiZIm8juZK4x2 -glUh/D7xvWL2uDXrgN+3lh7iGUW13LkFx5SMncbbo9TIwI57Z/XKvcnkVwquve+L -RfLFVc1Q5lD9lROv2rS86KTaN4LzYz3FKXi6dvMkpPAsUtfEQhMLkmISypQQq/1z -EJaqo7r85OjN7e0wKazlKZpOzJEa5FQLMVRjTRFhLFNbHXX/tAet2jw+umATKbw8 -hYgiuZ44TwSEd9JeIV/oSYWfI/3HetuYW0ru3caiztRF2NySNu8lcsWgNC7fIku9 -mcHjtSNzs91QN1Qlu7GQvvhpt6OWDirNDCW+49WGaQKBgQDg9SDhfF0jRYslgYbH -cqO4ggaFdHjrAAYpwnAgvanhFZL/zEqm5G1E7l/e2fCkJ9VOSFO0A208chvwMcr+ -dCjHE2tVdE81aQ2v/Eo83VdS1RcOV4Y75yPH48rMhxPaHvxWD/FFDbf0/P2mtPB7 -SZ3kIeZMkE1wxdaO3AKUbQoozwKBgQDDqYgg7kVtygyICE1mB8Hwp6nUxFTczG7y -4XcsDqMIrKmw+PbQluvkoHoStxeVrsTloDhkTjIrpmYLyAiazg+PUJdkd6xrfLSj -VV6X93W0S/1egEb1F1CGFxtk8v/PWH4K76EPL2vxXdxjywz3GWlrL9yDYaB2szzS -DqgwVMqx7QKBgDCD7UF0Bsoyl13RX3XoPXLvZ+SkR+e2q52Z94C4JskKVBeiwX7Y -yNAS8M4pBoMArDoj0xmBm69rlKbqtjLGbnzwrTdSzDpim7cWnBQgUFLm7gAD1Elb -AhZ8BCK0Bw4FnLoa2hfga4oEfdfUMgEE0W5/+SEOBgWKRUmuHUhRc911AoGAY2EN -YmSDYSM5wDIvVb5k9B3EtevOiqNPSw/XnsoEZtiEC/44JnQxdltIBY93bDBrk5IQ -cmoBM4h91kgQjshQwOMXMhFSwvmBKmCm/hrTbvMVytTutXfVD3ZXFKwT4DW7N0TF -ElhsxBh/YzRz7mG62JVjtFt2zDN3ld2Z8YpvtXUCgYEA4EJ4ObS5YyvcXAKHJFo6 -Fxmavyrf8LSm3MFA65uSnFvWukMVqqRMReQc5jvpxHKCis+XvnHzyOfL0gW9ZTi7 -tWGGbBi0TRJCa8BkvgngUZxOxUlMfg/7cVxOIB0TPoUSgxFd/+qVz4GZMvr0dPu7 -eAF7J/8ECVvb0wSPTUI1N3c= +MIIG/wIBADANBgkqhkiG9w0BAQEFAASCBukwggblAgEAAoIBgQCylKlLaKU+hOvJ +DfriTRLd+IthG5hv28I3A/CGjLICT0rDDtgaXd0uqloJAnjsgn5gMAcStpDW8Rm+ +t6LsrBL+5fBgkyU1r94Rvx0HHoyaZwBBouitVHw28hP3W+smddkqB1UxpGnTeL2B +gj3dVo/WTtRfO+0h0PKw1l98YE1pMTdqIwcOOE/ER0g4hvA/wrxuLhMvlVLMy/lL +58uctqaDUqryNyeerKbVkq4fJyCG5D2TwXVJ3i2DDh0xSt2Y10poZV4M4k8Su9Z5 +8zN2PSvYMT50aqF277v8BaOeYUApBE4kZGIJpo13ATGdEwpUFZ0Fri4zLYUZ1hWb +OC35sKo7OxWQ/+tefNUdgWHob6Vmy777jiYcLwxc3sS9rF3AJe0rMW83kCkR6hmy +A3250E137N/1QumHuT/Nj9rnI/lwt9jfaYkZjoAgT/C97m/mM83cYpGTdoGV1xNo +7G90MhP0di5FnVsrIaSnvkbGT9UgUWx0oVMjocifdG2qIhMI9psCAwEAAQKCAYBT +sHmaPmNaZj59jZCqp0YVQlpHWwBYQ5vD3pPE6oCttm0p9nXt/VkfenQRTthOtmT1 +POzDp00/feP7zeGLmqSYUjgRekPw4gdnN7Ip2PY5kdW77NWwDSzdLxuOS8Rq1MW9 +/Yu+ZPe3RBlDbT8C0IM+Atlh/BqIQ3zIxN4g0pzUlF0M33d6AYfYSzOcUhibOO7H +j84r+YXBNkIRgYKZYbutRXuZYaGuqejRpBj3voVu0d3Ntdb6lCWuClpB9HzfGN0c +RTv8g6UYO4sK3qyFn90ibIR/1GB9watvtoWVZqggiWeBzSWVWRsGEf9O+Cx4oJw1 +IphglhmhbgNksbj7bD24on/icldSOiVkoUemUOFmHWhCm4PnB1GmbD8YMfEdSbks +qDr1Ps1zg4mGOinVD/4cY7vuPFO/HCH07wfeaUGzRt4g0/yLr+XjVofOA3oowyxv +JAzr+niHA3lg5ecj4r7M68efwzN1OCyjMrVJw2RAzwvGxE+rm5NiT08SWlKQZnkC +gcEA4wvyLpIur/UB84nV3XVJ89UMNBLm++aTFzld047BLJtMaOhvNqx6Cl5c8VuW +l261KHjiVzpfNM3/A2LBQJcYkhX7avkqEXlj57cl+dCWAVwUzKmLJTPjfaTTZnYJ +xeN3dMYjJz2z2WtgvfvDoJLukVwIMmhTY8wtqqYyQBJ/l06pBsfw5TNvmVIOQHds +8ASOiFt+WRLk2bl9xrGGayqt3VV93KVRzF27cpjOgEcG74F3c0ZW9snERN7vIYwB +JfrlAoHBAMlahPwMP2TYylG8OzHe7EiehTekSO26LGh0Cq3wTGXYsK/q8hQCzL14 +kWW638vpwXL6L9ntvrd7hjzWRO3vX/VxnYEA6f0bpqHq1tZi6lzix5CTUN5McpDg +QnjenSJNrNjS1zEF8WeY9iLEuDI/M/iUW4y9R6s3WpgQhPDXpSvd2g3gMGRUYhxQ +Xna8auiJeYFq0oNaOxvJj+VeOfJ3ZMJttd+Y7gTOYZcbg3SdRb/kdxYki0RMD2hF +4ZvjJ6CTfwKBwQDiMqiZFTJGQwYqp4vWEmAW+I4r4xkUpWatoI2Fk5eI5T9+1PLX +uYXsho56NxEU1UrOg4Cb/p+TcBc8PErkGqR0BkpxDMOInTOXSrQe6lxIBoECVXc3 +HTbrmiay0a5y5GfCgxPKqIJhfcToAceoVjovv0y7S4yoxGZKuUEe7E8JY2iqRNAO +yOvKCCICv/hcN235E44RF+2/rDlOltagNej5tY6rIFkaDdgOF4bD7f9O5eEni1Bg +litfoesDtQP/3rECgcEAkQfvQ7D6tIPmbqsbJBfCr6fmoqZllT4FIJN84b50+OL0 +mTGsfjdqC4tdhx3sdu7/VPbaIqm5NmX10bowWgWSY7MbVME4yQPyqSwC5NbIonEC +d6N0mzoLR0kQ+Ai4u+2g82gicgAq2oj1uSNi3WZi48jQjHYFulCbo246o1NgeFFK +77WshYe2R1ioQfQDOU1URKCR0uTaMHClgfu112yiGd12JAD+aF3TM0kxDXz+sXI5 +SKy311DFxECZeXRLpcC3AoHBAJkNMJWTyPYbeVu+CTQkec8Uun233EkXa2kUNZc/ +5DuXDaK+A3DMgYRufTKSPpDHGaCZ1SYPInX1Uoe2dgVjWssRL2uitR4ENabDoAOA +ICVYXYYNagqQu5wwirF0QeaMXo1fjhuuHQh8GsMdXZvYEaAITZ9/NG5x/oY08+8H +kr78SMBOPy3XQn964uKG+e3JwpOG14GKABdAlrHKFXNWchu/6dgcYXB87mrC/GhO +zNwzC+QhFTZoOomFoqMgFWujng== -----END PRIVATE KEY----- diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index 7d06dc57c8e1..dc14e22ad121 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -201,9 +201,9 @@ def test_parse_cert(self): (('commonName', 'localhost'),)) ) # Note the next three asserts will fail if the keys are regenerated - self.assertEqual(p['notAfter'], asn1time('Jan 17 19:09:06 2028 GMT')) - self.assertEqual(p['notBefore'], asn1time('Jan 19 19:09:06 2018 GMT')) - self.assertEqual(p['serialNumber'], 'F9BA076D5B6ABD9B') + self.assertEqual(p['notAfter'], asn1time('Aug 26 14:23:15 2028 GMT')) + self.assertEqual(p['notBefore'], asn1time('Aug 29 14:23:15 2018 GMT')) + self.assertEqual(p['serialNumber'], '98A7CF88C74A32ED') self.assertEqual(p['subject'], ((('countryName', 'XY'),), (('localityName', 'Castle Anthrax'),), diff --git a/Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst b/Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst new file mode 100644 index 000000000000..1ca3c7d7996c --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-08-29-16-30-52.bpo-34542.9stVAW.rst @@ -0,0 +1 @@ +Use 3072 RSA keys and SHA-256 signature for test certs and keys. From webhook-mailer at python.org Thu Sep 20 07:10:54 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 20 Sep 2018 11:10:54 -0000 Subject: [Python-checkins] bpo-32215: Fix performance regression in sqlite3 (GH-8511) Message-ID: https://github.com/python/cpython/commit/8d1e190fc507a9e304f6817e761e9f628a23cbd8 commit: 8d1e190fc507a9e304f6817e761e9f628a23cbd8 branch: master author: Berker Peksag committer: GitHub date: 2018-09-20T14:10:49+03:00 summary: bpo-32215: Fix performance regression in sqlite3 (GH-8511) files: A Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst M Modules/_sqlite/statement.c diff --git a/Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst b/Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst new file mode 100644 index 000000000000..c097cf7310df --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst @@ -0,0 +1,2 @@ +Fix performance regression in :mod:`sqlite3` when a DML statement appeared +in a different line than the rest of the SQL query. diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c index 38690884227e..78033d8efcae 100644 --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -85,10 +85,10 @@ int pysqlite_statement_create(pysqlite_Statement* self, pysqlite_Connection* con continue; } - self->is_dml = (PyOS_strnicmp(p, "insert ", 7) == 0) - || (PyOS_strnicmp(p, "update ", 7) == 0) - || (PyOS_strnicmp(p, "delete ", 7) == 0) - || (PyOS_strnicmp(p, "replace ", 8) == 0); + self->is_dml = (PyOS_strnicmp(p, "insert", 6) == 0) + || (PyOS_strnicmp(p, "update", 6) == 0) + || (PyOS_strnicmp(p, "delete", 6) == 0) + || (PyOS_strnicmp(p, "replace", 7) == 0); break; } From webhook-mailer at python.org Thu Sep 20 07:14:38 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 20 Sep 2018 11:14:38 -0000 Subject: [Python-checkins] bpo-34743: Fix test_database_source_name under SQLite 3.7.9 (GH-9426) Message-ID: https://github.com/python/cpython/commit/b10a64d117de6121ea3e79c467c4107f8f399f3d commit: b10a64d117de6121ea3e79c467c4107f8f399f3d branch: master author: Berker Peksag committer: GitHub date: 2018-09-20T14:14:33+03:00 summary: bpo-34743: Fix test_database_source_name under SQLite 3.7.9 (GH-9426) files: M Modules/_sqlite/connection.c diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index 5b15d1d930a3..89a875189780 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -1594,6 +1594,12 @@ pysqlite_connection_backup(pysqlite_Connection *self, PyObject *args, PyObject * PyErr_SetString(pysqlite_OperationalError, sqlite3_errstr(rc)); #else switch (rc) { + case SQLITE_ERROR: + /* Description of SQLITE_ERROR in SQLite 3.7.14 and older + releases. */ + PyErr_SetString(pysqlite_OperationalError, + "SQL logic error or missing database"); + break; case SQLITE_READONLY: PyErr_SetString(pysqlite_OperationalError, "attempt to write a readonly database"); From webhook-mailer at python.org Thu Sep 20 11:09:08 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 20 Sep 2018 15:09:08 -0000 Subject: [Python-checkins] bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446) Message-ID: https://github.com/python/cpython/commit/8213eaddf3ce8e87564d2949454903a1484748b5 commit: 8213eaddf3ce8e87564d2949454903a1484748b5 branch: master author: Berker Peksag committer: GitHub date: 2018-09-20T18:08:57+03:00 summary: bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446) files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 19701cf388fe..ed0d46de6426 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1426,6 +1426,9 @@ def transient_internet(resource_name, *, timeout=30.0, errnos=()): ('EHOSTUNREACH', 113), ('ENETUNREACH', 101), ('ETIMEDOUT', 110), + # socket.create_connection() fails randomly with + # EADDRNOTAVAIL on Travis CI. + ('EADDRNOTAVAIL', 99), ] default_gai_errnos = [ ('EAI_AGAIN', -3), From webhook-mailer at python.org Thu Sep 20 11:23:52 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 20 Sep 2018 15:23:52 -0000 Subject: [Python-checkins] bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446) Message-ID: https://github.com/python/cpython/commit/476177005e8c8d4ece3070c8c378f8b8c068e76f commit: 476177005e8c8d4ece3070c8c378f8b8c068e76f branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Berker Peksag date: 2018-09-20T18:23:45+03:00 summary: bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446) (cherry picked from commit 8213eaddf3ce8e87564d2949454903a1484748b5) Co-authored-by: Berker Peksag files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 2ad563469ae3..4127b50d1fbb 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1408,6 +1408,9 @@ def transient_internet(resource_name, *, timeout=30.0, errnos=()): ('EHOSTUNREACH', 113), ('ENETUNREACH', 101), ('ETIMEDOUT', 110), + # socket.create_connection() fails randomly with + # EADDRNOTAVAIL on Travis CI. + ('EADDRNOTAVAIL', 99), ] default_gai_errnos = [ ('EAI_AGAIN', -3), From webhook-mailer at python.org Thu Sep 20 11:36:26 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 20 Sep 2018 15:36:26 -0000 Subject: [Python-checkins] bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446) Message-ID: https://github.com/python/cpython/commit/1eabe19c57938dd70b66b97239be337294e15cba commit: 1eabe19c57938dd70b66b97239be337294e15cba branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-20T08:36:23-07:00 summary: bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446) (cherry picked from commit 8213eaddf3ce8e87564d2949454903a1484748b5) Co-authored-by: Berker Peksag files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 9b3ebbadfe44..3f1962fe1cd2 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1411,6 +1411,9 @@ def transient_internet(resource_name, *, timeout=30.0, errnos=()): ('EHOSTUNREACH', 113), ('ENETUNREACH', 101), ('ETIMEDOUT', 110), + # socket.create_connection() fails randomly with + # EADDRNOTAVAIL on Travis CI. + ('EADDRNOTAVAIL', 99), ] default_gai_errnos = [ ('EAI_AGAIN', -3), From webhook-mailer at python.org Thu Sep 20 11:57:01 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 20 Sep 2018 15:57:01 -0000 Subject: [Python-checkins] bpo-32215: Fix performance regression in sqlite3 (GH-8511) Message-ID: https://github.com/python/cpython/commit/015cd0f5cb17b1b208a92e549cd665dc38f2f699 commit: 015cd0f5cb17b1b208a92e549cd665dc38f2f699 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Berker Peksag date: 2018-09-20T18:56:56+03:00 summary: bpo-32215: Fix performance regression in sqlite3 (GH-8511) (cherry picked from commit 8d1e190fc507a9e304f6817e761e9f628a23cbd8) Co-authored-by: Berker Peksag files: A Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst M Modules/_sqlite/statement.c diff --git a/Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst b/Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst new file mode 100644 index 000000000000..c097cf7310df --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst @@ -0,0 +1,2 @@ +Fix performance regression in :mod:`sqlite3` when a DML statement appeared +in a different line than the rest of the SQL query. diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c index 38690884227e..78033d8efcae 100644 --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -85,10 +85,10 @@ int pysqlite_statement_create(pysqlite_Statement* self, pysqlite_Connection* con continue; } - self->is_dml = (PyOS_strnicmp(p, "insert ", 7) == 0) - || (PyOS_strnicmp(p, "update ", 7) == 0) - || (PyOS_strnicmp(p, "delete ", 7) == 0) - || (PyOS_strnicmp(p, "replace ", 8) == 0); + self->is_dml = (PyOS_strnicmp(p, "insert", 6) == 0) + || (PyOS_strnicmp(p, "update", 6) == 0) + || (PyOS_strnicmp(p, "delete", 6) == 0) + || (PyOS_strnicmp(p, "replace", 7) == 0); break; } From webhook-mailer at python.org Thu Sep 20 12:06:08 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 20 Sep 2018 16:06:08 -0000 Subject: [Python-checkins] bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446) Message-ID: https://github.com/python/cpython/commit/170ea8ccd4235d28538ab713041502d07ad1cacd commit: 170ea8ccd4235d28538ab713041502d07ad1cacd branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-20T09:06:04-07:00 summary: bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446) (cherry picked from commit 8213eaddf3ce8e87564d2949454903a1484748b5) Co-authored-by: Berker Peksag files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 6da25971a9ce..c2cc009b9f48 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1151,6 +1151,9 @@ def transient_internet(resource_name, timeout=30.0, errnos=()): ('EHOSTUNREACH', 113), ('ENETUNREACH', 101), ('ETIMEDOUT', 110), + # socket.create_connection() fails randomly with + # EADDRNOTAVAIL on Travis CI. + ('EADDRNOTAVAIL', 99), ] default_gai_errnos = [ ('EAI_AGAIN', -3), From webhook-mailer at python.org Thu Sep 20 12:44:10 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Thu, 20 Sep 2018 16:44:10 -0000 Subject: [Python-checkins] bpo-33649: More improvements (GH-9439) Message-ID: https://github.com/python/cpython/commit/e247b46cba4f4d32ea96a15dbc36d73265171106 commit: e247b46cba4f4d32ea96a15dbc36d73265171106 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-20T12:43:59-04:00 summary: bpo-33649: More improvements (GH-9439) files: M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 92637327ae40..50d6ea4b1389 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -755,7 +755,7 @@ Watching file descriptors invoke *callback* with the specified arguments once *fd* is available for writing. - Use :func:`functools.partial` :ref:`to pass keywords + Use :func:`functools.partial` :ref:`to pass keyword arguments ` to *func*. .. method:: loop.remove_writer(fd) @@ -969,7 +969,7 @@ Unix signals Raise :exc:`ValueError` if the signal number is invalid or uncatchable. Raise :exc:`RuntimeError` if there is a problem setting up the handler. - Use :func:`functools.partial` :ref:`to pass keywords + Use :func:`functools.partial` :ref:`to pass keyword arguments ` to *func*. .. method:: loop.remove_signal_handler(sig) @@ -996,11 +996,52 @@ Executing code in thread or process pools The *executor* argument should be an :class:`concurrent.futures.Executor` instance. The default executor is used if *executor* is ``None``. - Use :func:`functools.partial` :ref:`to pass keywords - ` to *func*. + Example:: + + import asyncio + import concurrent.futures + + def blocking_io(): + # File operations (such as logging) can block the + # event loop: run them in a thread pool. + with open('/dev/urandom', 'rb') as f: + return f.read(100) + + def cpu_bound(): + # CPU-bound operations will block the event loop: + # in general it is preferable to run them in a + # process pool. + return sum(i * i for i in range(10 ** 7)) + + async def main(): + loop = asyncio.get_running_loop() + + ## Options: + + # 1. Run in the default loop's executor: + result = await loop.run_in_executor( + None, blocking_io) + print('default thread pool', result) + + # 2. Run in a custom thread pool: + with concurrent.futures.ThreadPoolExecutor() as pool: + result = await loop.run_in_executor( + pool, blocking_io) + print('custom thread pool', result) + + # 3. Run in a custom process pool: + with concurrent.futures.ProcessPoolExecutor() as pool: + result = await loop.run_in_executor( + pool, cpu_bound) + print('custom process pool', result) + + asyncio.run(main()) This method returns a :class:`asyncio.Future` object. + Use :func:`functools.partial` :ref:`to pass keyword arguments + ` to *func*. + .. versionchanged:: 3.5.3 :meth:`loop.run_in_executor` no longer configures the ``max_workers`` of the thread pool executor it creates, instead diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 85292a61ecd6..b33a7df4e822 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -109,50 +109,89 @@ To actually run a coroutine asyncio provides three main mechanisms: Awaitables ========== -We say that an object is an *awaitable* object if it can be used -in an :keyword:`await` expression. +We say that an object is an **awaitable** object if it can be used +in an :keyword:`await` expression. Many asyncio APIs are designed to +accept awaitables. +There are three main types of *awaitable* objects: +**coroutines**, **Tasks**, and **Futures**. -.. rubric:: Coroutines and Tasks -Python coroutines are *awaitables*:: +.. rubric:: Coroutines + +Python coroutines are *awaitables* and therefore can be awaited from +other coroutines:: + + import asyncio async def nested(): return 42 async def main(): - # Will print "42": - print(await nested()) + # Nothing happens if we just call "nested()". + # (a coroutine object is created but not awaited) + nested() + + # Let's do it differently now and await it: + print(await nested()) # will print "42". + + asyncio.run(main()) + +.. important:: + + In this documentation the term "coroutine" can be used for + two closely related concepts: + + * a *coroutine function*: an :keyword:`async def` function; + + * a *coroutine object*: an object returned by calling a + *coroutine function*. + +asyncio also supports legacy :ref:`generator-based +` coroutines. + + +.. rubric:: Tasks *Tasks* are used to schedule coroutines *concurrently*. -See the previous :ref:`section ` for an introduction -to coroutines and tasks. -Note that in this documentation the term "coroutine" can be used for -two closely related concepts: +When a coroutine is wrapped into a *Task* with functions like +:func:`asyncio.create_task` the coroutine is automatically +scheduled to run soon:: + + import asyncio + + async def nested(): + return 42 -* a *coroutine function*: an :keyword:`async def` function; + async def main(): + # Schedule nested() to run soon concurrently + # with "main()". + task = asyncio.create_task(nested()) -* a *coroutine object*: object returned by calling a - *coroutine function*. + # "task" can now be used to cancel "nested()", or + # can simply be awaited to wait until it is complete: + await task + + asyncio.run(main()) .. rubric:: Futures -There is a dedicated section about the :ref:`asyncio Future object -`, but the concept is fundamental to asyncio so -it needs a brief introduction in this section. +A :class:`Future` is a special **low-level** awaitable object that +represents an **eventual result** of an asynchronous operation. + +When a Future object is *awaited* it means that the coroutine will +wait until the Future is resolved in some other place. -A Future is a special **low-level** awaitable object that represents -an **eventual result** of an asynchronous operation. Future objects in asyncio are needed to allow callback-based code to be used with async/await. -Normally, **there is no need** to create Future objects at the +Normally **there is no need** to create Future objects at the application level code. Future objects, sometimes exposed by libraries and some asyncio -APIs, should be awaited:: +APIs, can be awaited:: async def main(): await function_that_returns_a_future_object() @@ -163,6 +202,9 @@ APIs, should be awaited:: some_python_coroutine() ) +A good example of a low-level function that returns a Future object +is :meth:`loop.run_in_executor`. + Running an asyncio Program ========================== @@ -192,8 +234,8 @@ Creating Tasks .. function:: create_task(coro, \*, name=None) - Wrap the *coro* :ref:`coroutine ` into a Task and - schedule its execution. Return the Task object. + Wrap the *coro* :ref:`coroutine ` into a :class:`Task` + and schedule its execution. Return the Task object. If *name* is not ``None``, it is set as the name of the task using :meth:`Task.set_name`. @@ -259,17 +301,17 @@ Sleeping Running Tasks Concurrently ========================== -.. awaitablefunction:: gather(\*fs, loop=None, return_exceptions=False) +.. awaitablefunction:: gather(\*aws, loop=None, return_exceptions=False) - Run :ref:`awaitable objects ` in the *fs* + Run :ref:`awaitable objects ` in the *aws* sequence *concurrently*. - If any awaitable in *fs* is a coroutine, it is automatically + If any awaitable in *aws* is a coroutine, it is automatically scheduled as a Task. If all awaitables are completed successfully, the result is an aggregate list of returned values. The order of result values - corresponds to the order of awaitables in *fs*. + corresponds to the order of awaitables in *aws*. If *return_exceptions* is ``True``, exceptions are treated the same as successful results, and aggregated in the result list. @@ -279,7 +321,7 @@ Running Tasks Concurrently If ``gather`` is *cancelled*, all submitted awaitables (that have not completed yet) are also *cancelled*. - If any Task or Future from the *fs* sequence is *cancelled*, it is + If any Task or Future from the *aws* sequence is *cancelled*, it is treated as if it raised :exc:`CancelledError` -- the ``gather()`` call is **not** cancelled in this case. This is to prevent the cancellation of one submitted Task/Future to cause other @@ -329,13 +371,13 @@ Running Tasks Concurrently Shielding Tasks From Cancellation ================================= -.. awaitablefunction:: shield(fut, \*, loop=None) +.. awaitablefunction:: shield(aw, \*, loop=None) Protect an :ref:`awaitable object ` from being :meth:`cancelled `. - *fut* can be a coroutine, a Task, or a Future-like object. If - *fut* is a coroutine it is automatically scheduled as a Task. + *aw* can be a coroutine, a Task, or a Future-like object. If + *aw* is a coroutine it is automatically scheduled as a Task. The statement:: @@ -367,12 +409,12 @@ Shielding Tasks From Cancellation Timeouts ======== -.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) +.. coroutinefunction:: wait_for(aw, timeout, \*, loop=None) - Wait for the *fut* :ref:`awaitable ` + Wait for the *aw* :ref:`awaitable ` to complete with a timeout. - If *fut* is a coroutine it is automatically scheduled as a Task. + If *aw* is a coroutine it is automatically scheduled as a Task. *timeout* can either be ``None`` or a float or int number of seconds to wait for. If *timeout* is ``None``, block until the future @@ -387,7 +429,7 @@ Timeouts The function will wait until the future is actually cancelled, so the total wait time may exceed the *timeout*. - If the wait is cancelled, the future *fut* is also cancelled. + If the wait is cancelled, the future *aw* is also cancelled. The *loop* argument is deprecated and scheduled for removal in Python 4.0. @@ -415,22 +457,22 @@ Timeouts # timeout! .. versionchanged:: 3.7 - When *fut* is cancelled due to a timeout, ``wait_for`` waits - for *fut* to be cancelled. Previously, it raised + When *aw* is cancelled due to a timeout, ``wait_for`` waits + for *aw* to be cancelled. Previously, it raised :exc:`asyncio.TimeoutError` immediately. Waiting Primitives ================== -.. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\ +.. coroutinefunction:: wait(aws, \*, loop=None, timeout=None,\ return_when=ALL_COMPLETED) - Run :ref:`awaitable objects ` in the *fs* + Run :ref:`awaitable objects ` in the *aws* sequence concurrently and block until the condition specified by *return_when*. - If any awaitable in *fs* is a coroutine, it is automatically + If any awaitable in *aws* is a coroutine, it is automatically scheduled as a Task. Returns two sets of Tasks/Futures: ``(done, pending)``. @@ -471,12 +513,12 @@ Waiting Primitives Usage:: - done, pending = await asyncio.wait(fs) + done, pending = await asyncio.wait(aws) -.. function:: as_completed(fs, \*, loop=None, timeout=None) +.. function:: as_completed(aws, \*, loop=None, timeout=None) - Run :ref:`awaitable objects ` in the *fs* + Run :ref:`awaitable objects ` in the *aws* set concurrently. Return an iterator of :class:`Future` objects. Each Future object returned represents the earliest result from the set of the remaining awaitables. @@ -486,7 +528,7 @@ Waiting Primitives Example:: - for f in as_completed(fs): + for f in as_completed(aws): earliest_result = await f # ... @@ -679,6 +721,52 @@ Task Object A Task is *done* when the wrapped coroutine either returned a value, raised an exception, or the Task was cancelled. + .. method:: result() + + Return the result of the Task. + + If the Task is *done*, the result of the wrapped coroutine + is returned (or if the coroutine raised an exception, that + exception is re-raised.) + + If the Task has been *cancelled*, this method raises + a :exc:`CancelledError` exception. + + If the Task's result isn't yet available, this method raises + a :exc:`InvalidStateError` exception. + + .. method:: exception() + + Return the exception of the Task. + + If the wrapped coroutine raised an exception that exception + is returned. If the wrapped coroutine returned normally + this method returns ``None``. + + If the Task has been *cancelled*, this method raises a + :exc:`CancelledError` exception. + + If the Task isn't *done* yet, this method raises an + :exc:`InvalidStateError` exception. + + .. method:: add_done_callback(callback, *, context=None) + + Add a callback to be run when the Task is *done*. + + This method should only be used in low-level callback-based code. + + See the documentation of :meth:`Future.add_done_callback` + for more details. + + .. method:: remove_done_callback(callback) + + Remove *callback* from the callbacks list. + + This method should only be used in low-level callback-based code. + + See the documentation of :meth:`Future.remove_done_callback` + for more details. + .. method:: get_stack(\*, limit=None) Return the list of stack frames for this Task. From webhook-mailer at python.org Thu Sep 20 12:53:11 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 20 Sep 2018 16:53:11 -0000 Subject: [Python-checkins] bpo-34754: Fix test_flush_return_value on FreeBSD (GH-9451) Message-ID: https://github.com/python/cpython/commit/bc854750589d4de0fd55693963964e0558b5c8ac commit: bc854750589d4de0fd55693963964e0558b5c8ac branch: master author: Berker Peksag committer: GitHub date: 2018-09-20T19:53:06+03:00 summary: bpo-34754: Fix test_flush_return_value on FreeBSD (GH-9451) Apparently, FreeBSD doesn't raise OSError when offset is not a multiple of mmap.PAGESIZE. files: M Lib/test/test_mmap.py diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py index d513810b35be..246fdf01f9cd 100644 --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -749,8 +749,9 @@ def test_flush_return_value(self): mm.write(b'python') result = mm.flush() self.assertIsNone(result) - if os.name != 'nt': - # 'offset' must be a multiple of mmap.PAGESIZE. + if sys.platform.startswith('linux'): + # 'offset' must be a multiple of mmap.PAGESIZE on Linux. + # See bpo-34754 for details. self.assertRaises(OSError, mm.flush, 1, len(b'python')) From webhook-mailer at python.org Thu Sep 20 12:57:23 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 20 Sep 2018 16:57:23 -0000 Subject: [Python-checkins] bpo-33649: More improvements (GH-9439) Message-ID: https://github.com/python/cpython/commit/8e5ef58c10a1154f824d5875c2d89794a800eadc commit: 8e5ef58c10a1154f824d5875c2d89794a800eadc branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-20T09:57:19-07:00 summary: bpo-33649: More improvements (GH-9439) (cherry picked from commit e247b46cba4f4d32ea96a15dbc36d73265171106) Co-authored-by: Yury Selivanov files: M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 30996308c80d..a61b971acbdc 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -749,7 +749,7 @@ Watching file descriptors invoke *callback* with the specified arguments once *fd* is available for writing. - Use :func:`functools.partial` :ref:`to pass keywords + Use :func:`functools.partial` :ref:`to pass keyword arguments ` to *func*. .. method:: loop.remove_writer(fd) @@ -963,7 +963,7 @@ Unix signals Raise :exc:`ValueError` if the signal number is invalid or uncatchable. Raise :exc:`RuntimeError` if there is a problem setting up the handler. - Use :func:`functools.partial` :ref:`to pass keywords + Use :func:`functools.partial` :ref:`to pass keyword arguments ` to *func*. .. method:: loop.remove_signal_handler(sig) @@ -990,11 +990,52 @@ Executing code in thread or process pools The *executor* argument should be an :class:`concurrent.futures.Executor` instance. The default executor is used if *executor* is ``None``. - Use :func:`functools.partial` :ref:`to pass keywords - ` to *func*. + Example:: + + import asyncio + import concurrent.futures + + def blocking_io(): + # File operations (such as logging) can block the + # event loop: run them in a thread pool. + with open('/dev/urandom', 'rb') as f: + return f.read(100) + + def cpu_bound(): + # CPU-bound operations will block the event loop: + # in general it is preferable to run them in a + # process pool. + return sum(i * i for i in range(10 ** 7)) + + async def main(): + loop = asyncio.get_running_loop() + + ## Options: + + # 1. Run in the default loop's executor: + result = await loop.run_in_executor( + None, blocking_io) + print('default thread pool', result) + + # 2. Run in a custom thread pool: + with concurrent.futures.ThreadPoolExecutor() as pool: + result = await loop.run_in_executor( + pool, blocking_io) + print('custom thread pool', result) + + # 3. Run in a custom process pool: + with concurrent.futures.ProcessPoolExecutor() as pool: + result = await loop.run_in_executor( + pool, cpu_bound) + print('custom process pool', result) + + asyncio.run(main()) This method returns a :class:`asyncio.Future` object. + Use :func:`functools.partial` :ref:`to pass keyword arguments + ` to *func*. + .. versionchanged:: 3.5.3 :meth:`loop.run_in_executor` no longer configures the ``max_workers`` of the thread pool executor it creates, instead diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 4b079e81a292..1b95592cd99e 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -109,50 +109,89 @@ To actually run a coroutine asyncio provides three main mechanisms: Awaitables ========== -We say that an object is an *awaitable* object if it can be used -in an :keyword:`await` expression. +We say that an object is an **awaitable** object if it can be used +in an :keyword:`await` expression. Many asyncio APIs are designed to +accept awaitables. +There are three main types of *awaitable* objects: +**coroutines**, **Tasks**, and **Futures**. -.. rubric:: Coroutines and Tasks -Python coroutines are *awaitables*:: +.. rubric:: Coroutines + +Python coroutines are *awaitables* and therefore can be awaited from +other coroutines:: + + import asyncio async def nested(): return 42 async def main(): - # Will print "42": - print(await nested()) + # Nothing happens if we just call "nested()". + # (a coroutine object is created but not awaited) + nested() + + # Let's do it differently now and await it: + print(await nested()) # will print "42". + + asyncio.run(main()) + +.. important:: + + In this documentation the term "coroutine" can be used for + two closely related concepts: + + * a *coroutine function*: an :keyword:`async def` function; + + * a *coroutine object*: an object returned by calling a + *coroutine function*. + +asyncio also supports legacy :ref:`generator-based +` coroutines. + + +.. rubric:: Tasks *Tasks* are used to schedule coroutines *concurrently*. -See the previous :ref:`section ` for an introduction -to coroutines and tasks. -Note that in this documentation the term "coroutine" can be used for -two closely related concepts: +When a coroutine is wrapped into a *Task* with functions like +:func:`asyncio.create_task` the coroutine is automatically +scheduled to run soon:: + + import asyncio + + async def nested(): + return 42 -* a *coroutine function*: an :keyword:`async def` function; + async def main(): + # Schedule nested() to run soon concurrently + # with "main()". + task = asyncio.create_task(nested()) -* a *coroutine object*: object returned by calling a - *coroutine function*. + # "task" can now be used to cancel "nested()", or + # can simply be awaited to wait until it is complete: + await task + + asyncio.run(main()) .. rubric:: Futures -There is a dedicated section about the :ref:`asyncio Future object -`, but the concept is fundamental to asyncio so -it needs a brief introduction in this section. +A :class:`Future` is a special **low-level** awaitable object that +represents an **eventual result** of an asynchronous operation. + +When a Future object is *awaited* it means that the coroutine will +wait until the Future is resolved in some other place. -A Future is a special **low-level** awaitable object that represents -an **eventual result** of an asynchronous operation. Future objects in asyncio are needed to allow callback-based code to be used with async/await. -Normally, **there is no need** to create Future objects at the +Normally **there is no need** to create Future objects at the application level code. Future objects, sometimes exposed by libraries and some asyncio -APIs, should be awaited:: +APIs, can be awaited:: async def main(): await function_that_returns_a_future_object() @@ -163,6 +202,9 @@ APIs, should be awaited:: some_python_coroutine() ) +A good example of a low-level function that returns a Future object +is :meth:`loop.run_in_executor`. + Running an asyncio Program ========================== @@ -192,8 +234,8 @@ Creating Tasks .. function:: create_task(coro) - Wrap the *coro* :ref:`coroutine ` into a Task and - schedule its execution. Return the Task object. + Wrap the *coro* :ref:`coroutine ` into a :class:`Task` + and schedule its execution. Return the Task object. The task is executed in the loop returned by :func:`get_running_loop`, :exc:`RuntimeError` is raised if there is no running loop in @@ -253,17 +295,17 @@ Sleeping Running Tasks Concurrently ========================== -.. awaitablefunction:: gather(\*fs, loop=None, return_exceptions=False) +.. awaitablefunction:: gather(\*aws, loop=None, return_exceptions=False) - Run :ref:`awaitable objects ` in the *fs* + Run :ref:`awaitable objects ` in the *aws* sequence *concurrently*. - If any awaitable in *fs* is a coroutine, it is automatically + If any awaitable in *aws* is a coroutine, it is automatically scheduled as a Task. If all awaitables are completed successfully, the result is an aggregate list of returned values. The order of result values - corresponds to the order of awaitables in *fs*. + corresponds to the order of awaitables in *aws*. If *return_exceptions* is ``True``, exceptions are treated the same as successful results, and aggregated in the result list. @@ -273,7 +315,7 @@ Running Tasks Concurrently If ``gather`` is *cancelled*, all submitted awaitables (that have not completed yet) are also *cancelled*. - If any Task or Future from the *fs* sequence is *cancelled*, it is + If any Task or Future from the *aws* sequence is *cancelled*, it is treated as if it raised :exc:`CancelledError` -- the ``gather()`` call is **not** cancelled in this case. This is to prevent the cancellation of one submitted Task/Future to cause other @@ -323,13 +365,13 @@ Running Tasks Concurrently Shielding Tasks From Cancellation ================================= -.. awaitablefunction:: shield(fut, \*, loop=None) +.. awaitablefunction:: shield(aw, \*, loop=None) Protect an :ref:`awaitable object ` from being :meth:`cancelled `. - *fut* can be a coroutine, a Task, or a Future-like object. If - *fut* is a coroutine it is automatically scheduled as a Task. + *aw* can be a coroutine, a Task, or a Future-like object. If + *aw* is a coroutine it is automatically scheduled as a Task. The statement:: @@ -361,12 +403,12 @@ Shielding Tasks From Cancellation Timeouts ======== -.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) +.. coroutinefunction:: wait_for(aw, timeout, \*, loop=None) - Wait for the *fut* :ref:`awaitable ` + Wait for the *aw* :ref:`awaitable ` to complete with a timeout. - If *fut* is a coroutine it is automatically scheduled as a Task. + If *aw* is a coroutine it is automatically scheduled as a Task. *timeout* can either be ``None`` or a float or int number of seconds to wait for. If *timeout* is ``None``, block until the future @@ -381,7 +423,7 @@ Timeouts The function will wait until the future is actually cancelled, so the total wait time may exceed the *timeout*. - If the wait is cancelled, the future *fut* is also cancelled. + If the wait is cancelled, the future *aw* is also cancelled. The *loop* argument is deprecated and scheduled for removal in Python 4.0. @@ -409,22 +451,22 @@ Timeouts # timeout! .. versionchanged:: 3.7 - When *fut* is cancelled due to a timeout, ``wait_for`` waits - for *fut* to be cancelled. Previously, it raised + When *aw* is cancelled due to a timeout, ``wait_for`` waits + for *aw* to be cancelled. Previously, it raised :exc:`asyncio.TimeoutError` immediately. Waiting Primitives ================== -.. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\ +.. coroutinefunction:: wait(aws, \*, loop=None, timeout=None,\ return_when=ALL_COMPLETED) - Run :ref:`awaitable objects ` in the *fs* + Run :ref:`awaitable objects ` in the *aws* sequence concurrently and block until the condition specified by *return_when*. - If any awaitable in *fs* is a coroutine, it is automatically + If any awaitable in *aws* is a coroutine, it is automatically scheduled as a Task. Returns two sets of Tasks/Futures: ``(done, pending)``. @@ -465,12 +507,12 @@ Waiting Primitives Usage:: - done, pending = await asyncio.wait(fs) + done, pending = await asyncio.wait(aws) -.. function:: as_completed(fs, \*, loop=None, timeout=None) +.. function:: as_completed(aws, \*, loop=None, timeout=None) - Run :ref:`awaitable objects ` in the *fs* + Run :ref:`awaitable objects ` in the *aws* set concurrently. Return an iterator of :class:`Future` objects. Each Future object returned represents the earliest result from the set of the remaining awaitables. @@ -480,7 +522,7 @@ Waiting Primitives Example:: - for f in as_completed(fs): + for f in as_completed(aws): earliest_result = await f # ... @@ -670,6 +712,52 @@ Task Object A Task is *done* when the wrapped coroutine either returned a value, raised an exception, or the Task was cancelled. + .. method:: result() + + Return the result of the Task. + + If the Task is *done*, the result of the wrapped coroutine + is returned (or if the coroutine raised an exception, that + exception is re-raised.) + + If the Task has been *cancelled*, this method raises + a :exc:`CancelledError` exception. + + If the Task's result isn't yet available, this method raises + a :exc:`InvalidStateError` exception. + + .. method:: exception() + + Return the exception of the Task. + + If the wrapped coroutine raised an exception that exception + is returned. If the wrapped coroutine returned normally + this method returns ``None``. + + If the Task has been *cancelled*, this method raises a + :exc:`CancelledError` exception. + + If the Task isn't *done* yet, this method raises an + :exc:`InvalidStateError` exception. + + .. method:: add_done_callback(callback, *, context=None) + + Add a callback to be run when the Task is *done*. + + This method should only be used in low-level callback-based code. + + See the documentation of :meth:`Future.add_done_callback` + for more details. + + .. method:: remove_done_callback(callback) + + Remove *callback* from the callbacks list. + + This method should only be used in low-level callback-based code. + + See the documentation of :meth:`Future.remove_done_callback` + for more details. + .. method:: get_stack(\*, limit=None) Return the list of stack frames for this Task. From webhook-mailer at python.org Thu Sep 20 13:19:53 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 20 Sep 2018 17:19:53 -0000 Subject: [Python-checkins] bpo-32215: Fix performance regression in sqlite3 (GH-8511) Message-ID: https://github.com/python/cpython/commit/4fb672ff96ecbb87aaf2ecc4f04aed76aafe63b1 commit: 4fb672ff96ecbb87aaf2ecc4f04aed76aafe63b1 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Berker Peksag date: 2018-09-20T20:19:50+03:00 summary: bpo-32215: Fix performance regression in sqlite3 (GH-8511) (cherry picked from commit 8d1e190fc507a9e304f6817e761e9f628a23cbd8) Co-authored-by: Berker Peksag files: A Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst M Modules/_sqlite/statement.c diff --git a/Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst b/Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst new file mode 100644 index 000000000000..c097cf7310df --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-28-12-08-53.bpo-32215.EU68SY.rst @@ -0,0 +1,2 @@ +Fix performance regression in :mod:`sqlite3` when a DML statement appeared +in a different line than the rest of the SQL query. diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c index 087375be9b63..de265963b084 100644 --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -85,10 +85,10 @@ int pysqlite_statement_create(pysqlite_Statement* self, pysqlite_Connection* con continue; } - self->is_dml = (PyOS_strnicmp(p, "insert ", 7) == 0) - || (PyOS_strnicmp(p, "update ", 7) == 0) - || (PyOS_strnicmp(p, "delete ", 7) == 0) - || (PyOS_strnicmp(p, "replace ", 8) == 0); + self->is_dml = (PyOS_strnicmp(p, "insert", 6) == 0) + || (PyOS_strnicmp(p, "update", 6) == 0) + || (PyOS_strnicmp(p, "delete", 6) == 0) + || (PyOS_strnicmp(p, "replace", 7) == 0); break; } From webhook-mailer at python.org Thu Sep 20 13:20:35 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 20 Sep 2018 17:20:35 -0000 Subject: [Python-checkins] bpo-34743: Fix test_database_source_name under SQLite 3.7.9 (GH-9426) Message-ID: https://github.com/python/cpython/commit/c56bbae5e92828a4c93be3b0e7a5072a31fdc587 commit: c56bbae5e92828a4c93be3b0e7a5072a31fdc587 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Berker Peksag date: 2018-09-20T20:20:32+03:00 summary: bpo-34743: Fix test_database_source_name under SQLite 3.7.9 (GH-9426) (cherry picked from commit b10a64d117de6121ea3e79c467c4107f8f399f3d) Co-authored-by: Berker Peksag files: M Modules/_sqlite/connection.c diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index a5191ed27e80..65f173c8c8af 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -1554,6 +1554,12 @@ pysqlite_connection_backup(pysqlite_Connection *self, PyObject *args, PyObject * PyErr_SetString(pysqlite_OperationalError, sqlite3_errstr(rc)); #else switch (rc) { + case SQLITE_ERROR: + /* Description of SQLITE_ERROR in SQLite 3.7.14 and older + releases. */ + PyErr_SetString(pysqlite_OperationalError, + "SQL logic error or missing database"); + break; case SQLITE_READONLY: PyErr_SetString(pysqlite_OperationalError, "attempt to write a readonly database"); From webhook-mailer at python.org Thu Sep 20 16:38:42 2018 From: webhook-mailer at python.org (Steve Dower) Date: Thu, 20 Sep 2018 20:38:42 -0000 Subject: [Python-checkins] bpo-34011: Fixes missing venv files and other tests (GH-9458) Message-ID: https://github.com/python/cpython/commit/f14c28f39766855420dd58d209da4ad847f3030e commit: f14c28f39766855420dd58d209da4ad847f3030e branch: master author: Steve Dower committer: GitHub date: 2018-09-20T13:38:34-07:00 summary: bpo-34011: Fixes missing venv files and other tests (GH-9458) files: M Lib/distutils/tests/test_bdist.py M Lib/distutils/tests/test_bdist_wininst.py M Lib/test/test_platform.py M Lib/test/test_site.py M Lib/test/test_sysconfig.py M Lib/test/test_venv.py M Lib/venv/__init__.py M Tools/msi/distutils.command.bdist_wininst.py diff --git a/Lib/distutils/tests/test_bdist.py b/Lib/distutils/tests/test_bdist.py index f762f5d98730..c80b3edc0220 100644 --- a/Lib/distutils/tests/test_bdist.py +++ b/Lib/distutils/tests/test_bdist.py @@ -39,6 +39,9 @@ def test_skip_build(self): for name in names: subcmd = cmd.get_finalized_command(name) + if getattr(subcmd, '_unsupported', False): + # command is not supported on this build + continue self.assertTrue(subcmd.skip_build, '%s should take --skip-build from bdist' % name) diff --git a/Lib/distutils/tests/test_bdist_wininst.py b/Lib/distutils/tests/test_bdist_wininst.py index 5d17ab19a9d8..4c19bbab219b 100644 --- a/Lib/distutils/tests/test_bdist_wininst.py +++ b/Lib/distutils/tests/test_bdist_wininst.py @@ -5,6 +5,8 @@ from distutils.command.bdist_wininst import bdist_wininst from distutils.tests import support + at unittest.skipIf(getattr(bdist_wininst, '_unsupported', False), + 'bdist_wininst is not supported in this install') class BuildWinInstTestCase(support.TempdirManager, support.LoggingSilencer, unittest.TestCase): diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py index fd6da313206d..c26e193e6536 100644 --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -3,6 +3,7 @@ import platform import subprocess import sys +import sysconfig import tempfile import unittest import warnings @@ -16,29 +17,34 @@ def test_architecture(self): @support.skip_unless_symlink def test_architecture_via_symlink(self): # issue3762 # On Windows, the EXE needs to know where pythonXY.dll and *.pyd is at - # so we add the directory to the path and PYTHONPATH. + # so we add the directory to the path, PYTHONHOME and PYTHONPATH. + env = None if sys.platform == "win32": - def restore_environ(old_env): - os.environ.clear() - os.environ.update(old_env) - - self.addCleanup(restore_environ, dict(os.environ)) - - os.environ["Path"] = "{};{}".format( - os.path.dirname(sys.executable), os.environ["Path"]) - os.environ["PYTHONPATH"] = os.path.dirname(sys.executable) - - def get(python): + env = {k.upper(): os.environ[k] for k in os.environ} + env["PATH"] = "{};{}".format( + os.path.dirname(sys.executable), env.get("PATH", "")) + env["PYTHONHOME"] = os.path.dirname(sys.executable) + if sysconfig.is_python_build(True): + env["PYTHONPATH"] = os.path.dirname(os.__file__) + + def get(python, env=None): cmd = [python, '-c', 'import platform; print(platform.architecture())'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE) - return p.communicate() + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, env=env) + r = p.communicate() + if p.returncode: + print(repr(r[0])) + print(repr(r[1]), file=sys.stderr) + self.fail('unexpected return code: {0} (0x{0:08X})' + .format(p.returncode)) + return r real = os.path.realpath(sys.executable) link = os.path.abspath(support.TESTFN) os.symlink(real, link) try: - self.assertEqual(get(real), get(link)) + self.assertEqual(get(real), get(link, env=env)) finally: os.remove(link) diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py index 742be1ec03d4..dc59e5917cfa 100644 --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -6,7 +6,8 @@ """ import unittest import test.support -from test.support import captured_stderr, TESTFN, EnvironmentVarGuard +from test.support import (captured_stderr, TESTFN, EnvironmentVarGuard, + change_cwd) import builtins import os import sys @@ -346,40 +347,47 @@ def test_abs_paths(self): # __file__ if abs_paths() does not get run. sys and builtins (the # only other modules imported before site.py runs) do not have # __file__ or __cached__ because they are built-in. - parent = os.path.relpath(os.path.dirname(os.__file__)) - env = os.environ.copy() - env['PYTHONPATH'] = parent - code = ('import os, sys', - # use ASCII to avoid locale issues with non-ASCII directories - 'os_file = os.__file__.encode("ascii", "backslashreplace")', - r'sys.stdout.buffer.write(os_file + b"\n")', - 'os_cached = os.__cached__.encode("ascii", "backslashreplace")', - r'sys.stdout.buffer.write(os_cached + b"\n")') - command = '\n'.join(code) - # First, prove that with -S (no 'import site'), the paths are - # relative. - proc = subprocess.Popen([sys.executable, '-S', '-c', command], - env=env, - stdout=subprocess.PIPE) - stdout, stderr = proc.communicate() - - self.assertEqual(proc.returncode, 0) - os__file__, os__cached__ = stdout.splitlines()[:2] - self.assertFalse(os.path.isabs(os__file__)) - self.assertFalse(os.path.isabs(os__cached__)) - # Now, with 'import site', it works. - proc = subprocess.Popen([sys.executable, '-c', command], - env=env, - stdout=subprocess.PIPE) - stdout, stderr = proc.communicate() - self.assertEqual(proc.returncode, 0) - os__file__, os__cached__ = stdout.splitlines()[:2] - self.assertTrue(os.path.isabs(os__file__), - "expected absolute path, got {}" - .format(os__file__.decode('ascii'))) - self.assertTrue(os.path.isabs(os__cached__), - "expected absolute path, got {}" - .format(os__cached__.decode('ascii'))) + try: + parent = os.path.relpath(os.path.dirname(os.__file__)) + cwd = os.getcwd() + except ValueError: + # Failure to get relpath probably means we need to chdir + # to the same drive. + cwd, parent = os.path.split(os.path.dirname(os.__file__)) + with change_cwd(cwd): + env = os.environ.copy() + env['PYTHONPATH'] = parent + code = ('import os, sys', + # use ASCII to avoid locale issues with non-ASCII directories + 'os_file = os.__file__.encode("ascii", "backslashreplace")', + r'sys.stdout.buffer.write(os_file + b"\n")', + 'os_cached = os.__cached__.encode("ascii", "backslashreplace")', + r'sys.stdout.buffer.write(os_cached + b"\n")') + command = '\n'.join(code) + # First, prove that with -S (no 'import site'), the paths are + # relative. + proc = subprocess.Popen([sys.executable, '-S', '-c', command], + env=env, + stdout=subprocess.PIPE) + stdout, stderr = proc.communicate() + + self.assertEqual(proc.returncode, 0) + os__file__, os__cached__ = stdout.splitlines()[:2] + self.assertFalse(os.path.isabs(os__file__)) + self.assertFalse(os.path.isabs(os__cached__)) + # Now, with 'import site', it works. + proc = subprocess.Popen([sys.executable, '-c', command], + env=env, + stdout=subprocess.PIPE) + stdout, stderr = proc.communicate() + self.assertEqual(proc.returncode, 0) + os__file__, os__cached__ = stdout.splitlines()[:2] + self.assertTrue(os.path.isabs(os__file__), + "expected absolute path, got {}" + .format(os__file__.decode('ascii'))) + self.assertTrue(os.path.isabs(os__cached__), + "expected absolute path, got {}" + .format(os__cached__.decode('ascii'))) def test_no_duplicate_paths(self): # No duplicate paths should exist in sys.path diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py index 20252be0a41b..1b1929885edd 100644 --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -235,21 +235,34 @@ def test_get_scheme_names(self): def test_symlink(self): # On Windows, the EXE needs to know where pythonXY.dll is at so we have # to add the directory to the path. + env = None if sys.platform == "win32": - os.environ["PATH"] = "{};{}".format( - os.path.dirname(sys.executable), os.environ["PATH"]) + env = {k.upper(): os.environ[k] for k in os.environ} + env["PATH"] = "{};{}".format( + os.path.dirname(sys.executable), env.get("PATH", "")) + # Requires PYTHONHOME as well since we locate stdlib from the + # EXE path and not the DLL path (which should be fixed) + env["PYTHONHOME"] = os.path.dirname(sys.executable) + if sysconfig.is_python_build(True): + env["PYTHONPATH"] = os.path.dirname(os.__file__) # Issue 7880 - def get(python): + def get(python, env=None): cmd = [python, '-c', 'import sysconfig; print(sysconfig.get_platform())'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, env=os.environ) - return p.communicate() + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, env=env) + out, err = p.communicate() + if p.returncode: + print((out, err)) + self.fail('Non-zero return code {0} (0x{0:08X})' + .format(p.returncode)) + return out, err real = os.path.realpath(sys.executable) link = os.path.abspath(TESTFN) os.symlink(real, link) try: - self.assertEqual(get(real), get(link)) + self.assertEqual(get(real), get(link, env)) finally: unlink(link) diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py index 9cea87e2fefa..c86f7a1a07ae 100644 --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -27,6 +27,17 @@ skipInVenv = unittest.skipIf(sys.prefix != sys.base_prefix, 'Test not appropriate in a venv') +def check_output(cmd, encoding=None): + p = subprocess.Popen(cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + encoding=encoding) + out, err = p.communicate() + if p.returncode: + raise subprocess.CalledProcessError( + p.returncode, cmd, None, out, err) + return out, err + class BaseTest(unittest.TestCase): """Base class for venv tests.""" maxDiff = 80 * 50 @@ -134,9 +145,7 @@ def test_prefixes(self): ('base_prefix', sys.prefix), ('base_exec_prefix', sys.exec_prefix)): cmd[2] = 'import sys; print(sys.%s)' % prefix - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output(cmd) self.assertEqual(out.strip(), expected.encode()) if sys.platform == 'win32': @@ -259,11 +268,10 @@ def test_executable(self): """ rmtree(self.env_dir) self.run_with_capture(venv.create, self.env_dir) - envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - cmd = [envpy, '-c', 'import sys; print(sys.executable)'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + envpy = os.path.join(os.path.realpath(self.env_dir), + self.bindir, self.exe) + out, err = check_output([envpy, '-c', + 'import sys; print(sys.executable)']) self.assertEqual(out.strip(), envpy.encode()) @unittest.skipUnless(can_symlink(), 'Needs symlinks') @@ -274,17 +282,16 @@ def test_executable_symlinks(self): rmtree(self.env_dir) builder = venv.EnvBuilder(clear=True, symlinks=True) builder.create(self.env_dir) - envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - cmd = [envpy, '-c', 'import sys; print(sys.executable)'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + envpy = os.path.join(os.path.realpath(self.env_dir), + self.bindir, self.exe) + out, err = check_output([envpy, '-c', + 'import sys; print(sys.executable)']) self.assertEqual(out.strip(), envpy.encode()) @unittest.skipUnless(os.name == 'nt', 'only relevant on Windows') def test_unicode_in_batch_file(self): """ - Test isolation from system site-packages + Test handling of Unicode paths """ rmtree(self.env_dir) env_dir = os.path.join(os.path.realpath(self.env_dir), '??????') @@ -292,12 +299,10 @@ def test_unicode_in_batch_file(self): builder.create(env_dir) activate = os.path.join(env_dir, self.bindir, 'activate.bat') envpy = os.path.join(env_dir, self.bindir, self.exe) - cmd = [activate, '&', self.exe, '-c', 'print(0)'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE, encoding='oem', - shell=True) - out, err = p.communicate() - print(err) + out, err = check_output( + [activate, '&', self.exe, '-c', 'print(0)'], + encoding='oem', + ) self.assertEqual(out.strip(), '0') @skipInVenv @@ -306,11 +311,8 @@ class EnsurePipTest(BaseTest): def assert_pip_not_installed(self): envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - try_import = 'try:\n import pip\nexcept ImportError:\n print("OK")' - cmd = [envpy, '-c', try_import] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output([envpy, '-c', + 'try:\n import pip\nexcept ImportError:\n print("OK")']) # We force everything to text, so unittest gives the detailed diff # if we get unexpected results err = err.decode("latin-1") # Force to text, prevent decoding errors @@ -388,11 +390,8 @@ def do_test_with_pip(self, system_site_packages): # Ensure pip is available in the virtual environment envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) # Ignore DeprecationWarning since pip code is not part of Python - cmd = [envpy, '-W', 'ignore::DeprecationWarning', '-I', - '-m', 'pip', '--version'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output([envpy, '-W', 'ignore::DeprecationWarning', '-I', + '-m', 'pip', '--version']) # We force everything to text, so unittest gives the detailed diff # if we get unexpected results err = err.decode("latin-1") # Force to text, prevent decoding errors @@ -406,12 +405,10 @@ def do_test_with_pip(self, system_site_packages): # http://bugs.python.org/issue19728 # Check the private uninstall command provided for the Windows # installers works (at least in a virtual environment) - cmd = [envpy, '-W', 'ignore::DeprecationWarning', '-I', - '-m', 'ensurepip._uninstall'] with EnvironmentVarGuard() as envvars: - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output([envpy, + '-W', 'ignore::DeprecationWarning', '-I', + '-m', 'ensurepip._uninstall']) # We force everything to text, so unittest gives the detailed diff # if we get unexpected results err = err.decode("latin-1") # Force to text, prevent decoding errors diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py index e0ab241f77c5..043420897e47 100644 --- a/Lib/venv/__init__.py +++ b/Lib/venv/__init__.py @@ -208,11 +208,9 @@ def setup_python(self, context): copier(context.env_exe, path, relative_symlinks_ok=True) if not os.path.islink(path): os.chmod(path, 0o755) - elif sysconfig.is_python_build(True): - # See bpo-34011. This copying code should only be needed when a - # venv is created from a source Python build (i.e. not an installed - # Python) - subdir = 'DLLs' + else: + # See bpo-34011. When using a proper install, we should only need to + # copy the top-level of DLLs. include = self.include_binary files = [f for f in os.listdir(dirname) if include(f)] for f in files: @@ -220,24 +218,28 @@ def setup_python(self, context): dst = os.path.join(binpath, f) if dst != context.env_exe: # already done, above copier(src, dst) - dirname = os.path.join(dirname, subdir) - if os.path.isdir(dirname): - files = [f for f in os.listdir(dirname) if include(f)] - for f in files: - src = os.path.join(dirname, f) - dst = os.path.join(binpath, f) - copier(src, dst) - # copy init.tcl over - for root, dirs, files in os.walk(context.python_dir): - if 'init.tcl' in files: - tcldir = os.path.basename(root) - tcldir = os.path.join(context.env_dir, 'Lib', tcldir) - if not os.path.exists(tcldir): - os.makedirs(tcldir) - src = os.path.join(root, 'init.tcl') - dst = os.path.join(tcldir, 'init.tcl') - shutil.copyfile(src, dst) - break + + # When creating from a build directory, we continue to copy all files. + if sysconfig.is_python_build(True): + subdir = 'DLLs' + dirname = os.path.join(dirname, subdir) + if os.path.isdir(dirname): + files = [f for f in os.listdir(dirname) if include(f)] + for f in files: + src = os.path.join(dirname, f) + dst = os.path.join(binpath, f) + copier(src, dst) + # copy init.tcl over + for root, dirs, files in os.walk(context.python_dir): + if 'init.tcl' in files: + tcldir = os.path.basename(root) + tcldir = os.path.join(context.env_dir, 'Lib', tcldir) + if not os.path.exists(tcldir): + os.makedirs(tcldir) + src = os.path.join(root, 'init.tcl') + dst = os.path.join(tcldir, 'init.tcl') + shutil.copyfile(src, dst) + break def _setup_pip(self, context): """Installs or upgrades pip in a virtual environment""" diff --git a/Tools/msi/distutils.command.bdist_wininst.py b/Tools/msi/distutils.command.bdist_wininst.py index d586e34fec82..548fdd0cc432 100644 --- a/Tools/msi/distutils.command.bdist_wininst.py +++ b/Tools/msi/distutils.command.bdist_wininst.py @@ -9,6 +9,9 @@ class bdist_wininst(Command): description = "create an executable installer for MS Windows" + # Marker for tests that we have the unsupported bdist_wininst + _unsupported = True + def initialize_options(self): pass From webhook-mailer at python.org Thu Sep 20 17:39:28 2018 From: webhook-mailer at python.org (Steve Dower) Date: Thu, 20 Sep 2018 21:39:28 -0000 Subject: [Python-checkins] Fixes tests requiring extra environment values on Windows (GH-9462) Message-ID: https://github.com/python/cpython/commit/a73e790c7894ab68311aef296074518db4778b46 commit: a73e790c7894ab68311aef296074518db4778b46 branch: 3.7 author: Steve Dower committer: GitHub date: 2018-09-20T14:39:21-07:00 summary: Fixes tests requiring extra environment values on Windows (GH-9462) files: M Lib/distutils/tests/test_bdist.py M Lib/distutils/tests/test_bdist_wininst.py M Lib/test/test_platform.py M Lib/test/test_site.py M Lib/test/test_sysconfig.py M Lib/test/test_venv.py M Tools/msi/distutils.command.bdist_wininst.py diff --git a/Lib/distutils/tests/test_bdist.py b/Lib/distutils/tests/test_bdist.py index f762f5d98730..c80b3edc0220 100644 --- a/Lib/distutils/tests/test_bdist.py +++ b/Lib/distutils/tests/test_bdist.py @@ -39,6 +39,9 @@ def test_skip_build(self): for name in names: subcmd = cmd.get_finalized_command(name) + if getattr(subcmd, '_unsupported', False): + # command is not supported on this build + continue self.assertTrue(subcmd.skip_build, '%s should take --skip-build from bdist' % name) diff --git a/Lib/distutils/tests/test_bdist_wininst.py b/Lib/distutils/tests/test_bdist_wininst.py index 5d17ab19a9d8..4c19bbab219b 100644 --- a/Lib/distutils/tests/test_bdist_wininst.py +++ b/Lib/distutils/tests/test_bdist_wininst.py @@ -5,6 +5,8 @@ from distutils.command.bdist_wininst import bdist_wininst from distutils.tests import support + at unittest.skipIf(getattr(bdist_wininst, '_unsupported', False), + 'bdist_wininst is not supported in this install') class BuildWinInstTestCase(support.TempdirManager, support.LoggingSilencer, unittest.TestCase): diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py index 509d8865c068..010ed6c6342b 100644 --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -3,6 +3,7 @@ import platform import subprocess import sys +import sysconfig import tempfile import unittest import warnings @@ -16,29 +17,34 @@ def test_architecture(self): @support.skip_unless_symlink def test_architecture_via_symlink(self): # issue3762 # On Windows, the EXE needs to know where pythonXY.dll and *.pyd is at - # so we add the directory to the path and PYTHONPATH. + # so we add the directory to the path, PYTHONHOME and PYTHONPATH. + env = None if sys.platform == "win32": - def restore_environ(old_env): - os.environ.clear() - os.environ.update(old_env) - - self.addCleanup(restore_environ, dict(os.environ)) - - os.environ["Path"] = "{};{}".format( - os.path.dirname(sys.executable), os.environ["Path"]) - os.environ["PYTHONPATH"] = os.path.dirname(sys.executable) - - def get(python): + env = {k.upper(): os.environ[k] for k in os.environ} + env["PATH"] = "{};{}".format( + os.path.dirname(sys.executable), env.get("PATH", "")) + env["PYTHONHOME"] = os.path.dirname(sys.executable) + if sysconfig.is_python_build(True): + env["PYTHONPATH"] = os.path.dirname(os.__file__) + + def get(python, env=None): cmd = [python, '-c', 'import platform; print(platform.architecture())'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE) - return p.communicate() + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, env=env) + r = p.communicate() + if p.returncode: + print(repr(r[0])) + print(repr(r[1]), file=sys.stderr) + self.fail('unexpected return code: {0} (0x{0:08X})' + .format(p.returncode)) + return r real = os.path.realpath(sys.executable) link = os.path.abspath(support.TESTFN) os.symlink(real, link) try: - self.assertEqual(get(real), get(link)) + self.assertEqual(get(real), get(link, env=env)) finally: os.remove(link) diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py index e3c9deebf08c..6cea58d934fb 100644 --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -6,7 +6,8 @@ """ import unittest import test.support -from test.support import captured_stderr, TESTFN, EnvironmentVarGuard +from test.support import (captured_stderr, TESTFN, EnvironmentVarGuard, + change_cwd) import builtins import os import sys @@ -348,40 +349,47 @@ def test_abs_paths(self): # __file__ if abs_paths() does not get run. sys and builtins (the # only other modules imported before site.py runs) do not have # __file__ or __cached__ because they are built-in. - parent = os.path.relpath(os.path.dirname(os.__file__)) - env = os.environ.copy() - env['PYTHONPATH'] = parent - code = ('import os, sys', - # use ASCII to avoid locale issues with non-ASCII directories - 'os_file = os.__file__.encode("ascii", "backslashreplace")', - r'sys.stdout.buffer.write(os_file + b"\n")', - 'os_cached = os.__cached__.encode("ascii", "backslashreplace")', - r'sys.stdout.buffer.write(os_cached + b"\n")') - command = '\n'.join(code) - # First, prove that with -S (no 'import site'), the paths are - # relative. - proc = subprocess.Popen([sys.executable, '-S', '-c', command], - env=env, - stdout=subprocess.PIPE) - stdout, stderr = proc.communicate() - - self.assertEqual(proc.returncode, 0) - os__file__, os__cached__ = stdout.splitlines()[:2] - self.assertFalse(os.path.isabs(os__file__)) - self.assertFalse(os.path.isabs(os__cached__)) - # Now, with 'import site', it works. - proc = subprocess.Popen([sys.executable, '-c', command], - env=env, - stdout=subprocess.PIPE) - stdout, stderr = proc.communicate() - self.assertEqual(proc.returncode, 0) - os__file__, os__cached__ = stdout.splitlines()[:2] - self.assertTrue(os.path.isabs(os__file__), - "expected absolute path, got {}" - .format(os__file__.decode('ascii'))) - self.assertTrue(os.path.isabs(os__cached__), - "expected absolute path, got {}" - .format(os__cached__.decode('ascii'))) + try: + parent = os.path.relpath(os.path.dirname(os.__file__)) + cwd = os.getcwd() + except ValueError: + # Failure to get relpath probably means we need to chdir + # to the same drive. + cwd, parent = os.path.split(os.path.dirname(os.__file__)) + with change_cwd(cwd): + env = os.environ.copy() + env['PYTHONPATH'] = parent + code = ('import os, sys', + # use ASCII to avoid locale issues with non-ASCII directories + 'os_file = os.__file__.encode("ascii", "backslashreplace")', + r'sys.stdout.buffer.write(os_file + b"\n")', + 'os_cached = os.__cached__.encode("ascii", "backslashreplace")', + r'sys.stdout.buffer.write(os_cached + b"\n")') + command = '\n'.join(code) + # First, prove that with -S (no 'import site'), the paths are + # relative. + proc = subprocess.Popen([sys.executable, '-S', '-c', command], + env=env, + stdout=subprocess.PIPE) + stdout, stderr = proc.communicate() + + self.assertEqual(proc.returncode, 0) + os__file__, os__cached__ = stdout.splitlines()[:2] + self.assertFalse(os.path.isabs(os__file__)) + self.assertFalse(os.path.isabs(os__cached__)) + # Now, with 'import site', it works. + proc = subprocess.Popen([sys.executable, '-c', command], + env=env, + stdout=subprocess.PIPE) + stdout, stderr = proc.communicate() + self.assertEqual(proc.returncode, 0) + os__file__, os__cached__ = stdout.splitlines()[:2] + self.assertTrue(os.path.isabs(os__file__), + "expected absolute path, got {}" + .format(os__file__.decode('ascii'))) + self.assertTrue(os.path.isabs(os__cached__), + "expected absolute path, got {}" + .format(os__cached__.decode('ascii'))) def test_no_duplicate_paths(self): # No duplicate paths should exist in sys.path diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py index 20252be0a41b..1b1929885edd 100644 --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -235,21 +235,34 @@ def test_get_scheme_names(self): def test_symlink(self): # On Windows, the EXE needs to know where pythonXY.dll is at so we have # to add the directory to the path. + env = None if sys.platform == "win32": - os.environ["PATH"] = "{};{}".format( - os.path.dirname(sys.executable), os.environ["PATH"]) + env = {k.upper(): os.environ[k] for k in os.environ} + env["PATH"] = "{};{}".format( + os.path.dirname(sys.executable), env.get("PATH", "")) + # Requires PYTHONHOME as well since we locate stdlib from the + # EXE path and not the DLL path (which should be fixed) + env["PYTHONHOME"] = os.path.dirname(sys.executable) + if sysconfig.is_python_build(True): + env["PYTHONPATH"] = os.path.dirname(os.__file__) # Issue 7880 - def get(python): + def get(python, env=None): cmd = [python, '-c', 'import sysconfig; print(sysconfig.get_platform())'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, env=os.environ) - return p.communicate() + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, env=env) + out, err = p.communicate() + if p.returncode: + print((out, err)) + self.fail('Non-zero return code {0} (0x{0:08X})' + .format(p.returncode)) + return out, err real = os.path.realpath(sys.executable) link = os.path.abspath(TESTFN) os.symlink(real, link) try: - self.assertEqual(get(real), get(link)) + self.assertEqual(get(real), get(link, env)) finally: unlink(link) diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py index 9cea87e2fefa..c86f7a1a07ae 100644 --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -27,6 +27,17 @@ skipInVenv = unittest.skipIf(sys.prefix != sys.base_prefix, 'Test not appropriate in a venv') +def check_output(cmd, encoding=None): + p = subprocess.Popen(cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + encoding=encoding) + out, err = p.communicate() + if p.returncode: + raise subprocess.CalledProcessError( + p.returncode, cmd, None, out, err) + return out, err + class BaseTest(unittest.TestCase): """Base class for venv tests.""" maxDiff = 80 * 50 @@ -134,9 +145,7 @@ def test_prefixes(self): ('base_prefix', sys.prefix), ('base_exec_prefix', sys.exec_prefix)): cmd[2] = 'import sys; print(sys.%s)' % prefix - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output(cmd) self.assertEqual(out.strip(), expected.encode()) if sys.platform == 'win32': @@ -259,11 +268,10 @@ def test_executable(self): """ rmtree(self.env_dir) self.run_with_capture(venv.create, self.env_dir) - envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - cmd = [envpy, '-c', 'import sys; print(sys.executable)'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + envpy = os.path.join(os.path.realpath(self.env_dir), + self.bindir, self.exe) + out, err = check_output([envpy, '-c', + 'import sys; print(sys.executable)']) self.assertEqual(out.strip(), envpy.encode()) @unittest.skipUnless(can_symlink(), 'Needs symlinks') @@ -274,17 +282,16 @@ def test_executable_symlinks(self): rmtree(self.env_dir) builder = venv.EnvBuilder(clear=True, symlinks=True) builder.create(self.env_dir) - envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - cmd = [envpy, '-c', 'import sys; print(sys.executable)'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + envpy = os.path.join(os.path.realpath(self.env_dir), + self.bindir, self.exe) + out, err = check_output([envpy, '-c', + 'import sys; print(sys.executable)']) self.assertEqual(out.strip(), envpy.encode()) @unittest.skipUnless(os.name == 'nt', 'only relevant on Windows') def test_unicode_in_batch_file(self): """ - Test isolation from system site-packages + Test handling of Unicode paths """ rmtree(self.env_dir) env_dir = os.path.join(os.path.realpath(self.env_dir), '??????') @@ -292,12 +299,10 @@ def test_unicode_in_batch_file(self): builder.create(env_dir) activate = os.path.join(env_dir, self.bindir, 'activate.bat') envpy = os.path.join(env_dir, self.bindir, self.exe) - cmd = [activate, '&', self.exe, '-c', 'print(0)'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE, encoding='oem', - shell=True) - out, err = p.communicate() - print(err) + out, err = check_output( + [activate, '&', self.exe, '-c', 'print(0)'], + encoding='oem', + ) self.assertEqual(out.strip(), '0') @skipInVenv @@ -306,11 +311,8 @@ class EnsurePipTest(BaseTest): def assert_pip_not_installed(self): envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - try_import = 'try:\n import pip\nexcept ImportError:\n print("OK")' - cmd = [envpy, '-c', try_import] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output([envpy, '-c', + 'try:\n import pip\nexcept ImportError:\n print("OK")']) # We force everything to text, so unittest gives the detailed diff # if we get unexpected results err = err.decode("latin-1") # Force to text, prevent decoding errors @@ -388,11 +390,8 @@ def do_test_with_pip(self, system_site_packages): # Ensure pip is available in the virtual environment envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) # Ignore DeprecationWarning since pip code is not part of Python - cmd = [envpy, '-W', 'ignore::DeprecationWarning', '-I', - '-m', 'pip', '--version'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output([envpy, '-W', 'ignore::DeprecationWarning', '-I', + '-m', 'pip', '--version']) # We force everything to text, so unittest gives the detailed diff # if we get unexpected results err = err.decode("latin-1") # Force to text, prevent decoding errors @@ -406,12 +405,10 @@ def do_test_with_pip(self, system_site_packages): # http://bugs.python.org/issue19728 # Check the private uninstall command provided for the Windows # installers works (at least in a virtual environment) - cmd = [envpy, '-W', 'ignore::DeprecationWarning', '-I', - '-m', 'ensurepip._uninstall'] with EnvironmentVarGuard() as envvars: - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output([envpy, + '-W', 'ignore::DeprecationWarning', '-I', + '-m', 'ensurepip._uninstall']) # We force everything to text, so unittest gives the detailed diff # if we get unexpected results err = err.decode("latin-1") # Force to text, prevent decoding errors diff --git a/Tools/msi/distutils.command.bdist_wininst.py b/Tools/msi/distutils.command.bdist_wininst.py index d586e34fec82..548fdd0cc432 100644 --- a/Tools/msi/distutils.command.bdist_wininst.py +++ b/Tools/msi/distutils.command.bdist_wininst.py @@ -9,6 +9,9 @@ class bdist_wininst(Command): description = "create an executable installer for MS Windows" + # Marker for tests that we have the unsupported bdist_wininst + _unsupported = True + def initialize_options(self): pass From webhook-mailer at python.org Thu Sep 20 19:10:50 2018 From: webhook-mailer at python.org (Steve Dower) Date: Thu, 20 Sep 2018 23:10:50 -0000 Subject: [Python-checkins] Fixes tests requiring extra environment values on Windows (GH-9463) Message-ID: https://github.com/python/cpython/commit/6c9e109c36594fa24dc2b6c881a55c5ac33307ea commit: 6c9e109c36594fa24dc2b6c881a55c5ac33307ea branch: 3.6 author: Steve Dower committer: GitHub date: 2018-09-20T16:10:40-07:00 summary: Fixes tests requiring extra environment values on Windows (GH-9463) files: M Lib/distutils/tests/test_bdist.py M Lib/distutils/tests/test_bdist_wininst.py M Lib/test/test_platform.py M Lib/test/test_site.py M Lib/test/test_sysconfig.py M Lib/test/test_venv.py M Tools/msi/distutils.command.bdist_wininst.py diff --git a/Lib/distutils/tests/test_bdist.py b/Lib/distutils/tests/test_bdist.py index f762f5d98730..c80b3edc0220 100644 --- a/Lib/distutils/tests/test_bdist.py +++ b/Lib/distutils/tests/test_bdist.py @@ -39,6 +39,9 @@ def test_skip_build(self): for name in names: subcmd = cmd.get_finalized_command(name) + if getattr(subcmd, '_unsupported', False): + # command is not supported on this build + continue self.assertTrue(subcmd.skip_build, '%s should take --skip-build from bdist' % name) diff --git a/Lib/distutils/tests/test_bdist_wininst.py b/Lib/distutils/tests/test_bdist_wininst.py index 5d17ab19a9d8..4c19bbab219b 100644 --- a/Lib/distutils/tests/test_bdist_wininst.py +++ b/Lib/distutils/tests/test_bdist_wininst.py @@ -5,6 +5,8 @@ from distutils.command.bdist_wininst import bdist_wininst from distutils.tests import support + at unittest.skipIf(getattr(bdist_wininst, '_unsupported', False), + 'bdist_wininst is not supported in this install') class BuildWinInstTestCase(support.TempdirManager, support.LoggingSilencer, unittest.TestCase): diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py index c8ba7ec35947..371cf0ad5e4a 100644 --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -3,6 +3,7 @@ import platform import subprocess import sys +import sysconfig import tempfile import unittest import warnings @@ -16,29 +17,34 @@ def test_architecture(self): @support.skip_unless_symlink def test_architecture_via_symlink(self): # issue3762 # On Windows, the EXE needs to know where pythonXY.dll and *.pyd is at - # so we add the directory to the path and PYTHONPATH. + # so we add the directory to the path, PYTHONHOME and PYTHONPATH. + env = None if sys.platform == "win32": - def restore_environ(old_env): - os.environ.clear() - os.environ.update(old_env) - - self.addCleanup(restore_environ, dict(os.environ)) - - os.environ["Path"] = "{};{}".format( - os.path.dirname(sys.executable), os.environ["Path"]) - os.environ["PYTHONPATH"] = os.path.dirname(sys.executable) - - def get(python): + env = {k.upper(): os.environ[k] for k in os.environ} + env["PATH"] = "{};{}".format( + os.path.dirname(sys.executable), env.get("PATH", "")) + env["PYTHONHOME"] = os.path.dirname(sys.executable) + if sysconfig.is_python_build(True): + env["PYTHONPATH"] = os.path.dirname(os.__file__) + + def get(python, env=None): cmd = [python, '-c', 'import platform; print(platform.architecture())'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE) - return p.communicate() + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, env=env) + r = p.communicate() + if p.returncode: + print(repr(r[0])) + print(repr(r[1]), file=sys.stderr) + self.fail('unexpected return code: {0} (0x{0:08X})' + .format(p.returncode)) + return r real = os.path.realpath(sys.executable) link = os.path.abspath(support.TESTFN) os.symlink(real, link) try: - self.assertEqual(get(real), get(link)) + self.assertEqual(get(real), get(link, env=env)) finally: os.remove(link) diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py index dce2aac909de..afdcf36f394d 100644 --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -6,7 +6,8 @@ """ import unittest import test.support -from test.support import captured_stderr, TESTFN, EnvironmentVarGuard +from test.support import (captured_stderr, TESTFN, EnvironmentVarGuard, + change_cwd) import builtins import os import sys @@ -345,40 +346,47 @@ def test_abs_paths(self): # __file__ if abs_paths() does not get run. sys and builtins (the # only other modules imported before site.py runs) do not have # __file__ or __cached__ because they are built-in. - parent = os.path.relpath(os.path.dirname(os.__file__)) - env = os.environ.copy() - env['PYTHONPATH'] = parent - code = ('import os, sys', - # use ASCII to avoid locale issues with non-ASCII directories - 'os_file = os.__file__.encode("ascii", "backslashreplace")', - r'sys.stdout.buffer.write(os_file + b"\n")', - 'os_cached = os.__cached__.encode("ascii", "backslashreplace")', - r'sys.stdout.buffer.write(os_cached + b"\n")') - command = '\n'.join(code) - # First, prove that with -S (no 'import site'), the paths are - # relative. - proc = subprocess.Popen([sys.executable, '-S', '-c', command], - env=env, - stdout=subprocess.PIPE) - stdout, stderr = proc.communicate() - - self.assertEqual(proc.returncode, 0) - os__file__, os__cached__ = stdout.splitlines()[:2] - self.assertFalse(os.path.isabs(os__file__)) - self.assertFalse(os.path.isabs(os__cached__)) - # Now, with 'import site', it works. - proc = subprocess.Popen([sys.executable, '-c', command], - env=env, - stdout=subprocess.PIPE) - stdout, stderr = proc.communicate() - self.assertEqual(proc.returncode, 0) - os__file__, os__cached__ = stdout.splitlines()[:2] - self.assertTrue(os.path.isabs(os__file__), - "expected absolute path, got {}" - .format(os__file__.decode('ascii'))) - self.assertTrue(os.path.isabs(os__cached__), - "expected absolute path, got {}" - .format(os__cached__.decode('ascii'))) + try: + parent = os.path.relpath(os.path.dirname(os.__file__)) + cwd = os.getcwd() + except ValueError: + # Failure to get relpath probably means we need to chdir + # to the same drive. + cwd, parent = os.path.split(os.path.dirname(os.__file__)) + with change_cwd(cwd): + env = os.environ.copy() + env['PYTHONPATH'] = parent + code = ('import os, sys', + # use ASCII to avoid locale issues with non-ASCII directories + 'os_file = os.__file__.encode("ascii", "backslashreplace")', + r'sys.stdout.buffer.write(os_file + b"\n")', + 'os_cached = os.__cached__.encode("ascii", "backslashreplace")', + r'sys.stdout.buffer.write(os_cached + b"\n")') + command = '\n'.join(code) + # First, prove that with -S (no 'import site'), the paths are + # relative. + proc = subprocess.Popen([sys.executable, '-S', '-c', command], + env=env, + stdout=subprocess.PIPE) + stdout, stderr = proc.communicate() + + self.assertEqual(proc.returncode, 0) + os__file__, os__cached__ = stdout.splitlines()[:2] + self.assertFalse(os.path.isabs(os__file__)) + self.assertFalse(os.path.isabs(os__cached__)) + # Now, with 'import site', it works. + proc = subprocess.Popen([sys.executable, '-c', command], + env=env, + stdout=subprocess.PIPE) + stdout, stderr = proc.communicate() + self.assertEqual(proc.returncode, 0) + os__file__, os__cached__ = stdout.splitlines()[:2] + self.assertTrue(os.path.isabs(os__file__), + "expected absolute path, got {}" + .format(os__file__.decode('ascii'))) + self.assertTrue(os.path.isabs(os__cached__), + "expected absolute path, got {}" + .format(os__cached__.decode('ascii'))) def test_no_duplicate_paths(self): # No duplicate paths should exist in sys.path diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py index a29ca96d1fc4..90e671908d95 100644 --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -243,21 +243,34 @@ def test_get_scheme_names(self): def test_symlink(self): # On Windows, the EXE needs to know where pythonXY.dll is at so we have # to add the directory to the path. + env = None if sys.platform == "win32": - os.environ["PATH"] = "{};{}".format( - os.path.dirname(sys.executable), os.environ["PATH"]) + env = {k.upper(): os.environ[k] for k in os.environ} + env["PATH"] = "{};{}".format( + os.path.dirname(sys.executable), env.get("PATH", "")) + # Requires PYTHONHOME as well since we locate stdlib from the + # EXE path and not the DLL path (which should be fixed) + env["PYTHONHOME"] = os.path.dirname(sys.executable) + if sysconfig.is_python_build(True): + env["PYTHONPATH"] = os.path.dirname(os.__file__) # Issue 7880 - def get(python): + def get(python, env=None): cmd = [python, '-c', 'import sysconfig; print(sysconfig.get_platform())'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, env=os.environ) - return p.communicate() + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, env=env) + out, err = p.communicate() + if p.returncode: + print((out, err)) + self.fail('Non-zero return code {0} (0x{0:08X})' + .format(p.returncode)) + return out, err real = os.path.realpath(sys.executable) link = os.path.abspath(TESTFN) os.symlink(real, link) try: - self.assertEqual(get(real), get(link)) + self.assertEqual(get(real), get(link, env)) finally: unlink(link) diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py index 2569a1fe49ef..d1ffb8132d81 100644 --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -32,6 +32,17 @@ skipInVenv = unittest.skipIf(sys.prefix != sys.base_prefix, 'Test not appropriate in a venv') +def check_output(cmd, encoding=None): + p = subprocess.Popen(cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + encoding=encoding) + out, err = p.communicate() + if p.returncode: + raise subprocess.CalledProcessError( + p.returncode, cmd, None, out, err) + return out, err + class BaseTest(unittest.TestCase): """Base class for venv tests.""" maxDiff = 80 * 50 @@ -139,9 +150,7 @@ def test_prefixes(self): ('base_prefix', sys.prefix), ('base_exec_prefix', sys.exec_prefix)): cmd[2] = 'import sys; print(sys.%s)' % prefix - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output(cmd) self.assertEqual(out.strip(), expected.encode()) if sys.platform == 'win32': @@ -264,11 +273,10 @@ def test_executable(self): """ rmtree(self.env_dir) self.run_with_capture(venv.create, self.env_dir) - envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - cmd = [envpy, '-c', 'import sys; print(sys.executable)'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + envpy = os.path.join(os.path.realpath(self.env_dir), + self.bindir, self.exe) + out, err = check_output([envpy, '-c', + 'import sys; print(sys.executable)']) self.assertEqual(out.strip(), envpy.encode()) @unittest.skipUnless(can_symlink(), 'Needs symlinks') @@ -279,17 +287,16 @@ def test_executable_symlinks(self): rmtree(self.env_dir) builder = venv.EnvBuilder(clear=True, symlinks=True) builder.create(self.env_dir) - envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - cmd = [envpy, '-c', 'import sys; print(sys.executable)'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + envpy = os.path.join(os.path.realpath(self.env_dir), + self.bindir, self.exe) + out, err = check_output([envpy, '-c', + 'import sys; print(sys.executable)']) self.assertEqual(out.strip(), envpy.encode()) @unittest.skipUnless(os.name == 'nt', 'only relevant on Windows') def test_unicode_in_batch_file(self): """ - Test isolation from system site-packages + Test handling of Unicode paths """ rmtree(self.env_dir) env_dir = os.path.join(os.path.realpath(self.env_dir), '??????') @@ -297,12 +304,10 @@ def test_unicode_in_batch_file(self): builder.create(env_dir) activate = os.path.join(env_dir, self.bindir, 'activate.bat') envpy = os.path.join(env_dir, self.bindir, self.exe) - cmd = [activate, '&', self.exe, '-c', 'print(0)'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE, encoding='oem', - shell=True) - out, err = p.communicate() - print(err) + out, err = check_output( + [activate, '&', self.exe, '-c', 'print(0)'], + encoding='oem', + ) self.assertEqual(out.strip(), '0') @skipInVenv @@ -311,11 +316,8 @@ class EnsurePipTest(BaseTest): def assert_pip_not_installed(self): envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - try_import = 'try:\n import pip\nexcept ImportError:\n print("OK")' - cmd = [envpy, '-c', try_import] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output([envpy, '-c', + 'try:\n import pip\nexcept ImportError:\n print("OK")']) # We force everything to text, so unittest gives the detailed diff # if we get unexpected results err = err.decode("latin-1") # Force to text, prevent decoding errors @@ -392,10 +394,7 @@ def do_test_with_pip(self, system_site_packages): self.fail(msg.format(exc, details)) # Ensure pip is available in the virtual environment envpy = os.path.join(os.path.realpath(self.env_dir), self.bindir, self.exe) - cmd = [envpy, '-Im', 'pip', '--version'] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output([envpy, '-Im', 'pip', '--version']) # We force everything to text, so unittest gives the detailed diff # if we get unexpected results err = err.decode("latin-1") # Force to text, prevent decoding errors @@ -409,11 +408,8 @@ def do_test_with_pip(self, system_site_packages): # http://bugs.python.org/issue19728 # Check the private uninstall command provided for the Windows # installers works (at least in a virtual environment) - cmd = [envpy, '-Im', 'ensurepip._uninstall'] with EnvironmentVarGuard() as envvars: - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - out, err = p.communicate() + out, err = check_output([envpy, '-Im', 'ensurepip._uninstall']) # We force everything to text, so unittest gives the detailed diff # if we get unexpected results err = err.decode("latin-1") # Force to text, prevent decoding errors diff --git a/Tools/msi/distutils.command.bdist_wininst.py b/Tools/msi/distutils.command.bdist_wininst.py index d586e34fec82..548fdd0cc432 100644 --- a/Tools/msi/distutils.command.bdist_wininst.py +++ b/Tools/msi/distutils.command.bdist_wininst.py @@ -9,6 +9,9 @@ class bdist_wininst(Command): description = "create an executable installer for MS Windows" + # Marker for tests that we have the unsupported bdist_wininst + _unsupported = True + def initialize_options(self): pass From webhook-mailer at python.org Thu Sep 20 21:36:49 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 21 Sep 2018 01:36:49 -0000 Subject: [Python-checkins] closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261) Message-ID: https://github.com/python/cpython/commit/a4ae828ee416a66d8c7bf5ee71d653c2cc6a26dd commit: a4ae828ee416a66d8c7bf5ee71d653c2cc6a26dd branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-20T18:36:40-07:00 summary: closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261) files: M Modules/_pickle.c diff --git a/Modules/_pickle.c b/Modules/_pickle.c index 2de70f5d9405..3588e33f0971 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -602,9 +602,9 @@ typedef struct { } PyMemoEntry; typedef struct { - Py_ssize_t mt_mask; - Py_ssize_t mt_used; - Py_ssize_t mt_allocated; + size_t mt_mask; + size_t mt_used; + size_t mt_allocated; PyMemoEntry *mt_table; } PyMemoTable; @@ -650,8 +650,8 @@ typedef struct UnpicklerObject { /* The unpickler memo is just an array of PyObject *s. Using a dict is unnecessary, since the keys are contiguous ints. */ PyObject **memo; - Py_ssize_t memo_size; /* Capacity of the memo array */ - Py_ssize_t memo_len; /* Number of objects in the memo */ + size_t memo_size; /* Capacity of the memo array */ + size_t memo_len; /* Number of objects in the memo */ PyObject *pers_func; /* persistent_load() method, can be NULL. */ PyObject *pers_func_self; /* borrowed reference to self if pers_func @@ -737,7 +737,6 @@ PyMemoTable_New(void) static PyMemoTable * PyMemoTable_Copy(PyMemoTable *self) { - Py_ssize_t i; PyMemoTable *new = PyMemoTable_New(); if (new == NULL) return NULL; @@ -754,7 +753,7 @@ PyMemoTable_Copy(PyMemoTable *self) PyErr_NoMemory(); return NULL; } - for (i = 0; i < self->mt_allocated; i++) { + for (size_t i = 0; i < self->mt_allocated; i++) { Py_XINCREF(self->mt_table[i].me_key); } memcpy(new->mt_table, self->mt_table, @@ -800,7 +799,7 @@ _PyMemoTable_Lookup(PyMemoTable *self, PyObject *key) { size_t i; size_t perturb; - size_t mask = (size_t)self->mt_mask; + size_t mask = self->mt_mask; PyMemoEntry *table = self->mt_table; PyMemoEntry *entry; Py_hash_t hash = (Py_hash_t)key >> 3; @@ -821,22 +820,24 @@ _PyMemoTable_Lookup(PyMemoTable *self, PyObject *key) /* Returns -1 on failure, 0 on success. */ static int -_PyMemoTable_ResizeTable(PyMemoTable *self, Py_ssize_t min_size) +_PyMemoTable_ResizeTable(PyMemoTable *self, size_t min_size) { PyMemoEntry *oldtable = NULL; PyMemoEntry *oldentry, *newentry; - Py_ssize_t new_size = MT_MINSIZE; - Py_ssize_t to_process; + size_t new_size = MT_MINSIZE; + size_t to_process; assert(min_size > 0); - /* Find the smallest valid table size >= min_size. */ - while (new_size < min_size && new_size > 0) - new_size <<= 1; - if (new_size <= 0) { + if (min_size > PY_SSIZE_T_MAX) { PyErr_NoMemory(); return -1; } + + /* Find the smallest valid table size >= min_size. */ + while (new_size < min_size) { + new_size <<= 1; + } /* new_size needs to be a power of two. */ assert((new_size & (new_size - 1)) == 0); @@ -909,10 +910,12 @@ PyMemoTable_Set(PyMemoTable *self, PyObject *key, Py_ssize_t value) * Very large memo tables (over 50K items) use doubling instead. * This may help applications with severe memory constraints. */ - if (!(self->mt_used * 3 >= (self->mt_mask + 1) * 2)) + if (SIZE_MAX / 3 >= self->mt_used && self->mt_used * 3 < self->mt_allocated * 2) { return 0; - return _PyMemoTable_ResizeTable(self, - (self->mt_used > 50000 ? 2 : 4) * self->mt_used); + } + // self->mt_used is always < PY_SSIZE_T_MAX, so this can't overflow. + size_t desired_size = (self->mt_used > 50000 ? 2 : 4) * self->mt_used; + return _PyMemoTable_ResizeTable(self, desired_size); } #undef MT_MINSIZE @@ -1376,9 +1379,9 @@ _Unpickler_Readline(UnpicklerObject *self, char **result) /* Returns -1 (with an exception set) on failure, 0 on success. The memo array will be modified in place. */ static int -_Unpickler_ResizeMemoList(UnpicklerObject *self, Py_ssize_t new_size) +_Unpickler_ResizeMemoList(UnpicklerObject *self, size_t new_size) { - Py_ssize_t i; + size_t i; assert(new_size > self->memo_size); @@ -1397,9 +1400,9 @@ _Unpickler_ResizeMemoList(UnpicklerObject *self, Py_ssize_t new_size) /* Returns NULL if idx is out of bounds. */ static PyObject * -_Unpickler_MemoGet(UnpicklerObject *self, Py_ssize_t idx) +_Unpickler_MemoGet(UnpicklerObject *self, size_t idx) { - if (idx < 0 || idx >= self->memo_size) + if (idx >= self->memo_size) return NULL; return self->memo[idx]; @@ -1408,7 +1411,7 @@ _Unpickler_MemoGet(UnpicklerObject *self, Py_ssize_t idx) /* Returns -1 (with an exception set) on failure, 0 on success. This takes its own reference to `value`. */ static int -_Unpickler_MemoPut(UnpicklerObject *self, Py_ssize_t idx, PyObject *value) +_Unpickler_MemoPut(UnpicklerObject *self, size_t idx, PyObject *value) { PyObject *old_item; @@ -4413,14 +4416,13 @@ static PyObject * _pickle_PicklerMemoProxy_copy_impl(PicklerMemoProxyObject *self) /*[clinic end generated code: output=bb83a919d29225ef input=b73043485ac30b36]*/ { - Py_ssize_t i; PyMemoTable *memo; PyObject *new_memo = PyDict_New(); if (new_memo == NULL) return NULL; memo = self->pickler->memo; - for (i = 0; i < memo->mt_allocated; ++i) { + for (size_t i = 0; i < memo->mt_allocated; ++i) { PyMemoEntry entry = memo->mt_table[i]; if (entry.me_key != NULL) { int status; @@ -6843,7 +6845,7 @@ static PyObject * _pickle_UnpicklerMemoProxy_copy_impl(UnpicklerMemoProxyObject *self) /*[clinic end generated code: output=e12af7e9bc1e4c77 input=97769247ce032c1d]*/ { - Py_ssize_t i; + size_t i; PyObject *new_memo = PyDict_New(); if (new_memo == NULL) return NULL; @@ -6994,8 +6996,7 @@ static int Unpickler_set_memo(UnpicklerObject *self, PyObject *obj) { PyObject **new_memo; - Py_ssize_t new_memo_size = 0; - Py_ssize_t i; + size_t new_memo_size = 0; if (obj == NULL) { PyErr_SetString(PyExc_TypeError, @@ -7012,7 +7013,7 @@ Unpickler_set_memo(UnpicklerObject *self, PyObject *obj) if (new_memo == NULL) return -1; - for (i = 0; i < new_memo_size; i++) { + for (size_t i = 0; i < new_memo_size; i++) { Py_XINCREF(unpickler->memo[i]); new_memo[i] = unpickler->memo[i]; } @@ -7060,8 +7061,7 @@ Unpickler_set_memo(UnpicklerObject *self, PyObject *obj) error: if (new_memo_size) { - i = new_memo_size; - while (--i >= 0) { + for (size_t i = new_memo_size - 1; i != SIZE_MAX; i--) { Py_XDECREF(new_memo[i]); } PyMem_FREE(new_memo); From webhook-mailer at python.org Thu Sep 20 21:52:40 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 21 Sep 2018 01:52:40 -0000 Subject: [Python-checkins] closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261) Message-ID: https://github.com/python/cpython/commit/ef4306b24c9034d6b37bb034e2ebe82e745d4b77 commit: ef4306b24c9034d6b37bb034e2ebe82e745d4b77 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-20T18:52:36-07:00 summary: closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261) (cherry picked from commit a4ae828ee416a66d8c7bf5ee71d653c2cc6a26dd) Co-authored-by: Benjamin Peterson files: M Modules/_pickle.c diff --git a/Modules/_pickle.c b/Modules/_pickle.c index fc119c0caa49..34ce38c5d3b2 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -600,9 +600,9 @@ typedef struct { } PyMemoEntry; typedef struct { - Py_ssize_t mt_mask; - Py_ssize_t mt_used; - Py_ssize_t mt_allocated; + size_t mt_mask; + size_t mt_used; + size_t mt_allocated; PyMemoEntry *mt_table; } PyMemoTable; @@ -648,8 +648,8 @@ typedef struct UnpicklerObject { /* The unpickler memo is just an array of PyObject *s. Using a dict is unnecessary, since the keys are contiguous ints. */ PyObject **memo; - Py_ssize_t memo_size; /* Capacity of the memo array */ - Py_ssize_t memo_len; /* Number of objects in the memo */ + size_t memo_size; /* Capacity of the memo array */ + size_t memo_len; /* Number of objects in the memo */ PyObject *pers_func; /* persistent_load() method, can be NULL. */ PyObject *pers_func_self; /* borrowed reference to self if pers_func @@ -735,7 +735,6 @@ PyMemoTable_New(void) static PyMemoTable * PyMemoTable_Copy(PyMemoTable *self) { - Py_ssize_t i; PyMemoTable *new = PyMemoTable_New(); if (new == NULL) return NULL; @@ -752,7 +751,7 @@ PyMemoTable_Copy(PyMemoTable *self) PyErr_NoMemory(); return NULL; } - for (i = 0; i < self->mt_allocated; i++) { + for (size_t i = 0; i < self->mt_allocated; i++) { Py_XINCREF(self->mt_table[i].me_key); } memcpy(new->mt_table, self->mt_table, @@ -798,7 +797,7 @@ _PyMemoTable_Lookup(PyMemoTable *self, PyObject *key) { size_t i; size_t perturb; - size_t mask = (size_t)self->mt_mask; + size_t mask = self->mt_mask; PyMemoEntry *table = self->mt_table; PyMemoEntry *entry; Py_hash_t hash = (Py_hash_t)key >> 3; @@ -819,22 +818,24 @@ _PyMemoTable_Lookup(PyMemoTable *self, PyObject *key) /* Returns -1 on failure, 0 on success. */ static int -_PyMemoTable_ResizeTable(PyMemoTable *self, Py_ssize_t min_size) +_PyMemoTable_ResizeTable(PyMemoTable *self, size_t min_size) { PyMemoEntry *oldtable = NULL; PyMemoEntry *oldentry, *newentry; - Py_ssize_t new_size = MT_MINSIZE; - Py_ssize_t to_process; + size_t new_size = MT_MINSIZE; + size_t to_process; assert(min_size > 0); - /* Find the smallest valid table size >= min_size. */ - while (new_size < min_size && new_size > 0) - new_size <<= 1; - if (new_size <= 0) { + if (min_size > PY_SSIZE_T_MAX) { PyErr_NoMemory(); return -1; } + + /* Find the smallest valid table size >= min_size. */ + while (new_size < min_size) { + new_size <<= 1; + } /* new_size needs to be a power of two. */ assert((new_size & (new_size - 1)) == 0); @@ -907,10 +908,12 @@ PyMemoTable_Set(PyMemoTable *self, PyObject *key, Py_ssize_t value) * Very large memo tables (over 50K items) use doubling instead. * This may help applications with severe memory constraints. */ - if (!(self->mt_used * 3 >= (self->mt_mask + 1) * 2)) + if (SIZE_MAX / 3 >= self->mt_used && self->mt_used * 3 < self->mt_allocated * 2) { return 0; - return _PyMemoTable_ResizeTable(self, - (self->mt_used > 50000 ? 2 : 4) * self->mt_used); + } + // self->mt_used is always < PY_SSIZE_T_MAX, so this can't overflow. + size_t desired_size = (self->mt_used > 50000 ? 2 : 4) * self->mt_used; + return _PyMemoTable_ResizeTable(self, desired_size); } #undef MT_MINSIZE @@ -1374,9 +1377,9 @@ _Unpickler_Readline(UnpicklerObject *self, char **result) /* Returns -1 (with an exception set) on failure, 0 on success. The memo array will be modified in place. */ static int -_Unpickler_ResizeMemoList(UnpicklerObject *self, Py_ssize_t new_size) +_Unpickler_ResizeMemoList(UnpicklerObject *self, size_t new_size) { - Py_ssize_t i; + size_t i; assert(new_size > self->memo_size); @@ -1395,9 +1398,9 @@ _Unpickler_ResizeMemoList(UnpicklerObject *self, Py_ssize_t new_size) /* Returns NULL if idx is out of bounds. */ static PyObject * -_Unpickler_MemoGet(UnpicklerObject *self, Py_ssize_t idx) +_Unpickler_MemoGet(UnpicklerObject *self, size_t idx) { - if (idx < 0 || idx >= self->memo_size) + if (idx >= self->memo_size) return NULL; return self->memo[idx]; @@ -1406,7 +1409,7 @@ _Unpickler_MemoGet(UnpicklerObject *self, Py_ssize_t idx) /* Returns -1 (with an exception set) on failure, 0 on success. This takes its own reference to `value`. */ static int -_Unpickler_MemoPut(UnpicklerObject *self, Py_ssize_t idx, PyObject *value) +_Unpickler_MemoPut(UnpicklerObject *self, size_t idx, PyObject *value) { PyObject *old_item; @@ -4415,14 +4418,13 @@ static PyObject * _pickle_PicklerMemoProxy_copy_impl(PicklerMemoProxyObject *self) /*[clinic end generated code: output=bb83a919d29225ef input=b73043485ac30b36]*/ { - Py_ssize_t i; PyMemoTable *memo; PyObject *new_memo = PyDict_New(); if (new_memo == NULL) return NULL; memo = self->pickler->memo; - for (i = 0; i < memo->mt_allocated; ++i) { + for (size_t i = 0; i < memo->mt_allocated; ++i) { PyMemoEntry entry = memo->mt_table[i]; if (entry.me_key != NULL) { int status; @@ -6855,7 +6857,7 @@ static PyObject * _pickle_UnpicklerMemoProxy_copy_impl(UnpicklerMemoProxyObject *self) /*[clinic end generated code: output=e12af7e9bc1e4c77 input=97769247ce032c1d]*/ { - Py_ssize_t i; + size_t i; PyObject *new_memo = PyDict_New(); if (new_memo == NULL) return NULL; @@ -7006,8 +7008,7 @@ static int Unpickler_set_memo(UnpicklerObject *self, PyObject *obj) { PyObject **new_memo; - Py_ssize_t new_memo_size = 0; - Py_ssize_t i; + size_t new_memo_size = 0; if (obj == NULL) { PyErr_SetString(PyExc_TypeError, @@ -7024,7 +7025,7 @@ Unpickler_set_memo(UnpicklerObject *self, PyObject *obj) if (new_memo == NULL) return -1; - for (i = 0; i < new_memo_size; i++) { + for (size_t i = 0; i < new_memo_size; i++) { Py_XINCREF(unpickler->memo[i]); new_memo[i] = unpickler->memo[i]; } @@ -7072,8 +7073,7 @@ Unpickler_set_memo(UnpicklerObject *self, PyObject *obj) error: if (new_memo_size) { - i = new_memo_size; - while (--i >= 0) { + for (size_t i = new_memo_size - 1; i != SIZE_MAX; i--) { Py_XDECREF(new_memo[i]); } PyMem_FREE(new_memo); From webhook-mailer at python.org Thu Sep 20 22:00:41 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 21 Sep 2018 02:00:41 -0000 Subject: [Python-checkins] closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261) Message-ID: https://github.com/python/cpython/commit/71a9c65e74a70b6ed39adc4ba81d311ac1aa2acc commit: 71a9c65e74a70b6ed39adc4ba81d311ac1aa2acc branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-20T19:00:37-07:00 summary: closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261) (cherry picked from commit a4ae828ee416a66d8c7bf5ee71d653c2cc6a26dd) Co-authored-by: Benjamin Peterson files: M Modules/_pickle.c diff --git a/Modules/_pickle.c b/Modules/_pickle.c index f5202f50c547..93bc1c6fee7f 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -596,9 +596,9 @@ typedef struct { } PyMemoEntry; typedef struct { - Py_ssize_t mt_mask; - Py_ssize_t mt_used; - Py_ssize_t mt_allocated; + size_t mt_mask; + size_t mt_used; + size_t mt_allocated; PyMemoEntry *mt_table; } PyMemoTable; @@ -644,8 +644,8 @@ typedef struct UnpicklerObject { /* The unpickler memo is just an array of PyObject *s. Using a dict is unnecessary, since the keys are contiguous ints. */ PyObject **memo; - Py_ssize_t memo_size; /* Capacity of the memo array */ - Py_ssize_t memo_len; /* Number of objects in the memo */ + size_t memo_size; /* Capacity of the memo array */ + size_t memo_len; /* Number of objects in the memo */ PyObject *pers_func; /* persistent_load() method, can be NULL. */ PyObject *pers_func_self; /* borrowed reference to self if pers_func @@ -731,7 +731,6 @@ PyMemoTable_New(void) static PyMemoTable * PyMemoTable_Copy(PyMemoTable *self) { - Py_ssize_t i; PyMemoTable *new = PyMemoTable_New(); if (new == NULL) return NULL; @@ -748,7 +747,7 @@ PyMemoTable_Copy(PyMemoTable *self) PyErr_NoMemory(); return NULL; } - for (i = 0; i < self->mt_allocated; i++) { + for (size_t i = 0; i < self->mt_allocated; i++) { Py_XINCREF(self->mt_table[i].me_key); } memcpy(new->mt_table, self->mt_table, @@ -794,7 +793,7 @@ _PyMemoTable_Lookup(PyMemoTable *self, PyObject *key) { size_t i; size_t perturb; - size_t mask = (size_t)self->mt_mask; + size_t mask = self->mt_mask; PyMemoEntry *table = self->mt_table; PyMemoEntry *entry; Py_hash_t hash = (Py_hash_t)key >> 3; @@ -816,22 +815,24 @@ _PyMemoTable_Lookup(PyMemoTable *self, PyObject *key) /* Returns -1 on failure, 0 on success. */ static int -_PyMemoTable_ResizeTable(PyMemoTable *self, Py_ssize_t min_size) +_PyMemoTable_ResizeTable(PyMemoTable *self, size_t min_size) { PyMemoEntry *oldtable = NULL; PyMemoEntry *oldentry, *newentry; - Py_ssize_t new_size = MT_MINSIZE; - Py_ssize_t to_process; + size_t new_size = MT_MINSIZE; + size_t to_process; assert(min_size > 0); - /* Find the smallest valid table size >= min_size. */ - while (new_size < min_size && new_size > 0) - new_size <<= 1; - if (new_size <= 0) { + if (min_size > PY_SSIZE_T_MAX) { PyErr_NoMemory(); return -1; } + + /* Find the smallest valid table size >= min_size. */ + while (new_size < min_size) { + new_size <<= 1; + } /* new_size needs to be a power of two. */ assert((new_size & (new_size - 1)) == 0); @@ -904,10 +905,12 @@ PyMemoTable_Set(PyMemoTable *self, PyObject *key, Py_ssize_t value) * Very large memo tables (over 50K items) use doubling instead. * This may help applications with severe memory constraints. */ - if (!(self->mt_used * 3 >= (self->mt_mask + 1) * 2)) + if (SIZE_MAX / 3 >= self->mt_used && self->mt_used * 3 < self->mt_allocated * 2) { return 0; - return _PyMemoTable_ResizeTable(self, - (self->mt_used > 50000 ? 2 : 4) * self->mt_used); + } + // self->mt_used is always < PY_SSIZE_T_MAX, so this can't overflow. + size_t desired_size = (self->mt_used > 50000 ? 2 : 4) * self->mt_used; + return _PyMemoTable_ResizeTable(self, desired_size); } #undef MT_MINSIZE @@ -1352,9 +1355,9 @@ _Unpickler_Readline(UnpicklerObject *self, char **result) /* Returns -1 (with an exception set) on failure, 0 on success. The memo array will be modified in place. */ static int -_Unpickler_ResizeMemoList(UnpicklerObject *self, Py_ssize_t new_size) +_Unpickler_ResizeMemoList(UnpicklerObject *self, size_t new_size) { - Py_ssize_t i; + size_t i; assert(new_size > self->memo_size); @@ -1373,9 +1376,9 @@ _Unpickler_ResizeMemoList(UnpicklerObject *self, Py_ssize_t new_size) /* Returns NULL if idx is out of bounds. */ static PyObject * -_Unpickler_MemoGet(UnpicklerObject *self, Py_ssize_t idx) +_Unpickler_MemoGet(UnpicklerObject *self, size_t idx) { - if (idx < 0 || idx >= self->memo_size) + if (idx >= self->memo_size) return NULL; return self->memo[idx]; @@ -1384,7 +1387,7 @@ _Unpickler_MemoGet(UnpicklerObject *self, Py_ssize_t idx) /* Returns -1 (with an exception set) on failure, 0 on success. This takes its own reference to `value`. */ static int -_Unpickler_MemoPut(UnpicklerObject *self, Py_ssize_t idx, PyObject *value) +_Unpickler_MemoPut(UnpicklerObject *self, size_t idx, PyObject *value) { PyObject *old_item; @@ -4328,14 +4331,13 @@ static PyObject * _pickle_PicklerMemoProxy_copy_impl(PicklerMemoProxyObject *self) /*[clinic end generated code: output=bb83a919d29225ef input=b73043485ac30b36]*/ { - Py_ssize_t i; PyMemoTable *memo; PyObject *new_memo = PyDict_New(); if (new_memo == NULL) return NULL; memo = self->pickler->memo; - for (i = 0; i < memo->mt_allocated; ++i) { + for (size_t i = 0; i < memo->mt_allocated; ++i) { PyMemoEntry entry = memo->mt_table[i]; if (entry.me_key != NULL) { int status; @@ -6764,7 +6766,7 @@ static PyObject * _pickle_UnpicklerMemoProxy_copy_impl(UnpicklerMemoProxyObject *self) /*[clinic end generated code: output=e12af7e9bc1e4c77 input=97769247ce032c1d]*/ { - Py_ssize_t i; + size_t i; PyObject *new_memo = PyDict_New(); if (new_memo == NULL) return NULL; @@ -6915,8 +6917,7 @@ static int Unpickler_set_memo(UnpicklerObject *self, PyObject *obj) { PyObject **new_memo; - Py_ssize_t new_memo_size = 0; - Py_ssize_t i; + size_t new_memo_size = 0; if (obj == NULL) { PyErr_SetString(PyExc_TypeError, @@ -6933,7 +6934,7 @@ Unpickler_set_memo(UnpicklerObject *self, PyObject *obj) if (new_memo == NULL) return -1; - for (i = 0; i < new_memo_size; i++) { + for (size_t i = 0; i < new_memo_size; i++) { Py_XINCREF(unpickler->memo[i]); new_memo[i] = unpickler->memo[i]; } @@ -6981,8 +6982,7 @@ Unpickler_set_memo(UnpicklerObject *self, PyObject *obj) error: if (new_memo_size) { - i = new_memo_size; - while (--i >= 0) { + for (size_t i = new_memo_size - 1; i != SIZE_MAX; i--) { Py_XDECREF(new_memo[i]); } PyMem_FREE(new_memo); From webhook-mailer at python.org Thu Sep 20 22:52:22 2018 From: webhook-mailer at python.org (Benjamin Peterson) Date: Fri, 21 Sep 2018 02:52:22 -0000 Subject: [Python-checkins] Simplify PyInit_timezone. (GH-9467) Message-ID: https://github.com/python/cpython/commit/c510c6b8b60f211793e0b84c317ea6974e8a6153 commit: c510c6b8b60f211793e0b84c317ea6974e8a6153 branch: master author: Benjamin Peterson committer: GitHub date: 2018-09-20T19:52:18-07:00 summary: Simplify PyInit_timezone. (GH-9467) Reduce the knotty preprocessor conditional logic, dedent unnecessarily nested code, and handle errors properly. The first edition of this change (afde1c1a05cc8a1e8adf6403c451f6708509a605) failed (bpo-34715) because FreeBSD doesn't define the timezone globals. That's why we're now checking for HAVE_DECL_TZNAME. files: M Modules/timemodule.c M PC/pyconfig.h diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 1a4cff23d65e..d162d93c9c63 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1522,7 +1522,7 @@ PyDoc_STRVAR(get_clock_info_doc, \n\ Get information of the specified clock."); -#if !defined(HAVE_TZNAME) || defined(__GLIBC__) || defined(__CYGWIN__) +#ifndef HAVE_DECL_TZNAME static void get_zone(char *zone, int n, struct tm *p) { @@ -1543,7 +1543,7 @@ get_gmtoff(time_t t, struct tm *p) return timegm(p) - t; #endif } -#endif /* !defined(HAVE_TZNAME) || defined(__GLIBC__) || defined(__CYGWIN__) */ +#endif // !HAVE_DECL_TZNAME static void PyInit_timezone(PyObject *m) { @@ -1563,7 +1563,7 @@ PyInit_timezone(PyObject *m) { And I'm lazy and hate C so nyer. */ -#if defined(HAVE_TZNAME) && !defined(__GLIBC__) && !defined(__CYGWIN__) +#ifdef HAVE_DECL_TZNAME PyObject *otz0, *otz1; tzset(); PyModule_AddIntConstant(m, "timezone", timezone); @@ -1574,54 +1574,52 @@ PyInit_timezone(PyObject *m) { #endif PyModule_AddIntConstant(m, "daylight", daylight); otz0 = PyUnicode_DecodeLocale(tzname[0], "surrogateescape"); - otz1 = PyUnicode_DecodeLocale(tzname[1], "surrogateescape"); - PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1)); -#else /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/ - { -#define YEAR ((time_t)((365 * 24 + 6) * 3600)) - time_t t; - struct tm p; - long janzone, julyzone; - char janname[10], julyname[10]; - t = (time((time_t *)0) / YEAR) * YEAR; - _PyTime_localtime(t, &p); - get_zone(janname, 9, &p); - janzone = -get_gmtoff(t, &p); - janname[9] = '\0'; - t += YEAR/2; - _PyTime_localtime(t, &p); - get_zone(julyname, 9, &p); - julyzone = -get_gmtoff(t, &p); - julyname[9] = '\0'; - - if( janzone < julyzone ) { - /* DST is reversed in the southern hemisphere */ - PyModule_AddIntConstant(m, "timezone", julyzone); - PyModule_AddIntConstant(m, "altzone", janzone); - PyModule_AddIntConstant(m, "daylight", - janzone != julyzone); - PyModule_AddObject(m, "tzname", - Py_BuildValue("(zz)", - julyname, janname)); - } else { - PyModule_AddIntConstant(m, "timezone", janzone); - PyModule_AddIntConstant(m, "altzone", julyzone); - PyModule_AddIntConstant(m, "daylight", - janzone != julyzone); - PyModule_AddObject(m, "tzname", - Py_BuildValue("(zz)", - janname, julyname)); - } + if (otz0 == NULL) { + return; } -#ifdef __CYGWIN__ - tzset(); - PyModule_AddIntConstant(m, "timezone", _timezone); - PyModule_AddIntConstant(m, "altzone", _timezone-3600); - PyModule_AddIntConstant(m, "daylight", _daylight); - PyModule_AddObject(m, "tzname", - Py_BuildValue("(zz)", _tzname[0], _tzname[1])); -#endif /* __CYGWIN__ */ -#endif /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/ + otz1 = PyUnicode_DecodeLocale(tzname[1], "surrogateescape"); + if (otz1 == NULL) { + Py_DECREF(otz0); + return; + } + PyObject *tzname_obj = Py_BuildValue("(NN)", otz0, otz1); + if (tzname_obj == NULL) + return; + PyModule_AddObject(m, "tzname", tzname_obj); +#else // !HAVE_DECL_TZNAME + static const time_t YEAR = (365 * 24 + 6) * 3600; + time_t t; + struct tm p; + long janzone, julyzone; + char janname[10], julyname[10]; + t = (time((time_t *)0) / YEAR) * YEAR; + _PyTime_localtime(t, &p); + get_zone(janname, 9, &p); + janzone = -get_gmtoff(t, &p); + janname[9] = '\0'; + t += YEAR/2; + _PyTime_localtime(t, &p); + get_zone(julyname, 9, &p); + julyzone = -get_gmtoff(t, &p); + julyname[9] = '\0'; + + PyObject *tzname_obj; + if (janzone < julyzone) { + /* DST is reversed in the southern hemisphere */ + PyModule_AddIntConstant(m, "timezone", julyzone); + PyModule_AddIntConstant(m, "altzone", janzone); + PyModule_AddIntConstant(m, "daylight", janzone != julyzone); + tzname_obj = Py_BuildValue("(zz)", julyname, janname); + } else { + PyModule_AddIntConstant(m, "timezone", janzone); + PyModule_AddIntConstant(m, "altzone", julyzone); + PyModule_AddIntConstant(m, "daylight", janzone != julyzone); + tzname_obj = Py_BuildValue("(zz)", janname, julyname); + } + if (tzname_obj == NULL) + return; + PyModule_AddObject(m, "tzname", tzname_obj); +#endif // !HAVE_DECL_TZNAME } diff --git a/PC/pyconfig.h b/PC/pyconfig.h index 388a3c64f18c..b5e3452460c4 100644 --- a/PC/pyconfig.h +++ b/PC/pyconfig.h @@ -396,6 +396,10 @@ Py_NO_ENABLE_SHARED to find out. Also support MS_NO_COREDLL for b/w compat */ /* Define to 1 if you have the header file. */ #define HAVE_DIRECT_H 1 +/* Define to 1 if you have the declaration of `tzname', and to 0 if you don't. + */ +#define HAVE_DECL_TZNAME 1 + /* Define if you have dirent.h. */ /* #define DIRENT 1 */ From webhook-mailer at python.org Fri Sep 21 02:09:51 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Fri, 21 Sep 2018 06:09:51 -0000 Subject: [Python-checkins] bpo-34735: Fix a memory leak in Modules/timemodule.c (GH-9418) Message-ID: https://github.com/python/cpython/commit/91e6c8717b7dcbcc46b189509de5f2d335819f37 commit: 91e6c8717b7dcbcc46b189509de5f2d335819f37 branch: master author: Zackery Spytz committer: Serhiy Storchaka date: 2018-09-21T09:09:48+03:00 summary: bpo-34735: Fix a memory leak in Modules/timemodule.c (GH-9418) There was a missing PyMem_Free(format) in time_strftime(). files: A Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst M Modules/timemodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst new file mode 100644 index 000000000000..8de08ec38637 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst @@ -0,0 +1 @@ +Fix a memory leak in Modules/timemodule.c. Patch by Zackery Spytz. diff --git a/Modules/timemodule.c b/Modules/timemodule.c index d162d93c9c63..f41d6fab95ba 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -801,6 +801,7 @@ time_strftime(PyObject *self, PyObject *args) if (outbuf[1] == L'y' && buf.tm_year < 0) { PyErr_SetString(PyExc_ValueError, "format %y requires year >= 1900 on AIX"); + PyMem_Free(format); return NULL; } } From webhook-mailer at python.org Fri Sep 21 02:11:35 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Fri, 21 Sep 2018 06:11:35 -0000 Subject: [Python-checkins] bpo-34755: Add few minor optimizations in _asynciomodule.c. (GH-9455) Message-ID: https://github.com/python/cpython/commit/fb3e9c00ed79f4d880ab9a67aab861eb3660ec75 commit: fb3e9c00ed79f4d880ab9a67aab861eb3660ec75 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-21T09:11:32+03:00 summary: bpo-34755: Add few minor optimizations in _asynciomodule.c. (GH-9455) files: M Modules/_asynciomodule.c diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 2ed033cacd85..fc91ebd854d0 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -10,6 +10,7 @@ module _asyncio /* identifiers used from some functions */ _Py_IDENTIFIER(__asyncio_running_event_loop__); +_Py_IDENTIFIER(_asyncio_future_blocking); _Py_IDENTIFIER(add_done_callback); _Py_IDENTIFIER(_all_tasks_compat); _Py_IDENTIFIER(call_soon); @@ -22,7 +23,6 @@ _Py_IDENTIFIER(throw); /* State of the _asyncio module */ static PyObject *asyncio_mod; -static PyObject *inspect_isgenerator; static PyObject *traceback_extract_stack; static PyObject *asyncio_get_event_loop_policy; static PyObject *asyncio_future_repr_info_func; @@ -1304,13 +1304,8 @@ FutureObj_repr(FutureObj *fut) return NULL; } - PyObject *rstr = NULL; - PyObject *type_name = PyObject_GetAttrString((PyObject*)Py_TYPE(fut), - "__name__"); - if (type_name != NULL) { - rstr = PyUnicode_FromFormat("<%S %U>", type_name, rinfo_s); - Py_DECREF(type_name); - } + PyObject *rstr = PyUnicode_FromFormat("<%s %U>", + _PyType_Name(Py_TYPE(fut)), rinfo_s); Py_DECREF(rinfo_s); return rstr; } @@ -1326,7 +1321,6 @@ FutureObj_finalize(FutureObj *fut) PyObject *error_type, *error_value, *error_traceback; PyObject *context; - PyObject *type_name; PyObject *message = NULL; PyObject *func; @@ -1344,14 +1338,8 @@ FutureObj_finalize(FutureObj *fut) goto finally; } - type_name = PyObject_GetAttrString((PyObject*)Py_TYPE(fut), "__name__"); - if (type_name == NULL) { - goto finally; - } - message = PyUnicode_FromFormat( - "%S exception was never retrieved", type_name); - Py_DECREF(type_name); + "%s exception was never retrieved", _PyType_Name(Py_TYPE(fut))); if (message == NULL) { goto finally; } @@ -1543,7 +1531,7 @@ static PyObject * FutureIter_send(futureiterobject *self, PyObject *unused) { /* Future.__iter__ doesn't care about values that are pushed to the - * generator, it just returns "self.result(). + * generator, it just returns self.result(). */ return FutureIter_iternext(self); } @@ -2702,163 +2690,142 @@ task_step_impl(TaskObj *task, PyObject *exc) goto different_loop; } - if (fut->fut_blocking) { - fut->fut_blocking = 0; + if (!fut->fut_blocking) { + goto yield_insteadof_yf; + } - /* result.add_done_callback(task._wakeup) */ - wrapper = TaskWakeupMethWrapper_new(task); - if (wrapper == NULL) { - goto fail; - } - res = future_add_done_callback( - (FutureObj*)result, wrapper, task->task_context); - Py_DECREF(wrapper); - if (res == NULL) { - goto fail; - } - Py_DECREF(res); + fut->fut_blocking = 0; - /* task._fut_waiter = result */ - task->task_fut_waiter = result; /* no incref is necessary */ + /* result.add_done_callback(task._wakeup) */ + wrapper = TaskWakeupMethWrapper_new(task); + if (wrapper == NULL) { + goto fail; + } + res = future_add_done_callback( + (FutureObj*)result, wrapper, task->task_context); + Py_DECREF(wrapper); + if (res == NULL) { + goto fail; + } + Py_DECREF(res); - if (task->task_must_cancel) { - PyObject *r; - r = future_cancel(fut); - if (r == NULL) { - return NULL; - } - if (r == Py_True) { - task->task_must_cancel = 0; - } - Py_DECREF(r); - } + /* task._fut_waiter = result */ + task->task_fut_waiter = result; /* no incref is necessary */ - Py_RETURN_NONE; + if (task->task_must_cancel) { + PyObject *r; + r = future_cancel(fut); + if (r == NULL) { + return NULL; + } + if (r == Py_True) { + task->task_must_cancel = 0; + } + Py_DECREF(r); } - else { - goto yield_insteadof_yf; + + Py_RETURN_NONE; + } + + /* Check if `result` is None */ + if (result == Py_None) { + /* Bare yield relinquishes control for one event loop iteration. */ + if (task_call_step_soon(task, NULL)) { + goto fail; } + return result; } /* Check if `result` is a Future-compatible object */ - o = PyObject_GetAttrString(result, "_asyncio_future_blocking"); - if (o == NULL) { - if (PyErr_ExceptionMatches(PyExc_AttributeError)) { - PyErr_Clear(); + if (_PyObject_LookupAttrId(result, &PyId__asyncio_future_blocking, &o) < 0) { + goto fail; + } + if (o != NULL && o != Py_None) { + /* `result` is a Future-compatible object */ + PyObject *wrapper; + PyObject *res; + + int blocking = PyObject_IsTrue(o); + Py_DECREF(o); + if (blocking < 0) { + goto fail; } - else { + + /* Check if `result` future is attached to a different loop */ + PyObject *oloop = get_future_loop(result); + if (oloop == NULL) { goto fail; } - } - else { - if (o == Py_None) { - Py_DECREF(o); + if (oloop != task->task_loop) { + Py_DECREF(oloop); + goto different_loop; } - else { - /* `result` is a Future-compatible object */ - PyObject *wrapper; - PyObject *res; + Py_DECREF(oloop); - int blocking = PyObject_IsTrue(o); - Py_DECREF(o); - if (blocking < 0) { - goto fail; - } + if (!blocking) { + goto yield_insteadof_yf; + } - /* Check if `result` future is attached to a different loop */ - PyObject *oloop = get_future_loop(result); - if (oloop == NULL) { - goto fail; - } - if (oloop != task->task_loop) { - Py_DECREF(oloop); - goto different_loop; - } - else { - Py_DECREF(oloop); - } + /* result._asyncio_future_blocking = False */ + if (_PyObject_SetAttrId( + result, &PyId__asyncio_future_blocking, Py_False) == -1) { + goto fail; + } - if (blocking) { - /* result._asyncio_future_blocking = False */ - if (PyObject_SetAttrString( - result, "_asyncio_future_blocking", Py_False) == -1) { - goto fail; - } + wrapper = TaskWakeupMethWrapper_new(task); + if (wrapper == NULL) { + goto fail; + } - wrapper = TaskWakeupMethWrapper_new(task); - if (wrapper == NULL) { - goto fail; - } + /* result.add_done_callback(task._wakeup) */ + PyObject *add_cb = _PyObject_GetAttrId( + result, &PyId_add_done_callback); + if (add_cb == NULL) { + Py_DECREF(wrapper); + goto fail; + } + PyObject *stack[2]; + stack[0] = wrapper; + stack[1] = (PyObject *)task->task_context; + res = _PyObject_FastCallKeywords( + add_cb, stack, 1, context_kwname); + Py_DECREF(add_cb); + Py_DECREF(wrapper); + if (res == NULL) { + goto fail; + } + Py_DECREF(res); - /* result.add_done_callback(task._wakeup) */ - PyObject *add_cb = _PyObject_GetAttrId( - result, &PyId_add_done_callback); - if (add_cb == NULL) { - Py_DECREF(wrapper); - goto fail; - } - PyObject *stack[2]; - stack[0] = wrapper; - stack[1] = (PyObject *)task->task_context; - res = _PyObject_FastCallKeywords( - add_cb, stack, 1, context_kwname); - Py_DECREF(add_cb); - Py_DECREF(wrapper); - if (res == NULL) { - goto fail; - } - Py_DECREF(res); - - /* task._fut_waiter = result */ - task->task_fut_waiter = result; /* no incref is necessary */ - - if (task->task_must_cancel) { - PyObject *r; - int is_true; - r = _PyObject_CallMethodId(result, &PyId_cancel, NULL); - if (r == NULL) { - return NULL; - } - is_true = PyObject_IsTrue(r); - Py_DECREF(r); - if (is_true < 0) { - return NULL; - } - else if (is_true) { - task->task_must_cancel = 0; - } - } + /* task._fut_waiter = result */ + task->task_fut_waiter = result; /* no incref is necessary */ - Py_RETURN_NONE; + if (task->task_must_cancel) { + PyObject *r; + int is_true; + r = _PyObject_CallMethodId(result, &PyId_cancel, NULL); + if (r == NULL) { + return NULL; } - else { - goto yield_insteadof_yf; + is_true = PyObject_IsTrue(r); + Py_DECREF(r); + if (is_true < 0) { + return NULL; + } + else if (is_true) { + task->task_must_cancel = 0; } } - } - /* Check if `result` is None */ - if (result == Py_None) { - /* Bare yield relinquishes control for one event loop iteration. */ - if (task_call_step_soon(task, NULL)) { - goto fail; - } - return result; + Py_RETURN_NONE; } + Py_XDECREF(o); /* Check if `result` is a generator */ - o = PyObject_CallFunctionObjArgs(inspect_isgenerator, result, NULL); - if (o == NULL) { - /* An exception in inspect.isgenerator */ + res = PyObject_IsInstance(result, (PyObject*)&PyGen_Type); + if (res < 0) { goto fail; } - res = PyObject_IsTrue(o); - Py_DECREF(o); - if (res == -1) { - /* An exception while checking if 'val' is True */ - goto fail; - } - if (res == 1) { + if (res) { /* `result` is a generator */ o = task_set_error_soon( task, PyExc_RuntimeError, @@ -2922,7 +2889,7 @@ task_step(TaskObj *task, PyObject *exc) return NULL; } else { - if(leave_task(task->task_loop, (PyObject*)task) < 0) { + if (leave_task(task->task_loop, (PyObject*)task) < 0) { Py_DECREF(res); return NULL; } @@ -3237,7 +3204,6 @@ static void module_free(void *m) { Py_CLEAR(asyncio_mod); - Py_CLEAR(inspect_isgenerator); Py_CLEAR(traceback_extract_stack); Py_CLEAR(asyncio_future_repr_info_func); Py_CLEAR(asyncio_get_event_loop_policy); @@ -3278,15 +3244,10 @@ module_init(void) } - context_kwname = PyTuple_New(1); + context_kwname = Py_BuildValue("(s)", "context"); if (context_kwname == NULL) { goto fail; } - PyObject *context_str = PyUnicode_FromString("context"); - if (context_str == NULL) { - goto fail; - } - PyTuple_SET_ITEM(context_kwname, 0, context_str); #define WITH_MOD(NAME) \ Py_CLEAR(module); \ @@ -3319,9 +3280,6 @@ module_init(void) WITH_MOD("asyncio.coroutines") GET_MOD_ATTR(asyncio_iscoroutine_func, "iscoroutine") - WITH_MOD("inspect") - GET_MOD_ATTR(inspect_isgenerator, "isgenerator") - WITH_MOD("traceback") GET_MOD_ATTR(traceback_extract_stack, "extract_stack") @@ -3388,7 +3346,7 @@ PyInit__asyncio(void) if (PyType_Ready(&TaskStepMethWrapper_Type) < 0) { return NULL; } - if(PyType_Ready(&TaskWakeupMethWrapper_Type) < 0) { + if (PyType_Ready(&TaskWakeupMethWrapper_Type) < 0) { return NULL; } if (PyType_Ready(&TaskType) < 0) { From webhook-mailer at python.org Fri Sep 21 03:41:49 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 21 Sep 2018 07:41:49 -0000 Subject: [Python-checkins] bpo-34735: Fix a memory leak in Modules/timemodule.c (GH-9418) Message-ID: https://github.com/python/cpython/commit/f37496e893d743dbd01c802c8e8cdc16498a1604 commit: f37496e893d743dbd01c802c8e8cdc16498a1604 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T00:41:42-07:00 summary: bpo-34735: Fix a memory leak in Modules/timemodule.c (GH-9418) There was a missing PyMem_Free(format) in time_strftime(). (cherry picked from commit 91e6c8717b7dcbcc46b189509de5f2d335819f37) Co-authored-by: Zackery Spytz files: A Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst M Modules/timemodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst new file mode 100644 index 000000000000..8de08ec38637 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst @@ -0,0 +1 @@ +Fix a memory leak in Modules/timemodule.c. Patch by Zackery Spytz. diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 05c4cbcb9c81..f66f098740b2 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -654,6 +654,7 @@ time_strftime(PyObject *self, PyObject *args) if (outbuf[1] == L'y' && buf.tm_year < 0) { PyErr_SetString(PyExc_ValueError, "format %y requires year >= 1900 on AIX"); + PyMem_Free(format); return NULL; } } From webhook-mailer at python.org Fri Sep 21 03:41:53 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 21 Sep 2018 07:41:53 -0000 Subject: [Python-checkins] bpo-34735: Fix a memory leak in Modules/timemodule.c (GH-9418) Message-ID: https://github.com/python/cpython/commit/975f3cb1f25406a9be019906227d53b23852f415 commit: 975f3cb1f25406a9be019906227d53b23852f415 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T00:41:50-07:00 summary: bpo-34735: Fix a memory leak in Modules/timemodule.c (GH-9418) There was a missing PyMem_Free(format) in time_strftime(). (cherry picked from commit 91e6c8717b7dcbcc46b189509de5f2d335819f37) Co-authored-by: Zackery Spytz files: A Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst M Modules/timemodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst new file mode 100644 index 000000000000..8de08ec38637 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-19-06-57-34.bpo-34735.-3mrSJ.rst @@ -0,0 +1 @@ +Fix a memory leak in Modules/timemodule.c. Patch by Zackery Spytz. diff --git a/Modules/timemodule.c b/Modules/timemodule.c index dbe2fbaf0796..7264ad616700 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -776,6 +776,7 @@ time_strftime(PyObject *self, PyObject *args) if (outbuf[1] == L'y' && buf.tm_year < 0) { PyErr_SetString(PyExc_ValueError, "format %y requires year >= 1900 on AIX"); + PyMem_Free(format); return NULL; } } From webhook-mailer at python.org Fri Sep 21 04:46:45 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Fri, 21 Sep 2018 08:46:45 -0000 Subject: [Python-checkins] Minor performance tweak for deque.index() with a start argument (GH-9440) Message-ID: https://github.com/python/cpython/commit/b46ad5431d2643f61e929c1ffec48766b2fafd75 commit: b46ad5431d2643f61e929c1ffec48766b2fafd75 branch: master author: Raymond Hettinger committer: GitHub date: 2018-09-21T01:46:41-07:00 summary: Minor performance tweak for deque.index() with a start argument (GH-9440) files: M Lib/test/test_deque.py M Modules/_collectionsmodule.c diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py index 921136069d77..51b66b76aca9 100644 --- a/Lib/test/test_deque.py +++ b/Lib/test/test_deque.py @@ -288,6 +288,14 @@ def test_index(self): else: self.assertEqual(d.index(element, start, stop), target) + # Test large start argument + d = deque(range(0, 10000, 10)) + for step in range(100): + i = d.index(8500, 700) + self.assertEqual(d[i], 8500) + # Repeat test with a different internal offset + d.rotate() + def test_index_bug_24913(self): d = deque('A' * 3) with self.assertRaises(ValueError): diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c index 935b4348a8ff..267cf07f1f72 100644 --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -1050,8 +1050,10 @@ deque_index(dequeobject *deque, PyObject *const *args, Py_ssize_t nargs) start = stop; assert(0 <= start && start <= stop && stop <= Py_SIZE(deque)); - /* XXX Replace this loop with faster code from deque_item() */ - for (i=0 ; irightlink; + } + for ( ; i < start ; i++) { index++; if (index == BLOCKLEN) { b = b->rightlink; From solipsis at pitrou.net Fri Sep 21 05:09:48 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 21 Sep 2018 09:09:48 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=5 Message-ID: <20180921090948.1.59346CA694B185F2@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_spawn leaked [2, -1, 0] memory blocks, sum=1 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog8Dxgvm', '--timeout', '7200'] From webhook-mailer at python.org Fri Sep 21 15:34:01 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Fri, 21 Sep 2018 19:34:01 -0000 Subject: [Python-checkins] bpo-34762: Fix contextvars C API to use PyObject* pointer types. (GH-9473) Message-ID: https://github.com/python/cpython/commit/2ec872b31e25cee1f983fe07991fb53f3fd1cbac commit: 2ec872b31e25cee1f983fe07991fb53f3fd1cbac branch: master author: Yury Selivanov committer: GitHub date: 2018-09-21T15:33:56-04:00 summary: bpo-34762: Fix contextvars C API to use PyObject* pointer types. (GH-9473) files: A Misc/NEWS.d/next/Core and Builtins/2018-09-21-11-06-56.bpo-34762.1nN53m.rst M Doc/c-api/contextvars.rst M Doc/whatsnew/3.7.rst M Include/context.h M Modules/_contextvarsmodule.c M Python/context.c diff --git a/Doc/c-api/contextvars.rst b/Doc/c-api/contextvars.rst index 4c33ba4f9944..c344c8d71ae3 100644 --- a/Doc/c-api/contextvars.rst +++ b/Doc/c-api/contextvars.rst @@ -5,6 +5,25 @@ Context Variables Objects ------------------------- +.. _contextvarsobjects_pointertype_change: +.. versionchanged:: 3.7.1 + + .. note:: + + In Python 3.7.1 the signatures of all context variables + C APIs were **changed** to use :c:type:`PyObject` pointers instead + of :c:type:`PyContext`, :c:type:`PyContextVar`, and + :c:type:`PyContextToken`, e.g.:: + + // in 3.7.0: + PyContext *PyContext_New(void); + + // in 3.7.1+: + PyObject *PyContext_New(void); + + See :issue:`34762` for more details. + + .. versionadded:: 3.7 This section details the public C API for the :mod:`contextvars` module. @@ -56,27 +75,27 @@ Type-check macros: Context object management functions: -.. c:function:: PyContext *PyContext_New(void) +.. c:function:: PyObject *PyContext_New(void) Create a new empty context object. Returns ``NULL`` if an error has occurred. -.. c:function:: PyContext *PyContext_Copy(PyContext *ctx) +.. c:function:: PyObject *PyContext_Copy(PyObject *ctx) Create a shallow copy of the passed *ctx* context object. Returns ``NULL`` if an error has occurred. -.. c:function:: PyContext *PyContext_CopyCurrent(void) +.. c:function:: PyObject *PyContext_CopyCurrent(void) Create a shallow copy of the current thread context. Returns ``NULL`` if an error has occurred. -.. c:function:: int PyContext_Enter(PyContext *ctx) +.. c:function:: int PyContext_Enter(PyObject *ctx) Set *ctx* as the current context for the current thread. Returns ``0`` on success, and ``-1`` on error. -.. c:function:: int PyContext_Exit(PyContext *ctx) +.. c:function:: int PyContext_Exit(PyObject *ctx) Deactivate the *ctx* context and restore the previous context as the current context for the current thread. Returns ``0`` on success, @@ -90,14 +109,14 @@ Context object management functions: Context variable functions: -.. c:function:: PyContextVar *PyContextVar_New(const char *name, PyObject *def) +.. c:function:: PyObject *PyContextVar_New(const char *name, PyObject *def) Create a new ``ContextVar`` object. The *name* parameter is used for introspection and debug purposes. The *def* parameter may optionally specify the default value for the context variable. If an error has occurred, this function returns ``NULL``. -.. c:function:: int PyContextVar_Get(PyContextVar *var, PyObject *default_value, PyObject **value) +.. c:function:: int PyContextVar_Get(PyObject *var, PyObject *default_value, PyObject **value) Get the value of a context variable. Returns ``-1`` if an error has occurred during lookup, and ``0`` if no error occurred, whether or not @@ -112,13 +131,13 @@ Context variable functions: If the value was found, the function will create a new reference to it. -.. c:function:: PyContextToken *PyContextVar_Set(PyContextVar *var, PyObject *value) +.. c:function:: PyObject *PyContextVar_Set(PyObject *var, PyObject *value) Set the value of *var* to *value* in the current context. Returns a - pointer to a :c:type:`PyContextToken` object, or ``NULL`` if an error + pointer to a :c:type:`PyObject` object, or ``NULL`` if an error has occurred. -.. c:function:: int PyContextVar_Reset(PyContextVar *var, PyContextToken *token) +.. c:function:: int PyContextVar_Reset(PyObject *var, PyObject *token) Reset the state of the *var* context variable to that it was in before :c:func:`PyContextVar_Set` that returned the *token* was called. diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index f53a0268738a..a2c5c284f255 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2494,3 +2494,7 @@ versions, it respected an ill-defined subset of those environment variables, while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before calling :c:func:`Py_Initialize`. + +In 3.7.1 the C API for Context Variables +:ref:`was updated ` to use +:c:type:`PyObject` pointers. See also :issue:`34762`. diff --git a/Include/context.h b/Include/context.h index 8b9f1292d75b..9581285247b3 100644 --- a/Include/context.h +++ b/Include/context.h @@ -22,19 +22,19 @@ typedef struct _pycontexttokenobject PyContextToken; #define PyContextToken_CheckExact(o) (Py_TYPE(o) == &PyContextToken_Type) -PyAPI_FUNC(PyContext *) PyContext_New(void); -PyAPI_FUNC(PyContext *) PyContext_Copy(PyContext *); -PyAPI_FUNC(PyContext *) PyContext_CopyCurrent(void); +PyAPI_FUNC(PyObject *) PyContext_New(void); +PyAPI_FUNC(PyObject *) PyContext_Copy(PyObject *); +PyAPI_FUNC(PyObject *) PyContext_CopyCurrent(void); -PyAPI_FUNC(int) PyContext_Enter(PyContext *); -PyAPI_FUNC(int) PyContext_Exit(PyContext *); +PyAPI_FUNC(int) PyContext_Enter(PyObject *); +PyAPI_FUNC(int) PyContext_Exit(PyObject *); /* Create a new context variable. default_value can be NULL. */ -PyAPI_FUNC(PyContextVar *) PyContextVar_New( +PyAPI_FUNC(PyObject *) PyContextVar_New( const char *name, PyObject *default_value); @@ -54,21 +54,19 @@ PyAPI_FUNC(PyContextVar *) PyContextVar_New( '*value' will be a new ref, if not NULL. */ PyAPI_FUNC(int) PyContextVar_Get( - PyContextVar *var, PyObject *default_value, PyObject **value); + PyObject *var, PyObject *default_value, PyObject **value); /* Set a new value for the variable. Returns NULL if an error occurs. */ -PyAPI_FUNC(PyContextToken *) PyContextVar_Set( - PyContextVar *var, PyObject *value); +PyAPI_FUNC(PyObject *) PyContextVar_Set(PyObject *var, PyObject *value); /* Reset a variable to its previous value. Returns 0 on success, -1 on error. */ -PyAPI_FUNC(int) PyContextVar_Reset( - PyContextVar *var, PyContextToken *token); +PyAPI_FUNC(int) PyContextVar_Reset(PyObject *var, PyObject *token); /* This method is exposed only for CPython tests. Don not use it. */ diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-21-11-06-56.bpo-34762.1nN53m.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-21-11-06-56.bpo-34762.1nN53m.rst new file mode 100644 index 000000000000..0cd47a40dd35 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-21-11-06-56.bpo-34762.1nN53m.rst @@ -0,0 +1 @@ +Fix contextvars C API to use PyObject* pointer types. diff --git a/Modules/_contextvarsmodule.c b/Modules/_contextvarsmodule.c index b7d112dd6018..71dd7fd8d08d 100644 --- a/Modules/_contextvarsmodule.c +++ b/Modules/_contextvarsmodule.c @@ -16,7 +16,7 @@ static PyObject * _contextvars_copy_context_impl(PyObject *module) /*[clinic end generated code: output=1fcd5da7225c4fa9 input=89bb9ae485888440]*/ { - return (PyObject *)PyContext_CopyCurrent(); + return PyContext_CopyCurrent(); } diff --git a/Python/context.c b/Python/context.c index c9658bae33c2..7344e968ccee 100644 --- a/Python/context.c +++ b/Python/context.c @@ -18,6 +18,28 @@ module _contextvars /*[clinic end generated code: output=da39a3ee5e6b4b0d input=a0955718c8b8cea6]*/ +#define ENSURE_Context(o, err_ret) \ + if (!PyContext_CheckExact(o)) { \ + PyErr_SetString(PyExc_TypeError, \ + "an instance of Context was expected"); \ + return err_ret; \ + } + +#define ENSURE_ContextVar(o, err_ret) \ + if (!PyContextVar_CheckExact(o)) { \ + PyErr_SetString(PyExc_TypeError, \ + "an instance of ContextVar was expected"); \ + return err_ret; \ + } + +#define ENSURE_ContextToken(o, err_ret) \ + if (!PyContextToken_CheckExact(o)) { \ + PyErr_SetString(PyExc_TypeError, \ + "an instance of Token was expected"); \ + return err_ret; \ + } + + /////////////////////////// Context API @@ -50,21 +72,23 @@ _PyContext_NewHamtForTests(void) } -PyContext * +PyObject * PyContext_New(void) { - return context_new_empty(); + return (PyObject *)context_new_empty(); } -PyContext * -PyContext_Copy(PyContext * ctx) +PyObject * +PyContext_Copy(PyObject * octx) { - return context_new_from_vars(ctx->ctx_vars); + ENSURE_Context(octx, NULL) + PyContext *ctx = (PyContext *)octx; + return (PyObject *)context_new_from_vars(ctx->ctx_vars); } -PyContext * +PyObject * PyContext_CopyCurrent(void) { PyContext *ctx = context_get(); @@ -72,13 +96,16 @@ PyContext_CopyCurrent(void) return NULL; } - return context_new_from_vars(ctx->ctx_vars); + return (PyObject *)context_new_from_vars(ctx->ctx_vars); } int -PyContext_Enter(PyContext *ctx) +PyContext_Enter(PyObject *octx) { + ENSURE_Context(octx, -1) + PyContext *ctx = (PyContext *)octx; + if (ctx->ctx_entered) { PyErr_Format(PyExc_RuntimeError, "cannot enter context: %R is already entered", ctx); @@ -100,8 +127,11 @@ PyContext_Enter(PyContext *ctx) int -PyContext_Exit(PyContext *ctx) +PyContext_Exit(PyObject *octx) { + ENSURE_Context(octx, -1) + PyContext *ctx = (PyContext *)octx; + if (!ctx->ctx_entered) { PyErr_Format(PyExc_RuntimeError, "cannot exit context: %R has not been entered", ctx); @@ -129,7 +159,7 @@ PyContext_Exit(PyContext *ctx) } -PyContextVar * +PyObject * PyContextVar_New(const char *name, PyObject *def) { PyObject *pyname = PyUnicode_FromString(name); @@ -138,14 +168,15 @@ PyContextVar_New(const char *name, PyObject *def) } PyContextVar *var = contextvar_new(pyname, def); Py_DECREF(pyname); - return var; + return (PyObject *)var; } int -PyContextVar_Get(PyContextVar *var, PyObject *def, PyObject **val) +PyContextVar_Get(PyObject *ovar, PyObject *def, PyObject **val) { - assert(PyContextVar_CheckExact(var)); + ENSURE_ContextVar(ovar, -1) + PyContextVar *var = (PyContextVar *)ovar; PyThreadState *ts = PyThreadState_GET(); assert(ts != NULL); @@ -204,9 +235,12 @@ PyContextVar_Get(PyContextVar *var, PyObject *def, PyObject **val) } -PyContextToken * -PyContextVar_Set(PyContextVar *var, PyObject *val) +PyObject * +PyContextVar_Set(PyObject *ovar, PyObject *val) { + ENSURE_ContextVar(ovar, NULL) + PyContextVar *var = (PyContextVar *)ovar; + if (!PyContextVar_CheckExact(var)) { PyErr_SetString( PyExc_TypeError, "an instance of ContextVar was expected"); @@ -233,13 +267,18 @@ PyContextVar_Set(PyContextVar *var, PyObject *val) return NULL; } - return tok; + return (PyObject *)tok; } int -PyContextVar_Reset(PyContextVar *var, PyContextToken *tok) +PyContextVar_Reset(PyObject *ovar, PyObject *otok) { + ENSURE_ContextVar(ovar, -1) + ENSURE_ContextToken(otok, -1) + PyContextVar *var = (PyContextVar *)ovar; + PyContextToken *tok = (PyContextToken *)otok; + if (tok->tok_used) { PyErr_Format(PyExc_RuntimeError, "%R has already been used once", tok); @@ -376,7 +415,7 @@ context_tp_new(PyTypeObject *type, PyObject *args, PyObject *kwds) PyExc_TypeError, "Context() does not accept any arguments"); return NULL; } - return (PyObject *)PyContext_New(); + return PyContext_New(); } static int @@ -587,14 +626,14 @@ context_run(PyContext *self, PyObject *const *args, return NULL; } - if (PyContext_Enter(self)) { + if (PyContext_Enter((PyObject *)self)) { return NULL; } PyObject *call_result = _PyObject_FastCallKeywords( args[0], args + 1, nargs - 1, kwnames); - if (PyContext_Exit(self)) { + if (PyContext_Exit((PyObject *)self)) { return NULL; } @@ -908,7 +947,7 @@ _contextvars_ContextVar_get_impl(PyContextVar *self, PyObject *default_value) } PyObject *val; - if (PyContextVar_Get(self, default_value, &val) < 0) { + if (PyContextVar_Get((PyObject *)self, default_value, &val) < 0) { return NULL; } @@ -937,7 +976,7 @@ static PyObject * _contextvars_ContextVar_set(PyContextVar *self, PyObject *value) /*[clinic end generated code: output=446ed5e820d6d60b input=c0a6887154227453]*/ { - return (PyObject *)PyContextVar_Set(self, value); + return PyContextVar_Set((PyObject *)self, value); } /*[clinic input] @@ -961,7 +1000,7 @@ _contextvars_ContextVar_reset(PyContextVar *self, PyObject *token) return NULL; } - if (PyContextVar_Reset(self, (PyContextToken *)token)) { + if (PyContextVar_Reset((PyObject *)self, token)) { return NULL; } From webhook-mailer at python.org Fri Sep 21 15:48:15 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 21 Sep 2018 19:48:15 -0000 Subject: [Python-checkins] bpo-34762: Fix contextvars C API to use PyObject* pointer types. (GH-9473) Message-ID: https://github.com/python/cpython/commit/187f2dd256a917c20bf55954d019fd35fb46ab08 commit: 187f2dd256a917c20bf55954d019fd35fb46ab08 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T12:48:10-07:00 summary: bpo-34762: Fix contextvars C API to use PyObject* pointer types. (GH-9473) (cherry picked from commit 2ec872b31e25cee1f983fe07991fb53f3fd1cbac) Co-authored-by: Yury Selivanov files: A Misc/NEWS.d/next/Core and Builtins/2018-09-21-11-06-56.bpo-34762.1nN53m.rst M Doc/c-api/contextvars.rst M Doc/whatsnew/3.7.rst M Include/context.h M Modules/_contextvarsmodule.c M Python/context.c diff --git a/Doc/c-api/contextvars.rst b/Doc/c-api/contextvars.rst index 4c33ba4f9944..c344c8d71ae3 100644 --- a/Doc/c-api/contextvars.rst +++ b/Doc/c-api/contextvars.rst @@ -5,6 +5,25 @@ Context Variables Objects ------------------------- +.. _contextvarsobjects_pointertype_change: +.. versionchanged:: 3.7.1 + + .. note:: + + In Python 3.7.1 the signatures of all context variables + C APIs were **changed** to use :c:type:`PyObject` pointers instead + of :c:type:`PyContext`, :c:type:`PyContextVar`, and + :c:type:`PyContextToken`, e.g.:: + + // in 3.7.0: + PyContext *PyContext_New(void); + + // in 3.7.1+: + PyObject *PyContext_New(void); + + See :issue:`34762` for more details. + + .. versionadded:: 3.7 This section details the public C API for the :mod:`contextvars` module. @@ -56,27 +75,27 @@ Type-check macros: Context object management functions: -.. c:function:: PyContext *PyContext_New(void) +.. c:function:: PyObject *PyContext_New(void) Create a new empty context object. Returns ``NULL`` if an error has occurred. -.. c:function:: PyContext *PyContext_Copy(PyContext *ctx) +.. c:function:: PyObject *PyContext_Copy(PyObject *ctx) Create a shallow copy of the passed *ctx* context object. Returns ``NULL`` if an error has occurred. -.. c:function:: PyContext *PyContext_CopyCurrent(void) +.. c:function:: PyObject *PyContext_CopyCurrent(void) Create a shallow copy of the current thread context. Returns ``NULL`` if an error has occurred. -.. c:function:: int PyContext_Enter(PyContext *ctx) +.. c:function:: int PyContext_Enter(PyObject *ctx) Set *ctx* as the current context for the current thread. Returns ``0`` on success, and ``-1`` on error. -.. c:function:: int PyContext_Exit(PyContext *ctx) +.. c:function:: int PyContext_Exit(PyObject *ctx) Deactivate the *ctx* context and restore the previous context as the current context for the current thread. Returns ``0`` on success, @@ -90,14 +109,14 @@ Context object management functions: Context variable functions: -.. c:function:: PyContextVar *PyContextVar_New(const char *name, PyObject *def) +.. c:function:: PyObject *PyContextVar_New(const char *name, PyObject *def) Create a new ``ContextVar`` object. The *name* parameter is used for introspection and debug purposes. The *def* parameter may optionally specify the default value for the context variable. If an error has occurred, this function returns ``NULL``. -.. c:function:: int PyContextVar_Get(PyContextVar *var, PyObject *default_value, PyObject **value) +.. c:function:: int PyContextVar_Get(PyObject *var, PyObject *default_value, PyObject **value) Get the value of a context variable. Returns ``-1`` if an error has occurred during lookup, and ``0`` if no error occurred, whether or not @@ -112,13 +131,13 @@ Context variable functions: If the value was found, the function will create a new reference to it. -.. c:function:: PyContextToken *PyContextVar_Set(PyContextVar *var, PyObject *value) +.. c:function:: PyObject *PyContextVar_Set(PyObject *var, PyObject *value) Set the value of *var* to *value* in the current context. Returns a - pointer to a :c:type:`PyContextToken` object, or ``NULL`` if an error + pointer to a :c:type:`PyObject` object, or ``NULL`` if an error has occurred. -.. c:function:: int PyContextVar_Reset(PyContextVar *var, PyContextToken *token) +.. c:function:: int PyContextVar_Reset(PyObject *var, PyObject *token) Reset the state of the *var* context variable to that it was in before :c:func:`PyContextVar_Set` that returned the *token* was called. diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index f53a0268738a..a2c5c284f255 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2494,3 +2494,7 @@ versions, it respected an ill-defined subset of those environment variables, while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before calling :c:func:`Py_Initialize`. + +In 3.7.1 the C API for Context Variables +:ref:`was updated ` to use +:c:type:`PyObject` pointers. See also :issue:`34762`. diff --git a/Include/context.h b/Include/context.h index 8b9f1292d75b..9581285247b3 100644 --- a/Include/context.h +++ b/Include/context.h @@ -22,19 +22,19 @@ typedef struct _pycontexttokenobject PyContextToken; #define PyContextToken_CheckExact(o) (Py_TYPE(o) == &PyContextToken_Type) -PyAPI_FUNC(PyContext *) PyContext_New(void); -PyAPI_FUNC(PyContext *) PyContext_Copy(PyContext *); -PyAPI_FUNC(PyContext *) PyContext_CopyCurrent(void); +PyAPI_FUNC(PyObject *) PyContext_New(void); +PyAPI_FUNC(PyObject *) PyContext_Copy(PyObject *); +PyAPI_FUNC(PyObject *) PyContext_CopyCurrent(void); -PyAPI_FUNC(int) PyContext_Enter(PyContext *); -PyAPI_FUNC(int) PyContext_Exit(PyContext *); +PyAPI_FUNC(int) PyContext_Enter(PyObject *); +PyAPI_FUNC(int) PyContext_Exit(PyObject *); /* Create a new context variable. default_value can be NULL. */ -PyAPI_FUNC(PyContextVar *) PyContextVar_New( +PyAPI_FUNC(PyObject *) PyContextVar_New( const char *name, PyObject *default_value); @@ -54,21 +54,19 @@ PyAPI_FUNC(PyContextVar *) PyContextVar_New( '*value' will be a new ref, if not NULL. */ PyAPI_FUNC(int) PyContextVar_Get( - PyContextVar *var, PyObject *default_value, PyObject **value); + PyObject *var, PyObject *default_value, PyObject **value); /* Set a new value for the variable. Returns NULL if an error occurs. */ -PyAPI_FUNC(PyContextToken *) PyContextVar_Set( - PyContextVar *var, PyObject *value); +PyAPI_FUNC(PyObject *) PyContextVar_Set(PyObject *var, PyObject *value); /* Reset a variable to its previous value. Returns 0 on success, -1 on error. */ -PyAPI_FUNC(int) PyContextVar_Reset( - PyContextVar *var, PyContextToken *token); +PyAPI_FUNC(int) PyContextVar_Reset(PyObject *var, PyObject *token); /* This method is exposed only for CPython tests. Don not use it. */ diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-21-11-06-56.bpo-34762.1nN53m.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-21-11-06-56.bpo-34762.1nN53m.rst new file mode 100644 index 000000000000..0cd47a40dd35 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-21-11-06-56.bpo-34762.1nN53m.rst @@ -0,0 +1 @@ +Fix contextvars C API to use PyObject* pointer types. diff --git a/Modules/_contextvarsmodule.c b/Modules/_contextvarsmodule.c index b7d112dd6018..71dd7fd8d08d 100644 --- a/Modules/_contextvarsmodule.c +++ b/Modules/_contextvarsmodule.c @@ -16,7 +16,7 @@ static PyObject * _contextvars_copy_context_impl(PyObject *module) /*[clinic end generated code: output=1fcd5da7225c4fa9 input=89bb9ae485888440]*/ { - return (PyObject *)PyContext_CopyCurrent(); + return PyContext_CopyCurrent(); } diff --git a/Python/context.c b/Python/context.c index 8ee048ccdd5c..ba64903a608e 100644 --- a/Python/context.c +++ b/Python/context.c @@ -18,6 +18,28 @@ module _contextvars /*[clinic end generated code: output=da39a3ee5e6b4b0d input=a0955718c8b8cea6]*/ +#define ENSURE_Context(o, err_ret) \ + if (!PyContext_CheckExact(o)) { \ + PyErr_SetString(PyExc_TypeError, \ + "an instance of Context was expected"); \ + return err_ret; \ + } + +#define ENSURE_ContextVar(o, err_ret) \ + if (!PyContextVar_CheckExact(o)) { \ + PyErr_SetString(PyExc_TypeError, \ + "an instance of ContextVar was expected"); \ + return err_ret; \ + } + +#define ENSURE_ContextToken(o, err_ret) \ + if (!PyContextToken_CheckExact(o)) { \ + PyErr_SetString(PyExc_TypeError, \ + "an instance of Token was expected"); \ + return err_ret; \ + } + + /////////////////////////// Context API @@ -50,21 +72,23 @@ _PyContext_NewHamtForTests(void) } -PyContext * +PyObject * PyContext_New(void) { - return context_new_empty(); + return (PyObject *)context_new_empty(); } -PyContext * -PyContext_Copy(PyContext * ctx) +PyObject * +PyContext_Copy(PyObject * octx) { - return context_new_from_vars(ctx->ctx_vars); + ENSURE_Context(octx, NULL) + PyContext *ctx = (PyContext *)octx; + return (PyObject *)context_new_from_vars(ctx->ctx_vars); } -PyContext * +PyObject * PyContext_CopyCurrent(void) { PyContext *ctx = context_get(); @@ -72,13 +96,16 @@ PyContext_CopyCurrent(void) return NULL; } - return context_new_from_vars(ctx->ctx_vars); + return (PyObject *)context_new_from_vars(ctx->ctx_vars); } int -PyContext_Enter(PyContext *ctx) +PyContext_Enter(PyObject *octx) { + ENSURE_Context(octx, -1) + PyContext *ctx = (PyContext *)octx; + if (ctx->ctx_entered) { PyErr_Format(PyExc_RuntimeError, "cannot enter context: %R is already entered", ctx); @@ -100,8 +127,11 @@ PyContext_Enter(PyContext *ctx) int -PyContext_Exit(PyContext *ctx) +PyContext_Exit(PyObject *octx) { + ENSURE_Context(octx, -1) + PyContext *ctx = (PyContext *)octx; + if (!ctx->ctx_entered) { PyErr_Format(PyExc_RuntimeError, "cannot exit context: %R has not been entered", ctx); @@ -129,7 +159,7 @@ PyContext_Exit(PyContext *ctx) } -PyContextVar * +PyObject * PyContextVar_New(const char *name, PyObject *def) { PyObject *pyname = PyUnicode_FromString(name); @@ -138,14 +168,15 @@ PyContextVar_New(const char *name, PyObject *def) } PyContextVar *var = contextvar_new(pyname, def); Py_DECREF(pyname); - return var; + return (PyObject *)var; } int -PyContextVar_Get(PyContextVar *var, PyObject *def, PyObject **val) +PyContextVar_Get(PyObject *ovar, PyObject *def, PyObject **val) { - assert(PyContextVar_CheckExact(var)); + ENSURE_ContextVar(ovar, -1) + PyContextVar *var = (PyContextVar *)ovar; PyThreadState *ts = PyThreadState_GET(); assert(ts != NULL); @@ -204,9 +235,12 @@ PyContextVar_Get(PyContextVar *var, PyObject *def, PyObject **val) } -PyContextToken * -PyContextVar_Set(PyContextVar *var, PyObject *val) +PyObject * +PyContextVar_Set(PyObject *ovar, PyObject *val) { + ENSURE_ContextVar(ovar, NULL) + PyContextVar *var = (PyContextVar *)ovar; + if (!PyContextVar_CheckExact(var)) { PyErr_SetString( PyExc_TypeError, "an instance of ContextVar was expected"); @@ -233,13 +267,18 @@ PyContextVar_Set(PyContextVar *var, PyObject *val) return NULL; } - return tok; + return (PyObject *)tok; } int -PyContextVar_Reset(PyContextVar *var, PyContextToken *tok) +PyContextVar_Reset(PyObject *ovar, PyObject *otok) { + ENSURE_ContextVar(ovar, -1) + ENSURE_ContextToken(otok, -1) + PyContextVar *var = (PyContextVar *)ovar; + PyContextToken *tok = (PyContextToken *)otok; + if (tok->tok_used) { PyErr_Format(PyExc_RuntimeError, "%R has already been used once", tok); @@ -376,7 +415,7 @@ context_tp_new(PyTypeObject *type, PyObject *args, PyObject *kwds) PyExc_TypeError, "Context() does not accept any arguments"); return NULL; } - return (PyObject *)PyContext_New(); + return PyContext_New(); } static int @@ -572,14 +611,14 @@ context_run(PyContext *self, PyObject *const *args, return NULL; } - if (PyContext_Enter(self)) { + if (PyContext_Enter((PyObject *)self)) { return NULL; } PyObject *call_result = _PyObject_FastCallKeywords( args[0], args + 1, nargs - 1, kwnames); - if (PyContext_Exit(self)) { + if (PyContext_Exit((PyObject *)self)) { return NULL; } @@ -885,7 +924,7 @@ _contextvars_ContextVar_get_impl(PyContextVar *self, PyObject *default_value) } PyObject *val; - if (PyContextVar_Get(self, default_value, &val) < 0) { + if (PyContextVar_Get((PyObject *)self, default_value, &val) < 0) { return NULL; } @@ -907,7 +946,7 @@ static PyObject * _contextvars_ContextVar_set(PyContextVar *self, PyObject *value) /*[clinic end generated code: output=446ed5e820d6d60b input=a2d88f57c6d86f7c]*/ { - return (PyObject *)PyContextVar_Set(self, value); + return PyContextVar_Set((PyObject *)self, value); } /*[clinic input] @@ -926,7 +965,7 @@ _contextvars_ContextVar_reset(PyContextVar *self, PyObject *token) return NULL; } - if (PyContextVar_Reset(self, (PyContextToken *)token)) { + if (PyContextVar_Reset((PyObject *)self, token)) { return NULL; } From webhook-mailer at python.org Fri Sep 21 16:18:25 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 21 Sep 2018 20:18:25 -0000 Subject: [Python-checkins] Make docs of exitcode for subprocess.getstatusoutput more clear. (GH-9477) Message-ID: https://github.com/python/cpython/commit/7d161726e4ddd2b2cdd7ac58a7e9e9ea3f57a807 commit: 7d161726e4ddd2b2cdd7ac58a7e9e9ea3f57a807 branch: master author: Xiang Zhang committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-21T13:18:20-07:00 summary: Make docs of exitcode for subprocess.getstatusoutput more clear. (GH-9477) Make it more accurate and not limited to UNIX. files: M Doc/library/subprocess.rst diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 6e602e72a1c8..aa889ed00b03 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1336,14 +1336,15 @@ handling consistency are valid for these functions. Windows support was added. The function now returns (exitcode, output) instead of (status, output) - as it did in Python 3.3.3 and earlier. See :func:`WEXITSTATUS`. + as it did in Python 3.3.3 and earlier. exitcode has the same value as + :attr:`~Popen.returncode`. .. function:: getoutput(cmd) Return output (stdout and stderr) of executing *cmd* in a shell. - Like :func:`getstatusoutput`, except the exit status is ignored and the return + Like :func:`getstatusoutput`, except the exit code is ignored and the return value is a string containing the command's output. Example:: >>> subprocess.getoutput('ls /bin/ls') From webhook-mailer at python.org Fri Sep 21 16:23:19 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Fri, 21 Sep 2018 20:23:19 -0000 Subject: [Python-checkins] bpo-33649: Fix gather() docs; fix title; few other nits. (GH-9475) Message-ID: https://github.com/python/cpython/commit/db1a80e97aa7217c561fb3627f70be1882de9534 commit: db1a80e97aa7217c561fb3627f70be1882de9534 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-21T16:23:15-04:00 summary: bpo-33649: Fix gather() docs; fix title; few other nits. (GH-9475) files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index b33a7df4e822..e995fb6391ad 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -129,7 +129,8 @@ other coroutines:: async def main(): # Nothing happens if we just call "nested()". - # (a coroutine object is created but not awaited) + # A coroutine object is created but not awaited, + # so it *won't run at all*. nested() # Let's do it differently now and await it: @@ -313,12 +314,15 @@ Running Tasks Concurrently aggregate list of returned values. The order of result values corresponds to the order of awaitables in *aws*. + If *return_exceptions* is ``False`` (default), the first + raised exception is immediately propagated to the task that + awaits on ``gather()``. Other awaitables in the *aws* sequence + **won't be cancelled** and will continue to run. + If *return_exceptions* is ``True``, exceptions are treated the same as successful results, and aggregated in the result list. - Otherwise, the first raised exception is immediately propagated - to the task that awaits on ``gather()``. - If ``gather`` is *cancelled*, all submitted awaitables + If ``gather()`` is *cancelled*, all submitted awaitables (that have not completed yet) are also *cancelled*. If any Task or Future from the *aws* sequence is *cancelled*, it is @@ -368,16 +372,15 @@ Running Tasks Concurrently propagated regardless of *return_exceptions*. -Shielding Tasks From Cancellation -================================= +Shielding From Cancellation +=========================== .. awaitablefunction:: shield(aw, \*, loop=None) Protect an :ref:`awaitable object ` from being :meth:`cancelled `. - *aw* can be a coroutine, a Task, or a Future-like object. If - *aw* is a coroutine it is automatically scheduled as a Task. + If *aw* is a coroutine it is automatically scheduled as a Task. The statement:: @@ -609,7 +612,7 @@ Task Object .. class:: Task(coro, \*, loop=None, name=None) - A :class:`Future`-like object that wraps a Python + A :class:`Future-like ` object that runs a Python :ref:`coroutine `. Not thread-safe. Tasks are used to run coroutines in event loops. @@ -831,7 +834,7 @@ Task Object is used to get the current loop. This method is **deprecated** and will be removed in - Python 3.9. Use the :func:`all_tasks` function instead. + Python 3.9. Use the :func:`asyncio.all_tasks` function instead. .. classmethod:: current_task(loop=None) @@ -841,7 +844,8 @@ Task Object is used to get the current loop. This method is **deprecated** and will be removed in - Python 3.9. Use the :func:`current_task` function instead. + Python 3.9. Use the :func:`asyncio.current_task` function + instead. .. _asyncio_generator_based_coro: From webhook-mailer at python.org Fri Sep 21 16:30:53 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 21 Sep 2018 20:30:53 -0000 Subject: [Python-checkins] Make docs of exitcode for subprocess.getstatusoutput more clear. (GH-9477) Message-ID: https://github.com/python/cpython/commit/29da8392a7ffa3e72212809e4838a11ee6e134d1 commit: 29da8392a7ffa3e72212809e4838a11ee6e134d1 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T13:30:50-07:00 summary: Make docs of exitcode for subprocess.getstatusoutput more clear. (GH-9477) Make it more accurate and not limited to UNIX. (cherry picked from commit 7d161726e4ddd2b2cdd7ac58a7e9e9ea3f57a807) Co-authored-by: Xiang Zhang files: M Doc/library/subprocess.rst diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index f624adbda4c8..dfb183aba547 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1199,14 +1199,15 @@ handling consistency are valid for these functions. Windows support was added. The function now returns (exitcode, output) instead of (status, output) - as it did in Python 3.3.3 and earlier. See :func:`WEXITSTATUS`. + as it did in Python 3.3.3 and earlier. exitcode has the same value as + :attr:`~Popen.returncode`. .. function:: getoutput(cmd) Return output (stdout and stderr) of executing *cmd* in a shell. - Like :func:`getstatusoutput`, except the exit status is ignored and the return + Like :func:`getstatusoutput`, except the exit code is ignored and the return value is a string containing the command's output. Example:: >>> subprocess.getoutput('ls /bin/ls') From webhook-mailer at python.org Fri Sep 21 16:34:51 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 21 Sep 2018 20:34:51 -0000 Subject: [Python-checkins] Make docs of exitcode for subprocess.getstatusoutput more clear. (GH-9477) Message-ID: https://github.com/python/cpython/commit/914086aa2fa15d357e5bce36943c52e1cd843445 commit: 914086aa2fa15d357e5bce36943c52e1cd843445 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T13:34:46-07:00 summary: Make docs of exitcode for subprocess.getstatusoutput more clear. (GH-9477) Make it more accurate and not limited to UNIX. (cherry picked from commit 7d161726e4ddd2b2cdd7ac58a7e9e9ea3f57a807) Co-authored-by: Xiang Zhang files: M Doc/library/subprocess.rst diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 19f6ff38bb2b..23beb52d3b9a 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1336,14 +1336,15 @@ handling consistency are valid for these functions. Windows support was added. The function now returns (exitcode, output) instead of (status, output) - as it did in Python 3.3.3 and earlier. See :func:`WEXITSTATUS`. + as it did in Python 3.3.3 and earlier. exitcode has the same value as + :attr:`~Popen.returncode`. .. function:: getoutput(cmd) Return output (stdout and stderr) of executing *cmd* in a shell. - Like :func:`getstatusoutput`, except the exit status is ignored and the return + Like :func:`getstatusoutput`, except the exit code is ignored and the return value is a string containing the command's output. Example:: >>> subprocess.getoutput('ls /bin/ls') From webhook-mailer at python.org Fri Sep 21 16:35:37 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Fri, 21 Sep 2018 20:35:37 -0000 Subject: [Python-checkins] bpo-33649: Fix gather() docs; fix title; few other nits. (GH-9475) (GH-9481) Message-ID: https://github.com/python/cpython/commit/e45662c28bfc84aa3674463a2995e45da4d63793 commit: e45662c28bfc84aa3674463a2995e45da4d63793 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Yury Selivanov date: 2018-09-21T16:35:34-04:00 summary: bpo-33649: Fix gather() docs; fix title; few other nits. (GH-9475) (GH-9481) (cherry picked from commit db1a80e97aa7217c561fb3627f70be1882de9534) Co-authored-by: Yury Selivanov files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 1b95592cd99e..bdb475a797db 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -129,7 +129,8 @@ other coroutines:: async def main(): # Nothing happens if we just call "nested()". - # (a coroutine object is created but not awaited) + # A coroutine object is created but not awaited, + # so it *won't run at all*. nested() # Let's do it differently now and await it: @@ -307,12 +308,15 @@ Running Tasks Concurrently aggregate list of returned values. The order of result values corresponds to the order of awaitables in *aws*. + If *return_exceptions* is ``False`` (default), the first + raised exception is immediately propagated to the task that + awaits on ``gather()``. Other awaitables in the *aws* sequence + **won't be cancelled** and will continue to run. + If *return_exceptions* is ``True``, exceptions are treated the same as successful results, and aggregated in the result list. - Otherwise, the first raised exception is immediately propagated - to the task that awaits on ``gather()``. - If ``gather`` is *cancelled*, all submitted awaitables + If ``gather()`` is *cancelled*, all submitted awaitables (that have not completed yet) are also *cancelled*. If any Task or Future from the *aws* sequence is *cancelled*, it is @@ -362,16 +366,15 @@ Running Tasks Concurrently propagated regardless of *return_exceptions*. -Shielding Tasks From Cancellation -================================= +Shielding From Cancellation +=========================== .. awaitablefunction:: shield(aw, \*, loop=None) Protect an :ref:`awaitable object ` from being :meth:`cancelled `. - *aw* can be a coroutine, a Task, or a Future-like object. If - *aw* is a coroutine it is automatically scheduled as a Task. + If *aw* is a coroutine it is automatically scheduled as a Task. The statement:: @@ -603,7 +606,7 @@ Task Object .. class:: Task(coro, \*, loop=None) - A :class:`Future`-like object that wraps a Python + A :class:`Future-like ` object that runs a Python :ref:`coroutine `. Not thread-safe. Tasks are used to run coroutines in event loops. @@ -800,7 +803,7 @@ Task Object is used to get the current loop. This method is **deprecated** and will be removed in - Python 3.9. Use the :func:`all_tasks` function instead. + Python 3.9. Use the :func:`asyncio.all_tasks` function instead. .. classmethod:: current_task(loop=None) @@ -810,7 +813,8 @@ Task Object is used to get the current loop. This method is **deprecated** and will be removed in - Python 3.9. Use the :func:`current_task` function instead. + Python 3.9. Use the :func:`asyncio.current_task` function + instead. .. _asyncio_generator_based_coro: From webhook-mailer at python.org Fri Sep 21 18:27:31 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 21 Sep 2018 22:27:31 -0000 Subject: [Python-checkins] bpo-32718: Make Activate.ps1 for venv cross-platform and available on all platforms (GH-9321) Message-ID: https://github.com/python/cpython/commit/d64ee1a5ba2007ae5fe085dd3495013d940a51bb commit: d64ee1a5ba2007ae5fe085dd3495013d940a51bb branch: master author: Brett Cannon committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-21T15:27:26-07:00 summary: bpo-32718: Make Activate.ps1 for venv cross-platform and available on all platforms (GH-9321) PowerShell Core 6.1 is the cross-platform port of Windows PowerShell. This change updates Activate.ps1 to not make Windows assumptions as well as installing it into the bin/Scripts directory on all operating systems. Requires PowerShell Core 6.1 for proper readline support once the shell has been activated for the virtual environment. files: A Lib/venv/scripts/common/Activate.ps1 A Misc/NEWS.d/next/Library/2018-09-14-12-38-49.bpo-32718.ICYQbt.rst D Lib/venv/scripts/nt/Activate.ps1 M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 1c129a704429..26928fbd55c0 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -136,6 +136,13 @@ pathlib contain characters unrepresentable at the OS level. (Contributed by Serhiy Storchaka in :issue:`33721`.) +venv +---- + +* :mod:`venv` now includes an ``Activate.ps1`` script on all platforms for + activating virtual environments under PowerShell Core 6.1. + (Contributed by Brett Cannon in :issue:`32718`.) + Optimizations ============= @@ -321,7 +328,7 @@ CPython bytecode changes * The interpreter loop has been simplified by moving the logic of unrolling the stack of blocks into the compiler. The compiler emits now explicit - instructions for adjusting the stack of values and calling the cleaning + instructions for adjusting the stack of values and calling the cleaning- up code for :keyword:`break`, :keyword:`continue` and :keyword:`return`. Removed opcodes :opcode:`BREAK_LOOP`, :opcode:`CONTINUE_LOOP`, diff --git a/Lib/venv/scripts/nt/Activate.ps1 b/Lib/venv/scripts/common/Activate.ps1 similarity index 86% rename from Lib/venv/scripts/nt/Activate.ps1 rename to Lib/venv/scripts/common/Activate.ps1 index bf60869e552e..de22962630aa 100644 --- a/Lib/venv/scripts/nt/Activate.ps1 +++ b/Lib/venv/scripts/common/Activate.ps1 @@ -1,3 +1,8 @@ +function Script:add-bin([string]$envPath) { + $binPath = Join-Path -Path $env:VIRTUAL_ENV -ChildPath '__VENV_BIN_NAME__' + return ($binPath, $envPath) -join [IO.Path]::PathSeparator +} + function global:deactivate ([switch]$NonDestructive) { # Revert to original values if (Test-Path function:_OLD_VIRTUAL_PROMPT) { @@ -48,4 +53,4 @@ if (Test-Path env:PYTHONHOME) { # Add the venv to the PATH copy-item env:PATH env:_OLD_VIRTUAL_PATH -$env:PATH = "$env:VIRTUAL_ENV\__VENV_BIN_NAME__;$env:PATH" +$env:PATH = add-bin $env:PATH diff --git a/Misc/NEWS.d/next/Library/2018-09-14-12-38-49.bpo-32718.ICYQbt.rst b/Misc/NEWS.d/next/Library/2018-09-14-12-38-49.bpo-32718.ICYQbt.rst new file mode 100644 index 000000000000..b60106a003d3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-14-12-38-49.bpo-32718.ICYQbt.rst @@ -0,0 +1,2 @@ +The Activate.ps1 script from venv works with PowerShell Core 6.1 and is now +available under all operating systems. From webhook-mailer at python.org Fri Sep 21 21:13:24 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Sat, 22 Sep 2018 01:13:24 -0000 Subject: [Python-checkins] bpo-34537: Fix test_gdb:test_strings with LC_ALL=C (GH-9483) Message-ID: https://github.com/python/cpython/commit/7279b5125e7c5d84a473d250b27d353cb7f6628e commit: 7279b5125e7c5d84a473d250b27d353cb7f6628e branch: master author: Elvis Pranskevichus committer: Victor Stinner date: 2018-09-21T18:13:16-07:00 summary: bpo-34537: Fix test_gdb:test_strings with LC_ALL=C (GH-9483) We cannot simply call locale.getpreferredencoding() here, as GDB might have been linked against a different version of Python with a different encoding and coercion policy with respect to PEP 538 and PEP 540. Thanks to Victor Stinner for a hint on how to fix this. files: A Misc/NEWS.d/next/Tests/2018-09-21-17-33-41.bpo-34537.GImYtZ.rst M Lib/test/test_gdb.py diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py index c2ca57a4a04f..93a2c7dd5758 100644 --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -321,7 +321,20 @@ def test_bytes(self): def test_strings(self): 'Verify the pretty-printing of unicode strings' - encoding = locale.getpreferredencoding() + # We cannot simply call locale.getpreferredencoding() here, + # as GDB might have been linked against a different version + # of Python with a different encoding and coercion policy + # with respect to PEP 538 and PEP 540. + out, err = run_gdb( + '--eval-command', + 'python import locale; print(locale.getpreferredencoding())') + + encoding = out.rstrip() + if err or not encoding: + raise RuntimeError( + f'unable to determine the preferred encoding ' + f'of embedded Python in GDB: {err}') + def check_repr(text): try: text.encode(encoding) diff --git a/Misc/NEWS.d/next/Tests/2018-09-21-17-33-41.bpo-34537.GImYtZ.rst b/Misc/NEWS.d/next/Tests/2018-09-21-17-33-41.bpo-34537.GImYtZ.rst new file mode 100644 index 000000000000..b64a6a762cdb --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-21-17-33-41.bpo-34537.GImYtZ.rst @@ -0,0 +1,2 @@ +Fix ``test_gdb.test_strings()`` when ``LC_ALL=C`` and GDB was compiled with +Python 3.6 or earlier. From webhook-mailer at python.org Fri Sep 21 21:29:40 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 01:29:40 -0000 Subject: [Python-checkins] bpo-34537: Fix test_gdb:test_strings with LC_ALL=C (GH-9483) Message-ID: https://github.com/python/cpython/commit/e5fde1f992e94f166415ab96d874ed1d2e0c8004 commit: e5fde1f992e94f166415ab96d874ed1d2e0c8004 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T18:29:34-07:00 summary: bpo-34537: Fix test_gdb:test_strings with LC_ALL=C (GH-9483) We cannot simply call locale.getpreferredencoding() here, as GDB might have been linked against a different version of Python with a different encoding and coercion policy with respect to PEP 538 and PEP 540. Thanks to Victor Stinner for a hint on how to fix this. (cherry picked from commit 7279b5125e7c5d84a473d250b27d353cb7f6628e) Co-authored-by: Elvis Pranskevichus files: A Misc/NEWS.d/next/Tests/2018-09-21-17-33-41.bpo-34537.GImYtZ.rst M Lib/test/test_gdb.py diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py index c2ca57a4a04f..93a2c7dd5758 100644 --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -321,7 +321,20 @@ def test_bytes(self): def test_strings(self): 'Verify the pretty-printing of unicode strings' - encoding = locale.getpreferredencoding() + # We cannot simply call locale.getpreferredencoding() here, + # as GDB might have been linked against a different version + # of Python with a different encoding and coercion policy + # with respect to PEP 538 and PEP 540. + out, err = run_gdb( + '--eval-command', + 'python import locale; print(locale.getpreferredencoding())') + + encoding = out.rstrip() + if err or not encoding: + raise RuntimeError( + f'unable to determine the preferred encoding ' + f'of embedded Python in GDB: {err}') + def check_repr(text): try: text.encode(encoding) diff --git a/Misc/NEWS.d/next/Tests/2018-09-21-17-33-41.bpo-34537.GImYtZ.rst b/Misc/NEWS.d/next/Tests/2018-09-21-17-33-41.bpo-34537.GImYtZ.rst new file mode 100644 index 000000000000..b64a6a762cdb --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2018-09-21-17-33-41.bpo-34537.GImYtZ.rst @@ -0,0 +1,2 @@ +Fix ``test_gdb.test_strings()`` when ``LC_ALL=C`` and GDB was compiled with +Python 3.6 or earlier. From webhook-mailer at python.org Fri Sep 21 21:31:20 2018 From: webhook-mailer at python.org (Guido van Rossum) Date: Sat, 22 Sep 2018 01:31:20 -0000 Subject: [Python-checkins] bpo-32117: Allow tuple unpacking in return and yield statements (gh-4509) Message-ID: https://github.com/python/cpython/commit/fd97d1f1af910a6222ea12aec42c456b64f9aee4 commit: fd97d1f1af910a6222ea12aec42c456b64f9aee4 branch: master author: David Cuthbert committer: Guido van Rossum date: 2018-09-21T18:31:15-07:00 summary: bpo-32117: Allow tuple unpacking in return and yield statements (gh-4509) Iterable unpacking is now allowed without parentheses in yield and return statements, e.g. ``yield 1, 2, 3, *rest``. Thanks to David Cuthbert for the change and jChapman for added tests. files: A Misc/NEWS.d/next/Core and Builtins/2017-11-22-15-43-14.bpo-32117.-vloh8.rst M Grammar/Grammar M Lib/test/test_grammar.py M Python/graminit.c diff --git a/Grammar/Grammar b/Grammar/Grammar index 7d3dd0b86dc6..e232df979e2d 100644 --- a/Grammar/Grammar +++ b/Grammar/Grammar @@ -50,7 +50,7 @@ pass_stmt: 'pass' flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt break_stmt: 'break' continue_stmt: 'continue' -return_stmt: 'return' [testlist] +return_stmt: 'return' [testlist_star_expr] yield_stmt: yield_expr raise_stmt: 'raise' [test ['from' test]] import_stmt: import_name | import_from @@ -147,4 +147,4 @@ comp_if: 'if' test_nocond [comp_iter] encoding_decl: NAME yield_expr: 'yield' [yield_arg] -yield_arg: 'from' test | testlist +yield_arg: 'from' test | testlist_star_expr diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py index 78918ae250c4..462e77a0be55 100644 --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -824,11 +824,17 @@ def test_inner(extra_burning_oil = 1, count=0): test_inner() def test_return(self): - # 'return' [testlist] + # 'return' [testlist_star_expr] def g1(): return def g2(): return 1 + def g3(): + z = [2, 3] + return 1, *z + g1() x = g2() + y = g3() + self.assertEqual(y, (1, 2, 3), "unparenthesized star expr return") check_syntax_error(self, "class foo:return 1") def test_break_in_finally(self): @@ -981,6 +987,9 @@ def g(): f((yield 1)) def g(): f((yield 1), 1) def g(): f((yield from ())) def g(): f((yield from ()), 1) + # Do not require parenthesis for tuple unpacking + def g(): rest = 4, 5, 6; yield 1, 2, 3, *rest + self.assertEquals(list(g()), [(1, 2, 3, 4, 5, 6)]) check_syntax_error(self, "def g(): f(yield 1)") check_syntax_error(self, "def g(): f(yield 1, 1)") check_syntax_error(self, "def g(): f(yield from ())") diff --git a/Misc/NEWS.d/next/Core and Builtins/2017-11-22-15-43-14.bpo-32117.-vloh8.rst b/Misc/NEWS.d/next/Core and Builtins/2017-11-22-15-43-14.bpo-32117.-vloh8.rst new file mode 100644 index 000000000000..9685680ee53e --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2017-11-22-15-43-14.bpo-32117.-vloh8.rst @@ -0,0 +1,3 @@ +Iterable unpacking is now allowed without parenthesis in yield and return +statements, e.g. ``yield 1, 2, 3, *rest``. Thanks to David Cuthbert for the +change and jChapman for added tests. diff --git a/Python/graminit.c b/Python/graminit.c index 5770e8f6a941..0a681f7b797f 100644 --- a/Python/graminit.c +++ b/Python/graminit.c @@ -613,7 +613,7 @@ static arc arcs_25_0[1] = { {75, 1}, }; static arc arcs_25_1[2] = { - {9, 2}, + {47, 2}, {0, 1}, }; static arc arcs_25_2[1] = { @@ -1900,7 +1900,7 @@ static state states_85[3] = { }; static arc arcs_86_0[2] = { {77, 1}, - {9, 2}, + {47, 2}, }; static arc arcs_86_1[1] = { {26, 2}, @@ -2087,7 +2087,7 @@ static dfa dfas[87] = { {341, "yield_expr", 0, 3, states_85, "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000"}, {342, "yield_arg", 0, 3, states_86, - "\000\040\200\000\000\000\000\000\000\040\010\000\000\000\020\002\000\300\220\050\037\000\000"}, + "\000\040\200\000\002\000\000\000\000\040\010\000\000\000\020\002\000\300\220\050\037\000\000"}, }; static label labels[177] = { {0, "EMPTY"}, From webhook-mailer at python.org Fri Sep 21 22:03:13 2018 From: webhook-mailer at python.org (Ethan Furman) Date: Sat, 22 Sep 2018 02:03:13 -0000 Subject: [Python-checkins] bpo-29577: Enum: mixin classes don't mix well with already mixed Enums (GH-9328) Message-ID: https://github.com/python/cpython/commit/5bdab641da0afd8aa581dfbde4f82d88d337c4b5 commit: 5bdab641da0afd8aa581dfbde4f82d88d337c4b5 branch: master author: Ethan Furman committer: GitHub date: 2018-09-21T19:03:09-07:00 summary: bpo-29577: Enum: mixin classes don't mix well with already mixed Enums (GH-9328) * bpo-29577: allow multiple mixin classes files: A Misc/NEWS.d/next/Library/2018-09-14-20-00-47.bpo-29577.RzwKFD.rst M Doc/library/enum.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 6408c0106040..702eacd0e98a 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -387,10 +387,17 @@ whatever value(s) were given to the enum member will be passed into those methods. See `Planet`_ for an example. -Restricted subclassing of enumerations --------------------------------------- +Restricted Enum subclassing +--------------------------- -Subclassing an enumeration is allowed only if the enumeration does not define +A new :class:`Enum` class must have one base Enum class, up to one concrete +data type, and as many :class:`object`-based mixin classes as needed. The +order of these base classes is:: + + def EnumName([mix-in, ...,] [data-type,] base-enum): + pass + +Also, subclassing an enumeration is allowed only if the enumeration does not define any members. So this is forbidden:: >>> class MoreColor(Color): diff --git a/Lib/enum.py b/Lib/enum.py index 02405c865b06..0ccb30d428e4 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -480,37 +480,25 @@ def _get_mixins_(bases): if not bases: return object, Enum - # double check that we are not subclassing a class with existing - # enumeration members; while we're at it, see if any other data - # type has been mixed in so we can use the correct __new__ - member_type = first_enum = None - for base in bases: - if (base is not Enum and - issubclass(base, Enum) and - base._member_names_): - raise TypeError("Cannot extend enumerations") - # base is now the last base in bases - if not issubclass(base, Enum): - raise TypeError("new enumerations must be created as " - "`ClassName([mixin_type,] enum_type)`") - - # get correct mix-in type (either mix-in type of Enum subclass, or - # first base if last base is Enum) - if not issubclass(bases[0], Enum): - member_type = bases[0] # first data type - first_enum = bases[-1] # enum type - else: - for base in bases[0].__mro__: - # most common: (IntEnum, int, Enum, object) - # possible: (, , - # , , - # ) - if issubclass(base, Enum): - if first_enum is None: - first_enum = base - else: - if member_type is None: - member_type = base + def _find_data_type(bases): + for chain in bases: + for base in chain.__mro__: + if base is object: + continue + elif '__new__' in base.__dict__: + if issubclass(base, Enum) and not hasattr(base, '__new_member__'): + continue + return base + + # ensure final parent class is an Enum derivative, find any concrete + # data type, and check that Enum has no members + first_enum = bases[-1] + if not issubclass(first_enum, Enum): + raise TypeError("new enumerations should be created as " + "`EnumName([mixin_type, ...] [data_type,] enum_type)`") + member_type = _find_data_type(bases) or object + if first_enum._member_names_: + raise TypeError("Cannot extend enumerations") return member_type, first_enum diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index b8efb835ce74..aadc11fcc49c 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -122,6 +122,22 @@ def test_is_dunder(self): '__', '___', '____', '_____',): self.assertFalse(enum._is_dunder(s)) +# for subclassing tests + +class classproperty: + + def __init__(self, fget=None, fset=None, fdel=None, doc=None): + self.fget = fget + self.fset = fset + self.fdel = fdel + if doc is None and fget is not None: + doc = fget.__doc__ + self.__doc__ = doc + + def __get__(self, instance, ownerclass): + return self.fget(ownerclass) + + # tests class TestEnum(unittest.TestCase): @@ -1730,6 +1746,102 @@ def _missing_(cls, item): else: raise Exception('Exception not raised.') + def test_multiple_mixin(self): + class MaxMixin: + @classproperty + def MAX(cls): + max = len(cls) + cls.MAX = max + return max + class StrMixin: + def __str__(self): + return self._name_.lower() + class SomeEnum(Enum): + def behavior(self): + return 'booyah' + class AnotherEnum(Enum): + def behavior(self): + return 'nuhuh!' + def social(self): + return "what's up?" + class Color(MaxMixin, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 3) + self.assertEqual(Color.MAX, 3) + self.assertEqual(str(Color.BLUE), 'Color.BLUE') + class Color(MaxMixin, StrMixin, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 3) + self.assertEqual(Color.MAX, 3) + self.assertEqual(str(Color.BLUE), 'blue') + class Color(StrMixin, MaxMixin, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 3) + self.assertEqual(Color.MAX, 3) + self.assertEqual(str(Color.BLUE), 'blue') + class CoolColor(StrMixin, SomeEnum, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(CoolColor.RED.value, 1) + self.assertEqual(CoolColor.GREEN.value, 2) + self.assertEqual(CoolColor.BLUE.value, 3) + self.assertEqual(str(CoolColor.BLUE), 'blue') + self.assertEqual(CoolColor.RED.behavior(), 'booyah') + class CoolerColor(StrMixin, AnotherEnum, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(CoolerColor.RED.value, 1) + self.assertEqual(CoolerColor.GREEN.value, 2) + self.assertEqual(CoolerColor.BLUE.value, 3) + self.assertEqual(str(CoolerColor.BLUE), 'blue') + self.assertEqual(CoolerColor.RED.behavior(), 'nuhuh!') + self.assertEqual(CoolerColor.RED.social(), "what's up?") + class CoolestColor(StrMixin, SomeEnum, AnotherEnum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(CoolestColor.RED.value, 1) + self.assertEqual(CoolestColor.GREEN.value, 2) + self.assertEqual(CoolestColor.BLUE.value, 3) + self.assertEqual(str(CoolestColor.BLUE), 'blue') + self.assertEqual(CoolestColor.RED.behavior(), 'booyah') + self.assertEqual(CoolestColor.RED.social(), "what's up?") + class ConfusedColor(StrMixin, AnotherEnum, SomeEnum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(ConfusedColor.RED.value, 1) + self.assertEqual(ConfusedColor.GREEN.value, 2) + self.assertEqual(ConfusedColor.BLUE.value, 3) + self.assertEqual(str(ConfusedColor.BLUE), 'blue') + self.assertEqual(ConfusedColor.RED.behavior(), 'nuhuh!') + self.assertEqual(ConfusedColor.RED.social(), "what's up?") + class ReformedColor(StrMixin, IntEnum, SomeEnum, AnotherEnum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(ReformedColor.RED.value, 1) + self.assertEqual(ReformedColor.GREEN.value, 2) + self.assertEqual(ReformedColor.BLUE.value, 3) + self.assertEqual(str(ReformedColor.BLUE), 'blue') + self.assertEqual(ReformedColor.RED.behavior(), 'booyah') + self.assertEqual(ConfusedColor.RED.social(), "what's up?") + self.assertTrue(issubclass(ReformedColor, int)) + class TestOrder(unittest.TestCase): @@ -2093,6 +2205,49 @@ class Bizarre(Flag): d = 6 self.assertEqual(repr(Bizarre(7)), '') + def test_multiple_mixin(self): + class AllMixin: + @classproperty + def ALL(cls): + members = list(cls) + all_value = None + if members: + all_value = members[0] + for member in members[1:]: + all_value |= member + cls.ALL = all_value + return all_value + class StrMixin: + def __str__(self): + return self._name_.lower() + class Color(AllMixin, Flag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'Color.BLUE') + class Color(AllMixin, StrMixin, Flag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'blue') + class Color(StrMixin, AllMixin, Flag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'blue') + @support.reap_threads def test_unique_composite(self): # override __eq__ to be identity only @@ -2468,6 +2623,49 @@ def test_bool(self): for f in Open: self.assertEqual(bool(f.value), bool(f)) + def test_multiple_mixin(self): + class AllMixin: + @classproperty + def ALL(cls): + members = list(cls) + all_value = None + if members: + all_value = members[0] + for member in members[1:]: + all_value |= member + cls.ALL = all_value + return all_value + class StrMixin: + def __str__(self): + return self._name_.lower() + class Color(AllMixin, IntFlag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'Color.BLUE') + class Color(AllMixin, StrMixin, IntFlag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'blue') + class Color(StrMixin, AllMixin, IntFlag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'blue') + @support.reap_threads def test_unique_composite(self): # override __eq__ to be identity only @@ -2553,6 +2751,7 @@ class Sillier(IntEnum): value = 4 + expected_help_output_with_docs = """\ Help on class Color in module %s: diff --git a/Misc/NEWS.d/next/Library/2018-09-14-20-00-47.bpo-29577.RzwKFD.rst b/Misc/NEWS.d/next/Library/2018-09-14-20-00-47.bpo-29577.RzwKFD.rst new file mode 100644 index 000000000000..bd71ac496a67 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-14-20-00-47.bpo-29577.RzwKFD.rst @@ -0,0 +1 @@ +Support multiple mixin classes when creating Enums. From webhook-mailer at python.org Sat Sep 22 00:42:33 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 04:42:33 -0000 Subject: [Python-checkins] bpo-34623: Mention CVE-2018-14647 in news entry (GH-9482) Message-ID: https://github.com/python/cpython/commit/026337a7101369297c8083047d2f3c6fc9dd1e2b commit: 026337a7101369297c8083047d2f3c6fc9dd1e2b branch: master author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-21T21:42:29-07:00 summary: bpo-34623: Mention CVE-2018-14647 in news entry (GH-9482) https://bugs.python.org/issue34623 files: M Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst diff --git a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst index 31ad92ef8582..cbaa4b750644 100644 --- a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst +++ b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst @@ -1,2 +1,2 @@ -The C accelerated _elementtree module now initializes hash randomization -salt from _Py_HashSecret instead of libexpat's default CSPRNG. +CVE-2018-14647: The C accelerated _elementtree module now initializes hash +randomization salt from _Py_HashSecret instead of libexpat's default CSPRNG. From webhook-mailer at python.org Sat Sep 22 00:44:15 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 04:44:15 -0000 Subject: [Python-checkins] bpo-34759: Fix error handling in ssl 'unwrap()' (GH-9468) Message-ID: https://github.com/python/cpython/commit/c0da582b227f311126e278b5553a7fa89c79b054 commit: c0da582b227f311126e278b5553a7fa89c79b054 branch: master author: Nathaniel J. Smith committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-21T21:44:12-07:00 summary: bpo-34759: Fix error handling in ssl 'unwrap()' (GH-9468) OpenSSL follows the convention that whenever you call a function, it returns an error indicator value; and if this value is negative, then you need to go look at the actual error code to see what happened. Commit c6fd1c1c3a introduced a small mistake in _ssl__SSLSocket_shutdown_impl: instead of checking whether the error indicator was negative, it started checking whether the actual error code was negative, and it turns out that the error codes are never negative. So the effect was that 'unwrap()' lost the ability to raise SSL errors. https://bugs.python.org/issue34759 files: M Lib/test/test_ssl.py M Modules/_ssl.c diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index e120a2f9adf2..b4cafc15c5cd 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1715,6 +1715,47 @@ def test_private_init(self): with self.assertRaisesRegex(TypeError, "public constructor"): ssl.SSLObject(bio, bio) + def test_unwrap(self): + client_ctx, server_ctx, hostname = testing_context() + c_in = ssl.MemoryBIO() + c_out = ssl.MemoryBIO() + s_in = ssl.MemoryBIO() + s_out = ssl.MemoryBIO() + client = client_ctx.wrap_bio(c_in, c_out, server_hostname=hostname) + server = server_ctx.wrap_bio(s_in, s_out, server_side=True) + + # Loop on the handshake for a bit to get it settled + for _ in range(5): + try: + client.do_handshake() + except ssl.SSLWantReadError: + pass + if c_out.pending: + s_in.write(c_out.read()) + try: + server.do_handshake() + except ssl.SSLWantReadError: + pass + if s_out.pending: + c_in.write(s_out.read()) + # Now the handshakes should be complete (don't raise WantReadError) + client.do_handshake() + server.do_handshake() + + # Now if we unwrap one side unilaterally, it should send close-notify + # and raise WantReadError: + with self.assertRaises(ssl.SSLWantReadError): + client.unwrap() + + # But server.unwrap() does not raise, because it reads the client's + # close-notify: + s_in.write(c_out.read()) + server.unwrap() + + # And now that the client gets the server's close-notify, it doesn't + # raise either. + c_in.write(s_out.read()) + client.unwrap() class SimpleBackgroundTests(unittest.TestCase): """Tests that connect to a simple server running in the background""" diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 4750b930c642..5b5d7dd445d2 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2583,9 +2583,9 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) break; } - if (err.ssl < 0) { + if (ret < 0) { Py_XDECREF(sock); - return PySSL_SetError(self, err.ssl, __FILE__, __LINE__); + return PySSL_SetError(self, ret, __FILE__, __LINE__); } if (sock) /* It's already INCREF'ed */ From webhook-mailer at python.org Sat Sep 22 00:57:05 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 04:57:05 -0000 Subject: [Python-checkins] [2.7] bpo-34623: Mention CVE-2018-14647 in news entry (GH-9482) (GH-9490) Message-ID: https://github.com/python/cpython/commit/10be1d3f802b874914b2a13eb41407c7a582d9b3 commit: 10be1d3f802b874914b2a13eb41407c7a582d9b3 branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T21:57:00-07:00 summary: [2.7] bpo-34623: Mention CVE-2018-14647 in news entry (GH-9482) (GH-9490) https://bugs.python.org/issue34623 (cherry picked from commit 026337a7101369297c8083047d2f3c6fc9dd1e2b) Co-authored-by: Christian Heimes https://bugs.python.org/issue34623 files: M Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst diff --git a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst index 31ad92ef8582..cbaa4b750644 100644 --- a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst +++ b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst @@ -1,2 +1,2 @@ -The C accelerated _elementtree module now initializes hash randomization -salt from _Py_HashSecret instead of libexpat's default CSPRNG. +CVE-2018-14647: The C accelerated _elementtree module now initializes hash +randomization salt from _Py_HashSecret instead of libexpat's default CSPRNG. From webhook-mailer at python.org Sat Sep 22 00:57:31 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 04:57:31 -0000 Subject: [Python-checkins] [3.6] bpo-34623: Mention CVE-2018-14647 in news entry (GH-9482) (GH-9489) Message-ID: https://github.com/python/cpython/commit/d1b336e530472f316b1d164d04626724c83b16d7 commit: d1b336e530472f316b1d164d04626724c83b16d7 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T21:57:28-07:00 summary: [3.6] bpo-34623: Mention CVE-2018-14647 in news entry (GH-9482) (GH-9489) https://bugs.python.org/issue34623 (cherry picked from commit 026337a7101369297c8083047d2f3c6fc9dd1e2b) Co-authored-by: Christian Heimes https://bugs.python.org/issue34623 files: M Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst diff --git a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst index 31ad92ef8582..cbaa4b750644 100644 --- a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst +++ b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst @@ -1,2 +1,2 @@ -The C accelerated _elementtree module now initializes hash randomization -salt from _Py_HashSecret instead of libexpat's default CSPRNG. +CVE-2018-14647: The C accelerated _elementtree module now initializes hash +randomization salt from _Py_HashSecret instead of libexpat's default CSPRNG. From webhook-mailer at python.org Sat Sep 22 00:57:49 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 04:57:49 -0000 Subject: [Python-checkins] [3.7] bpo-34623: Mention CVE-2018-14647 in news entry (GH-9482) (GH-9488) Message-ID: https://github.com/python/cpython/commit/5c3d8b2efda1b99abe09ad925f366c5695bd66fb commit: 5c3d8b2efda1b99abe09ad925f366c5695bd66fb branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T21:57:46-07:00 summary: [3.7] bpo-34623: Mention CVE-2018-14647 in news entry (GH-9482) (GH-9488) https://bugs.python.org/issue34623 (cherry picked from commit 026337a7101369297c8083047d2f3c6fc9dd1e2b) Co-authored-by: Christian Heimes https://bugs.python.org/issue34623 files: M Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst diff --git a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst index 31ad92ef8582..cbaa4b750644 100644 --- a/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst +++ b/Misc/NEWS.d/next/Security/2018-09-10-16-05-39.bpo-34623.Ua9jMv.rst @@ -1,2 +1,2 @@ -The C accelerated _elementtree module now initializes hash randomization -salt from _Py_HashSecret instead of libexpat's default CSPRNG. +CVE-2018-14647: The C accelerated _elementtree module now initializes hash +randomization salt from _Py_HashSecret instead of libexpat's default CSPRNG. From webhook-mailer at python.org Sat Sep 22 01:00:46 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 05:00:46 -0000 Subject: [Python-checkins] bpo-34759: Fix error handling in ssl 'unwrap()' (GH-9468) Message-ID: https://github.com/python/cpython/commit/c00f7037df3607c89323e68db3ab996b7df394de commit: c00f7037df3607c89323e68db3ab996b7df394de branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-21T22:00:42-07:00 summary: bpo-34759: Fix error handling in ssl 'unwrap()' (GH-9468) OpenSSL follows the convention that whenever you call a function, it returns an error indicator value; and if this value is negative, then you need to go look at the actual error code to see what happened. Commit c6fd1c1c3a introduced a small mistake in _ssl__SSLSocket_shutdown_impl: instead of checking whether the error indicator was negative, it started checking whether the actual error code was negative, and it turns out that the error codes are never negative. So the effect was that 'unwrap()' lost the ability to raise SSL errors. https://bugs.python.org/issue34759 (cherry picked from commit c0da582b227f311126e278b5553a7fa89c79b054) Co-authored-by: Nathaniel J. Smith files: M Lib/test/test_ssl.py M Modules/_ssl.c diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index 49a4d7295422..4d5925a9355e 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1735,6 +1735,47 @@ def test_private_init(self): with self.assertRaisesRegex(TypeError, "public constructor"): ssl.SSLObject(bio, bio) + def test_unwrap(self): + client_ctx, server_ctx, hostname = testing_context() + c_in = ssl.MemoryBIO() + c_out = ssl.MemoryBIO() + s_in = ssl.MemoryBIO() + s_out = ssl.MemoryBIO() + client = client_ctx.wrap_bio(c_in, c_out, server_hostname=hostname) + server = server_ctx.wrap_bio(s_in, s_out, server_side=True) + + # Loop on the handshake for a bit to get it settled + for _ in range(5): + try: + client.do_handshake() + except ssl.SSLWantReadError: + pass + if c_out.pending: + s_in.write(c_out.read()) + try: + server.do_handshake() + except ssl.SSLWantReadError: + pass + if s_out.pending: + c_in.write(s_out.read()) + # Now the handshakes should be complete (don't raise WantReadError) + client.do_handshake() + server.do_handshake() + + # Now if we unwrap one side unilaterally, it should send close-notify + # and raise WantReadError: + with self.assertRaises(ssl.SSLWantReadError): + client.unwrap() + + # But server.unwrap() does not raise, because it reads the client's + # close-notify: + s_in.write(c_out.read()) + server.unwrap() + + # And now that the client gets the server's close-notify, it doesn't + # raise either. + c_in.write(s_out.read()) + client.unwrap() class SimpleBackgroundTests(unittest.TestCase): """Tests that connect to a simple server running in the background""" diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 7ff616eacdce..faca3a27c562 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2584,9 +2584,9 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) break; } - if (err.ssl < 0) { + if (ret < 0) { Py_XDECREF(sock); - return PySSL_SetError(self, err.ssl, __FILE__, __LINE__); + return PySSL_SetError(self, ret, __FILE__, __LINE__); } if (sock) /* It's already INCREF'ed */ From webhook-mailer at python.org Sat Sep 22 01:10:10 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 05:10:10 -0000 Subject: [Python-checkins] [3.6] bpo-34759: Fix error handling in ssl 'unwrap()' (GH-9468) (GH-9492) Message-ID: https://github.com/python/cpython/commit/7529754d26f5e744ae25bee56fdc1937bcf08c7e commit: 7529754d26f5e744ae25bee56fdc1937bcf08c7e branch: 3.6 author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-21T22:10:06-07:00 summary: [3.6] bpo-34759: Fix error handling in ssl 'unwrap()' (GH-9468) (GH-9492) OpenSSL follows the convention that whenever you call a function, it returns an error indicator value; and if this value is negative, then you need to go look at the actual error code to see what happened. Commit c6fd1c1c3a introduced a small mistake in _ssl__SSLSocket_shutdown_impl: instead of checking whether the error indicator was negative, it started checking whether the actual error code was negative, and it turns out that the error codes are never negative. So the effect was that 'unwrap()' lost the ability to raise SSL errors. https://bugs.python.org/issue34759. (cherry picked from commit c0da582b227f311126e278b5553a7fa89c79b054) Co-authored-by: Nathaniel J. Smith https://bugs.python.org/issue34759 files: M Modules/_ssl.c diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 327f6ae7018b..2badf3172252 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2407,9 +2407,9 @@ _ssl__SSLSocket_shutdown_impl(PySSLSocket *self) break; } - if (err.ssl < 0) { + if (ret < 0) { Py_XDECREF(sock); - return PySSL_SetError(self, err.ssl, __FILE__, __LINE__); + return PySSL_SetError(self, ret, __FILE__, __LINE__); } if (sock) /* It's already INCREF'ed */ From webhook-mailer at python.org Sat Sep 22 01:26:36 2018 From: webhook-mailer at python.org (Ethan Furman) Date: Sat, 22 Sep 2018 05:26:36 -0000 Subject: [Python-checkins] [3.7] bpo-29577: Enum: mixin classes don't mix well with already mixed Enums (GH-9328) (GH-9486) Message-ID: https://github.com/python/cpython/commit/0c076caaa82a9c6596e1fe1dbe6384d53f30a1a3 commit: 0c076caaa82a9c6596e1fe1dbe6384d53f30a1a3 branch: 3.7 author: Ethan Furman committer: GitHub date: 2018-09-21T22:26:32-07:00 summary: [3.7] bpo-29577: Enum: mixin classes don't mix well with already mixed Enums (GH-9328) (GH-9486) * bpo-29577: allow multiple mixin classes files: A Misc/NEWS.d/next/Library/2018-09-14-20-00-47.bpo-29577.RzwKFD.rst M Doc/library/enum.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index a6aa863a1cc9..81e9766e429b 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -387,10 +387,17 @@ whatever value(s) were given to the enum member will be passed into those methods. See `Planet`_ for an example. -Restricted subclassing of enumerations --------------------------------------- +Restricted Enum subclassing +--------------------------- -Subclassing an enumeration is allowed only if the enumeration does not define +A new :class:`Enum` class must have one base Enum class, up to one concrete +data type, and as many :class:`object`-based mixin classes as needed. The +order of these base classes is:: + + def EnumName([mix-in, ...,] [data-type,] base-enum): + pass + +Also, subclassing an enumeration is allowed only if the enumeration does not define any members. So this is forbidden:: >>> class MoreColor(Color): diff --git a/Lib/enum.py b/Lib/enum.py index 69b41fe7cbc0..4e8a56818b3c 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -449,37 +449,25 @@ def _get_mixins_(bases): if not bases: return object, Enum - # double check that we are not subclassing a class with existing - # enumeration members; while we're at it, see if any other data - # type has been mixed in so we can use the correct __new__ - member_type = first_enum = None - for base in bases: - if (base is not Enum and - issubclass(base, Enum) and - base._member_names_): - raise TypeError("Cannot extend enumerations") - # base is now the last base in bases - if not issubclass(base, Enum): - raise TypeError("new enumerations must be created as " - "`ClassName([mixin_type,] enum_type)`") - - # get correct mix-in type (either mix-in type of Enum subclass, or - # first base if last base is Enum) - if not issubclass(bases[0], Enum): - member_type = bases[0] # first data type - first_enum = bases[-1] # enum type - else: - for base in bases[0].__mro__: - # most common: (IntEnum, int, Enum, object) - # possible: (, , - # , , - # ) - if issubclass(base, Enum): - if first_enum is None: - first_enum = base - else: - if member_type is None: - member_type = base + def _find_data_type(bases): + for chain in bases: + for base in chain.__mro__: + if base is object: + continue + elif '__new__' in base.__dict__: + if issubclass(base, Enum) and not hasattr(base, '__new_member__'): + continue + return base + + # ensure final parent class is an Enum derivative, find any concrete + # data type, and check that Enum has no members + first_enum = bases[-1] + if not issubclass(first_enum, Enum): + raise TypeError("new enumerations should be created as " + "`EnumName([mixin_type, ...] [data_type,] enum_type)`") + member_type = _find_data_type(bases) or object + if first_enum._member_names_: + raise TypeError("Cannot extend enumerations") return member_type, first_enum diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 4b1722894601..6c147d7ca6e1 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -120,6 +120,22 @@ def test_is_dunder(self): '__', '___', '____', '_____',): self.assertFalse(enum._is_dunder(s)) +# for subclassing tests + +class classproperty: + + def __init__(self, fget=None, fset=None, fdel=None, doc=None): + self.fget = fget + self.fset = fset + self.fdel = fdel + if doc is None and fget is not None: + doc = fget.__doc__ + self.__doc__ = doc + + def __get__(self, instance, ownerclass): + return self.fget(ownerclass) + + # tests class TestEnum(unittest.TestCase): @@ -1701,6 +1717,102 @@ class Dupes(Enum): third = auto() self.assertEqual([Dupes.first, Dupes.second, Dupes.third], list(Dupes)) + def test_multiple_mixin(self): + class MaxMixin: + @classproperty + def MAX(cls): + max = len(cls) + cls.MAX = max + return max + class StrMixin: + def __str__(self): + return self._name_.lower() + class SomeEnum(Enum): + def behavior(self): + return 'booyah' + class AnotherEnum(Enum): + def behavior(self): + return 'nuhuh!' + def social(self): + return "what's up?" + class Color(MaxMixin, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 3) + self.assertEqual(Color.MAX, 3) + self.assertEqual(str(Color.BLUE), 'Color.BLUE') + class Color(MaxMixin, StrMixin, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 3) + self.assertEqual(Color.MAX, 3) + self.assertEqual(str(Color.BLUE), 'blue') + class Color(StrMixin, MaxMixin, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 3) + self.assertEqual(Color.MAX, 3) + self.assertEqual(str(Color.BLUE), 'blue') + class CoolColor(StrMixin, SomeEnum, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(CoolColor.RED.value, 1) + self.assertEqual(CoolColor.GREEN.value, 2) + self.assertEqual(CoolColor.BLUE.value, 3) + self.assertEqual(str(CoolColor.BLUE), 'blue') + self.assertEqual(CoolColor.RED.behavior(), 'booyah') + class CoolerColor(StrMixin, AnotherEnum, Enum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(CoolerColor.RED.value, 1) + self.assertEqual(CoolerColor.GREEN.value, 2) + self.assertEqual(CoolerColor.BLUE.value, 3) + self.assertEqual(str(CoolerColor.BLUE), 'blue') + self.assertEqual(CoolerColor.RED.behavior(), 'nuhuh!') + self.assertEqual(CoolerColor.RED.social(), "what's up?") + class CoolestColor(StrMixin, SomeEnum, AnotherEnum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(CoolestColor.RED.value, 1) + self.assertEqual(CoolestColor.GREEN.value, 2) + self.assertEqual(CoolestColor.BLUE.value, 3) + self.assertEqual(str(CoolestColor.BLUE), 'blue') + self.assertEqual(CoolestColor.RED.behavior(), 'booyah') + self.assertEqual(CoolestColor.RED.social(), "what's up?") + class ConfusedColor(StrMixin, AnotherEnum, SomeEnum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(ConfusedColor.RED.value, 1) + self.assertEqual(ConfusedColor.GREEN.value, 2) + self.assertEqual(ConfusedColor.BLUE.value, 3) + self.assertEqual(str(ConfusedColor.BLUE), 'blue') + self.assertEqual(ConfusedColor.RED.behavior(), 'nuhuh!') + self.assertEqual(ConfusedColor.RED.social(), "what's up?") + class ReformedColor(StrMixin, IntEnum, SomeEnum, AnotherEnum): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(ReformedColor.RED.value, 1) + self.assertEqual(ReformedColor.GREEN.value, 2) + self.assertEqual(ReformedColor.BLUE.value, 3) + self.assertEqual(str(ReformedColor.BLUE), 'blue') + self.assertEqual(ReformedColor.RED.behavior(), 'booyah') + self.assertEqual(ConfusedColor.RED.social(), "what's up?") + self.assertTrue(issubclass(ReformedColor, int)) + class TestOrder(unittest.TestCase): @@ -2064,6 +2176,49 @@ class Bizarre(Flag): d = 6 self.assertEqual(repr(Bizarre(7)), '') + def test_multiple_mixin(self): + class AllMixin: + @classproperty + def ALL(cls): + members = list(cls) + all_value = None + if members: + all_value = members[0] + for member in members[1:]: + all_value |= member + cls.ALL = all_value + return all_value + class StrMixin: + def __str__(self): + return self._name_.lower() + class Color(AllMixin, Flag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'Color.BLUE') + class Color(AllMixin, StrMixin, Flag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'blue') + class Color(StrMixin, AllMixin, Flag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'blue') + @support.reap_threads def test_unique_composite(self): # override __eq__ to be identity only @@ -2439,6 +2594,49 @@ def test_bool(self): for f in Open: self.assertEqual(bool(f.value), bool(f)) + def test_multiple_mixin(self): + class AllMixin: + @classproperty + def ALL(cls): + members = list(cls) + all_value = None + if members: + all_value = members[0] + for member in members[1:]: + all_value |= member + cls.ALL = all_value + return all_value + class StrMixin: + def __str__(self): + return self._name_.lower() + class Color(AllMixin, IntFlag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'Color.BLUE') + class Color(AllMixin, StrMixin, IntFlag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'blue') + class Color(StrMixin, AllMixin, IntFlag): + RED = auto() + GREEN = auto() + BLUE = auto() + self.assertEqual(Color.RED.value, 1) + self.assertEqual(Color.GREEN.value, 2) + self.assertEqual(Color.BLUE.value, 4) + self.assertEqual(Color.ALL.value, 7) + self.assertEqual(str(Color.BLUE), 'blue') + @support.reap_threads def test_unique_composite(self): # override __eq__ to be identity only @@ -2524,6 +2722,7 @@ class Sillier(IntEnum): value = 4 + expected_help_output_with_docs = """\ Help on class Color in module %s: diff --git a/Misc/NEWS.d/next/Library/2018-09-14-20-00-47.bpo-29577.RzwKFD.rst b/Misc/NEWS.d/next/Library/2018-09-14-20-00-47.bpo-29577.RzwKFD.rst new file mode 100644 index 000000000000..bd71ac496a67 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-14-20-00-47.bpo-29577.RzwKFD.rst @@ -0,0 +1 @@ +Support multiple mixin classes when creating Enums. From solipsis at pitrou.net Sat Sep 22 05:07:54 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 22 Sep 2018 09:07:54 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=1 Message-ID: <20180922090754.1.C4E9C42AE15E8CFE@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [-7, 8, -7] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [2, -1, 1] memory blocks, sum=2 test_multiprocessing_forkserver leaked [-1, 2, 0] memory blocks, sum=1 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogRcf7Ic', '--timeout', '7200'] From webhook-mailer at python.org Sat Sep 22 11:13:57 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sat, 22 Sep 2018 15:13:57 -0000 Subject: [Python-checkins] Fix pickletools doc for NEWFALSE. (GH-9432) Message-ID: https://github.com/python/cpython/commit/488cfb78c8b81075942b5e4cc9630e7a6dd9dc28 commit: 488cfb78c8b81075942b5e4cc9630e7a6dd9dc28 branch: master author: Krzysztof Wroblewski committer: Serhiy Storchaka date: 2018-09-22T18:13:53+03:00 summary: Fix pickletools doc for NEWFALSE. (GH-9432) Also make docs for NEWFALSE and NEWTRUE more consistent with docs for other opcodes. files: M Lib/pickletools.py diff --git a/Lib/pickletools.py b/Lib/pickletools.py index 8486cbf84368..ed8bee36e8c5 100644 --- a/Lib/pickletools.py +++ b/Lib/pickletools.py @@ -1325,9 +1325,7 @@ def __init__(self, name, code, arg, stack_before=[], stack_after=[pybool], proto=2, - doc="""True. - - Push True onto the stack."""), + doc="Push True onto the stack."), I(name='NEWFALSE', code='\x89', @@ -1335,9 +1333,7 @@ def __init__(self, name, code, arg, stack_before=[], stack_after=[pybool], proto=2, - doc="""True. - - Push False onto the stack."""), + doc="Push False onto the stack."), # Ways to spell Unicode strings. From webhook-mailer at python.org Sat Sep 22 11:27:50 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 15:27:50 -0000 Subject: [Python-checkins] Fix pickletools doc for NEWFALSE. (GH-9432) Message-ID: https://github.com/python/cpython/commit/09f2eec05420a9df76de8ab65d1dd89e9d0aa4b7 commit: 09f2eec05420a9df76de8ab65d1dd89e9d0aa4b7 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-22T08:27:47-07:00 summary: Fix pickletools doc for NEWFALSE. (GH-9432) Also make docs for NEWFALSE and NEWTRUE more consistent with docs for other opcodes. (cherry picked from commit 488cfb78c8b81075942b5e4cc9630e7a6dd9dc28) Co-authored-by: Krzysztof Wroblewski files: M Lib/pickletools.py diff --git a/Lib/pickletools.py b/Lib/pickletools.py index 5e129b5b5637..c0415ec46b76 100644 --- a/Lib/pickletools.py +++ b/Lib/pickletools.py @@ -1354,9 +1354,7 @@ def __init__(self, name, code, arg, stack_before=[], stack_after=[pybool], proto=2, - doc="""True. - - Push True onto the stack."""), + doc="Push True onto the stack."), I(name='NEWFALSE', code='\x89', @@ -1364,9 +1362,7 @@ def __init__(self, name, code, arg, stack_before=[], stack_after=[pybool], proto=2, - doc="""True. - - Push False onto the stack."""), + doc="Push False onto the stack."), # Ways to spell Unicode strings. From webhook-mailer at python.org Sat Sep 22 11:39:00 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 22 Sep 2018 15:39:00 -0000 Subject: [Python-checkins] Fix pickletools doc for NEWFALSE. (GH-9432) Message-ID: https://github.com/python/cpython/commit/fef3a92be9e58aa4175daae98422f9b50b75ff84 commit: fef3a92be9e58aa4175daae98422f9b50b75ff84 branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-22T08:38:57-07:00 summary: Fix pickletools doc for NEWFALSE. (GH-9432) Also make docs for NEWFALSE and NEWTRUE more consistent with docs for other opcodes. (cherry picked from commit 488cfb78c8b81075942b5e4cc9630e7a6dd9dc28) Co-authored-by: Krzysztof Wroblewski files: M Lib/pickletools.py diff --git a/Lib/pickletools.py b/Lib/pickletools.py index c9366c8ce3ea..cc15540e554b 100644 --- a/Lib/pickletools.py +++ b/Lib/pickletools.py @@ -1048,9 +1048,7 @@ def __init__(self, name, code, arg, stack_before=[], stack_after=[pybool], proto=2, - doc="""True. - - Push True onto the stack."""), + doc="Push True onto the stack."), I(name='NEWFALSE', code='\x89', @@ -1058,9 +1056,7 @@ def __init__(self, name, code, arg, stack_before=[], stack_after=[pybool], proto=2, - doc="""True. - - Push False onto the stack."""), + doc="Push False onto the stack."), # Ways to spell Unicode strings. From webhook-mailer at python.org Sat Sep 22 13:48:26 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sat, 22 Sep 2018 17:48:26 -0000 Subject: [Python-checkins] =?utf-8?q?=5B2=2E7=5D_bpo-34472=3A_Add_data_de?= =?utf-8?q?scriptor_signature_to_zipfile_=28GH-8871=29_=28=D0=9F=D0=A0-940?= =?utf-8?q?7=29?= Message-ID: https://github.com/python/cpython/commit/6ec298114855b648a1f5fc4188ea3686a9d77fb3 commit: 6ec298114855b648a1f5fc4188ea3686a9d77fb3 branch: 2.7 author: Serhiy Storchaka committer: GitHub date: 2018-09-22T20:48:23+03:00 summary: [2.7] bpo-34472: Add data descriptor signature to zipfile (GH-8871) (??-9407) This makes streamed zips compatible with MacOS Archive Utility and other applications. (cherry picked from commit 4ba3b50bfe6d50cd82d208023ea23e203ab50589) Co-authored-by: Silas Sewell files: A Misc/NEWS.d/next/Library/2018-08-23-09-25-08.bpo-34472.cGyYrO.rst M Lib/zipfile.py M Misc/ACKS diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 0f890ac6f3ab..991a0add205d 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -131,6 +131,8 @@ class LargeZipFile(Exception): _CD64_DIRECTORY_SIZE = 8 _CD64_OFFSET_START_CENTDIR = 9 +_DD_SIGNATURE = 0x08074b50 + _EXTRA_FIELD_STRUCT = struct.Struct(' https://github.com/python/cpython/commit/ed21919d69ac22232cbc0dad0323477818112b6f commit: ed21919d69ac22232cbc0dad0323477818112b6f branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Serhiy Storchaka date: 2018-09-22T21:02:53+03:00 summary: bpo-34472: Add data descriptor signature to zipfile (GH-8871) (GH-9398) This makes streamed zips compatible with MacOS Archive Utility and other applications. (cherry picked from commit 4ba3b50bfe6d50cd82d208023ea23e203ab50589) Co-authored-by: Silas Sewell files: A Misc/NEWS.d/next/Library/2018-08-23-09-25-08.bpo-34472.cGyYrO.rst M Lib/zipfile.py M Misc/ACKS diff --git a/Lib/zipfile.py b/Lib/zipfile.py index bc757a37b5c4..5bb35870a689 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -164,6 +164,8 @@ class LargeZipFile(Exception): _CD64_DIRECTORY_SIZE = 8 _CD64_OFFSET_START_CENTDIR = 9 +_DD_SIGNATURE = 0x08074b50 + _EXTRA_FIELD_STRUCT = struct.Struct(' https://github.com/python/cpython/commit/44989bc2696320cf55ae6f329aaf58edd49d792a commit: 44989bc2696320cf55ae6f329aaf58edd49d792a branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Serhiy Storchaka date: 2018-09-22T21:03:04+03:00 summary: bpo-34472: Add data descriptor signature to zipfile (GH-8871) (GH-9399) This makes streamed zips compatible with MacOS Archive Utility and other applications. (cherry picked from commit 4ba3b50bfe6d50cd82d208023ea23e203ab50589) Co-authored-by: Silas Sewell files: A Misc/NEWS.d/next/Library/2018-08-23-09-25-08.bpo-34472.cGyYrO.rst M Lib/zipfile.py M Misc/ACKS diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 9f88512d9838..a43878575db8 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -159,6 +159,8 @@ class LargeZipFile(Exception): _CD64_DIRECTORY_SIZE = 8 _CD64_OFFSET_START_CENTDIR = 9 +_DD_SIGNATURE = 0x08074b50 + _EXTRA_FIELD_STRUCT = struct.Struct(' https://github.com/python/cpython/commit/69d0bc1430d2e9cddf0b39054ddcb86dbbe7927e commit: 69d0bc1430d2e9cddf0b39054ddcb86dbbe7927e branch: 2.7 author: Serhiy Storchaka committer: GitHub date: 2018-09-22T21:34:16+03:00 summary: [2.7] bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113). (GH-9500) (cherry picked from commit e0e5065daef36dafe10a46eaa8b7800274d73062) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst M Lib/multiprocessing/managers.py M Lib/test/test_multiprocessing.py diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py index f2cee0c38ce9..118812c8ce7e 100644 --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -1059,10 +1059,13 @@ def __imul__(self, value): DictProxy = MakeProxyType('DictProxy', ( - '__contains__', '__delitem__', '__getitem__', '__len__', + '__contains__', '__delitem__', '__getitem__', '__iter__', '__len__', '__setitem__', 'clear', 'copy', 'get', 'has_key', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values' )) +DictProxy._method_to_typeid_ = { + '__iter__': 'Iterator', + } ArrayProxy = MakeProxyType('ArrayProxy', ( diff --git a/Lib/test/test_multiprocessing.py b/Lib/test/test_multiprocessing.py index 42ccbcdf675d..ff299feed894 100644 --- a/Lib/test/test_multiprocessing.py +++ b/Lib/test/test_multiprocessing.py @@ -1135,6 +1135,16 @@ def test_list(self): a.append('hello') self.assertEqual(f[:], [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 'hello']]) + def test_list_iter(self): + a = self.list(range(10)) + it = iter(a) + self.assertEqual(list(it), range(10)) + self.assertEqual(list(it), []) # exhausted + # list modified during iteration + it = iter(a) + a[0] = 100 + self.assertEqual(next(it), 100) + def test_dict(self): d = self.dict() indices = range(65, 70) @@ -1145,6 +1155,19 @@ def test_dict(self): self.assertEqual(sorted(d.values()), [chr(i) for i in indices]) self.assertEqual(sorted(d.items()), [(i, chr(i)) for i in indices]) + def test_dict_iter(self): + d = self.dict() + indices = range(65, 70) + for i in indices: + d[i] = chr(i) + it = iter(d) + self.assertEqual(list(it), indices) + self.assertEqual(list(it), []) # exhausted + # dictionary changed size during iteration + it = iter(d) + d.clear() + self.assertRaises(RuntimeError, next, it) + def test_namespace(self): n = self.Namespace() n.name = 'Bob' diff --git a/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst b/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst new file mode 100644 index 000000000000..bffb355ea2ab --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-08-12-57-07.bpo-34610.wmoP5j.rst @@ -0,0 +1 @@ +Fixed iterator of :class:`multiprocessing.managers.DictProxy`. From webhook-mailer at python.org Sat Sep 22 21:13:14 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 23 Sep 2018 01:13:14 -0000 Subject: [Python-checkins] bpo-32117: Iterable unpacking in return and yield documentation (GH-9487) Message-ID: https://github.com/python/cpython/commit/8fabae3b00b2ccffd9f7bf4736734ae584ac5829 commit: 8fabae3b00b2ccffd9f7bf4736734ae584ac5829 branch: master author: jChapman committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-22T18:13:10-07:00 summary: bpo-32117: Iterable unpacking in return and yield documentation (GH-9487) News entry clean up, added to what's new Requested by @gvanrossum in https://github.com/python/cpython/pull/4509 https://bugs.python.org/issue32117 files: M Doc/whatsnew/3.8.rst M Misc/NEWS.d/next/Core and Builtins/2017-11-22-15-43-14.bpo-32117.-vloh8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 26928fbd55c0..c464346987b9 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -103,6 +103,10 @@ Other Language Changes never intended to permit more than a bare name on the left-hand side of a keyword argument assignment term. See :issue:`34641`. +* Iterable unpacking is now allowed without parentheses in :keyword:`yield` + and :keyword:`return` statements. + (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.) + New Modules =========== diff --git a/Misc/NEWS.d/next/Core and Builtins/2017-11-22-15-43-14.bpo-32117.-vloh8.rst b/Misc/NEWS.d/next/Core and Builtins/2017-11-22-15-43-14.bpo-32117.-vloh8.rst index 9685680ee53e..8add90c5b70a 100644 --- a/Misc/NEWS.d/next/Core and Builtins/2017-11-22-15-43-14.bpo-32117.-vloh8.rst +++ b/Misc/NEWS.d/next/Core and Builtins/2017-11-22-15-43-14.bpo-32117.-vloh8.rst @@ -1,3 +1,3 @@ -Iterable unpacking is now allowed without parenthesis in yield and return +Iterable unpacking is now allowed without parentheses in yield and return statements, e.g. ``yield 1, 2, 3, *rest``. Thanks to David Cuthbert for the -change and jChapman for added tests. +change and Jordan Chapman for added tests. From webhook-mailer at python.org Sun Sep 23 02:13:03 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sun, 23 Sep 2018 06:13:03 -0000 Subject: [Python-checkins] bpo-34421: Improve distutils logging for non-ASCII strings. (GH-9126) Message-ID: https://github.com/python/cpython/commit/4b860fd777e983f5d2a6bd1288e2b53099c6a803 commit: 4b860fd777e983f5d2a6bd1288e2b53099c6a803 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-23T09:12:59+03:00 summary: bpo-34421: Improve distutils logging for non-ASCII strings. (GH-9126) Use "backslashreplace" instead of "unicode-escape". It is not implementation depended and escapes only non-encodable characters. Also simplify the code. files: M Lib/distutils/log.py M Lib/distutils/tests/test_log.py diff --git a/Lib/distutils/log.py b/Lib/distutils/log.py index 3a6602bc8b8e..8ef6b28ea2ec 100644 --- a/Lib/distutils/log.py +++ b/Lib/distutils/log.py @@ -27,14 +27,13 @@ def _log(self, level, msg, args): stream = sys.stderr else: stream = sys.stdout - if stream.errors == 'strict': + try: + stream.write('%s\n' % msg) + except UnicodeEncodeError: # emulate backslashreplace error handler encoding = stream.encoding msg = msg.encode(encoding, "backslashreplace").decode(encoding) - try: stream.write('%s\n' % msg) - except UnicodeEncodeError: - stream.write('%s\n' % msg.encode('unicode-escape').decode('ascii')) stream.flush() def log(self, level, msg, *args): diff --git a/Lib/distutils/tests/test_log.py b/Lib/distutils/tests/test_log.py index 0c2ad7a4268d..22c26246ca9d 100644 --- a/Lib/distutils/tests/test_log.py +++ b/Lib/distutils/tests/test_log.py @@ -3,33 +3,39 @@ import sys import unittest from tempfile import NamedTemporaryFile -from test.support import run_unittest +from test.support import swap_attr, run_unittest from distutils import log class TestLog(unittest.TestCase): def test_non_ascii(self): - # Issue #8663: test that non-ASCII text is escaped with - # backslashreplace error handler (stream use ASCII encoding and strict - # error handler) - old_stdout = sys.stdout - old_stderr = sys.stderr - old_threshold = log.set_threshold(log.DEBUG) - try: - with NamedTemporaryFile(mode="w+", encoding='ascii') as stdout, \ - NamedTemporaryFile(mode="w+", encoding='ascii') as stderr: - sys.stdout = stdout - sys.stderr = stderr - log.debug("debug:\xe9") - log.fatal("fatal:\xe9") + # Issues #8663, #34421: test that non-encodable text is escaped with + # backslashreplace error handler and encodable non-ASCII text is + # output as is. + for errors in ('strict', 'backslashreplace', 'surrogateescape', + 'replace', 'ignore'): + with self.subTest(errors=errors), \ + NamedTemporaryFile("w+", encoding='cp437', errors=errors) as stdout, \ + NamedTemporaryFile("w+", encoding='cp437', errors=errors) as stderr: + old_threshold = log.set_threshold(log.DEBUG) + try: + with swap_attr(sys, 'stdout', stdout), \ + swap_attr(sys, 'stderr', stderr): + log.debug('D?bug\tM?ss?ge') + log.fatal('F?tal\t?rr?r') + finally: + log.set_threshold(old_threshold) + stdout.seek(0) - self.assertEqual(stdout.read().rstrip(), "debug:\\xe9") + self.assertEqual(stdout.read().rstrip(), + 'D?bug\tM?ss?ge' if errors == 'replace' else + 'D?bug\tMssge' if errors == 'ignore' else + 'D?bug\tM\\u0117ss\\xe3ge') stderr.seek(0) - self.assertEqual(stderr.read().rstrip(), "fatal:\\xe9") - finally: - log.set_threshold(old_threshold) - sys.stdout = old_stdout - sys.stderr = old_stderr + self.assertEqual(stderr.read().rstrip(), + 'F?tal\t?rr?r' if errors == 'replace' else + 'F?tal\trrr' if errors == 'ignore' else + 'F?tal\t\\xc8rr\\u014dr') def test_suite(): return unittest.makeSuite(TestLog) From webhook-mailer at python.org Sun Sep 23 02:32:34 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 23 Sep 2018 06:32:34 -0000 Subject: [Python-checkins] bpo-34670: Add TLS 1.3 post handshake auth (GH-9460) Message-ID: https://github.com/python/cpython/commit/9fb051f032c36b9f6086b79086b4d6b7755a3d70 commit: 9fb051f032c36b9f6086b79086b4d6b7755a3d70 branch: master author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-22T23:32:31-07:00 summary: bpo-34670: Add TLS 1.3 post handshake auth (GH-9460) Add SSLContext.post_handshake_auth and SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake authentication. Signed-off-by: Christian Heimes q https://bugs.python.org/issue34670 files: A Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst M .travis.yml M Doc/library/ssl.rst M Doc/whatsnew/3.8.rst M Lib/ssl.py M Lib/test/test_ssl.py M Modules/_ssl.c M Modules/clinic/_ssl.c.h M Tools/ssl/multissltests.py diff --git a/.travis.yml b/.travis.yml index 90989f51af68..d1a6da70e508 100644 --- a/.travis.yml +++ b/.travis.yml @@ -12,7 +12,7 @@ cache: env: global: - - OPENSSL=1.1.0h + - OPENSSL=1.1.0i - OPENSSL_DIR="$HOME/multissl/openssl/${OPENSSL}" - PATH="${OPENSSL_DIR}/bin:$PATH" # Use -O3 because we don't use debugger on Travis-CI diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 99d1d774b4ae..a8cbe236d866 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1314,6 +1314,26 @@ SSL sockets also have the following additional methods and attributes: returned socket should always be used for further communication with the other side of the connection, rather than the original socket. +.. method:: SSLSocket.verify_client_post_handshake() + + Requests post-handshake authentication (PHA) from a TLS 1.3 client. PHA + can only be initiated for a TLS 1.3 connection from a server-side socket, + after the initial TLS handshake and with PHA enabled on both sides, see + :attr:`SSLContext.post_handshake_auth`. + + The method does not perform a cert exchange immediately. The server-side + sends a CertificateRequest during the next write event and expects the + client to respond with a certificate on the next read event. + + If any precondition isn't met (e.g. not TLS 1.3, PHA not enabled), an + :exc:`SSLError` is raised. + + .. versionadded:: 3.8 + + .. note:: + Only available with OpenSSL 1.1.1 and TLS 1.3 enabled. Without TLS 1.3 + support, the method raises :exc:`NotImplementedError`. + .. method:: SSLSocket.version() Return the actual SSL protocol version negotiated by the connection @@ -1929,6 +1949,28 @@ to speed up repeated connections from the same clients. >>> ssl.create_default_context().options # doctest: +SKIP +.. attribute:: SSLContext.post_handshake_auth + + Enable TLS 1.3 post-handshake client authentication. Post-handshake auth + is disabled by default and a server can only request a TLS client + certificate during the initial handshake. When enabled, a server may + request a TLS client certificate at any time after the handshake. + + When enabled on client-side sockets, the client signals the server that + it supports post-handshake authentication. + + When enabled on server-side sockets, :attr:`SSLContext.verify_mode` must + be set to :data:`CERT_OPTIONAL` or :data:`CERT_REQUIRED`, too. The + actual client cert exchange is delayed until + :meth:`SSLSocket.verify_client_post_handshake` is called and some I/O is + performed. + + .. versionadded:: 3.8 + + .. note:: + Only available with OpenSSL 1.1.1 and TLS 1.3 enabled. Without TLS 1.3 + support, the property value is None and can't be modified + .. attribute:: SSLContext.protocol The protocol version chosen when constructing the context. This attribute diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index c464346987b9..9aaaa761cc8a 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -140,6 +140,14 @@ pathlib contain characters unrepresentable at the OS level. (Contributed by Serhiy Storchaka in :issue:`33721`.) +ssl +--- + +Added :attr:`SSLContext.post_handshake_auth` to enable and +:meth:`ssl.SSLSocket.verify_client_post_handshake` to initiate TLS 1.3 +post-handshake authentication. +(Contributed by Christian Heimes in :issue:`34670`.) + venv ---- @@ -147,7 +155,6 @@ venv activating virtual environments under PowerShell Core 6.1. (Contributed by Brett Cannon in :issue:`32718`.) - Optimizations ============= diff --git a/Lib/ssl.py b/Lib/ssl.py index fa7c152ade91..c7b493250558 100644 --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -777,6 +777,9 @@ def version(self): current SSL channel. """ return self._sslobj.version() + def verify_client_post_handshake(self): + return self._sslobj.verify_client_post_handshake() + class SSLSocket(socket): """This class implements a subtype of socket.socket that wraps @@ -1094,6 +1097,12 @@ def unwrap(self): else: raise ValueError("No SSL wrapper around " + str(self)) + def verify_client_post_handshake(self): + if self._sslobj: + return self._sslobj.verify_client_post_handshake() + else: + raise ValueError("No SSL wrapper around " + str(self)) + def _real_close(self): self._sslobj = None super()._real_close() diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index b4cafc15c5cd..6fd2002e5eec 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -218,7 +218,7 @@ def testing_context(server_cert=SIGNED_CERTFILE): server_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) server_context.load_cert_chain(server_cert) - client_context.load_verify_locations(SIGNING_CA) + server_context.load_verify_locations(SIGNING_CA) return client_context, server_context, hostname @@ -2262,6 +2262,23 @@ def run(self): sys.stdout.write(" server: read CB tls-unique from client, sending our CB data...\n") data = self.sslconn.get_channel_binding("tls-unique") self.write(repr(data).encode("us-ascii") + b"\n") + elif stripped == b'PHA': + if support.verbose and self.server.connectionchatty: + sys.stdout.write(" server: initiating post handshake auth\n") + try: + self.sslconn.verify_client_post_handshake() + except ssl.SSLError as e: + self.write(repr(e).encode("us-ascii") + b"\n") + else: + self.write(b"OK\n") + elif stripped == b'HASCERT': + if self.sslconn.getpeercert() is not None: + self.write(b'TRUE\n') + else: + self.write(b'FALSE\n') + elif stripped == b'GETCERT': + cert = self.sslconn.getpeercert() + self.write(repr(cert).encode("us-ascii") + b"\n") else: if (support.verbose and self.server.connectionchatty): @@ -4148,6 +4165,179 @@ def test_session_handling(self): 'Session refers to a different SSLContext.') + at unittest.skipUnless(ssl.HAS_TLSv1_3, "Test needs TLS 1.3") +class TestPostHandshakeAuth(unittest.TestCase): + def test_pha_setter(self): + protocols = [ + ssl.PROTOCOL_TLS, ssl.PROTOCOL_TLS_SERVER, ssl.PROTOCOL_TLS_CLIENT + ] + for protocol in protocols: + ctx = ssl.SSLContext(protocol) + self.assertEqual(ctx.post_handshake_auth, False) + + ctx.post_handshake_auth = True + self.assertEqual(ctx.post_handshake_auth, True) + + ctx.verify_mode = ssl.CERT_REQUIRED + self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) + self.assertEqual(ctx.post_handshake_auth, True) + + ctx.post_handshake_auth = False + self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) + self.assertEqual(ctx.post_handshake_auth, False) + + ctx.verify_mode = ssl.CERT_OPTIONAL + ctx.post_handshake_auth = True + self.assertEqual(ctx.verify_mode, ssl.CERT_OPTIONAL) + self.assertEqual(ctx.post_handshake_auth, True) + + def test_pha_required(self): + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + # PHA method just returns true when cert is already available + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'GETCERT') + cert_text = s.recv(4096).decode('us-ascii') + self.assertIn('Python Software Foundation CA', cert_text) + + def test_pha_required_nocert(self): + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'PHA') + # receive CertificateRequest + self.assertEqual(s.recv(1024), b'OK\n') + # send empty Certificate + Finish + s.write(b'HASCERT') + # receive alert + with self.assertRaisesRegex( + ssl.SSLError, + 'tlsv13 alert certificate required'): + s.recv(1024) + + def test_pha_optional(self): + if support.verbose: + sys.stdout.write("\n") + + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + # check CERT_OPTIONAL + server_context.verify_mode = ssl.CERT_OPTIONAL + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + + def test_pha_optional_nocert(self): + if support.verbose: + sys.stdout.write("\n") + + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_OPTIONAL + client_context.post_handshake_auth = True + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + # optional doens't fail when client does not have a cert + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + + def test_pha_no_pha_client(self): + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + with self.assertRaisesRegex(ssl.SSLError, 'not server'): + s.verify_client_post_handshake() + s.write(b'PHA') + self.assertIn(b'extension not received', s.recv(1024)) + + def test_pha_no_pha_server(self): + # server doesn't have PHA enabled, cert is requested in handshake + client_context, server_context, hostname = testing_context() + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + # PHA doesn't fail if there is already a cert + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + + def test_pha_not_tls13(self): + # TLS 1.2 + client_context, server_context, hostname = testing_context() + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.maximum_version = ssl.TLSVersion.TLSv1_2 + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + # PHA fails for TLS != 1.3 + s.write(b'PHA') + self.assertIn(b'WRONG_SSL_VERSION', s.recv(1024)) + + def test_main(verbose=False): if support.verbose: import warnings @@ -4183,6 +4373,7 @@ def test_main(verbose=False): tests = [ ContextTests, BasicSocketTests, SSLErrorTests, MemoryBIOTests, SSLObjectTests, SimpleBackgroundTests, ThreadedTests, + TestPostHandshakeAuth ] if support.is_resource_enabled('network'): diff --git a/Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst b/Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst new file mode 100644 index 000000000000..c1a61293faa3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst @@ -0,0 +1,3 @@ +Add SSLContext.post_handshake_auth and +SSLSocket.verify_client_post_handshake for TLS 1.3's post +handshake authentication feature. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 5b5d7dd445d2..99d4ecceaf01 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -421,6 +421,9 @@ typedef struct { */ unsigned int hostflags; int protocol; +#ifdef TLS1_3_VERSION + int post_handshake_auth; +#endif } PySSLContext; typedef struct { @@ -2643,6 +2646,30 @@ _ssl__SSLSocket_get_channel_binding_impl(PySSLSocket *self, return PyBytes_FromStringAndSize(buf, len); } +/*[clinic input] +_ssl._SSLSocket.verify_client_post_handshake + +Initiate TLS 1.3 post-handshake authentication +[clinic start generated code]*/ + +static PyObject * +_ssl__SSLSocket_verify_client_post_handshake_impl(PySSLSocket *self) +/*[clinic end generated code: output=532147f3b1341425 input=6bfa874810a3d889]*/ +{ +#ifdef TLS1_3_VERSION + int err = SSL_verify_client_post_handshake(self->ssl); + if (err == 0) + return _setSSLError(NULL, 0, __FILE__, __LINE__); + else + Py_RETURN_NONE; +#else + PyErr_SetString(PyExc_NotImplementedError, + "Post-handshake auth is not supported by your " + "OpenSSL version."); + return NULL; +#endif +} + #ifdef OPENSSL_VERSION_1_1 static SSL_SESSION* @@ -2819,6 +2846,7 @@ static PyMethodDef PySSLMethods[] = { _SSL__SSLSOCKET_SELECTED_ALPN_PROTOCOL_METHODDEF _SSL__SSLSOCKET_COMPRESSION_METHODDEF _SSL__SSLSOCKET_SHUTDOWN_METHODDEF + _SSL__SSLSOCKET_VERIFY_CLIENT_POST_HANDSHAKE_METHODDEF {NULL, NULL} }; @@ -2862,7 +2890,7 @@ static PyTypeObject PySSLSocket_Type = { */ static int -_set_verify_mode(SSL_CTX *ctx, enum py_ssl_cert_requirements n) +_set_verify_mode(PySSLContext *self, enum py_ssl_cert_requirements n) { int mode; int (*verify_cb)(int, X509_STORE_CTX *) = NULL; @@ -2882,9 +2910,13 @@ _set_verify_mode(SSL_CTX *ctx, enum py_ssl_cert_requirements n) "invalid value for verify_mode"); return -1; } +#ifdef TLS1_3_VERSION + if (self->post_handshake_auth) + mode |= SSL_VERIFY_POST_HANDSHAKE; +#endif /* keep current verify cb */ - verify_cb = SSL_CTX_get_verify_callback(ctx); - SSL_CTX_set_verify(ctx, mode, verify_cb); + verify_cb = SSL_CTX_get_verify_callback(self->ctx); + SSL_CTX_set_verify(self->ctx, mode, verify_cb); return 0; } @@ -2966,13 +2998,13 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) /* Don't check host name by default */ if (proto_version == PY_SSL_VERSION_TLS_CLIENT) { self->check_hostname = 1; - if (_set_verify_mode(self->ctx, PY_SSL_CERT_REQUIRED) == -1) { + if (_set_verify_mode(self, PY_SSL_CERT_REQUIRED) == -1) { Py_DECREF(self); return NULL; } } else { self->check_hostname = 0; - if (_set_verify_mode(self->ctx, PY_SSL_CERT_NONE) == -1) { + if (_set_verify_mode(self, PY_SSL_CERT_NONE) == -1) { Py_DECREF(self); return NULL; } @@ -3065,6 +3097,11 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) #endif X509_VERIFY_PARAM_set_hostflags(params, self->hostflags); +#ifdef TLS1_3_VERSION + self->post_handshake_auth = 0; + SSL_CTX_set_post_handshake_auth(self->ctx, self->post_handshake_auth); +#endif + return (PyObject *)self; } @@ -3319,7 +3356,10 @@ _ssl__SSLContext__set_alpn_protocols_impl(PySSLContext *self, static PyObject * get_verify_mode(PySSLContext *self, void *c) { - switch (SSL_CTX_get_verify_mode(self->ctx)) { + /* ignore SSL_VERIFY_CLIENT_ONCE and SSL_VERIFY_POST_HANDSHAKE */ + int mask = (SSL_VERIFY_NONE | SSL_VERIFY_PEER | + SSL_VERIFY_FAIL_IF_NO_PEER_CERT); + switch (SSL_CTX_get_verify_mode(self->ctx) & mask) { case SSL_VERIFY_NONE: return PyLong_FromLong(PY_SSL_CERT_NONE); case SSL_VERIFY_PEER: @@ -3344,7 +3384,7 @@ set_verify_mode(PySSLContext *self, PyObject *arg, void *c) "check_hostname is enabled."); return -1; } - return _set_verify_mode(self->ctx, n); + return _set_verify_mode(self, n); } static PyObject * @@ -3550,7 +3590,7 @@ set_check_hostname(PySSLContext *self, PyObject *arg, void *c) if (check_hostname && SSL_CTX_get_verify_mode(self->ctx) == SSL_VERIFY_NONE) { /* check_hostname = True sets verify_mode = CERT_REQUIRED */ - if (_set_verify_mode(self->ctx, PY_SSL_CERT_REQUIRED) == -1) { + if (_set_verify_mode(self, PY_SSL_CERT_REQUIRED) == -1) { return -1; } } @@ -3558,6 +3598,43 @@ set_check_hostname(PySSLContext *self, PyObject *arg, void *c) return 0; } +static PyObject * +get_post_handshake_auth(PySSLContext *self, void *c) { +#if TLS1_3_VERSION + return PyBool_FromLong(self->post_handshake_auth); +#else + Py_RETURN_NONE; +#endif +} + +#if TLS1_3_VERSION +static int +set_post_handshake_auth(PySSLContext *self, PyObject *arg, void *c) { + int (*verify_cb)(int, X509_STORE_CTX *) = NULL; + int mode = SSL_CTX_get_verify_mode(self->ctx); + int pha = PyObject_IsTrue(arg); + + if (pha == -1) { + return -1; + } + self->post_handshake_auth = pha; + + /* client-side socket setting, ignored by server-side */ + SSL_CTX_set_post_handshake_auth(self->ctx, pha); + + /* server-side socket setting, ignored by client-side */ + verify_cb = SSL_CTX_get_verify_callback(self->ctx); + if (pha) { + mode |= SSL_VERIFY_POST_HANDSHAKE; + } else { + mode ^= SSL_VERIFY_POST_HANDSHAKE; + } + SSL_CTX_set_verify(self->ctx, mode, verify_cb); + + return 0; +} +#endif + static PyObject * get_protocol(PySSLContext *self, void *c) { return PyLong_FromLong(self->protocol); @@ -4461,6 +4538,13 @@ static PyGetSetDef context_getsetlist[] = { (setter) set_sni_callback, PySSLContext_sni_callback_doc}, {"options", (getter) get_options, (setter) set_options, NULL}, + {"post_handshake_auth", (getter) get_post_handshake_auth, +#ifdef TLS1_3_VERSION + (setter) set_post_handshake_auth, +#else + NULL, +#endif + NULL}, {"protocol", (getter) get_protocol, NULL, NULL}, {"verify_flags", (getter) get_verify_flags, diff --git a/Modules/clinic/_ssl.c.h b/Modules/clinic/_ssl.c.h index 5ba34eca594f..186e6432c2e3 100644 --- a/Modules/clinic/_ssl.c.h +++ b/Modules/clinic/_ssl.c.h @@ -342,6 +342,24 @@ _ssl__SSLSocket_get_channel_binding(PySSLSocket *self, PyObject *const *args, Py return return_value; } +PyDoc_STRVAR(_ssl__SSLSocket_verify_client_post_handshake__doc__, +"verify_client_post_handshake($self, /)\n" +"--\n" +"\n" +"Initiate TLS 1.3 post-handshake authentication"); + +#define _SSL__SSLSOCKET_VERIFY_CLIENT_POST_HANDSHAKE_METHODDEF \ + {"verify_client_post_handshake", (PyCFunction)_ssl__SSLSocket_verify_client_post_handshake, METH_NOARGS, _ssl__SSLSocket_verify_client_post_handshake__doc__}, + +static PyObject * +_ssl__SSLSocket_verify_client_post_handshake_impl(PySSLSocket *self); + +static PyObject * +_ssl__SSLSocket_verify_client_post_handshake(PySSLSocket *self, PyObject *Py_UNUSED(ignored)) +{ + return _ssl__SSLSocket_verify_client_post_handshake_impl(self); +} + static PyObject * _ssl__SSLContext_impl(PyTypeObject *type, int proto_version); @@ -1175,4 +1193,4 @@ _ssl_enum_crls(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObje #ifndef _SSL_ENUM_CRLS_METHODDEF #define _SSL_ENUM_CRLS_METHODDEF #endif /* !defined(_SSL_ENUM_CRLS_METHODDEF) */ -/*[clinic end generated code: output=e2417fee28666f7c input=a9049054013a1b77]*/ +/*[clinic end generated code: output=c4e73b70ac3618ba input=a9049054013a1b77]*/ diff --git a/Tools/ssl/multissltests.py b/Tools/ssl/multissltests.py index c4ebe317797d..759f5f4a3e78 100755 --- a/Tools/ssl/multissltests.py +++ b/Tools/ssl/multissltests.py @@ -45,16 +45,16 @@ ] OPENSSL_RECENT_VERSIONS = [ - "1.0.2o", - "1.1.0h", - # "1.1.1-pre7", + "1.0.2p", + "1.1.0i", + "1.1.1", ] LIBRESSL_OLD_VERSIONS = [ ] LIBRESSL_RECENT_VERSIONS = [ - "2.7.3", + "2.7.4", ] # store files in ../multissl From webhook-mailer at python.org Sun Sep 23 03:22:55 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 23 Sep 2018 07:22:55 -0000 Subject: [Python-checkins] [3.7] bpo-34670: Add TLS 1.3 post handshake auth (GH-9460) (GH-9505) Message-ID: https://github.com/python/cpython/commit/2756ef31656399a120589b7aa19c32e2b91a4758 commit: 2756ef31656399a120589b7aa19c32e2b91a4758 branch: 3.7 author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-23T00:22:52-07:00 summary: [3.7] bpo-34670: Add TLS 1.3 post handshake auth (GH-9460) (GH-9505) Add SSLContext.post_handshake_auth and SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake authentication. Signed-off-by: Christian Heimes q https://bugs.python.org/issue34670. (cherry picked from commit 9fb051f032c36b9f6086b79086b4d6b7755a3d70) Co-authored-by: Christian Heimes https://bugs.python.org/issue34670 files: A Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst M .travis.yml M Doc/library/ssl.rst M Doc/whatsnew/3.7.rst M Lib/ssl.py M Lib/test/test_ssl.py M Modules/_ssl.c M Modules/clinic/_ssl.c.h M Tools/ssl/multissltests.py diff --git a/.travis.yml b/.travis.yml index 8b1219636677..27a240f1e684 100644 --- a/.travis.yml +++ b/.travis.yml @@ -12,7 +12,7 @@ cache: env: global: - - OPENSSL=1.1.0h + - OPENSSL=1.1.0i - OPENSSL_DIR="$HOME/multissl/openssl/${OPENSSL}" - PATH="${OPENSSL_DIR}/bin:$PATH" # Use -O3 because we don't use debugger on Travis-CI diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 99d1d774b4ae..dd2fccc2e4bc 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1314,6 +1314,26 @@ SSL sockets also have the following additional methods and attributes: returned socket should always be used for further communication with the other side of the connection, rather than the original socket. +.. method:: SSLSocket.verify_client_post_handshake() + + Requests post-handshake authentication (PHA) from a TLS 1.3 client. PHA + can only be initiated for a TLS 1.3 connection from a server-side socket, + after the initial TLS handshake and with PHA enabled on both sides, see + :attr:`SSLContext.post_handshake_auth`. + + The method does not perform a cert exchange immediately. The server-side + sends a CertificateRequest during the next write event and expects the + client to respond with a certificate on the next read event. + + If any precondition isn't met (e.g. not TLS 1.3, PHA not enabled), an + :exc:`SSLError` is raised. + + .. versionadded:: 3.7.1 + + .. note:: + Only available with OpenSSL 1.1.1 and TLS 1.3 enabled. Without TLS 1.3 + support, the method raises :exc:`NotImplementedError`. + .. method:: SSLSocket.version() Return the actual SSL protocol version negotiated by the connection @@ -1929,6 +1949,28 @@ to speed up repeated connections from the same clients. >>> ssl.create_default_context().options # doctest: +SKIP +.. attribute:: SSLContext.post_handshake_auth + + Enable TLS 1.3 post-handshake client authentication. Post-handshake auth + is disabled by default and a server can only request a TLS client + certificate during the initial handshake. When enabled, a server may + request a TLS client certificate at any time after the handshake. + + When enabled on client-side sockets, the client signals the server that + it supports post-handshake authentication. + + When enabled on server-side sockets, :attr:`SSLContext.verify_mode` must + be set to :data:`CERT_OPTIONAL` or :data:`CERT_REQUIRED`, too. The + actual client cert exchange is delayed until + :meth:`SSLSocket.verify_client_post_handshake` is called and some I/O is + performed. + + .. versionadded:: 3.7.1 + + .. note:: + Only available with OpenSSL 1.1.1 and TLS 1.3 enabled. Without TLS 1.3 + support, the property value is None and can't be modified + .. attribute:: SSLContext.protocol The protocol version chosen when constructing the context. This attribute diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index a2c5c284f255..132ed00f7321 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1350,6 +1350,10 @@ Supported protocols are indicated by serveral new flags, such as :data:`~ssl.HAS_TLSv1_1`. (Contributed by Christian Heimes in :issue:`32609`.) +Added :attr:`SSLContext.post_handshake_auth` to enable and +:meth:`ssl.SSLSocket.verify_client_post_handshake` to initiate TLS 1.3 +post-handshake authentication. +(Contributed by Christian Heimes in :issue:`34670`.) string ------ diff --git a/Lib/ssl.py b/Lib/ssl.py index fdd161574434..2f21937b8ec2 100644 --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -777,6 +777,9 @@ def version(self): current SSL channel. """ return self._sslobj.version() + def verify_client_post_handshake(self): + return self._sslobj.verify_client_post_handshake() + class SSLSocket(socket): """This class implements a subtype of socket.socket that wraps @@ -1094,6 +1097,12 @@ def unwrap(self): else: raise ValueError("No SSL wrapper around " + str(self)) + def verify_client_post_handshake(self): + if self._sslobj: + return self._sslobj.verify_client_post_handshake() + else: + raise ValueError("No SSL wrapper around " + str(self)) + def _real_close(self): self._sslobj = None super()._real_close() diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index 4d5925a9355e..d4132c5043c7 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -234,7 +234,7 @@ def testing_context(server_cert=SIGNED_CERTFILE): server_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) server_context.load_cert_chain(server_cert) - client_context.load_verify_locations(SIGNING_CA) + server_context.load_verify_locations(SIGNING_CA) return client_context, server_context, hostname @@ -2282,6 +2282,23 @@ def run(self): sys.stdout.write(" server: read CB tls-unique from client, sending our CB data...\n") data = self.sslconn.get_channel_binding("tls-unique") self.write(repr(data).encode("us-ascii") + b"\n") + elif stripped == b'PHA': + if support.verbose and self.server.connectionchatty: + sys.stdout.write(" server: initiating post handshake auth\n") + try: + self.sslconn.verify_client_post_handshake() + except ssl.SSLError as e: + self.write(repr(e).encode("us-ascii") + b"\n") + else: + self.write(b"OK\n") + elif stripped == b'HASCERT': + if self.sslconn.getpeercert() is not None: + self.write(b'TRUE\n') + else: + self.write(b'FALSE\n') + elif stripped == b'GETCERT': + cert = self.sslconn.getpeercert() + self.write(repr(cert).encode("us-ascii") + b"\n") else: if (support.verbose and self.server.connectionchatty): @@ -4175,6 +4192,179 @@ def test_session_handling(self): 'Session refers to a different SSLContext.') + at unittest.skipUnless(ssl.HAS_TLSv1_3, "Test needs TLS 1.3") +class TestPostHandshakeAuth(unittest.TestCase): + def test_pha_setter(self): + protocols = [ + ssl.PROTOCOL_TLS, ssl.PROTOCOL_TLS_SERVER, ssl.PROTOCOL_TLS_CLIENT + ] + for protocol in protocols: + ctx = ssl.SSLContext(protocol) + self.assertEqual(ctx.post_handshake_auth, False) + + ctx.post_handshake_auth = True + self.assertEqual(ctx.post_handshake_auth, True) + + ctx.verify_mode = ssl.CERT_REQUIRED + self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) + self.assertEqual(ctx.post_handshake_auth, True) + + ctx.post_handshake_auth = False + self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) + self.assertEqual(ctx.post_handshake_auth, False) + + ctx.verify_mode = ssl.CERT_OPTIONAL + ctx.post_handshake_auth = True + self.assertEqual(ctx.verify_mode, ssl.CERT_OPTIONAL) + self.assertEqual(ctx.post_handshake_auth, True) + + def test_pha_required(self): + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + # PHA method just returns true when cert is already available + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'GETCERT') + cert_text = s.recv(4096).decode('us-ascii') + self.assertIn('Python Software Foundation CA', cert_text) + + def test_pha_required_nocert(self): + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'PHA') + # receive CertificateRequest + self.assertEqual(s.recv(1024), b'OK\n') + # send empty Certificate + Finish + s.write(b'HASCERT') + # receive alert + with self.assertRaisesRegex( + ssl.SSLError, + 'tlsv13 alert certificate required'): + s.recv(1024) + + def test_pha_optional(self): + if support.verbose: + sys.stdout.write("\n") + + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + # check CERT_OPTIONAL + server_context.verify_mode = ssl.CERT_OPTIONAL + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + + def test_pha_optional_nocert(self): + if support.verbose: + sys.stdout.write("\n") + + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_OPTIONAL + client_context.post_handshake_auth = True + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + # optional doens't fail when client does not have a cert + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + + def test_pha_no_pha_client(self): + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + with self.assertRaisesRegex(ssl.SSLError, 'not server'): + s.verify_client_post_handshake() + s.write(b'PHA') + self.assertIn(b'extension not received', s.recv(1024)) + + def test_pha_no_pha_server(self): + # server doesn't have PHA enabled, cert is requested in handshake + client_context, server_context, hostname = testing_context() + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + # PHA doesn't fail if there is already a cert + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + + def test_pha_not_tls13(self): + # TLS 1.2 + client_context, server_context, hostname = testing_context() + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.maximum_version = ssl.TLSVersion.TLSv1_2 + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + # PHA fails for TLS != 1.3 + s.write(b'PHA') + self.assertIn(b'WRONG_SSL_VERSION', s.recv(1024)) + + def test_main(verbose=False): if support.verbose: import warnings @@ -4218,6 +4408,7 @@ def test_main(verbose=False): tests = [ ContextTests, BasicSocketTests, SSLErrorTests, MemoryBIOTests, SSLObjectTests, SimpleBackgroundTests, ThreadedTests, + TestPostHandshakeAuth ] if support.is_resource_enabled('network'): diff --git a/Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst b/Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst new file mode 100644 index 000000000000..c1a61293faa3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst @@ -0,0 +1,3 @@ +Add SSLContext.post_handshake_auth and +SSLSocket.verify_client_post_handshake for TLS 1.3's post +handshake authentication feature. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index faca3a27c562..3c9d46e43d94 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -421,6 +421,9 @@ typedef struct { */ unsigned int hostflags; int protocol; +#ifdef TLS1_3_VERSION + int post_handshake_auth; +#endif } PySSLContext; typedef struct { @@ -2644,6 +2647,30 @@ _ssl__SSLSocket_get_channel_binding_impl(PySSLSocket *self, return PyBytes_FromStringAndSize(buf, len); } +/*[clinic input] +_ssl._SSLSocket.verify_client_post_handshake + +Initiate TLS 1.3 post-handshake authentication +[clinic start generated code]*/ + +static PyObject * +_ssl__SSLSocket_verify_client_post_handshake_impl(PySSLSocket *self) +/*[clinic end generated code: output=532147f3b1341425 input=6bfa874810a3d889]*/ +{ +#ifdef TLS1_3_VERSION + int err = SSL_verify_client_post_handshake(self->ssl); + if (err == 0) + return _setSSLError(NULL, 0, __FILE__, __LINE__); + else + Py_RETURN_NONE; +#else + PyErr_SetString(PyExc_NotImplementedError, + "Post-handshake auth is not supported by your " + "OpenSSL version."); + return NULL; +#endif +} + #ifdef OPENSSL_VERSION_1_1 static SSL_SESSION* @@ -2820,6 +2847,7 @@ static PyMethodDef PySSLMethods[] = { _SSL__SSLSOCKET_SELECTED_ALPN_PROTOCOL_METHODDEF _SSL__SSLSOCKET_COMPRESSION_METHODDEF _SSL__SSLSOCKET_SHUTDOWN_METHODDEF + _SSL__SSLSOCKET_VERIFY_CLIENT_POST_HANDSHAKE_METHODDEF {NULL, NULL} }; @@ -2863,7 +2891,7 @@ static PyTypeObject PySSLSocket_Type = { */ static int -_set_verify_mode(SSL_CTX *ctx, enum py_ssl_cert_requirements n) +_set_verify_mode(PySSLContext *self, enum py_ssl_cert_requirements n) { int mode; int (*verify_cb)(int, X509_STORE_CTX *) = NULL; @@ -2883,9 +2911,13 @@ _set_verify_mode(SSL_CTX *ctx, enum py_ssl_cert_requirements n) "invalid value for verify_mode"); return -1; } +#ifdef TLS1_3_VERSION + if (self->post_handshake_auth) + mode |= SSL_VERIFY_POST_HANDSHAKE; +#endif /* keep current verify cb */ - verify_cb = SSL_CTX_get_verify_callback(ctx); - SSL_CTX_set_verify(ctx, mode, verify_cb); + verify_cb = SSL_CTX_get_verify_callback(self->ctx); + SSL_CTX_set_verify(self->ctx, mode, verify_cb); return 0; } @@ -2967,13 +2999,13 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) /* Don't check host name by default */ if (proto_version == PY_SSL_VERSION_TLS_CLIENT) { self->check_hostname = 1; - if (_set_verify_mode(self->ctx, PY_SSL_CERT_REQUIRED) == -1) { + if (_set_verify_mode(self, PY_SSL_CERT_REQUIRED) == -1) { Py_DECREF(self); return NULL; } } else { self->check_hostname = 0; - if (_set_verify_mode(self->ctx, PY_SSL_CERT_NONE) == -1) { + if (_set_verify_mode(self, PY_SSL_CERT_NONE) == -1) { Py_DECREF(self); return NULL; } @@ -3066,6 +3098,11 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) #endif X509_VERIFY_PARAM_set_hostflags(params, self->hostflags); +#ifdef TLS1_3_VERSION + self->post_handshake_auth = 0; + SSL_CTX_set_post_handshake_auth(self->ctx, self->post_handshake_auth); +#endif + return (PyObject *)self; } @@ -3320,7 +3357,10 @@ _ssl__SSLContext__set_alpn_protocols_impl(PySSLContext *self, static PyObject * get_verify_mode(PySSLContext *self, void *c) { - switch (SSL_CTX_get_verify_mode(self->ctx)) { + /* ignore SSL_VERIFY_CLIENT_ONCE and SSL_VERIFY_POST_HANDSHAKE */ + int mask = (SSL_VERIFY_NONE | SSL_VERIFY_PEER | + SSL_VERIFY_FAIL_IF_NO_PEER_CERT); + switch (SSL_CTX_get_verify_mode(self->ctx) & mask) { case SSL_VERIFY_NONE: return PyLong_FromLong(PY_SSL_CERT_NONE); case SSL_VERIFY_PEER: @@ -3345,7 +3385,7 @@ set_verify_mode(PySSLContext *self, PyObject *arg, void *c) "check_hostname is enabled."); return -1; } - return _set_verify_mode(self->ctx, n); + return _set_verify_mode(self, n); } static PyObject * @@ -3551,7 +3591,7 @@ set_check_hostname(PySSLContext *self, PyObject *arg, void *c) if (check_hostname && SSL_CTX_get_verify_mode(self->ctx) == SSL_VERIFY_NONE) { /* check_hostname = True sets verify_mode = CERT_REQUIRED */ - if (_set_verify_mode(self->ctx, PY_SSL_CERT_REQUIRED) == -1) { + if (_set_verify_mode(self, PY_SSL_CERT_REQUIRED) == -1) { return -1; } } @@ -3559,6 +3599,43 @@ set_check_hostname(PySSLContext *self, PyObject *arg, void *c) return 0; } +static PyObject * +get_post_handshake_auth(PySSLContext *self, void *c) { +#if TLS1_3_VERSION + return PyBool_FromLong(self->post_handshake_auth); +#else + Py_RETURN_NONE; +#endif +} + +#if TLS1_3_VERSION +static int +set_post_handshake_auth(PySSLContext *self, PyObject *arg, void *c) { + int (*verify_cb)(int, X509_STORE_CTX *) = NULL; + int mode = SSL_CTX_get_verify_mode(self->ctx); + int pha = PyObject_IsTrue(arg); + + if (pha == -1) { + return -1; + } + self->post_handshake_auth = pha; + + /* client-side socket setting, ignored by server-side */ + SSL_CTX_set_post_handshake_auth(self->ctx, pha); + + /* server-side socket setting, ignored by client-side */ + verify_cb = SSL_CTX_get_verify_callback(self->ctx); + if (pha) { + mode |= SSL_VERIFY_POST_HANDSHAKE; + } else { + mode ^= SSL_VERIFY_POST_HANDSHAKE; + } + SSL_CTX_set_verify(self->ctx, mode, verify_cb); + + return 0; +} +#endif + static PyObject * get_protocol(PySSLContext *self, void *c) { return PyLong_FromLong(self->protocol); @@ -4462,6 +4539,13 @@ static PyGetSetDef context_getsetlist[] = { (setter) set_sni_callback, PySSLContext_sni_callback_doc}, {"options", (getter) get_options, (setter) set_options, NULL}, + {"post_handshake_auth", (getter) get_post_handshake_auth, +#ifdef TLS1_3_VERSION + (setter) set_post_handshake_auth, +#else + NULL, +#endif + NULL}, {"protocol", (getter) get_protocol, NULL, NULL}, {"verify_flags", (getter) get_verify_flags, diff --git a/Modules/clinic/_ssl.c.h b/Modules/clinic/_ssl.c.h index 5ba34eca594f..186e6432c2e3 100644 --- a/Modules/clinic/_ssl.c.h +++ b/Modules/clinic/_ssl.c.h @@ -342,6 +342,24 @@ _ssl__SSLSocket_get_channel_binding(PySSLSocket *self, PyObject *const *args, Py return return_value; } +PyDoc_STRVAR(_ssl__SSLSocket_verify_client_post_handshake__doc__, +"verify_client_post_handshake($self, /)\n" +"--\n" +"\n" +"Initiate TLS 1.3 post-handshake authentication"); + +#define _SSL__SSLSOCKET_VERIFY_CLIENT_POST_HANDSHAKE_METHODDEF \ + {"verify_client_post_handshake", (PyCFunction)_ssl__SSLSocket_verify_client_post_handshake, METH_NOARGS, _ssl__SSLSocket_verify_client_post_handshake__doc__}, + +static PyObject * +_ssl__SSLSocket_verify_client_post_handshake_impl(PySSLSocket *self); + +static PyObject * +_ssl__SSLSocket_verify_client_post_handshake(PySSLSocket *self, PyObject *Py_UNUSED(ignored)) +{ + return _ssl__SSLSocket_verify_client_post_handshake_impl(self); +} + static PyObject * _ssl__SSLContext_impl(PyTypeObject *type, int proto_version); @@ -1175,4 +1193,4 @@ _ssl_enum_crls(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObje #ifndef _SSL_ENUM_CRLS_METHODDEF #define _SSL_ENUM_CRLS_METHODDEF #endif /* !defined(_SSL_ENUM_CRLS_METHODDEF) */ -/*[clinic end generated code: output=e2417fee28666f7c input=a9049054013a1b77]*/ +/*[clinic end generated code: output=c4e73b70ac3618ba input=a9049054013a1b77]*/ diff --git a/Tools/ssl/multissltests.py b/Tools/ssl/multissltests.py index c4ebe317797d..759f5f4a3e78 100755 --- a/Tools/ssl/multissltests.py +++ b/Tools/ssl/multissltests.py @@ -45,16 +45,16 @@ ] OPENSSL_RECENT_VERSIONS = [ - "1.0.2o", - "1.1.0h", - # "1.1.1-pre7", + "1.0.2p", + "1.1.0i", + "1.1.1", ] LIBRESSL_OLD_VERSIONS = [ ] LIBRESSL_RECENT_VERSIONS = [ - "2.7.3", + "2.7.4", ] # store files in ../multissl From webhook-mailer at python.org Sun Sep 23 03:23:04 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 23 Sep 2018 07:23:04 -0000 Subject: [Python-checkins] [3.6] bpo-34670: Add TLS 1.3 post handshake auth (GH-9460) (GH-9507) Message-ID: https://github.com/python/cpython/commit/94812f717dde8b11a9ad9c0fd5be66ff9bd53f58 commit: 94812f717dde8b11a9ad9c0fd5be66ff9bd53f58 branch: 3.6 author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-23T00:23:01-07:00 summary: [3.6] bpo-34670: Add TLS 1.3 post handshake auth (GH-9460) (GH-9507) Add SSLContext.post_handshake_auth and SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake authentication. Signed-off-by: Christian Heimes q https://bugs.python.org/issue34670. (cherry picked from commit 9fb051f032c36b9f6086b79086b4d6b7755a3d70) Co-authored-by: Christian Heimes https://bugs.python.org/issue34670 files: A Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst M Doc/library/ssl.rst M Doc/whatsnew/3.6.rst M Lib/ssl.py M Lib/test/test_ssl.py M Modules/_ssl.c M Modules/clinic/_ssl.c.h M Tools/ssl/multissltests.py diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 376d467cb056..a85be1a744ab 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1210,6 +1210,26 @@ SSL sockets also have the following additional methods and attributes: returned socket should always be used for further communication with the other side of the connection, rather than the original socket. +.. method:: SSLSocket.verify_client_post_handshake() + + Requests post-handshake authentication (PHA) from a TLS 1.3 client. PHA + can only be initiated for a TLS 1.3 connection from a server-side socket, + after the initial TLS handshake and with PHA enabled on both sides, see + :attr:`SSLContext.post_handshake_auth`. + + The method does not perform a cert exchange immediately. The server-side + sends a CertificateRequest during the next write event and expects the + client to respond with a certificate on the next read event. + + If any precondition isn't met (e.g. not TLS 1.3, PHA not enabled), an + :exc:`SSLError` is raised. + + .. versionadded:: 3.6.7 + + .. note:: + Only available with OpenSSL 1.1.1 and TLS 1.3 enabled. Without TLS 1.3 + support, the method raises :exc:`NotImplementedError`. + .. method:: SSLSocket.version() Return the actual SSL protocol version negotiated by the connection @@ -1693,6 +1713,28 @@ to speed up repeated connections from the same clients. >>> ssl.create_default_context().options +.. attribute:: SSLContext.post_handshake_auth + + Enable TLS 1.3 post-handshake client authentication. Post-handshake auth + is disabled by default and a server can only request a TLS client + certificate during the initial handshake. When enabled, a server may + request a TLS client certificate at any time after the handshake. + + When enabled on client-side sockets, the client signals the server that + it supports post-handshake authentication. + + When enabled on server-side sockets, :attr:`SSLContext.verify_mode` must + be set to :data:`CERT_OPTIONAL` or :data:`CERT_REQUIRED`, too. The + actual client cert exchange is delayed until + :meth:`SSLSocket.verify_client_post_handshake` is called and some I/O is + performed. + + .. versionadded:: 3.6.7 + + .. note:: + Only available with OpenSSL 1.1.1 and TLS 1.3 enabled. Without TLS 1.3 + support, the property value is None and can't be modified + .. attribute:: SSLContext.protocol The protocol version chosen when constructing the context. This attribute diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 4d63bbe73a29..d375cff1af94 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1462,6 +1462,10 @@ Server and client-side specific TLS protocols for :class:`~ssl.SSLContext` were added. (Contributed by Christian Heimes in :issue:`28085`.) +Added :attr:`SSLContext.post_handshake_auth` to enable and +:meth:`ssl.SSLSocket.verify_client_post_handshake` to initiate TLS 1.3 +post-handshake authentication. +(Contributed by Christian Heimes in :issue:`34670`.) statistics ---------- diff --git a/Lib/ssl.py b/Lib/ssl.py index 1f3a31a9b796..4130cc7d1a27 100644 --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -714,6 +714,9 @@ def version(self): current SSL channel. """ return self._sslobj.version() + def verify_client_post_handshake(self): + return self._sslobj.verify_client_post_handshake() + class SSLSocket(socket): """This class implements a subtype of socket.socket that wraps @@ -1054,6 +1057,12 @@ def unwrap(self): else: raise ValueError("No SSL wrapper around " + str(self)) + def verify_client_post_handshake(self): + if self._sslobj: + return self._sslobj.verify_client_post_handshake() + else: + raise ValueError("No SSL wrapper around " + str(self)) + def _real_close(self): self._sslobj = None socket._real_close(self) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index 4ba988fd57ee..2f0b6a75e96f 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1976,6 +1976,24 @@ def run(self): sys.stdout.write(" server: read CB tls-unique from client, sending our CB data...\n") data = self.sslconn.get_channel_binding("tls-unique") self.write(repr(data).encode("us-ascii") + b"\n") + elif stripped == b'PHA': + if support.verbose and self.server.connectionchatty: + sys.stdout.write( + " server: initiating post handshake auth\n") + try: + self.sslconn.verify_client_post_handshake() + except ssl.SSLError as e: + self.write(repr(e).encode("us-ascii") + b"\n") + else: + self.write(b"OK\n") + elif stripped == b'HASCERT': + if self.sslconn.getpeercert() is not None: + self.write(b'TRUE\n') + else: + self.write(b'FALSE\n') + elif stripped == b'GETCERT': + cert = self.sslconn.getpeercert() + self.write(repr(cert).encode("us-ascii") + b"\n") else: if (support.verbose and self.server.connectionchatty): @@ -3629,6 +3647,194 @@ def test_session_handling(self): 'Session refers to a different SSLContext.') +def testing_context(): + """Create context + + client_context, server_context, hostname = testing_context() + """ + client_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + client_context.load_verify_locations(SIGNING_CA) + + server_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + server_context.load_cert_chain(SIGNED_CERTFILE) + server_context.load_verify_locations(SIGNING_CA) + + return client_context, server_context, 'localhost' + + + at unittest.skipUnless(ssl.HAS_TLSv1_3, "Test needs TLS 1.3") +class TestPostHandshakeAuth(unittest.TestCase): + def test_pha_setter(self): + protocols = [ + ssl.PROTOCOL_TLS, ssl.PROTOCOL_TLS_SERVER, ssl.PROTOCOL_TLS_CLIENT + ] + for protocol in protocols: + ctx = ssl.SSLContext(protocol) + self.assertEqual(ctx.post_handshake_auth, False) + + ctx.post_handshake_auth = True + self.assertEqual(ctx.post_handshake_auth, True) + + ctx.verify_mode = ssl.CERT_REQUIRED + self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) + self.assertEqual(ctx.post_handshake_auth, True) + + ctx.post_handshake_auth = False + self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) + self.assertEqual(ctx.post_handshake_auth, False) + + ctx.verify_mode = ssl.CERT_OPTIONAL + ctx.post_handshake_auth = True + self.assertEqual(ctx.verify_mode, ssl.CERT_OPTIONAL) + self.assertEqual(ctx.post_handshake_auth, True) + + def test_pha_required(self): + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + # PHA method just returns true when cert is already available + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'GETCERT') + cert_text = s.recv(4096).decode('us-ascii') + self.assertIn('Python Software Foundation CA', cert_text) + + def test_pha_required_nocert(self): + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'PHA') + # receive CertificateRequest + self.assertEqual(s.recv(1024), b'OK\n') + # send empty Certificate + Finish + s.write(b'HASCERT') + # receive alert + with self.assertRaisesRegex( + ssl.SSLError, + 'tlsv13 alert certificate required'): + s.recv(1024) + + def test_pha_optional(self): + if support.verbose: + sys.stdout.write("\n") + + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + # check CERT_OPTIONAL + server_context.verify_mode = ssl.CERT_OPTIONAL + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + + def test_pha_optional_nocert(self): + if support.verbose: + sys.stdout.write("\n") + + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_OPTIONAL + client_context.post_handshake_auth = True + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + # optional doens't fail when client does not have a cert + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'FALSE\n') + + def test_pha_no_pha_client(self): + client_context, server_context, hostname = testing_context() + server_context.post_handshake_auth = True + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + with self.assertRaisesRegex(ssl.SSLError, 'not server'): + s.verify_client_post_handshake() + s.write(b'PHA') + self.assertIn(b'extension not received', s.recv(1024)) + + def test_pha_no_pha_server(self): + # server doesn't have PHA enabled, cert is requested in handshake + client_context, server_context, hostname = testing_context() + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + # PHA doesn't fail if there is already a cert + s.write(b'PHA') + self.assertEqual(s.recv(1024), b'OK\n') + s.write(b'HASCERT') + self.assertEqual(s.recv(1024), b'TRUE\n') + + def test_pha_not_tls13(self): + # TLS 1.2 + client_context, server_context, hostname = testing_context() + server_context.verify_mode = ssl.CERT_REQUIRED + client_context.options |= ssl.OP_NO_TLSv1_3 + client_context.post_handshake_auth = True + client_context.load_cert_chain(SIGNED_CERTFILE) + + server = ThreadedEchoServer(context=server_context, chatty=False) + with server: + with client_context.wrap_socket(socket.socket(), + server_hostname=hostname) as s: + s.connect((HOST, server.port)) + # PHA fails for TLS != 1.3 + s.write(b'PHA') + self.assertIn(b'WRONG_SSL_VERSION', s.recv(1024)) + + def test_main(verbose=False): if support.verbose: import warnings @@ -3681,6 +3887,7 @@ def test_main(verbose=False): thread_info = support.threading_setup() if thread_info: tests.append(ThreadedTests) + tests.append(TestPostHandshakeAuth) try: support.run_unittest(*tests) diff --git a/Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst b/Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst new file mode 100644 index 000000000000..c1a61293faa3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-14-14-29-45.bpo-34670.17XwGB.rst @@ -0,0 +1,3 @@ +Add SSLContext.post_handshake_auth and +SSLSocket.verify_client_post_handshake for TLS 1.3's post +handshake authentication feature. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 2badf3172252..b0cfbdc96c07 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -323,6 +323,9 @@ typedef struct { PyObject *set_hostname; #endif int check_hostname; +#ifdef TLS1_3_VERSION + int post_handshake_auth; +#endif } PySSLContext; typedef struct { @@ -2456,6 +2459,30 @@ _ssl__SSLSocket_tls_unique_cb_impl(PySSLSocket *self) return retval; } +/*[clinic input] +_ssl._SSLSocket.verify_client_post_handshake + +Initiate TLS 1.3 post-handshake authentication +[clinic start generated code]*/ + +static PyObject * +_ssl__SSLSocket_verify_client_post_handshake_impl(PySSLSocket *self) +/*[clinic end generated code: output=532147f3b1341425 input=6bfa874810a3d889]*/ +{ +#ifdef TLS1_3_VERSION + int err = SSL_verify_client_post_handshake(self->ssl); + if (err == 0) + return _setSSLError(NULL, 0, __FILE__, __LINE__); + else + Py_RETURN_NONE; +#else + PyErr_SetString(PyExc_NotImplementedError, + "Post-handshake auth is not supported by your " + "OpenSSL version."); + return NULL; +#endif +} + #ifdef OPENSSL_VERSION_1_1 static SSL_SESSION* @@ -2632,6 +2659,7 @@ static PyMethodDef PySSLMethods[] = { _SSL__SSLSOCKET_COMPRESSION_METHODDEF _SSL__SSLSOCKET_SHUTDOWN_METHODDEF _SSL__SSLSOCKET_TLS_UNIQUE_CB_METHODDEF + _SSL__SSLSOCKET_VERIFY_CLIENT_POST_HANDSHAKE_METHODDEF {NULL, NULL} }; @@ -2675,7 +2703,7 @@ static PyTypeObject PySSLSocket_Type = { */ static int -_set_verify_mode(SSL_CTX *ctx, enum py_ssl_cert_requirements n) +_set_verify_mode(PySSLContext *self, enum py_ssl_cert_requirements n) { int mode; int (*verify_cb)(int, X509_STORE_CTX *) = NULL; @@ -2695,9 +2723,13 @@ _set_verify_mode(SSL_CTX *ctx, enum py_ssl_cert_requirements n) "invalid value for verify_mode"); return -1; } +#ifdef TLS1_3_VERSION + if (self->post_handshake_auth) + mode |= SSL_VERIFY_POST_HANDSHAKE; +#endif /* keep current verify cb */ - verify_cb = SSL_CTX_get_verify_callback(ctx); - SSL_CTX_set_verify(ctx, mode, verify_cb); + verify_cb = SSL_CTX_get_verify_callback(self->ctx); + SSL_CTX_set_verify(self->ctx, mode, verify_cb); return 0; } @@ -2776,13 +2808,13 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) /* Don't check host name by default */ if (proto_version == PY_SSL_VERSION_TLS_CLIENT) { self->check_hostname = 1; - if (_set_verify_mode(self->ctx, PY_SSL_CERT_REQUIRED) == -1) { + if (_set_verify_mode(self, PY_SSL_CERT_REQUIRED) == -1) { Py_DECREF(self); return NULL; } } else { self->check_hostname = 0; - if (_set_verify_mode(self->ctx, PY_SSL_CERT_NONE) == -1) { + if (_set_verify_mode(self, PY_SSL_CERT_NONE) == -1) { Py_DECREF(self); return NULL; } @@ -2871,6 +2903,11 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) } #endif +#ifdef TLS1_3_VERSION + self->post_handshake_auth = 0; + SSL_CTX_set_post_handshake_auth(self->ctx, self->post_handshake_auth); +#endif + return (PyObject *)self; } @@ -3125,7 +3162,10 @@ _ssl__SSLContext__set_alpn_protocols_impl(PySSLContext *self, static PyObject * get_verify_mode(PySSLContext *self, void *c) { - switch (SSL_CTX_get_verify_mode(self->ctx)) { + /* ignore SSL_VERIFY_CLIENT_ONCE and SSL_VERIFY_POST_HANDSHAKE */ + int mask = (SSL_VERIFY_NONE | SSL_VERIFY_PEER | + SSL_VERIFY_FAIL_IF_NO_PEER_CERT); + switch (SSL_CTX_get_verify_mode(self->ctx) & mask) { case SSL_VERIFY_NONE: return PyLong_FromLong(PY_SSL_CERT_NONE); case SSL_VERIFY_PEER: @@ -3150,7 +3190,7 @@ set_verify_mode(PySSLContext *self, PyObject *arg, void *c) "check_hostname is enabled."); return -1; } - return _set_verify_mode(self->ctx, n); + return _set_verify_mode(self, n); } static PyObject * @@ -3247,6 +3287,42 @@ set_check_hostname(PySSLContext *self, PyObject *arg, void *c) return 0; } +static PyObject * +get_post_handshake_auth(PySSLContext *self, void *c) { +#if TLS1_3_VERSION + return PyBool_FromLong(self->post_handshake_auth); +#else + Py_RETURN_NONE; +#endif +} + +#if TLS1_3_VERSION +static int +set_post_handshake_auth(PySSLContext *self, PyObject *arg, void *c) { + int (*verify_cb)(int, X509_STORE_CTX *) = NULL; + int mode = SSL_CTX_get_verify_mode(self->ctx); + int pha = PyObject_IsTrue(arg); + + if (pha == -1) { + return -1; + } + self->post_handshake_auth = pha; + + /* client-side socket setting, ignored by server-side */ + SSL_CTX_set_post_handshake_auth(self->ctx, pha); + + /* server-side socket setting, ignored by client-side */ + verify_cb = SSL_CTX_get_verify_callback(self->ctx); + if (pha) { + mode |= SSL_VERIFY_POST_HANDSHAKE; + } else { + mode ^= SSL_VERIFY_POST_HANDSHAKE; + } + SSL_CTX_set_verify(self->ctx, mode, verify_cb); + + return 0; +} +#endif typedef struct { PyThreadState *thread_state; @@ -4118,6 +4194,13 @@ static PyGetSetDef context_getsetlist[] = { (setter) set_check_hostname, NULL}, {"options", (getter) get_options, (setter) set_options, NULL}, + {"post_handshake_auth", (getter) get_post_handshake_auth, +#ifdef TLS1_3_VERSION + (setter) set_post_handshake_auth, +#else + NULL, +#endif + NULL}, {"verify_flags", (getter) get_verify_flags, (setter) set_verify_flags, NULL}, {"verify_mode", (getter) get_verify_mode, diff --git a/Modules/clinic/_ssl.c.h b/Modules/clinic/_ssl.c.h index eabe2aa06686..c9206a684efd 100644 --- a/Modules/clinic/_ssl.c.h +++ b/Modules/clinic/_ssl.c.h @@ -329,6 +329,24 @@ _ssl__SSLSocket_tls_unique_cb(PySSLSocket *self, PyObject *Py_UNUSED(ignored)) return _ssl__SSLSocket_tls_unique_cb_impl(self); } +PyDoc_STRVAR(_ssl__SSLSocket_verify_client_post_handshake__doc__, +"verify_client_post_handshake($self, /)\n" +"--\n" +"\n" +"Initiate TLS 1.3 post-handshake authentication"); + +#define _SSL__SSLSOCKET_VERIFY_CLIENT_POST_HANDSHAKE_METHODDEF \ + {"verify_client_post_handshake", (PyCFunction)_ssl__SSLSocket_verify_client_post_handshake, METH_NOARGS, _ssl__SSLSocket_verify_client_post_handshake__doc__}, + +static PyObject * +_ssl__SSLSocket_verify_client_post_handshake_impl(PySSLSocket *self); + +static PyObject * +_ssl__SSLSocket_verify_client_post_handshake(PySSLSocket *self, PyObject *Py_UNUSED(ignored)) +{ + return _ssl__SSLSocket_verify_client_post_handshake_impl(self); +} + static PyObject * _ssl__SSLContext_impl(PyTypeObject *type, int proto_version); @@ -1168,4 +1186,4 @@ _ssl_enum_crls(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kw #ifndef _SSL_ENUM_CRLS_METHODDEF #define _SSL_ENUM_CRLS_METHODDEF #endif /* !defined(_SSL_ENUM_CRLS_METHODDEF) */ -/*[clinic end generated code: output=c79fb0dfd3c90784 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=a832758678f4d934 input=a9049054013a1b77]*/ diff --git a/Tools/ssl/multissltests.py b/Tools/ssl/multissltests.py index 9d668d4202ab..2264c9331042 100755 --- a/Tools/ssl/multissltests.py +++ b/Tools/ssl/multissltests.py @@ -47,9 +47,9 @@ ] OPENSSL_RECENT_VERSIONS = [ - "1.0.2o", - "1.1.0h", - # "1.1.1-pre7", + "1.0.2p", + "1.1.0i", + "1.1.1", ] LIBRESSL_OLD_VERSIONS = [ @@ -58,7 +58,7 @@ ] LIBRESSL_RECENT_VERSIONS = [ - "2.7.3", + "2.7.4", ] # store files in ../multissl From webhook-mailer at python.org Sun Sep 23 03:31:56 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sun, 23 Sep 2018 07:31:56 -0000 Subject: [Python-checkins] bpo-34421: Improve distutils logging for non-ASCII strings. (GH-9126) (GH-9506) Message-ID: https://github.com/python/cpython/commit/c73df53569f86d0c7742bafa55958c53d57a02e4 commit: c73df53569f86d0c7742bafa55958c53d57a02e4 branch: 3.7 author: Serhiy Storchaka committer: GitHub date: 2018-09-23T10:31:53+03:00 summary: bpo-34421: Improve distutils logging for non-ASCII strings. (GH-9126) (GH-9506) Use "backslashreplace" instead of "unicode-escape". It is not implementation depended and escapes only non-encodable characters. Also simplify the code. (cherry picked from commit 4b860fd) files: M Lib/distutils/log.py M Lib/distutils/tests/test_log.py diff --git a/Lib/distutils/log.py b/Lib/distutils/log.py index 3a6602bc8b8e..8ef6b28ea2ec 100644 --- a/Lib/distutils/log.py +++ b/Lib/distutils/log.py @@ -27,14 +27,13 @@ def _log(self, level, msg, args): stream = sys.stderr else: stream = sys.stdout - if stream.errors == 'strict': + try: + stream.write('%s\n' % msg) + except UnicodeEncodeError: # emulate backslashreplace error handler encoding = stream.encoding msg = msg.encode(encoding, "backslashreplace").decode(encoding) - try: stream.write('%s\n' % msg) - except UnicodeEncodeError: - stream.write('%s\n' % msg.encode('unicode-escape').decode('ascii')) stream.flush() def log(self, level, msg, *args): diff --git a/Lib/distutils/tests/test_log.py b/Lib/distutils/tests/test_log.py index 0c2ad7a4268d..75cf900617be 100644 --- a/Lib/distutils/tests/test_log.py +++ b/Lib/distutils/tests/test_log.py @@ -1,35 +1,43 @@ """Tests for distutils.log""" +import io import sys import unittest -from tempfile import NamedTemporaryFile -from test.support import run_unittest +from test.support import swap_attr, run_unittest from distutils import log class TestLog(unittest.TestCase): def test_non_ascii(self): - # Issue #8663: test that non-ASCII text is escaped with - # backslashreplace error handler (stream use ASCII encoding and strict - # error handler) - old_stdout = sys.stdout - old_stderr = sys.stderr - old_threshold = log.set_threshold(log.DEBUG) - try: - with NamedTemporaryFile(mode="w+", encoding='ascii') as stdout, \ - NamedTemporaryFile(mode="w+", encoding='ascii') as stderr: - sys.stdout = stdout - sys.stderr = stderr - log.debug("debug:\xe9") - log.fatal("fatal:\xe9") + # Issues #8663, #34421: test that non-encodable text is escaped with + # backslashreplace error handler and encodable non-ASCII text is + # output as is. + for errors in ('strict', 'backslashreplace', 'surrogateescape', + 'replace', 'ignore'): + with self.subTest(errors=errors): + stdout = io.TextIOWrapper(io.BytesIO(), + encoding='cp437', errors=errors) + stderr = io.TextIOWrapper(io.BytesIO(), + encoding='cp437', errors=errors) + old_threshold = log.set_threshold(log.DEBUG) + try: + with swap_attr(sys, 'stdout', stdout), \ + swap_attr(sys, 'stderr', stderr): + log.debug('D?bug\tM?ss?ge') + log.fatal('F?tal\t?rr?r') + finally: + log.set_threshold(old_threshold) + stdout.seek(0) - self.assertEqual(stdout.read().rstrip(), "debug:\\xe9") + self.assertEqual(stdout.read().rstrip(), + 'D?bug\tM?ss?ge' if errors == 'replace' else + 'D?bug\tMssge' if errors == 'ignore' else + 'D?bug\tM\\u0117ss\\xe3ge') stderr.seek(0) - self.assertEqual(stderr.read().rstrip(), "fatal:\\xe9") - finally: - log.set_threshold(old_threshold) - sys.stdout = old_stdout - sys.stderr = old_stderr + self.assertEqual(stderr.read().rstrip(), + 'F?tal\t?rr?r' if errors == 'replace' else + 'F?tal\trrr' if errors == 'ignore' else + 'F?tal\t\\xc8rr\\u014dr') def test_suite(): return unittest.makeSuite(TestLog) From webhook-mailer at python.org Sun Sep 23 03:50:37 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 23 Sep 2018 07:50:37 -0000 Subject: [Python-checkins] bpo-17239: Disable external entities in SAX parser (GH-9217) Message-ID: https://github.com/python/cpython/commit/17b1d5d4e36aa57a9b25a0e694affbd1ee637e45 commit: 17b1d5d4e36aa57a9b25a0e694affbd1ee637e45 branch: master author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-23T00:50:25-07:00 summary: bpo-17239: Disable external entities in SAX parser (GH-9217) The SAX parser no longer processes general external entities by default to increase security. Before, the parser created network connections to fetch remote files or loaded local files from the file system for DTD and entities. Signed-off-by: Christian Heimes https://bugs.python.org/issue17239 files: A Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst M Doc/library/xml.dom.pulldom.rst M Doc/library/xml.rst M Doc/library/xml.sax.rst M Doc/whatsnew/3.8.rst M Lib/test/test_pulldom.py M Lib/test/test_sax.py M Lib/test/test_xml_etree.py M Lib/xml/sax/expatreader.py diff --git a/Doc/library/xml.dom.pulldom.rst b/Doc/library/xml.dom.pulldom.rst index 56f545c0e6d8..eb2b16bd6c0b 100644 --- a/Doc/library/xml.dom.pulldom.rst +++ b/Doc/library/xml.dom.pulldom.rst @@ -25,6 +25,20 @@ events until either processing is finished or an error condition occurs. maliciously constructed data. If you need to parse untrusted or unauthenticated data see :ref:`xml-vulnerabilities`. +.. versionchanged:: 3.8 + + The SAX parser no longer processes general external entities by default to + increase security by default. To enable processing of external entities, + pass a custom parser instance in:: + + from xml.dom.pulldom import parse + from xml.sax import make_parser + from xml.sax.handler import feature_external_ges + + parser = make_parser() + parser.setFeature(feature_external_ges, True) + parse(filename, parser=parser) + Example:: diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index 63c24f80ac87..9b8ba6b17c85 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -65,8 +65,8 @@ kind sax etree minidom p ========================= ============== =============== ============== ============== ============== billion laughs **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** quadratic blowup **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** -external entity expansion **Vulnerable** Safe (1) Safe (2) **Vulnerable** Safe (3) -`DTD`_ retrieval **Vulnerable** Safe Safe **Vulnerable** Safe +external entity expansion Safe (4) Safe (1) Safe (2) Safe (4) Safe (3) +`DTD`_ retrieval Safe (4) Safe Safe Safe (4) Safe decompression bomb Safe Safe Safe Safe **Vulnerable** ========================= ============== =============== ============== ============== ============== @@ -75,6 +75,8 @@ decompression bomb Safe Safe Safe S 2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns the unexpanded entity verbatim. 3. :mod:`xmlrpclib` doesn't expand external entities and omits them. +4. Since Python 3.8.0, external general entities are no longer processed by + default since Python. billion laughs / exponential entity expansion diff --git a/Doc/library/xml.sax.rst b/Doc/library/xml.sax.rst index 78d6633e098b..aa3ea9bfc55a 100644 --- a/Doc/library/xml.sax.rst +++ b/Doc/library/xml.sax.rst @@ -24,6 +24,14 @@ the SAX API. constructed data. If you need to parse untrusted or unauthenticated data see :ref:`xml-vulnerabilities`. +.. versionchanged:: 3.8 + + The SAX parser no longer processes general external entities by default + to increase security. Before, the parser created network connections + to fetch remote files or loaded local files from the file + system for DTD and entities. The feature can be enabled again with method + :meth:`~xml.sax.xmlreader.XMLReader.setFeature` on the parser object + and argument :data:`~xml.sax.handler.feature_external_ges`. The convenience functions are: diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 9aaaa761cc8a..e37a70f32d99 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -155,6 +155,15 @@ venv activating virtual environments under PowerShell Core 6.1. (Contributed by Brett Cannon in :issue:`32718`.) +xml +--- + +* As mitigation against DTD and external entity retrieval, the + :mod:`xml.dom.minidom` and mod:`xml.sax` modules no longer process + external entities by default. + (Contributed by Christian Heimes in :issue:`17239`.) + + Optimizations ============= @@ -333,6 +342,9 @@ Changes in the Python API * :class:`uuid.UUID` now uses ``__slots__``, therefore instances can no longer be weak-referenced and attributes can no longer be added. +* :mod:`xml.dom.minidom` and mod:`xml.sax` modules no longer process + external entities by default. + (Contributed by Christian Heimes in :issue:`17239`.) CPython bytecode changes ------------------------ diff --git a/Lib/test/test_pulldom.py b/Lib/test/test_pulldom.py index f454098c65b1..4a1bad3442b2 100644 --- a/Lib/test/test_pulldom.py +++ b/Lib/test/test_pulldom.py @@ -3,6 +3,7 @@ import xml.sax from xml.sax.xmlreader import AttributesImpl +from xml.sax.handler import feature_external_ges from xml.dom import pulldom from test.support import findfile @@ -166,6 +167,12 @@ def test_getitem_deprecation(self): # This should have returned 'END_ELEMENT'. self.assertEqual(parser[-1][0], pulldom.START_DOCUMENT) + def test_external_ges_default(self): + parser = pulldom.parseString(SMALL_SAMPLE) + saxparser = parser.parser + ges = saxparser.getFeature(feature_external_ges) + self.assertEqual(ges, False) + class ThoroughTestCase(unittest.TestCase): """Test the hard-to-reach parts of pulldom.""" diff --git a/Lib/test/test_sax.py b/Lib/test/test_sax.py index 2eb62905ffa8..3044960a0ed1 100644 --- a/Lib/test/test_sax.py +++ b/Lib/test/test_sax.py @@ -13,13 +13,14 @@ from xml.sax.saxutils import XMLGenerator, escape, unescape, quoteattr, \ XMLFilterBase, prepare_input_source from xml.sax.expatreader import create_parser -from xml.sax.handler import feature_namespaces +from xml.sax.handler import feature_namespaces, feature_external_ges from xml.sax.xmlreader import InputSource, AttributesImpl, AttributesNSImpl from io import BytesIO, StringIO import codecs import gc import os.path import shutil +from urllib.error import URLError from test import support from test.support import findfile, run_unittest, TESTFN @@ -911,6 +912,18 @@ def notationDecl(self, name, publicId, systemId): def unparsedEntityDecl(self, name, publicId, systemId, ndata): self._entities.append((name, publicId, systemId, ndata)) + + class TestEntityRecorder: + def __init__(self): + self.entities = [] + + def resolveEntity(self, publicId, systemId): + self.entities.append((publicId, systemId)) + source = InputSource() + source.setPublicId(publicId) + source.setSystemId(systemId) + return source + def test_expat_dtdhandler(self): parser = create_parser() handler = self.TestDTDHandler() @@ -927,6 +940,32 @@ def test_expat_dtdhandler(self): [("GIF", "-//CompuServe//NOTATION Graphics Interchange Format 89a//EN", None)]) self.assertEqual(handler._entities, [("img", None, "expat.gif", "GIF")]) + def test_expat_external_dtd_enabled(self): + parser = create_parser() + parser.setFeature(feature_external_ges, True) + resolver = self.TestEntityRecorder() + parser.setEntityResolver(resolver) + + with self.assertRaises(URLError): + parser.feed( + '\n' + ) + self.assertEqual( + resolver.entities, [(None, 'unsupported://non-existing')] + ) + + def test_expat_external_dtd_default(self): + parser = create_parser() + resolver = self.TestEntityRecorder() + parser.setEntityResolver(resolver) + + parser.feed( + '\n' + ) + parser.feed('') + parser.close() + self.assertEqual(resolver.entities, []) + # ===== EntityResolver support class TestEntityResolver: @@ -936,8 +975,9 @@ def resolveEntity(self, publicId, systemId): inpsrc.setByteStream(BytesIO(b"")) return inpsrc - def test_expat_entityresolver(self): + def test_expat_entityresolver_enabled(self): parser = create_parser() + parser.setFeature(feature_external_ges, True) parser.setEntityResolver(self.TestEntityResolver()) result = BytesIO() parser.setContentHandler(XMLGenerator(result)) @@ -951,6 +991,22 @@ def test_expat_entityresolver(self): self.assertEqual(result.getvalue(), start + b"") + def test_expat_entityresolver_default(self): + parser = create_parser() + self.assertEqual(parser.getFeature(feature_external_ges), False) + parser.setEntityResolver(self.TestEntityResolver()) + result = BytesIO() + parser.setContentHandler(XMLGenerator(result)) + + parser.feed('\n') + parser.feed(']>\n') + parser.feed('&test;') + parser.close() + + self.assertEqual(result.getvalue(), start + + b"") + # ===== Attributes support class AttrGatherer(ContentHandler): diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index a52529051bf0..ecb910f04f59 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -91,6 +91,12 @@ &entity; """ +EXTERNAL_ENTITY_XML = """\ + +]> +&entity; +""" def checkwarnings(*filters, quiet=False): def decorator(test): @@ -861,6 +867,13 @@ def test_entity(self): root = parser.close() self.serialize_check(root, 'text') + # 4) external (SYSTEM) entity + + with self.assertRaises(ET.ParseError) as cm: + ET.XML(EXTERNAL_ENTITY_XML) + self.assertEqual(str(cm.exception), + 'undefined entity &entity;: line 4, column 10') + def test_namespace(self): # Test namespace issues. diff --git a/Lib/xml/sax/expatreader.py b/Lib/xml/sax/expatreader.py index 421358fa5bc7..5066ffc2fa51 100644 --- a/Lib/xml/sax/expatreader.py +++ b/Lib/xml/sax/expatreader.py @@ -95,7 +95,7 @@ def __init__(self, namespaceHandling=0, bufsize=2**16-20): self._lex_handler_prop = None self._parsing = 0 self._entity_stack = [] - self._external_ges = 1 + self._external_ges = 0 self._interning = None # XMLReader methods diff --git a/Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst b/Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst new file mode 100644 index 000000000000..8dd0fe8c1b53 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst @@ -0,0 +1,3 @@ +The xml.sax and xml.dom.minidom parsers no longer processes external +entities by default. External DTD and ENTITY declarations no longer +load files or create network connections. From webhook-mailer at python.org Sun Sep 23 03:54:02 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 23 Sep 2018 07:54:02 -0000 Subject: [Python-checkins] bpo-34421: Improve distutils logging for non-ASCII strings. (GH-9126) (GH-9506) Message-ID: https://github.com/python/cpython/commit/0b67995bfa45393585e2e0017c82c706c4a04b04 commit: 0b67995bfa45393585e2e0017c82c706c4a04b04 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-23T00:53:58-07:00 summary: bpo-34421: Improve distutils logging for non-ASCII strings. (GH-9126) (GH-9506) Use "backslashreplace" instead of "unicode-escape". It is not implementation depended and escapes only non-encodable characters. Also simplify the code. (cherry picked from commit 4b860fd) (cherry picked from commit c73df53569f86d0c7742bafa55958c53d57a02e4) Co-authored-by: Serhiy Storchaka files: M Lib/distutils/log.py M Lib/distutils/tests/test_log.py diff --git a/Lib/distutils/log.py b/Lib/distutils/log.py index 3a6602bc8b8e..8ef6b28ea2ec 100644 --- a/Lib/distutils/log.py +++ b/Lib/distutils/log.py @@ -27,14 +27,13 @@ def _log(self, level, msg, args): stream = sys.stderr else: stream = sys.stdout - if stream.errors == 'strict': + try: + stream.write('%s\n' % msg) + except UnicodeEncodeError: # emulate backslashreplace error handler encoding = stream.encoding msg = msg.encode(encoding, "backslashreplace").decode(encoding) - try: stream.write('%s\n' % msg) - except UnicodeEncodeError: - stream.write('%s\n' % msg.encode('unicode-escape').decode('ascii')) stream.flush() def log(self, level, msg, *args): diff --git a/Lib/distutils/tests/test_log.py b/Lib/distutils/tests/test_log.py index 0c2ad7a4268d..75cf900617be 100644 --- a/Lib/distutils/tests/test_log.py +++ b/Lib/distutils/tests/test_log.py @@ -1,35 +1,43 @@ """Tests for distutils.log""" +import io import sys import unittest -from tempfile import NamedTemporaryFile -from test.support import run_unittest +from test.support import swap_attr, run_unittest from distutils import log class TestLog(unittest.TestCase): def test_non_ascii(self): - # Issue #8663: test that non-ASCII text is escaped with - # backslashreplace error handler (stream use ASCII encoding and strict - # error handler) - old_stdout = sys.stdout - old_stderr = sys.stderr - old_threshold = log.set_threshold(log.DEBUG) - try: - with NamedTemporaryFile(mode="w+", encoding='ascii') as stdout, \ - NamedTemporaryFile(mode="w+", encoding='ascii') as stderr: - sys.stdout = stdout - sys.stderr = stderr - log.debug("debug:\xe9") - log.fatal("fatal:\xe9") + # Issues #8663, #34421: test that non-encodable text is escaped with + # backslashreplace error handler and encodable non-ASCII text is + # output as is. + for errors in ('strict', 'backslashreplace', 'surrogateescape', + 'replace', 'ignore'): + with self.subTest(errors=errors): + stdout = io.TextIOWrapper(io.BytesIO(), + encoding='cp437', errors=errors) + stderr = io.TextIOWrapper(io.BytesIO(), + encoding='cp437', errors=errors) + old_threshold = log.set_threshold(log.DEBUG) + try: + with swap_attr(sys, 'stdout', stdout), \ + swap_attr(sys, 'stderr', stderr): + log.debug('D?bug\tM?ss?ge') + log.fatal('F?tal\t?rr?r') + finally: + log.set_threshold(old_threshold) + stdout.seek(0) - self.assertEqual(stdout.read().rstrip(), "debug:\\xe9") + self.assertEqual(stdout.read().rstrip(), + 'D?bug\tM?ss?ge' if errors == 'replace' else + 'D?bug\tMssge' if errors == 'ignore' else + 'D?bug\tM\\u0117ss\\xe3ge') stderr.seek(0) - self.assertEqual(stderr.read().rstrip(), "fatal:\\xe9") - finally: - log.set_threshold(old_threshold) - sys.stdout = old_stdout - sys.stderr = old_stderr + self.assertEqual(stderr.read().rstrip(), + 'F?tal\t?rr?r' if errors == 'replace' else + 'F?tal\trrr' if errors == 'ignore' else + 'F?tal\t\\xc8rr\\u014dr') def test_suite(): return unittest.makeSuite(TestLog) From solipsis at pitrou.net Sun Sep 23 05:09:38 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 23 Sep 2018 09:09:38 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=9 Message-ID: <20180923090938.1.14AD7E2ACE55AE14@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [0, 7, 0] memory blocks, sum=7 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_forkserver leaked [0, 0, -2] memory blocks, sum=-2 test_multiprocessing_spawn leaked [1, -2, 1] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogDosEfW', '--timeout', '7200'] From webhook-mailer at python.org Sun Sep 23 07:10:17 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sun, 23 Sep 2018 11:10:17 -0000 Subject: [Python-checkins] Use in-memory streams instead of NamedTemporaryFile. (GH-9508) Message-ID: https://github.com/python/cpython/commit/24b447edf204a674f9e164ea6d553562c21de1a4 commit: 24b447edf204a674f9e164ea6d553562c21de1a4 branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-23T14:10:07+03:00 summary: Use in-memory streams instead of NamedTemporaryFile. (GH-9508) files: M Lib/distutils/tests/test_log.py diff --git a/Lib/distutils/tests/test_log.py b/Lib/distutils/tests/test_log.py index 22c26246ca9d..75cf900617be 100644 --- a/Lib/distutils/tests/test_log.py +++ b/Lib/distutils/tests/test_log.py @@ -1,8 +1,8 @@ """Tests for distutils.log""" +import io import sys import unittest -from tempfile import NamedTemporaryFile from test.support import swap_attr, run_unittest from distutils import log @@ -14,9 +14,11 @@ def test_non_ascii(self): # output as is. for errors in ('strict', 'backslashreplace', 'surrogateescape', 'replace', 'ignore'): - with self.subTest(errors=errors), \ - NamedTemporaryFile("w+", encoding='cp437', errors=errors) as stdout, \ - NamedTemporaryFile("w+", encoding='cp437', errors=errors) as stderr: + with self.subTest(errors=errors): + stdout = io.TextIOWrapper(io.BytesIO(), + encoding='cp437', errors=errors) + stderr = io.TextIOWrapper(io.BytesIO(), + encoding='cp437', errors=errors) old_threshold = log.set_threshold(log.DEBUG) try: with swap_attr(sys, 'stdout', stdout), \ From webhook-mailer at python.org Sun Sep 23 08:23:23 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 23 Sep 2018 12:23:23 -0000 Subject: [Python-checkins] bpo-34548: IDLE: use configured theme colors in TextView (GH-9008) Message-ID: https://github.com/python/cpython/commit/c87d9f406bb23657c1b4cd63017bb7bd7693a1fb commit: c87d9f406bb23657c1b4cd63017bb7bd7693a1fb branch: master author: Tal Einat committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-23T05:23:15-07:00 summary: bpo-34548: IDLE: use configured theme colors in TextView (GH-9008) https://bugs.python.org/issue34548 files: A Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst M Lib/idlelib/colorizer.py M Lib/idlelib/textview.py diff --git a/Lib/idlelib/colorizer.py b/Lib/idlelib/colorizer.py index d626d23121df..d4e592b6ebe0 100644 --- a/Lib/idlelib/colorizer.py +++ b/Lib/idlelib/colorizer.py @@ -31,11 +31,12 @@ def make_pat(): prog = re.compile(make_pat(), re.S) idprog = re.compile(r"\s+(\w+)", re.S) -def color_config(text): # Called from htest, Editor, and Turtle Demo. - '''Set color opitons of Text widget. +def color_config(text): + """Set color options of Text widget. - Should be called whenever ColorDelegator is called. - ''' + If ColorDelegator is used, this should be called first. + """ + # Called from htest, TextFrame, Editor, and Turtledemo. # Not automatic because ColorDelegator does not know 'text'. theme = idleConf.CurrentTheme() normal_colors = idleConf.GetHighlight(theme, 'normal') @@ -50,6 +51,7 @@ def color_config(text): # Called from htest, Editor, and Turtle Demo. inactiveselectbackground=select_colors['background'], # new in 8.5 ) + class ColorDelegator(Delegator): def __init__(self): @@ -285,6 +287,7 @@ def _color_delegator(parent): # htest # d = ColorDelegator() p.insertfilter(d) + if __name__ == "__main__": from unittest import main main('idlelib.idle_test.test_colorizer', verbosity=2, exit=False) diff --git a/Lib/idlelib/textview.py b/Lib/idlelib/textview.py index 75b24703b4c3..464e6ac6b94e 100644 --- a/Lib/idlelib/textview.py +++ b/Lib/idlelib/textview.py @@ -5,6 +5,8 @@ from tkinter.ttk import Frame, Scrollbar, Button from tkinter.messagebox import showerror +from idlelib.colorizer import color_config + class TextFrame(Frame): "Display text with scrollbar." @@ -18,12 +20,9 @@ def __init__(self, parent, rawtext): super().__init__(parent) self['relief'] = 'sunken' self['height'] = 700 - # TODO: get fg/bg from theme. - self.bg = '#ffffff' - self.fg = '#000000' - self.text = text = Text(self, wrap='word', highlightthickness=0, - fg=self.fg, bg=self.bg) + self.text = text = Text(self, wrap='word', highlightthickness=0) + color_config(text) self.scroll = scroll = Scrollbar(self, orient='vertical', takefocus=False, command=text.yview) text['yscrollcommand'] = scroll.set diff --git a/Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst b/Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst new file mode 100644 index 000000000000..237c0c788c2c --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst @@ -0,0 +1 @@ +Use configured color theme for read-only text views. From webhook-mailer at python.org Sun Sep 23 09:31:40 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 23 Sep 2018 13:31:40 -0000 Subject: [Python-checkins] bpo-34548: IDLE: use configured theme colors in TextView (GH-9008) Message-ID: https://github.com/python/cpython/commit/aeadf59e457ca0ee20983eb1ed78b348f0b453e3 commit: aeadf59e457ca0ee20983eb1ed78b348f0b453e3 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-23T06:31:35-07:00 summary: bpo-34548: IDLE: use configured theme colors in TextView (GH-9008) https://bugs.python.org/issue34548 (cherry picked from commit c87d9f406bb23657c1b4cd63017bb7bd7693a1fb) Co-authored-by: Tal Einat files: A Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst M Lib/idlelib/colorizer.py M Lib/idlelib/textview.py diff --git a/Lib/idlelib/colorizer.py b/Lib/idlelib/colorizer.py index d626d23121df..d4e592b6ebe0 100644 --- a/Lib/idlelib/colorizer.py +++ b/Lib/idlelib/colorizer.py @@ -31,11 +31,12 @@ def make_pat(): prog = re.compile(make_pat(), re.S) idprog = re.compile(r"\s+(\w+)", re.S) -def color_config(text): # Called from htest, Editor, and Turtle Demo. - '''Set color opitons of Text widget. +def color_config(text): + """Set color options of Text widget. - Should be called whenever ColorDelegator is called. - ''' + If ColorDelegator is used, this should be called first. + """ + # Called from htest, TextFrame, Editor, and Turtledemo. # Not automatic because ColorDelegator does not know 'text'. theme = idleConf.CurrentTheme() normal_colors = idleConf.GetHighlight(theme, 'normal') @@ -50,6 +51,7 @@ def color_config(text): # Called from htest, Editor, and Turtle Demo. inactiveselectbackground=select_colors['background'], # new in 8.5 ) + class ColorDelegator(Delegator): def __init__(self): @@ -285,6 +287,7 @@ def _color_delegator(parent): # htest # d = ColorDelegator() p.insertfilter(d) + if __name__ == "__main__": from unittest import main main('idlelib.idle_test.test_colorizer', verbosity=2, exit=False) diff --git a/Lib/idlelib/textview.py b/Lib/idlelib/textview.py index 75b24703b4c3..464e6ac6b94e 100644 --- a/Lib/idlelib/textview.py +++ b/Lib/idlelib/textview.py @@ -5,6 +5,8 @@ from tkinter.ttk import Frame, Scrollbar, Button from tkinter.messagebox import showerror +from idlelib.colorizer import color_config + class TextFrame(Frame): "Display text with scrollbar." @@ -18,12 +20,9 @@ def __init__(self, parent, rawtext): super().__init__(parent) self['relief'] = 'sunken' self['height'] = 700 - # TODO: get fg/bg from theme. - self.bg = '#ffffff' - self.fg = '#000000' - self.text = text = Text(self, wrap='word', highlightthickness=0, - fg=self.fg, bg=self.bg) + self.text = text = Text(self, wrap='word', highlightthickness=0) + color_config(text) self.scroll = scroll = Scrollbar(self, orient='vertical', takefocus=False, command=text.yview) text['yscrollcommand'] = scroll.set diff --git a/Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst b/Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst new file mode 100644 index 000000000000..237c0c788c2c --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst @@ -0,0 +1 @@ +Use configured color theme for read-only text views. From webhook-mailer at python.org Sun Sep 23 09:31:54 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 23 Sep 2018 13:31:54 -0000 Subject: [Python-checkins] bpo-34548: IDLE: use configured theme colors in TextView (GH-9008) Message-ID: https://github.com/python/cpython/commit/6b48f9854e2ae35fd74bcd359451eb37ae65f798 commit: 6b48f9854e2ae35fd74bcd359451eb37ae65f798 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-23T06:31:51-07:00 summary: bpo-34548: IDLE: use configured theme colors in TextView (GH-9008) https://bugs.python.org/issue34548 (cherry picked from commit c87d9f406bb23657c1b4cd63017bb7bd7693a1fb) Co-authored-by: Tal Einat files: A Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst M Lib/idlelib/colorizer.py M Lib/idlelib/textview.py diff --git a/Lib/idlelib/colorizer.py b/Lib/idlelib/colorizer.py index 58cced249ac2..f2de9fc45515 100644 --- a/Lib/idlelib/colorizer.py +++ b/Lib/idlelib/colorizer.py @@ -31,11 +31,12 @@ def make_pat(): prog = re.compile(make_pat(), re.S) idprog = re.compile(r"\s+(\w+)", re.S) -def color_config(text): # Called from htest, Editor, and Turtle Demo. - '''Set color opitons of Text widget. +def color_config(text): + """Set color options of Text widget. - Should be called whenever ColorDelegator is called. - ''' + If ColorDelegator is used, this should be called first. + """ + # Called from htest, TextFrame, Editor, and Turtledemo. # Not automatic because ColorDelegator does not know 'text'. theme = idleConf.CurrentTheme() normal_colors = idleConf.GetHighlight(theme, 'normal') @@ -50,6 +51,7 @@ def color_config(text): # Called from htest, Editor, and Turtle Demo. inactiveselectbackground=select_colors['background'], # new in 8.5 ) + class ColorDelegator(Delegator): def __init__(self): @@ -285,6 +287,7 @@ def _color_delegator(parent): # htest # d = ColorDelegator() p.insertfilter(d) + if __name__ == "__main__": from unittest import main main('idlelib.idle_test.test_colorizer', verbosity=2, exit=False) diff --git a/Lib/idlelib/textview.py b/Lib/idlelib/textview.py index 75b24703b4c3..464e6ac6b94e 100644 --- a/Lib/idlelib/textview.py +++ b/Lib/idlelib/textview.py @@ -5,6 +5,8 @@ from tkinter.ttk import Frame, Scrollbar, Button from tkinter.messagebox import showerror +from idlelib.colorizer import color_config + class TextFrame(Frame): "Display text with scrollbar." @@ -18,12 +20,9 @@ def __init__(self, parent, rawtext): super().__init__(parent) self['relief'] = 'sunken' self['height'] = 700 - # TODO: get fg/bg from theme. - self.bg = '#ffffff' - self.fg = '#000000' - self.text = text = Text(self, wrap='word', highlightthickness=0, - fg=self.fg, bg=self.bg) + self.text = text = Text(self, wrap='word', highlightthickness=0) + color_config(text) self.scroll = scroll = Scrollbar(self, orient='vertical', takefocus=False, command=text.yview) text['yscrollcommand'] = scroll.set diff --git a/Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst b/Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst new file mode 100644 index 000000000000..237c0c788c2c --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2018-09-22-20-25-07.bpo-34548.7pBzjg.rst @@ -0,0 +1 @@ +Use configured color theme for read-only text views. From webhook-mailer at python.org Sun Sep 23 20:35:03 2018 From: webhook-mailer at python.org (Lisa Roach) Date: Mon, 24 Sep 2018 00:35:03 -0000 Subject: [Python-checkins] bpo-34659: Adds initial kwarg to itertools.accumulate() (GH-9345) Message-ID: https://github.com/python/cpython/commit/9718b59ee5f2416cdb8116ea5837b062faf0d9f8 commit: 9718b59ee5f2416cdb8116ea5837b062faf0d9f8 branch: master author: Lisa Roach committer: GitHub date: 2018-09-23T17:34:59-07:00 summary: bpo-34659: Adds initial kwarg to itertools.accumulate() (GH-9345) files: A Misc/NEWS.d/next/Library/2018-09-16-17-04-16.bpo-34659.CWemzH.rst M Doc/library/itertools.rst M Lib/test/test_itertools.py M Modules/clinic/itertoolsmodule.c.h M Modules/itertoolsmodule.c diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 959424ff9143..b1513cd8b1b9 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -86,29 +86,38 @@ The following module functions all construct and return iterators. Some provide streams of infinite length, so they should only be accessed by functions or loops that truncate the stream. -.. function:: accumulate(iterable[, func]) +.. function:: accumulate(iterable[, func, *, initial=None]) Make an iterator that returns accumulated sums, or accumulated results of other binary functions (specified via the optional - *func* argument). If *func* is supplied, it should be a function + *func* argument). + + If *func* is supplied, it should be a function of two arguments. Elements of the input *iterable* may be any type that can be accepted as arguments to *func*. (For example, with the default operation of addition, elements may be any addable type including :class:`~decimal.Decimal` or - :class:`~fractions.Fraction`.) If the input iterable is empty, the - output iterable will also be empty. + :class:`~fractions.Fraction`.) + + Usually, the number of elements output matches the input iterable. + However, if the keyword argument *initial* is provided, the + accumulation leads off with the *initial* value so that the output + has one more element than the input iterable. Roughly equivalent to:: - def accumulate(iterable, func=operator.add): + def accumulate(iterable, func=operator.add, *, initial=None): 'Return running totals' # accumulate([1,2,3,4,5]) --> 1 3 6 10 15 + # accumulate([1,2,3,4,5], initial=100) --> 100 101 103 106 110 115 # accumulate([1,2,3,4,5], operator.mul) --> 1 2 6 24 120 it = iter(iterable) - try: - total = next(it) - except StopIteration: - return + total = initial + if initial is None: + try: + total = next(it) + except StopIteration: + return yield total for element in it: total = func(total, element) @@ -152,6 +161,9 @@ loops that truncate the stream. .. versionchanged:: 3.3 Added the optional *func* parameter. + .. versionchanged:: 3.8 + Added the optional *initial* parameter. + .. function:: chain(*iterables) Make an iterator that returns elements from the first iterable until it is diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py index cbbb4c4f71d3..ea060a98a5ee 100644 --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -147,6 +147,12 @@ def test_accumulate(self): list(accumulate(s, chr)) # unary-operation for proto in range(pickle.HIGHEST_PROTOCOL + 1): self.pickletest(proto, accumulate(range(10))) # test pickling + self.pickletest(proto, accumulate(range(10), initial=7)) + self.assertEqual(list(accumulate([10, 5, 1], initial=None)), [10, 15, 16]) + self.assertEqual(list(accumulate([10, 5, 1], initial=100)), [100, 110, 115, 116]) + self.assertEqual(list(accumulate([], initial=100)), [100]) + with self.assertRaises(TypeError): + list(accumulate([10, 20], 100)) def test_chain(self): diff --git a/Misc/NEWS.d/next/Library/2018-09-16-17-04-16.bpo-34659.CWemzH.rst b/Misc/NEWS.d/next/Library/2018-09-16-17-04-16.bpo-34659.CWemzH.rst new file mode 100644 index 000000000000..3b7925aafd4e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-16-17-04-16.bpo-34659.CWemzH.rst @@ -0,0 +1 @@ +Add an optional *initial* argument to itertools.accumulate(). diff --git a/Modules/clinic/itertoolsmodule.c.h b/Modules/clinic/itertoolsmodule.c.h index 94df96c0b7e4..476adc1f5c5d 100644 --- a/Modules/clinic/itertoolsmodule.c.h +++ b/Modules/clinic/itertoolsmodule.c.h @@ -382,29 +382,30 @@ itertools_permutations(PyTypeObject *type, PyObject *args, PyObject *kwargs) } PyDoc_STRVAR(itertools_accumulate__doc__, -"accumulate(iterable, func=None)\n" +"accumulate(iterable, func=None, *, initial=None)\n" "--\n" "\n" "Return series of accumulated sums (or other binary function results)."); static PyObject * itertools_accumulate_impl(PyTypeObject *type, PyObject *iterable, - PyObject *binop); + PyObject *binop, PyObject *initial); static PyObject * itertools_accumulate(PyTypeObject *type, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static const char * const _keywords[] = {"iterable", "func", NULL}; - static _PyArg_Parser _parser = {"O|O:accumulate", _keywords, 0}; + static const char * const _keywords[] = {"iterable", "func", "initial", NULL}; + static _PyArg_Parser _parser = {"O|O$O:accumulate", _keywords, 0}; PyObject *iterable; PyObject *binop = Py_None; + PyObject *initial = Py_None; if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, - &iterable, &binop)) { + &iterable, &binop, &initial)) { goto exit; } - return_value = itertools_accumulate_impl(type, iterable, binop); + return_value = itertools_accumulate_impl(type, iterable, binop, initial); exit: return return_value; @@ -509,4 +510,4 @@ itertools_count(PyTypeObject *type, PyObject *args, PyObject *kwargs) exit: return return_value; } -/*[clinic end generated code: output=d9eb9601bd3296ef input=a9049054013a1b77]*/ +/*[clinic end generated code: output=c8c47b766deeffc3 input=a9049054013a1b77]*/ diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index ec8f0ae14206..89c0280c9d35 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -3475,6 +3475,7 @@ typedef struct { PyObject *total; PyObject *it; PyObject *binop; + PyObject *initial; } accumulateobject; static PyTypeObject accumulate_type; @@ -3484,18 +3485,19 @@ static PyTypeObject accumulate_type; itertools.accumulate.__new__ iterable: object func as binop: object = None + * + initial: object = None Return series of accumulated sums (or other binary function results). [clinic start generated code]*/ static PyObject * itertools_accumulate_impl(PyTypeObject *type, PyObject *iterable, - PyObject *binop) -/*[clinic end generated code: output=514d0fb30ba14d55 input=6d9d16aaa1d3cbfc]*/ + PyObject *binop, PyObject *initial) +/*[clinic end generated code: output=66da2650627128f8 input=c4ce20ac59bf7ffd]*/ { PyObject *it; accumulateobject *lz; - /* Get iterator. */ it = PyObject_GetIter(iterable); if (it == NULL) @@ -3514,6 +3516,8 @@ itertools_accumulate_impl(PyTypeObject *type, PyObject *iterable, } lz->total = NULL; lz->it = it; + Py_XINCREF(initial); + lz->initial = initial; return (PyObject *)lz; } @@ -3524,6 +3528,7 @@ accumulate_dealloc(accumulateobject *lz) Py_XDECREF(lz->binop); Py_XDECREF(lz->total); Py_XDECREF(lz->it); + Py_XDECREF(lz->initial); Py_TYPE(lz)->tp_free(lz); } @@ -3533,6 +3538,7 @@ accumulate_traverse(accumulateobject *lz, visitproc visit, void *arg) Py_VISIT(lz->binop); Py_VISIT(lz->it); Py_VISIT(lz->total); + Py_VISIT(lz->initial); return 0; } @@ -3541,6 +3547,13 @@ accumulate_next(accumulateobject *lz) { PyObject *val, *newtotal; + if (lz->initial != Py_None) { + lz->total = lz->initial; + Py_INCREF(Py_None); + lz->initial = Py_None; + Py_INCREF(lz->total); + return lz->total; + } val = (*Py_TYPE(lz->it)->tp_iternext)(lz->it); if (val == NULL) return NULL; @@ -3567,6 +3580,19 @@ accumulate_next(accumulateobject *lz) static PyObject * accumulate_reduce(accumulateobject *lz, PyObject *Py_UNUSED(ignored)) { + if (lz->initial != Py_None) { + PyObject *it; + + assert(lz->total == NULL); + if (PyType_Ready(&chain_type) < 0) + return NULL; + it = PyObject_CallFunction((PyObject *)&chain_type, "(O)O", + lz->initial, lz->it); + if (it == NULL) + return NULL; + return Py_BuildValue("O(NO)O", Py_TYPE(lz), + it, lz->binop?lz->binop:Py_None, Py_None); + } if (lz->total == Py_None) { PyObject *it; From webhook-mailer at python.org Mon Sep 24 04:39:13 2018 From: webhook-mailer at python.org (Petr Viktorin) Date: Mon, 24 Sep 2018 08:39:13 -0000 Subject: [Python-checkins] Migrate datetime.date.fromtimestamp to Argument Clinic (GH-8535) Message-ID: https://github.com/python/cpython/commit/a0fd7f1b55a1d76842fa2c6b5777a39cdcf2bb5e commit: a0fd7f1b55a1d76842fa2c6b5777a39cdcf2bb5e branch: master author: Tim Hoffmann <2836374+timhoffm at users.noreply.github.com> committer: Petr Viktorin date: 2018-09-24T10:39:02+02:00 summary: Migrate datetime.date.fromtimestamp to Argument Clinic (GH-8535) files: A Misc/NEWS.d/next/Documentation/2018-07-28-17-17-42.bpo-20177.cOZJWp.rst M Misc/ACKS M Modules/_datetimemodule.c M Modules/clinic/_datetimemodule.c.h diff --git a/Misc/ACKS b/Misc/ACKS index 96563f62a1a8..272130f4e643 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -672,6 +672,7 @@ Benjamin Hodgson Joerg-Cyril Hoehle Gregor Hoffleit Chris Hoffman +Tim Hoffmann Stefan Hoffmeister Albert Hofkamp Chris Hogan diff --git a/Misc/NEWS.d/next/Documentation/2018-07-28-17-17-42.bpo-20177.cOZJWp.rst b/Misc/NEWS.d/next/Documentation/2018-07-28-17-17-42.bpo-20177.cOZJWp.rst new file mode 100644 index 000000000000..35592a64a8da --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-07-28-17-17-42.bpo-20177.cOZJWp.rst @@ -0,0 +1 @@ +Migrate datetime.date.fromtimestamp to Argument Clinic. Patch by Tim Hoffmann. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index 3ba700bbf852..b7c59f1bd862 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -23,8 +23,9 @@ /*[clinic input] module datetime class datetime.datetime "PyDateTime_DateTime *" "&PyDateTime_DateTimeType" +class datetime.date "PyDateTime_Date *" "&PyDateTime_DateType" [clinic start generated code]*/ -/*[clinic end generated code: output=da39a3ee5e6b4b0d input=78142cb64b9e98bc]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=25138ad6a696b785]*/ #include "clinic/_datetimemodule.c.h" @@ -2788,9 +2789,8 @@ date_new(PyTypeObject *type, PyObject *args, PyObject *kw) return self; } -/* Return new date from localtime(t). */ static PyObject * -date_local_from_object(PyObject *cls, PyObject *obj) +date_fromtimestamp(PyObject *cls, PyObject *obj) { struct tm tm; time_t t; @@ -2835,19 +2835,26 @@ date_today(PyObject *cls, PyObject *dummy) return result; } -/* Return new date from given timestamp (Python timestamp -- a double). */ +/*[clinic input] + at classmethod +datetime.date.fromtimestamp + + timestamp: object + / + +Create a date from a POSIX timestamp. + +The timestamp is a number, e.g. created via time.time(), that is interpreted +as local time. +[clinic start generated code]*/ + static PyObject * -date_fromtimestamp(PyObject *cls, PyObject *args) +datetime_date_fromtimestamp(PyTypeObject *type, PyObject *timestamp) +/*[clinic end generated code: output=fd045fda58168869 input=eabb3fe7f40491fe]*/ { - PyObject *timestamp; - PyObject *result = NULL; - - if (PyArg_ParseTuple(args, "O:fromtimestamp", ×tamp)) - result = date_local_from_object(cls, timestamp); - return result; + return date_fromtimestamp((PyObject *) type, timestamp); } - /* Return new date from proleptic Gregorian ordinal. Raises ValueError if * the ordinal is out of range. */ @@ -3193,11 +3200,7 @@ date_reduce(PyDateTime_Date *self, PyObject *arg) static PyMethodDef date_methods[] = { /* Class methods: */ - - {"fromtimestamp", (PyCFunction)date_fromtimestamp, METH_VARARGS | - METH_CLASS, - PyDoc_STR("timestamp -> local date from a POSIX timestamp (like " - "time.time()).")}, + DATETIME_DATE_FROMTIMESTAMP_METHODDEF {"fromordinal", (PyCFunction)date_fromordinal, METH_VARARGS | METH_CLASS, diff --git a/Modules/clinic/_datetimemodule.c.h b/Modules/clinic/_datetimemodule.c.h index 4d9204958157..0f917ddd2e18 100644 --- a/Modules/clinic/_datetimemodule.c.h +++ b/Modules/clinic/_datetimemodule.c.h @@ -2,6 +2,18 @@ preserve [clinic start generated code]*/ +PyDoc_STRVAR(datetime_date_fromtimestamp__doc__, +"fromtimestamp($type, timestamp, /)\n" +"--\n" +"\n" +"Create a date from a POSIX timestamp.\n" +"\n" +"The timestamp is a number, e.g. created via time.time(), that is interpreted\n" +"as local time."); + +#define DATETIME_DATE_FROMTIMESTAMP_METHODDEF \ + {"fromtimestamp", (PyCFunction)datetime_date_fromtimestamp, METH_O|METH_CLASS, datetime_date_fromtimestamp__doc__}, + PyDoc_STRVAR(datetime_datetime_now__doc__, "now($type, /, tz=None)\n" "--\n" @@ -36,4 +48,4 @@ datetime_datetime_now(PyTypeObject *type, PyObject *const *args, Py_ssize_t narg exit: return return_value; } -/*[clinic end generated code: output=1fc05897ab239b3f input=a9049054013a1b77]*/ +/*[clinic end generated code: output=7fd14bd67749da23 input=a9049054013a1b77]*/ From solipsis at pitrou.net Mon Sep 24 05:08:33 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 24 Sep 2018 09:08:33 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=-2 Message-ID: <20180924090833.1.87DE8AA98025E222@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [-7, 1, 0] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_forkserver leaked [-2, 1, 1] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogX3OSDa', '--timeout', '7200'] From webhook-mailer at python.org Mon Sep 24 05:51:26 2018 From: webhook-mailer at python.org (Carol Willing) Date: Mon, 24 Sep 2018 09:51:26 -0000 Subject: [Python-checkins] bpo-34728: Remove deprecate *loop* argument in asyncio.sleep (GH-9415) Message-ID: https://github.com/python/cpython/commit/558c49bcf3a8543d64a68de836b5d855efd56696 commit: 558c49bcf3a8543d64a68de836b5d855efd56696 branch: master author: Jo?o J?nior committer: Carol Willing date: 2018-09-24T05:51:22-04:00 summary: bpo-34728: Remove deprecate *loop* argument in asyncio.sleep (GH-9415) * Insert the warn in the asyncio.sleep when the loop argument is used * Insert the warn in the asyncio.wait and asyncio.wait_for when the loop argument is used * Better format of the code * Add news file * change calls for get_event_loop() to calls for get_running_loop() * Change message to be more clear in News * Improve the comments in test_tasks files: A Misc/NEWS.d/next/Library/2018-09-20-16-55-43.bpo-34728.CUE8LU.rst M Lib/asyncio/tasks.py M Lib/test/test_asyncio/test_tasks.py diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py index 7121aa65da0e..b52aad8c422d 100644 --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -382,7 +382,11 @@ def create_task(coro, *, name=None): raise ValueError(f'Invalid return_when value: {return_when}') if loop is None: - loop = events.get_event_loop() + loop = events.get_running_loop() + else: + warnings.warn("The loop argument is deprecated and scheduled for" + "removal in Python 4.0.", + DeprecationWarning, stacklevel=2) fs = {ensure_future(f, loop=loop) for f in set(fs)} @@ -408,7 +412,11 @@ def _release_waiter(waiter, *args): This function is a coroutine. """ if loop is None: - loop = events.get_event_loop() + loop = events.get_running_loop() + else: + warnings.warn("The loop argument is deprecated and scheduled for" + "removal in Python 4.0.", + DeprecationWarning, stacklevel=2) if timeout is None: return await fut @@ -585,7 +593,12 @@ def __sleep0(): return result if loop is None: - loop = events.get_event_loop() + loop = events.get_running_loop() + else: + warnings.warn("The loop argument is deprecated and scheduled for" + "removal in Python 4.0.", + DeprecationWarning, stacklevel=2) + future = loop.create_future() h = loop.call_later(delay, futures._set_result_unless_cancelled, diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index 40d1953532bc..ea54706edc8e 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -3220,6 +3220,35 @@ def coro(): self.loop.run_until_complete(coro()) self.assertEqual(result, 11) + def test_loop_argument_is_deprecated(self): + # Remove test when loop argument is removed in Python 4.0 + with self.assertWarns(DeprecationWarning): + self.loop.run_until_complete(asyncio.sleep(0.01, loop=self.loop)) + + +class WaitTests(test_utils.TestCase): + def setUp(self): + super().setUp() + self.loop = asyncio.new_event_loop() + asyncio.set_event_loop(None) + + def tearDown(self): + self.loop.close() + self.loop = None + super().tearDown() + + def test_loop_argument_is_deprecated_in_wait(self): + # Remove test when loop argument is removed in Python 4.0 + with self.assertWarns(DeprecationWarning): + self.loop.run_until_complete( + asyncio.wait([coroutine_function()], loop=self.loop)) + + def test_loop_argument_is_deprecated_in_wait_for(self): + # Remove test when loop argument is removed in Python 4.0 + with self.assertWarns(DeprecationWarning): + self.loop.run_until_complete( + asyncio.wait_for(coroutine_function(), 0.01, loop=self.loop)) + class CompatibilityTests(test_utils.TestCase): # Tests for checking a bridge between old-styled coroutines diff --git a/Misc/NEWS.d/next/Library/2018-09-20-16-55-43.bpo-34728.CUE8LU.rst b/Misc/NEWS.d/next/Library/2018-09-20-16-55-43.bpo-34728.CUE8LU.rst new file mode 100644 index 000000000000..e06eb0f45c9f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-20-16-55-43.bpo-34728.CUE8LU.rst @@ -0,0 +1,2 @@ +Add deprecation warning when `loop` is used in methods: `asyncio.sleep`, +`asyncio.wait` and `asyncio.wait_for`. From webhook-mailer at python.org Mon Sep 24 06:42:43 2018 From: webhook-mailer at python.org (Petr Viktorin) Date: Mon, 24 Sep 2018 10:42:43 -0000 Subject: [Python-checkins] bpo-24937: Replace the extension module porting HOWTO by links to external projects (GH-9317) Message-ID: https://github.com/python/cpython/commit/2d3ff2b5ea6c903973f99d2155c9c1b60591dceb commit: 2d3ff2b5ea6c903973f99d2155c9c1b60591dceb branch: master author: Petr Viktorin committer: GitHub date: 2018-09-24T12:42:33+02:00 summary: bpo-24937: Replace the extension module porting HOWTO by links to external projects (GH-9317) files: D Doc/includes/capsulethunk.h M Doc/howto/cporting.rst M Doc/tools/susp-ignored.csv diff --git a/Doc/howto/cporting.rst b/Doc/howto/cporting.rst index 7cacb0aecd8b..b638e32f5d0e 100644 --- a/Doc/howto/cporting.rst +++ b/Doc/howto/cporting.rst @@ -6,252 +6,21 @@ Porting Extension Modules to Python 3 ************************************* -:author: Benjamin Peterson - - -.. topic:: Abstract - - Although changing the C-API was not one of Python 3's objectives, - the many Python-level changes made leaving Python 2's API intact - impossible. In fact, some changes such as :func:`int` and - :func:`long` unification are more obvious on the C level. This - document endeavors to document incompatibilities and how they can - be worked around. - - -Conditional compilation -======================= - -The easiest way to compile only some code for Python 3 is to check -if :c:macro:`PY_MAJOR_VERSION` is greater than or equal to 3. :: - - #if PY_MAJOR_VERSION >= 3 - #define IS_PY3K - #endif - -API functions that are not present can be aliased to their equivalents within -conditional blocks. - - -Changes to Object APIs -====================== - -Python 3 merged together some types with similar functions while cleanly -separating others. - - -str/unicode Unification ------------------------ - -Python 3's :func:`str` type is equivalent to Python 2's :func:`unicode`; the C -functions are called ``PyUnicode_*`` for both. The old 8-bit string type has become -:func:`bytes`, with C functions called ``PyBytes_*``. Python 2.6 and later provide a compatibility header, -:file:`bytesobject.h`, mapping ``PyBytes`` names to ``PyString`` ones. For best -compatibility with Python 3, :c:type:`PyUnicode` should be used for textual data and -:c:type:`PyBytes` for binary data. It's also important to remember that -:c:type:`PyBytes` and :c:type:`PyUnicode` in Python 3 are not interchangeable like -:c:type:`PyString` and :c:type:`PyUnicode` are in Python 2. The following example -shows best practices with regards to :c:type:`PyUnicode`, :c:type:`PyString`, -and :c:type:`PyBytes`. :: - - #include "stdlib.h" - #include "Python.h" - #include "bytesobject.h" - - /* text example */ - static PyObject * - say_hello(PyObject *self, PyObject *args) { - PyObject *name, *result; - - if (!PyArg_ParseTuple(args, "U:say_hello", &name)) - return NULL; - - result = PyUnicode_FromFormat("Hello, %S!", name); - return result; - } - - /* just a forward */ - static char * do_encode(PyObject *); - - /* bytes example */ - static PyObject * - encode_object(PyObject *self, PyObject *args) { - char *encoded; - PyObject *result, *myobj; - - if (!PyArg_ParseTuple(args, "O:encode_object", &myobj)) - return NULL; - - encoded = do_encode(myobj); - if (encoded == NULL) - return NULL; - result = PyBytes_FromString(encoded); - free(encoded); - return result; - } - - -long/int Unification --------------------- - -Python 3 has only one integer type, :func:`int`. But it actually -corresponds to Python 2's :func:`long` type?the :func:`int` type -used in Python 2 was removed. In the C-API, ``PyInt_*`` functions -are replaced by their ``PyLong_*`` equivalents. - - -Module initialization and state -=============================== - -Python 3 has a revamped extension module initialization system. (See -:pep:`3121`.) Instead of storing module state in globals, they should -be stored in an interpreter specific structure. Creating modules that -act correctly in both Python 2 and Python 3 is tricky. The following -simple example demonstrates how. :: - - #include "Python.h" - - struct module_state { - PyObject *error; - }; - - #if PY_MAJOR_VERSION >= 3 - #define GETSTATE(m) ((struct module_state*)PyModule_GetState(m)) - #else - #define GETSTATE(m) (&_state) - static struct module_state _state; - #endif - - static PyObject * - error_out(PyObject *m) { - struct module_state *st = GETSTATE(m); - PyErr_SetString(st->error, "something bad happened"); - return NULL; - } - - static PyMethodDef myextension_methods[] = { - {"error_out", (PyCFunction)error_out, METH_NOARGS, NULL}, - {NULL, NULL} - }; - - #if PY_MAJOR_VERSION >= 3 - - static int myextension_traverse(PyObject *m, visitproc visit, void *arg) { - Py_VISIT(GETSTATE(m)->error); - return 0; - } - - static int myextension_clear(PyObject *m) { - Py_CLEAR(GETSTATE(m)->error); - return 0; - } - - - static struct PyModuleDef moduledef = { - PyModuleDef_HEAD_INIT, - "myextension", - NULL, - sizeof(struct module_state), - myextension_methods, - NULL, - myextension_traverse, - myextension_clear, - NULL - }; - - #define INITERROR return NULL - - PyMODINIT_FUNC - PyInit_myextension(void) - - #else - #define INITERROR return - - void - initmyextension(void) - #endif - { - #if PY_MAJOR_VERSION >= 3 - PyObject *module = PyModule_Create(&moduledef); - #else - PyObject *module = Py_InitModule("myextension", myextension_methods); - #endif - - if (module == NULL) - INITERROR; - struct module_state *st = GETSTATE(module); - - st->error = PyErr_NewException("myextension.Error", NULL, NULL); - if (st->error == NULL) { - Py_DECREF(module); - INITERROR; - } - - #if PY_MAJOR_VERSION >= 3 - return module; - #endif - } - - -CObject replaced with Capsule -============================= - -The :c:type:`Capsule` object was introduced in Python 3.1 and 2.7 to replace -:c:type:`CObject`. CObjects were useful, -but the :c:type:`CObject` API was problematic: it didn't permit distinguishing -between valid CObjects, which allowed mismatched CObjects to crash the -interpreter, and some of its APIs relied on undefined behavior in C. -(For further reading on the rationale behind Capsules, please see :issue:`5630`.) - -If you're currently using CObjects, and you want to migrate to 3.1 or newer, -you'll need to switch to Capsules. -:c:type:`CObject` was deprecated in 3.1 and 2.7 and completely removed in -Python 3.2. If you only support 2.7, or 3.1 and above, you -can simply switch to :c:type:`Capsule`. If you need to support Python 3.0, -or versions of Python earlier than 2.7, -you'll have to support both CObjects and Capsules. -(Note that Python 3.0 is no longer supported, and it is not recommended -for production use.) - -The following example header file :file:`capsulethunk.h` may -solve the problem for you. Simply write your code against the -:c:type:`Capsule` API and include this header file after -:file:`Python.h`. Your code will automatically use Capsules -in versions of Python with Capsules, and switch to CObjects -when Capsules are unavailable. - -:file:`capsulethunk.h` simulates Capsules using CObjects. However, -:c:type:`CObject` provides no place to store the capsule's "name". As a -result the simulated :c:type:`Capsule` objects created by :file:`capsulethunk.h` -behave slightly differently from real Capsules. Specifically: - - * The name parameter passed in to :c:func:`PyCapsule_New` is ignored. - - * The name parameter passed in to :c:func:`PyCapsule_IsValid` and - :c:func:`PyCapsule_GetPointer` is ignored, and no error checking - of the name is performed. - - * :c:func:`PyCapsule_GetName` always returns NULL. - - * :c:func:`PyCapsule_SetName` always raises an exception and - returns failure. (Since there's no way to store a name - in a CObject, noisy failure of :c:func:`PyCapsule_SetName` - was deemed preferable to silent failure here. If this is - inconvenient, feel free to modify your local - copy as you see fit.) - -You can find :file:`capsulethunk.h` in the Python source distribution -as :source:`Doc/includes/capsulethunk.h`. We also include it here for -your convenience: - -.. literalinclude:: ../includes/capsulethunk.h - - - -Other options -============= - -If you are writing a new extension module, you might consider `Cython -`_. It translates a Python-like language to C. The -extension modules it creates are compatible with Python 3 and Python 2. - +We recommend the following resources for porting extension modules to Python 3: + +* The `Migrating C extensions`_ chapter from + *Supporting Python 3: An in-depth guide*, a book on moving from Python 2 + to Python 3 in general, guides the reader through porting an extension + module. +* The `Porting guide`_ from the *py3c* project provides opinionated + suggestions with supporting code. +* The `Cython`_ and `CFFI`_ libraries offer abstractions over + Python's C API. + Extensions generally need to be re-written to use one of them, + but the library then handles differences between various Python + versions and implementations. + +.. _Migrating C extensions: http://python3porting.com/cextensions.html +.. _Porting guide: https://py3c.readthedocs.io/en/latest/guide.html +.. _Cython: http://cython.org/ +.. _CFFI: https://cffi.readthedocs.io/en/latest/ diff --git a/Doc/includes/capsulethunk.h b/Doc/includes/capsulethunk.h deleted file mode 100644 index 6b20564f1397..000000000000 --- a/Doc/includes/capsulethunk.h +++ /dev/null @@ -1,134 +0,0 @@ -#ifndef __CAPSULETHUNK_H -#define __CAPSULETHUNK_H - -#if ( (PY_VERSION_HEX < 0x02070000) \ - || ((PY_VERSION_HEX >= 0x03000000) \ - && (PY_VERSION_HEX < 0x03010000)) ) - -#define __PyCapsule_GetField(capsule, field, default_value) \ - ( PyCapsule_CheckExact(capsule) \ - ? (((PyCObject *)capsule)->field) \ - : (default_value) \ - ) \ - -#define __PyCapsule_SetField(capsule, field, value) \ - ( PyCapsule_CheckExact(capsule) \ - ? (((PyCObject *)capsule)->field = value), 1 \ - : 0 \ - ) \ - - -#define PyCapsule_Type PyCObject_Type - -#define PyCapsule_CheckExact(capsule) (PyCObject_Check(capsule)) -#define PyCapsule_IsValid(capsule, name) (PyCObject_Check(capsule)) - - -#define PyCapsule_New(pointer, name, destructor) \ - (PyCObject_FromVoidPtr(pointer, destructor)) - - -#define PyCapsule_GetPointer(capsule, name) \ - (PyCObject_AsVoidPtr(capsule)) - -/* Don't call PyCObject_SetPointer here, it fails if there's a destructor */ -#define PyCapsule_SetPointer(capsule, pointer) \ - __PyCapsule_SetField(capsule, cobject, pointer) - - -#define PyCapsule_GetDestructor(capsule) \ - __PyCapsule_GetField(capsule, destructor) - -#define PyCapsule_SetDestructor(capsule, dtor) \ - __PyCapsule_SetField(capsule, destructor, dtor) - - -/* - * Sorry, there's simply no place - * to store a Capsule "name" in a CObject. - */ -#define PyCapsule_GetName(capsule) NULL - -static int -PyCapsule_SetName(PyObject *capsule, const char *unused) -{ - unused = unused; - PyErr_SetString(PyExc_NotImplementedError, - "can't use PyCapsule_SetName with CObjects"); - return 1; -} - - - -#define PyCapsule_GetContext(capsule) \ - __PyCapsule_GetField(capsule, descr) - -#define PyCapsule_SetContext(capsule, context) \ - __PyCapsule_SetField(capsule, descr, context) - - -static void * -PyCapsule_Import(const char *name, int no_block) -{ - PyObject *object = NULL; - void *return_value = NULL; - char *trace; - size_t name_length = (strlen(name) + 1) * sizeof(char); - char *name_dup = (char *)PyMem_MALLOC(name_length); - - if (!name_dup) { - return NULL; - } - - memcpy(name_dup, name, name_length); - - trace = name_dup; - while (trace) { - char *dot = strchr(trace, '.'); - if (dot) { - *dot++ = '\0'; - } - - if (object == NULL) { - if (no_block) { - object = PyImport_ImportModuleNoBlock(trace); - } else { - object = PyImport_ImportModule(trace); - if (!object) { - PyErr_Format(PyExc_ImportError, - "PyCapsule_Import could not " - "import module \"%s\"", trace); - } - } - } else { - PyObject *object2 = PyObject_GetAttrString(object, trace); - Py_DECREF(object); - object = object2; - } - if (!object) { - goto EXIT; - } - - trace = dot; - } - - if (PyCObject_Check(object)) { - PyCObject *cobject = (PyCObject *)object; - return_value = cobject->cobject; - } else { - PyErr_Format(PyExc_AttributeError, - "PyCapsule_Import \"%s\" is not valid", - name); - } - -EXIT: - Py_XDECREF(object); - if (name_dup) { - PyMem_FREE(name_dup); - } - return return_value; -} - -#endif /* #if PY_VERSION_HEX < 0x02070000 */ - -#endif /* __CAPSULETHUNK_H */ diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv index 33cd48f3d81e..bb3310bac75f 100644 --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -15,8 +15,6 @@ faq/programming,,::,for x in sequence[::-1]: faq/programming,,:reduce,"print((lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y," faq/programming,,:reduce,"Sx=Sx,Sy=Sy:reduce(lambda x,y:x+y,map(lambda x,xc=Ru,yc=yc,Ru=Ru,Ro=Ro," faq/windows,,:bd8afb90ebf2,"Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32 bit (Intel)] on win32" -howto/cporting,,:encode,"if (!PyArg_ParseTuple(args, ""O:encode_object"", &myobj))" -howto/cporting,,:say,"if (!PyArg_ParseTuple(args, ""U:say_hello"", &name))" howto/curses,,:black,"colors when it activates color mode. They are: 0:black, 1:red," howto/curses,,:red,"colors when it activates color mode. They are: 0:black, 1:red," howto/curses,,:green,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" From webhook-mailer at python.org Mon Sep 24 07:44:55 2018 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 24 Sep 2018 11:44:55 -0000 Subject: [Python-checkins] bpo-34582: Update syntax of Azure Pipelines builds (GH-9521) Message-ID: https://github.com/python/cpython/commit/57675090b0fe7d6c7d72e56384dc2ff6798f1723 commit: 57675090b0fe7d6c7d72e56384dc2ff6798f1723 branch: master author: Steve Dower committer: GitHub date: 2018-09-24T07:44:50-04:00 summary: bpo-34582: Update syntax of Azure Pipelines builds (GH-9521) files: A .azure-pipelines/ci.yml A .azure-pipelines/docker-steps.yml A .azure-pipelines/docs-steps.yml A .azure-pipelines/macos-steps.yml A .azure-pipelines/posix-deps.sh A .azure-pipelines/posix-steps.yml A .azure-pipelines/pr.yml A .azure-pipelines/prebuild-checks.yml A .azure-pipelines/windows-steps.yml D .vsts/docs-release.yml D .vsts/docs.yml D .vsts/install_deps.sh D .vsts/linux-buildbot.yml D .vsts/linux-coverage.yml D .vsts/linux-pr.yml D .vsts/macos-buildbot.yml D .vsts/macos-pr.yml D .vsts/windows-buildbot.yml D .vsts/windows-pr.yml M Lib/test/test_zlib.py diff --git a/.azure-pipelines/ci.yml b/.azure-pipelines/ci.yml new file mode 100644 index 000000000000..f56254e38024 --- /dev/null +++ b/.azure-pipelines/ci.yml @@ -0,0 +1,136 @@ +variables: + manylinux: false + coverage: false + +jobs: +- job: Prebuild + displayName: Pre-build checks + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./prebuild-checks.yml + + +- job: Docs_PR + displayName: Docs PR + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['docs.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./docs-steps.yml + parameters: + upload: true + + +- job: macOS_CI_Tests + displayName: macOS CI Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + variables: + testRunTitle: '$(build.sourceBranchName)-macos' + testRunPlatform: macos + + pool: + vmImage: xcode9-macos10.13 + + steps: + - template: ./macos-steps.yml + + +- job: Ubuntu_CI_Tests + displayName: Ubuntu CI Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(build.sourceBranchName)-linux' + testRunPlatform: linux + openssl_version: 1.1.0g + + steps: + - template: ./posix-steps.yml + + +- job: ManyLinux1_CI_Tests + displayName: ManyLinux1 CI Tests + dependsOn: Prebuild + condition: | + and( + and( + succeeded(), + eq(variables['manylinux'], 'true') + ), + eq(dependencies.Prebuild.outputs['tests.run'], 'true') + ) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(build.sourceBranchName)-manylinux1' + testRunPlatform: manylinux1 + imageName: 'dockcross/manylinux-x64' + + steps: + - template: ./docker-steps.yml + + +- job: Ubuntu_Coverage_CI_Tests + displayName: Ubuntu CI Tests (coverage) + dependsOn: Prebuild + condition: | + and( + and( + succeeded(), + eq(variables['coverage'], 'true') + ), + eq(dependencies.Prebuild.outputs['tests.run'], 'true') + ) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(Build.SourceBranchName)-linux-coverage' + testRunPlatform: linux-coverage + openssl_version: 1.1.0g + + steps: + - template: ./posix-steps.yml + parameters: + coverage: true + + +- job: Windows_CI_Tests + displayName: Windows CI Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: vs2017-win2017 + + strategy: + matrix: + win32: + arch: win32 + buildOpt: + testRunTitle: '$(Build.SourceBranchName)-win32' + testRunPlatform: win32 + win64: + arch: amd64 + buildOpt: '-p x64' + testRunTitle: '$(Build.SourceBranchName)-win64' + testRunPlatform: win64 + maxParallel: 2 + + steps: + - template: ./windows-steps.yml diff --git a/.azure-pipelines/docker-steps.yml b/.azure-pipelines/docker-steps.yml new file mode 100644 index 000000000000..ba4dfd72dd8b --- /dev/null +++ b/.azure-pipelines/docker-steps.yml @@ -0,0 +1,76 @@ +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- ${{ if ne(parameters.targetBranch, '') }}: + - script: | + git fetch -q origin ${{ parameters.targetbranch }} + if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qvE '(\.rst$|^Doc|^Misc)' + then + echo "Only docs were updated, stopping build process." + echo "##vso[task.setvariable variable=DocOnly]true" + exit + fi + displayName: Detect doc-only changes + +- task: docker at 0 + displayName: 'Configure CPython (debug)' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: './configure --with-pydebug' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: docker at 0 + displayName: 'Build CPython' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: 'make -s -j4' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: docker at 0 + displayName: 'Display build info' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: 'make pythoninfo' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: docker at 0 + displayName: 'Tests' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: 'make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=/build/test-results.xml"' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.azure-pipelines/docs-steps.yml b/.azure-pipelines/docs-steps.yml new file mode 100644 index 000000000000..c0404aebdcc5 --- /dev/null +++ b/.azure-pipelines/docs-steps.yml @@ -0,0 +1,46 @@ +parameters: + latex: false + upload: false + +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- task: UsePythonVersion at 0 + displayName: 'Use Python 3.6 or later' + inputs: + versionSpec: '>=3.6' + +- script: python -m pip install sphinx~=1.6.1 blurb python-docs-theme + displayName: 'Install build dependencies' + +- ${{ if ne(parameters.latex, 'true') }}: + - script: make check suspicious html PYTHON=python + workingDirectory: '$(build.sourcesDirectory)/Doc' + displayName: 'Build documentation' + +- ${{ if eq(parameters.latex, 'true') }}: + - script: sudo apt-get update && sudo apt-get install -qy --force-yes texlive-full + displayName: 'Install LaTeX' + + - script: make dist PYTHON=python SPHINXBUILD='python -m sphinx' BLURB='python -m blurb' + workingDirectory: '$(build.sourcesDirectory)/Doc' + displayName: 'Build documentation' + +- ${{ if eq(parameters.upload, 'true') }}: + - task: PublishBuildArtifacts at 1 + displayName: 'Publish docs' + + inputs: + PathToPublish: '$(build.sourcesDirectory)/Doc/build' + ArtifactName: docs + publishLocation: Container + + - ${{ if eq(parameters.latex, 'true') }}: + - task: PublishBuildArtifacts at 1 + displayName: 'Publish dist' + inputs: + PathToPublish: '$(build.sourcesDirectory)/Doc/dist' + ArtifactName: docs_dist + publishLocation: Container diff --git a/.vsts/macos-buildbot.yml b/.azure-pipelines/macos-steps.yml similarity index 57% rename from .vsts/macos-buildbot.yml rename to .azure-pipelines/macos-steps.yml index d9c190c46131..647081689454 100644 --- a/.vsts/macos-buildbot.yml +++ b/.azure-pipelines/macos-steps.yml @@ -1,30 +1,9 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted macOS - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -#variables: - steps: - checkout: self clean: true fetchDepth: 5 -- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl --prefix=/opt/python-vsts +- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl --prefix=/opt/python-azdev displayName: 'Configure CPython (debug)' - script: make -s -j4 @@ -41,6 +20,6 @@ steps: inputs: testResultsFiles: '$(build.binariesDirectory)/test-results.xml' mergeTestResults: true - testRunTitle: '$(build.sourceBranchName)-macOS' - platform: macOS + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) condition: succeededOrFailed() diff --git a/.vsts/install_deps.sh b/.azure-pipelines/posix-deps.sh similarity index 51% rename from .vsts/install_deps.sh rename to .azure-pipelines/posix-deps.sh index b1fa576047e6..a57210756601 100755 --- a/.vsts/install_deps.sh +++ b/.azure-pipelines/posix-deps.sh @@ -1,4 +1,4 @@ -sudo apt-get update || true +sudo apt-get update sudo apt-get -yq install \ build-essential \ @@ -17,3 +17,10 @@ sudo apt-get -yq install \ libffi-dev \ uuid-dev \ xvfb + +if [ ! -z "$1" ] +then + echo ##vso[task.prependpath]$PWD/multissl/openssl/$1 + echo ##vso[task.setvariable variable=OPENSSL_DIR]$PWD/multissl/openssl/$1 + python3 Tools/ssl/multissltests.py --steps=library --base-directory $PWD/multissl --openssl $1 --system Linux +fi diff --git a/.azure-pipelines/posix-steps.yml b/.azure-pipelines/posix-steps.yml new file mode 100644 index 000000000000..429381a7e242 --- /dev/null +++ b/.azure-pipelines/posix-steps.yml @@ -0,0 +1,63 @@ +parameters: + coverage: false + +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- script: ./.azure-pipelines/posix-deps.sh $(openssl_version) + displayName: 'Install dependencies' + +- script: ./configure --with-pydebug + displayName: 'Configure CPython (debug)' + +- script: make -s -j4 + displayName: 'Build CPython' + +- ${{ if eq(parameters.coverage, 'true') }}: + - script: ./python -m venv venv && ./venv/bin/python -m pip install -U coverage + displayName: 'Set up virtual environment' + + - script: ./venv/bin/python -m test.pythoninfo + displayName: 'Display build info' + + - script: | + xvfb-run ./venv/bin/python -m coverage run --pylib -m test \ + --fail-env-changed \ + -uall,-cpu \ + --junit-xml=$(build.binariesDirectory)/test-results.xml" \ + -x test_multiprocessing_fork \ + -x test_multiprocessing_forkserver \ + -x test_multiprocessing_spawn \ + -x test_concurrent_futures + displayName: 'Tests with coverage' + + - script: ./venv/bin/python -m coverage xml + displayName: 'Generate coverage.xml' + + - script: source ./venv/bin/activate && bash <(curl -s https://codecov.io/bash) + displayName: 'Publish code coverage results' + + +- ${{ if ne(parameters.coverage, 'true') }}: + - script: make pythoninfo + displayName: 'Display build info' + + - script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" + displayName: 'Tests' + + +- script: python Tools/scripts/patchcheck.py --travis true + displayName: 'Run patchcheck.py' + condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) + + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) + condition: succeededOrFailed() diff --git a/.azure-pipelines/pr.yml b/.azure-pipelines/pr.yml new file mode 100644 index 000000000000..653f55b69b13 --- /dev/null +++ b/.azure-pipelines/pr.yml @@ -0,0 +1,86 @@ +jobs: +- job: Prebuild + displayName: Pre-build checks + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./prebuild-checks.yml + + +- job: Docs_PR + displayName: Docs PR + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['docs.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./docs-steps.yml + + +- job: macOS_PR_Tests + displayName: macOS PR Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + variables: + testRunTitle: '$(system.pullRequest.TargetBranch)-macos' + testRunPlatform: macos + + pool: + vmImage: xcode9-macos10.13 + + steps: + - template: ./macos-steps.yml + parameters: + targetBranch: $(System.PullRequest.TargetBranch) + + +- job: Ubuntu_PR_Tests + displayName: Ubuntu PR Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(system.pullRequest.TargetBranch)-linux' + testRunPlatform: linux + openssl_version: 1.1.0g + + steps: + - template: ./posix-steps.yml + parameters: + targetBranch: $(System.PullRequest.TargetBranch) + + +- job: Windows_PR_Tests + displayName: Windows PR Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: vs2017-win2017 + + strategy: + matrix: + win32: + arch: win32 + buildOpt: + testRunTitle: '$(System.PullRequest.TargetBranch)-win32' + testRunPlatform: win32 + win64: + arch: amd64 + buildOpt: '-p x64' + testRunTitle: '$(System.PullRequest.TargetBranch)-win64' + testRunPlatform: win64 + maxParallel: 2 + + steps: + - template: ./windows-steps.yml + parameters: + targetBranch: $(System.PullRequest.TargetBranch) diff --git a/.azure-pipelines/prebuild-checks.yml b/.azure-pipelines/prebuild-checks.yml new file mode 100644 index 000000000000..30ff642d1267 --- /dev/null +++ b/.azure-pipelines/prebuild-checks.yml @@ -0,0 +1,36 @@ +steps: +- checkout: self + fetchDepth: 5 + +- script: echo "##vso[task.setvariable variable=diffTarget]HEAD~1" + displayName: Set default diff target + +- script: | + git fetch -q origin $(System.PullRequest.TargetBranch) + echo "##vso[task.setvariable variable=diffTarget]HEAD \$(git merge-base HEAD FETCH_HEAD)" + displayName: Fetch comparison tree + condition: and(succeeded(), variables['System.PullRequest.TargetBranch']) + +- script: | + if ! git diff --name-only $(diffTarget) | grep -qE '(\.rst$|^Doc|^Misc)' + then + echo "No docs were updated: docs.run=false" + echo "##vso[task.setvariable variable=run;isOutput=true]false" + else + echo "Docs were updated: docs.run=true" + echo "##vso[task.setvariable variable=run;isOutput=true]true" + fi + displayName: Detect documentation changes + name: docs + +- script: | + if ! git diff --name-only $(diffTarget) | grep -qvE '(\.rst$|^Doc|^Misc)' + then + echo "Only docs were updated: tests.run=false" + echo "##vso[task.setvariable variable=run;isOutput=true]false" + else + echo "Code was updated: tests.run=true" + echo "##vso[task.setvariable variable=run;isOutput=true]true" + fi + displayName: Detect source changes + name: tests diff --git a/.azure-pipelines/windows-steps.yml b/.azure-pipelines/windows-steps.yml new file mode 100644 index 000000000000..d8d5f1753a07 --- /dev/null +++ b/.azure-pipelines/windows-steps.yml @@ -0,0 +1,32 @@ +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- powershell: | + # Relocate build outputs outside of source directory to make cleaning faster + Write-Host '##vso[task.setvariable variable=Py_IntDir]$(Build.BinariesDirectory)\obj' + # UNDONE: Do not build to a different directory because of broken tests + Write-Host '##vso[task.setvariable variable=Py_OutDir]$(Build.SourcesDirectory)\PCbuild' + Write-Host '##vso[task.setvariable variable=EXTERNAL_DIR]$(Build.BinariesDirectory)\externals' + displayName: Update build locations + +- script: PCbuild\build.bat -e $(buildOpt) + displayName: 'Build CPython' + +- script: python.bat -m test.pythoninfo + displayName: 'Display build info' + +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" + displayName: 'Tests' + env: + PREFIX: $(Py_OutDir)\$(arch) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' + mergeTestResults: true + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) + condition: succeededOrFailed() diff --git a/.vsts/docs-release.yml b/.vsts/docs-release.yml deleted file mode 100644 index e90428a42494..000000000000 --- a/.vsts/docs-release.yml +++ /dev/null @@ -1,43 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted Linux Preview - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: sudo apt-get update && sudo apt-get install -qy --force-yes texlive-full - displayName: 'Install LaTeX' - -- task: UsePythonVersion at 0 - displayName: 'Use Python 3.6 or later' - inputs: - versionSpec: '>=3.6' - -- script: python -m pip install sphinx blurb python-docs-theme - displayName: 'Install build dependencies' - -- script: make dist PYTHON=python SPHINXBUILD='python -m sphinx' BLURB='python -m blurb' - workingDirectory: '$(build.sourcesDirectory)/Doc' - displayName: 'Build documentation' - -- task: PublishBuildArtifacts at 1 - displayName: 'Publish build' - inputs: - PathToPublish: '$(build.sourcesDirectory)/Doc/build' - ArtifactName: build - publishLocation: Container - -- task: PublishBuildArtifacts at 1 - displayName: 'Publish dist' - inputs: - PathToPublish: '$(build.sourcesDirectory)/Doc/dist' - ArtifactName: dist - publishLocation: Container diff --git a/.vsts/docs.yml b/.vsts/docs.yml deleted file mode 100644 index 0be07b31dfcc..000000000000 --- a/.vsts/docs.yml +++ /dev/null @@ -1,58 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - include: - - Doc/* - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qE '(\.rst$|^Doc|^Misc)' - then - echo "No docs were updated, stopping build process." - echo "##vso[task.setvariable variable=NoDocs]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -- task: UsePythonVersion at 0 - displayName: 'Use Python 3.6 or later' - inputs: - versionSpec: '>=3.6' - condition: and(succeeded(), ne(variables['NoDocs'], 'true')) - -- script: python -m pip install sphinx~=1.6.1 blurb python-docs-theme - displayName: 'Install build dependencies' - condition: and(succeeded(), ne(variables['NoDocs'], 'true')) - -- script: make check suspicious html PYTHON=python - workingDirectory: '$(build.sourcesDirectory)/Doc' - displayName: 'Build documentation' - condition: and(succeeded(), ne(variables['NoDocs'], 'true')) - -- task: PublishBuildArtifacts at 1 - displayName: 'Publish build' - condition: and(and(succeeded(), ne(variables['Build.Reason'], 'PullRequest')), ne(variables['NoDocs'], 'true')) - inputs: - PathToPublish: '$(build.sourcesDirectory)/Doc/build' - ArtifactName: build - publishLocation: Container diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml deleted file mode 100644 index 417125ae3a46..000000000000 --- a/.vsts/linux-buildbot.yml +++ /dev/null @@ -1,59 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Copy-pasted from linux-deps.yml until template support arrives - OPENSSL: 1.1.0g - OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" - - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -#- template: linux-deps.yml - -- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) - displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: ./.vsts/install_deps.sh - displayName: 'Install dependencies' -- script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux - displayName: 'python multissltests.py' - -- script: ./configure --with-pydebug - displayName: 'Configure CPython (debug)' - -- script: make -s -j4 - displayName: 'Build CPython' - -- script: make pythoninfo - displayName: 'Display build info' - -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" - displayName: 'Tests' - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(build.binariesDirectory)/test-results.xml' - mergeTestResults: true - testRunTitle: '$(build.sourceBranchName)-linux' - platform: linux - condition: succeededOrFailed() diff --git a/.vsts/linux-coverage.yml b/.vsts/linux-coverage.yml deleted file mode 100644 index cc03e4258ab4..000000000000 --- a/.vsts/linux-coverage.yml +++ /dev/null @@ -1,75 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Copy-pasted from linux-deps.yml until template support arrives - OPENSSL: 1.1.0g - OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qvE '(\.rst$|^Doc|^Misc)' - then - echo "Only docs were updated, stopping build process." - echo "##vso[task.setvariable variable=DocOnly]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -#- template: linux-deps.yml - -- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) - displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: ./.vsts/install_deps.sh - displayName: 'Install dependencies' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux - displayName: 'python multissltests.py' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - - -- script: ./configure --with-pydebug - displayName: 'Configure CPython (debug)' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make -s -j4 - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./python -m venv venv && ./venv/bin/python -m pip install -U coverage - displayName: 'Set up virtual environment' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./venv/bin/python -m test.pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: xvfb-run ./venv/bin/python -m coverage run --pylib -m test --fail-env-changed -uall,-cpu -x test_multiprocessing_fork -x test_multiprocessing_forkserver -x test_multiprocessing_spawn -x test_concurrent_futures - displayName: 'Tests with coverage' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: source ./venv/bin/activate && bash <(curl -s https://codecov.io/bash) - displayName: 'Publish code coverage results' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml deleted file mode 100644 index d11a4f06e4e1..000000000000 --- a/.vsts/linux-pr.yml +++ /dev/null @@ -1,84 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Copy-pasted from linux-deps.yml until template support arrives - OPENSSL: 1.1.0g - OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qvE '(\.rst$|^Doc|^Misc)' - then - echo "Only docs were updated, stopping build process." - echo "##vso[task.setvariable variable=DocOnly]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -#- template: linux-deps.yml - -- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) - displayName: 'Add $(OPENSSL_DIR) to PATH' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./.vsts/install_deps.sh - displayName: 'Install dependencies' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux - displayName: 'python multissltests.py' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - - -- script: ./configure --with-pydebug - displayName: 'Configure CPython (debug)' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make -s -j4 - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -# Run patchcheck and fail if anything is discovered -- script: ./python Tools/scripts/patchcheck.py --travis true - displayName: 'Run patchcheck.py' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" - displayName: 'Tests' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(build.binariesDirectory)/test-results.xml' - mergeTestResults: true - testRunTitle: '$(system.pullRequest.targetBranch)-linux' - platform: linux - condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml deleted file mode 100644 index 69b619e47577..000000000000 --- a/.vsts/macos-pr.yml +++ /dev/null @@ -1,64 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted macOS - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - changes = $(git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD)) - echo "Files changed:" - echo "$changes" - if ! echo "$changes" | grep -qvE '(\.rst$)|(^Doc)|(^Misc)' - then - echo "Only docs were updated, stopping build process." - echo "##vso[task.setvariable variable=DocOnly]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl --prefix=/opt/python-vsts - displayName: 'Configure CPython (debug)' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make -s -j4 - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" - displayName: 'Tests' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(build.binariesDirectory)/test-results.xml' - mergeTestResults: true - testRunTitle: '$(system.pullRequest.targetBranch)-macOS' - platform: macOS - condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/windows-buildbot.yml b/.vsts/windows-buildbot.yml deleted file mode 100644 index 15aebeda4077..000000000000 --- a/.vsts/windows-buildbot.yml +++ /dev/null @@ -1,58 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted VS2017 - parallel: 2 - matrix: - amd64: - buildOpt: -p x64 - outDirSuffix: amd64 - win32: - buildOpt: - outDirSuffix: win32 - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Relocate build outputs outside of source directory to make cleaning faster - Py_IntDir: $(Build.BinariesDirectory)\obj - # UNDONE: Do not build to a different directory because of broken tests - Py_OutDir: $(Build.SourcesDirectory)\PCbuild - EXTERNAL_DIR: $(Build.BinariesDirectory)\externals - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: PCbuild\build.bat -e $(buildOpt) - displayName: 'Build CPython' - -- script: python.bat -m test.pythoninfo - displayName: 'Display build info' - -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" - displayName: 'Tests' - env: - PREFIX: $(Py_OutDir)\$(outDirSuffix) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' - mergeTestResults: true - testRunTitle: '$(Build.SourceBranchName)-$(outDirSuffix)' - platform: $(outDirSuffix) - condition: succeededOrFailed() diff --git a/.vsts/windows-pr.yml b/.vsts/windows-pr.yml deleted file mode 100644 index 7134120d6414..000000000000 --- a/.vsts/windows-pr.yml +++ /dev/null @@ -1,70 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted VS2017 - parallel: 2 - matrix: - amd64: - buildOpt: -p x64 - outDirSuffix: amd64 - win32: - buildOpt: - outDirSuffix: win32 - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Relocate build outputs outside of source directory to make cleaning faster - Py_IntDir: $(Build.BinariesDirectory)\obj - # UNDONE: Do not build to a different directory because of broken tests - Py_OutDir: $(Build.SourcesDirectory)\PCbuild - EXTERNAL_DIR: $(Build.BinariesDirectory)\externals - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- powershell: | - git fetch -q origin $(System.PullRequest.TargetBranch) - if (-not (git diff --name-only HEAD (git merge-base HEAD FETCH_HEAD) | sls -NotMatch '(\.rst$)|(^Doc)|(^Misc)')) { - Write-Host 'Only docs were updated. Skipping build' - Write-Host '##vso[task.setvariable variable=DocOnly]true' - } - displayName: Detect doc-only changes - condition: and(succeeded(), variables['System.PullRequest.TargetBranch']) - -- script: PCbuild\build.bat -e $(buildOpt) - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: python.bat -m test.pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" - displayName: 'Tests' - env: - PREFIX: $(Py_OutDir)\$(outDirSuffix) - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' - mergeTestResults: true - testRunTitle: '$(System.PullRequest.TargetBranch)-$(outDirSuffix)' - platform: $(outDirSuffix) - condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py index 4871d60d756c..bf5d64ceb298 100644 --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -436,18 +436,29 @@ def test_flushes(self): # Test flush() with the various options, using all the # different levels in order to provide more variations. sync_opt = ['Z_NO_FLUSH', 'Z_SYNC_FLUSH', 'Z_FULL_FLUSH', - 'Z_PARTIAL_FLUSH', 'Z_BLOCK'] + 'Z_PARTIAL_FLUSH'] + + ver = tuple(int(v) for v in zlib.ZLIB_RUNTIME_VERSION.split('.')) + # Z_BLOCK has a known failure prior to 1.2.5.3 + if ver >= (1, 2, 5, 3): + sync_opt.append('Z_BLOCK') + sync_opt = [getattr(zlib, opt) for opt in sync_opt if hasattr(zlib, opt)] data = HAMLET_SCENE * 8 for sync in sync_opt: for level in range(10): - obj = zlib.compressobj( level ) - a = obj.compress( data[:3000] ) - b = obj.flush( sync ) - c = obj.compress( data[3000:] ) - d = obj.flush() + try: + obj = zlib.compressobj( level ) + a = obj.compress( data[:3000] ) + b = obj.flush( sync ) + c = obj.compress( data[3000:] ) + d = obj.flush() + except: + print("Error for flush mode={}, level={}" + .format(sync, level)) + raise self.assertEqual(zlib.decompress(b''.join([a,b,c,d])), data, ("Decompress failed: flush " "mode=%i, level=%i") % (sync, level)) From webhook-mailer at python.org Mon Sep 24 08:03:04 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 24 Sep 2018 12:03:04 -0000 Subject: [Python-checkins] bpo-34783: Fix Py_Main() (GH-9526) Message-ID: https://github.com/python/cpython/commit/ddc163df25191af5bf8000091dbf8f6500b1d7c9 commit: ddc163df25191af5bf8000091dbf8f6500b1d7c9 branch: 3.7 author: Victor Stinner committer: GitHub date: 2018-09-24T05:03:01-07:00 summary: bpo-34783: Fix Py_Main() (GH-9526) Fix a crash with musl libc (on Alpine Linux) when the script filename specified on the command line doesn't exist. pymain_open_filename() now gets the current core configuration from the interpreter state. Modify the code to make it closer to the master branch: * Rename _Py_CommandLineDetails to _PyCmdline * Remove _PyMain.config: replaced with a local variable 'local_config' in pymain_init() * Reorganize pymain_main(): move code using the "local config" into pymain_init() * As soon as possible, switch from the local config to the core configuration attached to the interpreter. files: A Misc/NEWS.d/next/Core and Builtins/2018-09-24-11-31-23.bpo-34783.O79cwo.rst M Modules/main.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-24-11-31-23.bpo-34783.O79cwo.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-24-11-31-23.bpo-34783.O79cwo.rst new file mode 100644 index 000000000000..e1b6c1ced238 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-24-11-31-23.bpo-34783.O79cwo.rst @@ -0,0 +1,2 @@ +Fix a crash with musl libc (on Alpine Linux) when the script filename +specified on the command line doesn't exist. diff --git a/Modules/main.c b/Modules/main.c index 0f7498d6104f..7771d2750097 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -459,7 +459,7 @@ typedef struct { int legacy_windows_stdio; /* Py_LegacyWindowsStdioFlag, PYTHONLEGACYWINDOWSSTDIO */ #endif -} _Py_CommandLineDetails; +} _PyCmdline; /* Structure used by Py_Main() to pass data to subfunctions */ typedef struct { @@ -481,14 +481,10 @@ typedef struct { wchar_t *command; /* -c argument */ wchar_t *module; /* -m argument */ - _PyCoreConfig config; - PyObject *main_importer_path; } _PyMain; -#define _PyMain_INIT \ - {.config = _PyCoreConfig_INIT, \ - .err = _Py_INIT_OK()} +#define _PyMain_INIT {.err = _Py_INIT_OK()} /* Note: _PyMain_INIT sets other fields to 0/NULL */ @@ -522,7 +518,8 @@ clear_wstrlist(int len, wchar_t **list) static int -pymain_init_cmdline_argv(_PyMain *pymain, _Py_CommandLineDetails *cmdline) +pymain_init_cmdline_argv(_PyMain *pymain, _PyCoreConfig *config, + _PyCmdline *cmdline) { assert(cmdline->argv == NULL); @@ -561,8 +558,8 @@ pymain_init_cmdline_argv(_PyMain *pymain, _Py_CommandLineDetails *cmdline) else { program = L""; } - pymain->config.program = pymain_wstrdup(pymain, program); - if (pymain->config.program == NULL) { + config->program = pymain_wstrdup(pymain, program); + if (config->program == NULL) { return -1; } @@ -571,7 +568,7 @@ pymain_init_cmdline_argv(_PyMain *pymain, _Py_CommandLineDetails *cmdline) static void -pymain_clear_cmdline(_PyMain *pymain, _Py_CommandLineDetails *cmdline) +pymain_clear_cmdline(_PyMain *pymain, _PyCmdline *cmdline) { PyMemAllocatorEx old_alloc; _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc); @@ -609,14 +606,14 @@ pymain_clear_pymain(_PyMain *pymain) } static void -pymain_clear_config(_PyMain *pymain) +pymain_clear_config(_PyCoreConfig *config) { /* Clear core config with the memory allocator used by pymain_read_conf() */ PyMemAllocatorEx old_alloc; _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc); - _PyCoreConfig_Clear(&pymain->config); + _PyCoreConfig_Clear(config); PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &old_alloc); } @@ -653,8 +650,6 @@ pymain_free_raw(_PyMain *pymain) Py_Initialize()-Py_Finalize() can be called multiple times. */ _PyPathConfig_Clear(&_Py_path_config); - pymain_clear_config(pymain); - /* Force the allocator used by pymain_read_conf() */ PyMemAllocatorEx old_alloc; _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc); @@ -740,10 +735,9 @@ pymain_wstrlist_append(_PyMain *pymain, int *len, wchar_t ***list, const wchar_t Return 1 if parsing failed. Set pymain->err and return -1 on other errors. */ static int -pymain_parse_cmdline_impl(_PyMain *pymain, _Py_CommandLineDetails *cmdline) +pymain_parse_cmdline_impl(_PyMain *pymain, _PyCoreConfig *config, + _PyCmdline *cmdline) { - _PyCoreConfig *config = &pymain->config; - _PyOS_ResetGetOpt(); do { int longindex = -1; @@ -985,7 +979,7 @@ config_add_warnings_optlist(_PyCoreConfig *config, int len, wchar_t **options) static _PyInitError -config_init_warnoptions(_PyCoreConfig *config, _Py_CommandLineDetails *cmdline) +config_init_warnoptions(_PyCoreConfig *config, _PyCmdline *cmdline) { _PyInitError err; @@ -1056,7 +1050,7 @@ config_init_warnoptions(_PyCoreConfig *config, _Py_CommandLineDetails *cmdline) Return 0 on success. Set pymain->err and return -1 on error. */ static _PyInitError -cmdline_init_env_warnoptions(_Py_CommandLineDetails *cmdline) +cmdline_init_env_warnoptions(_PyCmdline *cmdline) { if (Py_IgnoreEnvironmentFlag) { return _Py_INIT_OK(); @@ -1252,7 +1246,8 @@ copy_wstrlist(int len, wchar_t **list) static int -pymain_init_core_argv(_PyMain *pymain, _Py_CommandLineDetails *cmdline) +pymain_init_core_argv(_PyMain *pymain, _PyCoreConfig *config, + _PyCmdline *cmdline) { /* Copy argv to be able to modify it (to force -c/-m) */ int argc = pymain->argc - _PyOS_optind; @@ -1295,8 +1290,8 @@ pymain_init_core_argv(_PyMain *pymain, _Py_CommandLineDetails *cmdline) argv[0] = arg0; } - pymain->config.argc = argc; - pymain->config.argv = argv; + config->argc = argc; + config->argv = argv; return 0; } @@ -1324,7 +1319,7 @@ wstrlist_as_pylist(int len, wchar_t **list) static int -pymain_compute_path0(_PyMain *pymain, PyObject **path0) +pymain_compute_path0(_PyMain *pymain, _PyCoreConfig *config, PyObject **path0) { if (pymain->main_importer_path != NULL) { /* Let pymain_run_main_from_importer() adjust sys.path[0] later */ @@ -1337,8 +1332,7 @@ pymain_compute_path0(_PyMain *pymain, PyObject **path0) return 0; } - *path0 = _PyPathConfig_ComputeArgv0(pymain->config.argc, - pymain->config.argv); + *path0 = _PyPathConfig_ComputeArgv0(config->argc, config->argv); if (*path0 == NULL) { pymain->err = _Py_INIT_NO_MEMORY(); return -1; @@ -1384,7 +1378,7 @@ _PyCoreConfig_GetGlobalConfig(_PyCoreConfig *config) /* Get Py_xxx global configuration variables */ static void -cmdline_get_global_config(_Py_CommandLineDetails *cmdline) +cmdline_get_global_config(_PyCmdline *cmdline) { cmdline->bytes_warning = Py_BytesWarningFlag; cmdline->debug = Py_DebugFlag; @@ -1420,7 +1414,7 @@ _PyCoreConfig_SetGlobalConfig(const _PyCoreConfig *config) /* Set Py_xxx global configuration variables */ static void -cmdline_set_global_config(_Py_CommandLineDetails *cmdline) +cmdline_set_global_config(_PyCmdline *cmdline) { Py_BytesWarningFlag = cmdline->bytes_warning; Py_DebugFlag = cmdline->debug; @@ -1468,6 +1462,7 @@ pymain_import_readline(_PyMain *pymain) static FILE* pymain_open_filename(_PyMain *pymain) { + const _PyCoreConfig *config = &_PyGILState_GetInterpreterStateUnsafe()->core_config; FILE* fp; fp = _Py_wfopen(pymain->filename, L"r"); @@ -1481,7 +1476,7 @@ pymain_open_filename(_PyMain *pymain) else cfilename = ""; fprintf(stderr, "%ls: can't open file '%s': [Errno %d] %s\n", - pymain->config.program, cfilename, err, strerror(err)); + config->program, cfilename, err, strerror(err)); PyMem_RawFree(cfilename_buffer); pymain->status = 2; return NULL; @@ -1504,7 +1499,7 @@ pymain_open_filename(_PyMain *pymain) S_ISDIR(sb.st_mode)) { fprintf(stderr, "%ls: '%ls' is a directory, cannot continue\n", - pymain->config.program, pymain->filename); + config->program, pymain->filename); fclose(fp); pymain->status = 1; return NULL; @@ -1571,14 +1566,15 @@ pymain_repl(_PyMain *pymain, PyCompilerFlags *cf) Return 0 on success. Set pymain->err and return -1 on failure. */ static int -pymain_parse_cmdline(_PyMain *pymain, _Py_CommandLineDetails *cmdline) +pymain_parse_cmdline(_PyMain *pymain, _PyCoreConfig *config, + _PyCmdline *cmdline) { - int res = pymain_parse_cmdline_impl(pymain, cmdline); + int res = pymain_parse_cmdline_impl(pymain, config, cmdline); if (res < 0) { return -1; } if (res) { - pymain_usage(1, pymain->config.program); + pymain_usage(1, config->program); pymain->status = 2; return 1; } @@ -1716,7 +1712,7 @@ get_env_flag(int *flag, const char *name) static void -cmdline_get_env_flags(_Py_CommandLineDetails *cmdline) +cmdline_get_env_flags(_PyCmdline *cmdline) { get_env_flag(&cmdline->debug, "PYTHONDEBUG"); get_env_flag(&cmdline->verbosity, "PYTHONVERBOSE"); @@ -1738,7 +1734,7 @@ cmdline_get_env_flags(_Py_CommandLineDetails *cmdline) void _Py_Initialize_ReadEnvVarsNoAlloc(void) { - _Py_CommandLineDetails cmdline; + _PyCmdline cmdline; memset(&cmdline, 0, sizeof(cmdline)); cmdline_get_global_config(&cmdline); @@ -1926,17 +1922,17 @@ config_read_complex_options(_PyCoreConfig *config) Return 1 if Python is done and must exit. Set pymain->err and return -1 on error. */ static int -pymain_read_conf_impl(_PyMain *pymain, _Py_CommandLineDetails *cmdline) +pymain_read_conf_impl(_PyMain *pymain, _PyCoreConfig *config, + _PyCmdline *cmdline) { _PyInitError err; - int res = pymain_parse_cmdline(pymain, cmdline); + int res = pymain_parse_cmdline(pymain, config, cmdline); if (res != 0) { return res; } /* Set Py_IgnoreEnvironmentFlag for Py_GETENV() */ - _PyCoreConfig *config = &pymain->config; Py_IgnoreEnvironmentFlag = config->ignore_environment || cmdline->isolated; /* Get environment variables */ @@ -1956,7 +1952,7 @@ pymain_read_conf_impl(_PyMain *pymain, _Py_CommandLineDetails *cmdline) } #endif - if (pymain_init_core_argv(pymain, cmdline) < 0) { + if (pymain_init_core_argv(pymain, config, cmdline) < 0) { return -1; } @@ -1985,10 +1981,9 @@ pymain_read_conf_impl(_PyMain *pymain, _Py_CommandLineDetails *cmdline) /* Read the configuration, but initialize also the LC_CTYPE locale: enable UTF-8 mode (PEP 540) and/or coerce the C locale (PEP 538) */ static int -pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) +pymain_read_conf(_PyMain *pymain, _PyCoreConfig *config, _PyCmdline *cmdline) { int init_utf8_mode = Py_UTF8Mode; - _PyCoreConfig *config = &pymain->config; _PyCoreConfig save_config = _PyCoreConfig_INIT; int res = -1; @@ -2026,11 +2021,11 @@ pymain_read_conf(_PyMain *pymain, _Py_CommandLineDetails *cmdline) functions depend on Py_UTF8Mode. */ Py_UTF8Mode = config->utf8_mode; - if (pymain_init_cmdline_argv(pymain, cmdline) < 0) { + if (pymain_init_cmdline_argv(pymain, config, cmdline) < 0) { goto done; } - int conf_res = pymain_read_conf_impl(pymain, cmdline); + int conf_res = pymain_read_conf_impl(pymain, config, cmdline); if (conf_res != 0) { res = conf_res; goto done; @@ -2587,7 +2582,7 @@ pymain_init_python_main(_PyMain *pymain, PyInterpreterState *interp) static int -pymain_init_sys_path(_PyMain *pymain) +pymain_init_sys_path(_PyMain *pymain, _PyCoreConfig *config) { if (pymain->filename != NULL) { /* If filename is a package (ex: directory or ZIP file) which contains @@ -2598,12 +2593,10 @@ pymain_init_sys_path(_PyMain *pymain) } PyObject *path0; - if (pymain_compute_path0(pymain, &path0) < 0) { + if (pymain_compute_path0(pymain, config, &path0) < 0) { return -1; } - pymain_clear_config(pymain); - if (path0 != NULL) { if (pymain_update_sys_path(pymain, path0) < 0) { Py_DECREF(path0); @@ -2637,32 +2630,16 @@ pymain_run_python(_PyMain *pymain) } -static void -pymain_init(_PyMain *pymain) -{ - /* 754 requires that FP exceptions run in "no stop" mode by default, - * and until C vendors implement C99's ways to control FP exceptions, - * Python requires non-stop mode. Alas, some platforms enable FP - * exceptions by default. Here we disable them. - */ -#ifdef __FreeBSD__ - fedisableexcept(FE_OVERFLOW); -#endif - - pymain->config._disable_importlib = 0; - pymain->config.install_signal_handlers = 1; -} - - static int -pymain_cmdline_impl(_PyMain *pymain, _Py_CommandLineDetails *cmdline) +pymain_cmdline_impl(_PyMain *pymain, _PyCoreConfig *config, + _PyCmdline *cmdline) { pymain->err = _PyRuntime_Initialize(); if (_Py_INIT_FAILED(pymain->err)) { return -1; } - int res = pymain_read_conf(pymain, cmdline); + int res = pymain_read_conf(pymain, config, cmdline); if (res < 0) { return -1; } @@ -2672,7 +2649,7 @@ pymain_cmdline_impl(_PyMain *pymain, _Py_CommandLineDetails *cmdline) } if (cmdline->print_help) { - pymain_usage(0, pymain->config.program); + pymain_usage(0, config->program); return 1; } @@ -2690,7 +2667,7 @@ pymain_cmdline_impl(_PyMain *pymain, _Py_CommandLineDetails *cmdline) } orig_argc = pymain->argc; - _PyInitError err = config_init_warnoptions(&pymain->config, cmdline); + _PyInitError err = config_init_warnoptions(config, cmdline); if (_Py_INIT_FAILED(err)) { pymain->err = err; return -1; @@ -2708,10 +2685,10 @@ pymain_cmdline_impl(_PyMain *pymain, _Py_CommandLineDetails *cmdline) * Environment variables * Py_xxx global configuration variables - _Py_CommandLineDetails is a temporary structure used to prioritize these + _PyCmdline is a temporary structure used to prioritize these variables. */ static int -pymain_cmdline(_PyMain *pymain) +pymain_cmdline(_PyMain *pymain, _PyCoreConfig *config) { /* Force default allocator, since pymain_free() and pymain_clear_config() must use the same allocator than this function. */ @@ -2722,16 +2699,15 @@ pymain_cmdline(_PyMain *pymain) PyMem_GetAllocator(PYMEM_DOMAIN_RAW, &default_alloc); #endif - _Py_CommandLineDetails cmdline; + _PyCmdline cmdline; memset(&cmdline, 0, sizeof(cmdline)); cmdline_get_global_config(&cmdline); - _PyCoreConfig_GetGlobalConfig(&pymain->config); - int res = pymain_cmdline_impl(pymain, &cmdline); + int res = pymain_cmdline_impl(pymain, config, &cmdline); cmdline_set_global_config(&cmdline); - _PyCoreConfig_SetGlobalConfig(&pymain->config); + _PyCoreConfig_SetGlobalConfig(config); if (Py_IsolatedFlag) { Py_IgnoreEnvironmentFlag = 1; Py_NoUserSiteDirectory = 1; @@ -2751,33 +2727,62 @@ pymain_cmdline(_PyMain *pymain) static int -pymain_main(_PyMain *pymain) +pymain_init(_PyMain *pymain) { - pymain_init(pymain); + _PyCoreConfig local_config = _PyCoreConfig_INIT; + _PyCoreConfig *config = &local_config; + + /* 754 requires that FP exceptions run in "no stop" mode by default, + * and until C vendors implement C99's ways to control FP exceptions, + * Python requires non-stop mode. Alas, some platforms enable FP + * exceptions by default. Here we disable them. + */ +#ifdef __FreeBSD__ + fedisableexcept(FE_OVERFLOW); +#endif + + config->_disable_importlib = 0; + config->install_signal_handlers = 1; + _PyCoreConfig_GetGlobalConfig(config); - int res = pymain_cmdline(pymain); + int res = pymain_cmdline(pymain, config); if (res < 0) { _Py_FatalInitError(pymain->err); } if (res == 1) { - goto done; + pymain_clear_config(&local_config); + return res; } pymain_init_stdio(pymain); PyInterpreterState *interp; - pymain->err = _Py_InitializeCore(&interp, &pymain->config); + pymain->err = _Py_InitializeCore(&interp, config); if (_Py_INIT_FAILED(pymain->err)) { _Py_FatalInitError(pymain->err); } + pymain_clear_config(&local_config); + config = &interp->core_config; + if (pymain_init_python_main(pymain, interp) < 0) { _Py_FatalInitError(pymain->err); } - if (pymain_init_sys_path(pymain) < 0) { + if (pymain_init_sys_path(pymain, config) < 0) { _Py_FatalInitError(pymain->err); } + return 0; +} + + +static int +pymain_main(_PyMain *pymain) +{ + int res = pymain_init(pymain); + if (res == 1) { + goto done; + } pymain_run_python(pymain); From webhook-mailer at python.org Mon Sep 24 08:04:36 2018 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 24 Sep 2018 12:04:36 -0000 Subject: [Python-checkins] Update Azure Pipelines badge (GH-9529) Message-ID: https://github.com/python/cpython/commit/680d26e3b6b3fcd2deadd392d837325e2d1bb652 commit: 680d26e3b6b3fcd2deadd392d837325e2d1bb652 branch: master author: Steve Dower committer: GitHub date: 2018-09-24T08:04:33-04:00 summary: Update Azure Pipelines badge (GH-9529) files: M README.rst diff --git a/README.rst b/README.rst index c65f6e29d413..970f62e69389 100644 --- a/README.rst +++ b/README.rst @@ -9,16 +9,8 @@ This is Python version 3.8.0 alpha 0 :alt: CPython build status on Appveyor :target: https://ci.appveyor.com/project/python/cpython/branch/master -.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Linux-Buildbot?branchName=master&label=Linux - :alt: CPython build status on Azure DevOps (Linux) - :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=6&branchName=master - -.. image:: https://dev.azure.com/python/cpython/_apis/build/status/macOS-Buildbot?branchName=master&label=macOS - :alt: CPython build status on Azure DevOps (macOS) - :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=5&branchName=master - -.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Windows-Buildbot?branchName=master&label=Windows - :alt: CPython build status on Azure DevOps (Windows) +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Azure%20Pipelines%20CI?branchName=master + :alt: CPython build status on Azure DevOps :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=4&branchName=master .. image:: https://codecov.io/gh/python/cpython/branch/master/graph/badge.svg From webhook-mailer at python.org Mon Sep 24 08:38:36 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 24 Sep 2018 12:38:36 -0000 Subject: [Python-checkins] [3.6] bpo-17239: Disable external entities in SAX parser (GH-9217) (GH-9512) Message-ID: https://github.com/python/cpython/commit/582d188e6e3487180891f1fc457a80dec8be26a8 commit: 582d188e6e3487180891f1fc457a80dec8be26a8 branch: 3.6 author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-24T05:38:31-07:00 summary: [3.6] bpo-17239: Disable external entities in SAX parser (GH-9217) (GH-9512) The SAX parser no longer processes general external entities by default to increase security. Before, the parser created network connections to fetch remote files or loaded local files from the file system for DTD and entities. Signed-off-by: Christian Heimes https://bugs.python.org/issue17239. (cherry picked from commit 17b1d5d4e36aa57a9b25a0e694affbd1ee637e45) Co-authored-by: Christian Heimes https://bugs.python.org/issue17239 files: A Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst M Doc/library/xml.dom.pulldom.rst M Doc/library/xml.rst M Doc/library/xml.sax.rst M Doc/whatsnew/3.6.rst M Lib/test/test_pulldom.py M Lib/test/test_sax.py M Lib/test/test_xml_etree.py M Lib/xml/sax/expatreader.py diff --git a/Doc/library/xml.dom.pulldom.rst b/Doc/library/xml.dom.pulldom.rst index 5c0f469ad7a5..2504409e7f87 100644 --- a/Doc/library/xml.dom.pulldom.rst +++ b/Doc/library/xml.dom.pulldom.rst @@ -25,6 +25,20 @@ events until either processing is finished or an error condition occurs. maliciously constructed data. If you need to parse untrusted or unauthenticated data see :ref:`xml-vulnerabilities`. +.. versionchanged:: 3.6.7 + + The SAX parser no longer processes general external entities by default to + increase security by default. To enable processing of external entities, + pass a custom parser instance in:: + + from xml.dom.pulldom import parse + from xml.sax import make_parser + from xml.sax.handler import feature_external_ges + + parser = make_parser() + parser.setFeature(feature_external_ges, True) + parse(filename, parser=parser) + Example:: diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index 63c24f80ac87..9b8ba6b17c85 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -65,8 +65,8 @@ kind sax etree minidom p ========================= ============== =============== ============== ============== ============== billion laughs **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** quadratic blowup **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** -external entity expansion **Vulnerable** Safe (1) Safe (2) **Vulnerable** Safe (3) -`DTD`_ retrieval **Vulnerable** Safe Safe **Vulnerable** Safe +external entity expansion Safe (4) Safe (1) Safe (2) Safe (4) Safe (3) +`DTD`_ retrieval Safe (4) Safe Safe Safe (4) Safe decompression bomb Safe Safe Safe Safe **Vulnerable** ========================= ============== =============== ============== ============== ============== @@ -75,6 +75,8 @@ decompression bomb Safe Safe Safe S 2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns the unexpanded entity verbatim. 3. :mod:`xmlrpclib` doesn't expand external entities and omits them. +4. Since Python 3.8.0, external general entities are no longer processed by + default since Python. billion laughs / exponential entity expansion diff --git a/Doc/library/xml.sax.rst b/Doc/library/xml.sax.rst index 78d6633e098b..1a8f183a945f 100644 --- a/Doc/library/xml.sax.rst +++ b/Doc/library/xml.sax.rst @@ -24,6 +24,14 @@ the SAX API. constructed data. If you need to parse untrusted or unauthenticated data see :ref:`xml-vulnerabilities`. +.. versionchanged:: 3.6.7 + + The SAX parser no longer processes general external entities by default + to increase security. Before, the parser created network connections + to fetch remote files or loaded local files from the file + system for DTD and entities. The feature can be enabled again with method + :meth:`~xml.sax.xmlreader.XMLReader.setFeature` on the parser object + and argument :data:`~xml.sax.handler.feature_external_ges`. The convenience functions are: diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index d375cff1af94..08d2d24294d0 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -2055,6 +2055,15 @@ connected to and thus what Python interpreter will be used by the virtual environment. (Contributed by Brett Cannon in :issue:`25154`.) +xml +--- + +* As mitigation against DTD and external entity retrieval, the + :mod:`xml.dom.minidom` and mod:`xml.sax` modules no longer process + external entities by default. + (Contributed by Christian Heimes in :issue:`17239`.) + + Deprecated functions and types of the C API ------------------------------------------- @@ -2163,7 +2172,7 @@ Changes in the Python API * The functions in the :mod:`compileall` module now return booleans instead of ``1`` or ``0`` to represent success or failure, respectively. Thanks to - booleans being a subclass of integers, this should only be an issue if you + booleans being a subclass of integers, this should only be an issue if you7 were doing identity checks for ``1`` or ``0``. See :issue:`25768`. * Reading the :attr:`~urllib.parse.SplitResult.port` attribute of @@ -2408,3 +2417,10 @@ Notable changes in Python 3.6.5 The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` locale in some cases. (Contributed by Victor Stinner in :issue:`31900`.) + + +Notable changes in Python 3.6.7 +=============================== + +:mod:`xml.dom.minidom` and mod:`xml.sax` modules no longer process +external entities by default. See also :issue:`17239`. diff --git a/Lib/test/test_pulldom.py b/Lib/test/test_pulldom.py index 3d89e3adda26..6dc51e4371d0 100644 --- a/Lib/test/test_pulldom.py +++ b/Lib/test/test_pulldom.py @@ -3,6 +3,7 @@ import xml.sax from xml.sax.xmlreader import AttributesImpl +from xml.sax.handler import feature_external_ges from xml.dom import pulldom from test.support import findfile @@ -159,6 +160,12 @@ def test_end_document(self): self.fail( "Ran out of events, but should have received END_DOCUMENT") + def test_external_ges_default(self): + parser = pulldom.parseString(SMALL_SAMPLE) + saxparser = parser.parser + ges = saxparser.getFeature(feature_external_ges) + self.assertEqual(ges, False) + class ThoroughTestCase(unittest.TestCase): """Test the hard-to-reach parts of pulldom.""" diff --git a/Lib/test/test_sax.py b/Lib/test/test_sax.py index 2eb62905ffa8..3044960a0ed1 100644 --- a/Lib/test/test_sax.py +++ b/Lib/test/test_sax.py @@ -13,13 +13,14 @@ from xml.sax.saxutils import XMLGenerator, escape, unescape, quoteattr, \ XMLFilterBase, prepare_input_source from xml.sax.expatreader import create_parser -from xml.sax.handler import feature_namespaces +from xml.sax.handler import feature_namespaces, feature_external_ges from xml.sax.xmlreader import InputSource, AttributesImpl, AttributesNSImpl from io import BytesIO, StringIO import codecs import gc import os.path import shutil +from urllib.error import URLError from test import support from test.support import findfile, run_unittest, TESTFN @@ -911,6 +912,18 @@ def notationDecl(self, name, publicId, systemId): def unparsedEntityDecl(self, name, publicId, systemId, ndata): self._entities.append((name, publicId, systemId, ndata)) + + class TestEntityRecorder: + def __init__(self): + self.entities = [] + + def resolveEntity(self, publicId, systemId): + self.entities.append((publicId, systemId)) + source = InputSource() + source.setPublicId(publicId) + source.setSystemId(systemId) + return source + def test_expat_dtdhandler(self): parser = create_parser() handler = self.TestDTDHandler() @@ -927,6 +940,32 @@ def test_expat_dtdhandler(self): [("GIF", "-//CompuServe//NOTATION Graphics Interchange Format 89a//EN", None)]) self.assertEqual(handler._entities, [("img", None, "expat.gif", "GIF")]) + def test_expat_external_dtd_enabled(self): + parser = create_parser() + parser.setFeature(feature_external_ges, True) + resolver = self.TestEntityRecorder() + parser.setEntityResolver(resolver) + + with self.assertRaises(URLError): + parser.feed( + '\n' + ) + self.assertEqual( + resolver.entities, [(None, 'unsupported://non-existing')] + ) + + def test_expat_external_dtd_default(self): + parser = create_parser() + resolver = self.TestEntityRecorder() + parser.setEntityResolver(resolver) + + parser.feed( + '\n' + ) + parser.feed('') + parser.close() + self.assertEqual(resolver.entities, []) + # ===== EntityResolver support class TestEntityResolver: @@ -936,8 +975,9 @@ def resolveEntity(self, publicId, systemId): inpsrc.setByteStream(BytesIO(b"")) return inpsrc - def test_expat_entityresolver(self): + def test_expat_entityresolver_enabled(self): parser = create_parser() + parser.setFeature(feature_external_ges, True) parser.setEntityResolver(self.TestEntityResolver()) result = BytesIO() parser.setContentHandler(XMLGenerator(result)) @@ -951,6 +991,22 @@ def test_expat_entityresolver(self): self.assertEqual(result.getvalue(), start + b"") + def test_expat_entityresolver_default(self): + parser = create_parser() + self.assertEqual(parser.getFeature(feature_external_ges), False) + parser.setEntityResolver(self.TestEntityResolver()) + result = BytesIO() + parser.setContentHandler(XMLGenerator(result)) + + parser.feed('\n') + parser.feed(']>\n') + parser.feed('&test;') + parser.close() + + self.assertEqual(result.getvalue(), start + + b"") + # ===== Attributes support class AttrGatherer(ContentHandler): diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index c14f8397ae7d..2b8c5e598656 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -90,6 +90,12 @@ &entity; """ +EXTERNAL_ENTITY_XML = """\ + +]> +&entity; +""" class ModuleTest(unittest.TestCase): def test_sanity(self): @@ -846,6 +852,13 @@ def test_entity(self): root = parser.close() self.serialize_check(root, 'text') + # 4) external (SYSTEM) entity + + with self.assertRaises(ET.ParseError) as cm: + ET.XML(EXTERNAL_ENTITY_XML) + self.assertEqual(str(cm.exception), + 'undefined entity &entity;: line 4, column 10') + def test_namespace(self): # Test namespace issues. diff --git a/Lib/xml/sax/expatreader.py b/Lib/xml/sax/expatreader.py index 421358fa5bc7..5066ffc2fa51 100644 --- a/Lib/xml/sax/expatreader.py +++ b/Lib/xml/sax/expatreader.py @@ -95,7 +95,7 @@ def __init__(self, namespaceHandling=0, bufsize=2**16-20): self._lex_handler_prop = None self._parsing = 0 self._entity_stack = [] - self._external_ges = 1 + self._external_ges = 0 self._interning = None # XMLReader methods diff --git a/Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst b/Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst new file mode 100644 index 000000000000..8dd0fe8c1b53 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst @@ -0,0 +1,3 @@ +The xml.sax and xml.dom.minidom parsers no longer processes external +entities by default. External DTD and ENTITY declarations no longer +load files or create network connections. From webhook-mailer at python.org Mon Sep 24 08:38:40 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 24 Sep 2018 12:38:40 -0000 Subject: [Python-checkins] [3.7] bpo-17239: Disable external entities in SAX parser (GH-9217) (GH-9511) Message-ID: https://github.com/python/cpython/commit/394e55a9279d17240ef6fe85d3b4ea3fe7b6dff5 commit: 394e55a9279d17240ef6fe85d3b4ea3fe7b6dff5 branch: 3.7 author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-24T05:38:37-07:00 summary: [3.7] bpo-17239: Disable external entities in SAX parser (GH-9217) (GH-9511) The SAX parser no longer processes general external entities by default to increase security. Before, the parser created network connections to fetch remote files or loaded local files from the file system for DTD and entities. Signed-off-by: Christian Heimes https://bugs.python.org/issue17239. (cherry picked from commit 17b1d5d4e36aa57a9b25a0e694affbd1ee637e45) Co-authored-by: Christian Heimes https://bugs.python.org/issue17239 files: A Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst M Doc/library/xml.dom.pulldom.rst M Doc/library/xml.rst M Doc/library/xml.sax.rst M Doc/whatsnew/3.7.rst M Lib/test/test_pulldom.py M Lib/test/test_sax.py M Lib/test/test_xml_etree.py M Lib/xml/sax/expatreader.py diff --git a/Doc/library/xml.dom.pulldom.rst b/Doc/library/xml.dom.pulldom.rst index 5c0f469ad7a5..b4974f904d28 100644 --- a/Doc/library/xml.dom.pulldom.rst +++ b/Doc/library/xml.dom.pulldom.rst @@ -25,6 +25,20 @@ events until either processing is finished or an error condition occurs. maliciously constructed data. If you need to parse untrusted or unauthenticated data see :ref:`xml-vulnerabilities`. +.. versionchanged:: 3.7.1 + + The SAX parser no longer processes general external entities by default to + increase security by default. To enable processing of external entities, + pass a custom parser instance in:: + + from xml.dom.pulldom import parse + from xml.sax import make_parser + from xml.sax.handler import feature_external_ges + + parser = make_parser() + parser.setFeature(feature_external_ges, True) + parse(filename, parser=parser) + Example:: diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index 63c24f80ac87..9b8ba6b17c85 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -65,8 +65,8 @@ kind sax etree minidom p ========================= ============== =============== ============== ============== ============== billion laughs **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** quadratic blowup **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** **Vulnerable** -external entity expansion **Vulnerable** Safe (1) Safe (2) **Vulnerable** Safe (3) -`DTD`_ retrieval **Vulnerable** Safe Safe **Vulnerable** Safe +external entity expansion Safe (4) Safe (1) Safe (2) Safe (4) Safe (3) +`DTD`_ retrieval Safe (4) Safe Safe Safe (4) Safe decompression bomb Safe Safe Safe Safe **Vulnerable** ========================= ============== =============== ============== ============== ============== @@ -75,6 +75,8 @@ decompression bomb Safe Safe Safe S 2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns the unexpanded entity verbatim. 3. :mod:`xmlrpclib` doesn't expand external entities and omits them. +4. Since Python 3.8.0, external general entities are no longer processed by + default since Python. billion laughs / exponential entity expansion diff --git a/Doc/library/xml.sax.rst b/Doc/library/xml.sax.rst index 78d6633e098b..952090c339f4 100644 --- a/Doc/library/xml.sax.rst +++ b/Doc/library/xml.sax.rst @@ -24,6 +24,14 @@ the SAX API. constructed data. If you need to parse untrusted or unauthenticated data see :ref:`xml-vulnerabilities`. +.. versionchanged:: 3.7.1 + + The SAX parser no longer processes general external entities by default + to increase security. Before, the parser created network connections + to fetch remote files or loaded local files from the file + system for DTD and entities. The feature can be enabled again with method + :meth:`~xml.sax.xmlreader.XMLReader.setFeature` on the parser object + and argument :data:`~xml.sax.handler.feature_external_ges`. The convenience functions are: diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 132ed00f7321..e988aecf9683 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1573,6 +1573,15 @@ at the interactive prompt. See :ref:`whatsnew37-pep565` for details. (Contributed by Nick Coghlan in :issue:`31975`.) +xml +--- + +As mitigation against DTD and external entity retrieval, the +:mod:`xml.dom.minidom` and mod:`xml.sax` modules no longer process +external entities by default. +(Contributed by Christian Heimes in :issue:`17239`.) + + xml.etree --------- @@ -2502,3 +2511,6 @@ calling :c:func:`Py_Initialize`. In 3.7.1 the C API for Context Variables :ref:`was updated ` to use :c:type:`PyObject` pointers. See also :issue:`34762`. + +:mod:`xml.dom.minidom` and mod:`xml.sax` modules no longer process +external entities by default. See also :issue:`17239`. diff --git a/Lib/test/test_pulldom.py b/Lib/test/test_pulldom.py index 3d89e3adda26..6dc51e4371d0 100644 --- a/Lib/test/test_pulldom.py +++ b/Lib/test/test_pulldom.py @@ -3,6 +3,7 @@ import xml.sax from xml.sax.xmlreader import AttributesImpl +from xml.sax.handler import feature_external_ges from xml.dom import pulldom from test.support import findfile @@ -159,6 +160,12 @@ def test_end_document(self): self.fail( "Ran out of events, but should have received END_DOCUMENT") + def test_external_ges_default(self): + parser = pulldom.parseString(SMALL_SAMPLE) + saxparser = parser.parser + ges = saxparser.getFeature(feature_external_ges) + self.assertEqual(ges, False) + class ThoroughTestCase(unittest.TestCase): """Test the hard-to-reach parts of pulldom.""" diff --git a/Lib/test/test_sax.py b/Lib/test/test_sax.py index 2eb62905ffa8..3044960a0ed1 100644 --- a/Lib/test/test_sax.py +++ b/Lib/test/test_sax.py @@ -13,13 +13,14 @@ from xml.sax.saxutils import XMLGenerator, escape, unescape, quoteattr, \ XMLFilterBase, prepare_input_source from xml.sax.expatreader import create_parser -from xml.sax.handler import feature_namespaces +from xml.sax.handler import feature_namespaces, feature_external_ges from xml.sax.xmlreader import InputSource, AttributesImpl, AttributesNSImpl from io import BytesIO, StringIO import codecs import gc import os.path import shutil +from urllib.error import URLError from test import support from test.support import findfile, run_unittest, TESTFN @@ -911,6 +912,18 @@ def notationDecl(self, name, publicId, systemId): def unparsedEntityDecl(self, name, publicId, systemId, ndata): self._entities.append((name, publicId, systemId, ndata)) + + class TestEntityRecorder: + def __init__(self): + self.entities = [] + + def resolveEntity(self, publicId, systemId): + self.entities.append((publicId, systemId)) + source = InputSource() + source.setPublicId(publicId) + source.setSystemId(systemId) + return source + def test_expat_dtdhandler(self): parser = create_parser() handler = self.TestDTDHandler() @@ -927,6 +940,32 @@ def test_expat_dtdhandler(self): [("GIF", "-//CompuServe//NOTATION Graphics Interchange Format 89a//EN", None)]) self.assertEqual(handler._entities, [("img", None, "expat.gif", "GIF")]) + def test_expat_external_dtd_enabled(self): + parser = create_parser() + parser.setFeature(feature_external_ges, True) + resolver = self.TestEntityRecorder() + parser.setEntityResolver(resolver) + + with self.assertRaises(URLError): + parser.feed( + '\n' + ) + self.assertEqual( + resolver.entities, [(None, 'unsupported://non-existing')] + ) + + def test_expat_external_dtd_default(self): + parser = create_parser() + resolver = self.TestEntityRecorder() + parser.setEntityResolver(resolver) + + parser.feed( + '\n' + ) + parser.feed('') + parser.close() + self.assertEqual(resolver.entities, []) + # ===== EntityResolver support class TestEntityResolver: @@ -936,8 +975,9 @@ def resolveEntity(self, publicId, systemId): inpsrc.setByteStream(BytesIO(b"")) return inpsrc - def test_expat_entityresolver(self): + def test_expat_entityresolver_enabled(self): parser = create_parser() + parser.setFeature(feature_external_ges, True) parser.setEntityResolver(self.TestEntityResolver()) result = BytesIO() parser.setContentHandler(XMLGenerator(result)) @@ -951,6 +991,22 @@ def test_expat_entityresolver(self): self.assertEqual(result.getvalue(), start + b"") + def test_expat_entityresolver_default(self): + parser = create_parser() + self.assertEqual(parser.getFeature(feature_external_ges), False) + parser.setEntityResolver(self.TestEntityResolver()) + result = BytesIO() + parser.setContentHandler(XMLGenerator(result)) + + parser.feed('\n') + parser.feed(']>\n') + parser.feed('&test;') + parser.close() + + self.assertEqual(result.getvalue(), start + + b"") + # ===== Attributes support class AttrGatherer(ContentHandler): diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index e11397585d97..088a04e98b60 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -91,6 +91,12 @@ &entity; """ +EXTERNAL_ENTITY_XML = """\ + +]> +&entity; +""" def checkwarnings(*filters, quiet=False): def decorator(test): @@ -861,6 +867,13 @@ def test_entity(self): root = parser.close() self.serialize_check(root, 'text') + # 4) external (SYSTEM) entity + + with self.assertRaises(ET.ParseError) as cm: + ET.XML(EXTERNAL_ENTITY_XML) + self.assertEqual(str(cm.exception), + 'undefined entity &entity;: line 4, column 10') + def test_namespace(self): # Test namespace issues. diff --git a/Lib/xml/sax/expatreader.py b/Lib/xml/sax/expatreader.py index 421358fa5bc7..5066ffc2fa51 100644 --- a/Lib/xml/sax/expatreader.py +++ b/Lib/xml/sax/expatreader.py @@ -95,7 +95,7 @@ def __init__(self, namespaceHandling=0, bufsize=2**16-20): self._lex_handler_prop = None self._parsing = 0 self._entity_stack = [] - self._external_ges = 1 + self._external_ges = 0 self._interning = None # XMLReader methods diff --git a/Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst b/Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst new file mode 100644 index 000000000000..8dd0fe8c1b53 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst @@ -0,0 +1,3 @@ +The xml.sax and xml.dom.minidom parsers no longer processes external +entities by default. External DTD and ENTITY declarations no longer +load files or create network connections. From webhook-mailer at python.org Mon Sep 24 08:43:39 2018 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 24 Sep 2018 12:43:39 -0000 Subject: [Python-checkins] bpo-34582: Update syntax of Azure Pipelines builds (GH-9521) Message-ID: https://github.com/python/cpython/commit/657fdf2eba617ade49052de91cafeb373b652077 commit: 657fdf2eba617ade49052de91cafeb373b652077 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Steve Dower date: 2018-09-24T08:43:33-04:00 summary: bpo-34582: Update syntax of Azure Pipelines builds (GH-9521) files: A .azure-pipelines/ci.yml A .azure-pipelines/docker-steps.yml A .azure-pipelines/docs-steps.yml A .azure-pipelines/macos-steps.yml A .azure-pipelines/posix-deps.sh A .azure-pipelines/posix-steps.yml A .azure-pipelines/pr.yml A .azure-pipelines/prebuild-checks.yml A .azure-pipelines/windows-steps.yml D .vsts/docs-release.yml D .vsts/docs.yml D .vsts/install_deps.sh D .vsts/linux-buildbot.yml D .vsts/linux-coverage.yml D .vsts/linux-pr.yml D .vsts/macos-buildbot.yml D .vsts/macos-pr.yml D .vsts/windows-buildbot.yml D .vsts/windows-pr.yml M Lib/test/test_zlib.py diff --git a/.azure-pipelines/ci.yml b/.azure-pipelines/ci.yml new file mode 100644 index 000000000000..f56254e38024 --- /dev/null +++ b/.azure-pipelines/ci.yml @@ -0,0 +1,136 @@ +variables: + manylinux: false + coverage: false + +jobs: +- job: Prebuild + displayName: Pre-build checks + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./prebuild-checks.yml + + +- job: Docs_PR + displayName: Docs PR + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['docs.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./docs-steps.yml + parameters: + upload: true + + +- job: macOS_CI_Tests + displayName: macOS CI Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + variables: + testRunTitle: '$(build.sourceBranchName)-macos' + testRunPlatform: macos + + pool: + vmImage: xcode9-macos10.13 + + steps: + - template: ./macos-steps.yml + + +- job: Ubuntu_CI_Tests + displayName: Ubuntu CI Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(build.sourceBranchName)-linux' + testRunPlatform: linux + openssl_version: 1.1.0g + + steps: + - template: ./posix-steps.yml + + +- job: ManyLinux1_CI_Tests + displayName: ManyLinux1 CI Tests + dependsOn: Prebuild + condition: | + and( + and( + succeeded(), + eq(variables['manylinux'], 'true') + ), + eq(dependencies.Prebuild.outputs['tests.run'], 'true') + ) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(build.sourceBranchName)-manylinux1' + testRunPlatform: manylinux1 + imageName: 'dockcross/manylinux-x64' + + steps: + - template: ./docker-steps.yml + + +- job: Ubuntu_Coverage_CI_Tests + displayName: Ubuntu CI Tests (coverage) + dependsOn: Prebuild + condition: | + and( + and( + succeeded(), + eq(variables['coverage'], 'true') + ), + eq(dependencies.Prebuild.outputs['tests.run'], 'true') + ) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(Build.SourceBranchName)-linux-coverage' + testRunPlatform: linux-coverage + openssl_version: 1.1.0g + + steps: + - template: ./posix-steps.yml + parameters: + coverage: true + + +- job: Windows_CI_Tests + displayName: Windows CI Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: vs2017-win2017 + + strategy: + matrix: + win32: + arch: win32 + buildOpt: + testRunTitle: '$(Build.SourceBranchName)-win32' + testRunPlatform: win32 + win64: + arch: amd64 + buildOpt: '-p x64' + testRunTitle: '$(Build.SourceBranchName)-win64' + testRunPlatform: win64 + maxParallel: 2 + + steps: + - template: ./windows-steps.yml diff --git a/.azure-pipelines/docker-steps.yml b/.azure-pipelines/docker-steps.yml new file mode 100644 index 000000000000..ba4dfd72dd8b --- /dev/null +++ b/.azure-pipelines/docker-steps.yml @@ -0,0 +1,76 @@ +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- ${{ if ne(parameters.targetBranch, '') }}: + - script: | + git fetch -q origin ${{ parameters.targetbranch }} + if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qvE '(\.rst$|^Doc|^Misc)' + then + echo "Only docs were updated, stopping build process." + echo "##vso[task.setvariable variable=DocOnly]true" + exit + fi + displayName: Detect doc-only changes + +- task: docker at 0 + displayName: 'Configure CPython (debug)' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: './configure --with-pydebug' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: docker at 0 + displayName: 'Build CPython' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: 'make -s -j4' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: docker at 0 + displayName: 'Display build info' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: 'make pythoninfo' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: docker at 0 + displayName: 'Tests' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: 'make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=/build/test-results.xml"' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.azure-pipelines/docs-steps.yml b/.azure-pipelines/docs-steps.yml new file mode 100644 index 000000000000..c0404aebdcc5 --- /dev/null +++ b/.azure-pipelines/docs-steps.yml @@ -0,0 +1,46 @@ +parameters: + latex: false + upload: false + +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- task: UsePythonVersion at 0 + displayName: 'Use Python 3.6 or later' + inputs: + versionSpec: '>=3.6' + +- script: python -m pip install sphinx~=1.6.1 blurb python-docs-theme + displayName: 'Install build dependencies' + +- ${{ if ne(parameters.latex, 'true') }}: + - script: make check suspicious html PYTHON=python + workingDirectory: '$(build.sourcesDirectory)/Doc' + displayName: 'Build documentation' + +- ${{ if eq(parameters.latex, 'true') }}: + - script: sudo apt-get update && sudo apt-get install -qy --force-yes texlive-full + displayName: 'Install LaTeX' + + - script: make dist PYTHON=python SPHINXBUILD='python -m sphinx' BLURB='python -m blurb' + workingDirectory: '$(build.sourcesDirectory)/Doc' + displayName: 'Build documentation' + +- ${{ if eq(parameters.upload, 'true') }}: + - task: PublishBuildArtifacts at 1 + displayName: 'Publish docs' + + inputs: + PathToPublish: '$(build.sourcesDirectory)/Doc/build' + ArtifactName: docs + publishLocation: Container + + - ${{ if eq(parameters.latex, 'true') }}: + - task: PublishBuildArtifacts at 1 + displayName: 'Publish dist' + inputs: + PathToPublish: '$(build.sourcesDirectory)/Doc/dist' + ArtifactName: docs_dist + publishLocation: Container diff --git a/.vsts/macos-buildbot.yml b/.azure-pipelines/macos-steps.yml similarity index 57% rename from .vsts/macos-buildbot.yml rename to .azure-pipelines/macos-steps.yml index d9c190c46131..647081689454 100644 --- a/.vsts/macos-buildbot.yml +++ b/.azure-pipelines/macos-steps.yml @@ -1,30 +1,9 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted macOS - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -#variables: - steps: - checkout: self clean: true fetchDepth: 5 -- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl --prefix=/opt/python-vsts +- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl --prefix=/opt/python-azdev displayName: 'Configure CPython (debug)' - script: make -s -j4 @@ -41,6 +20,6 @@ steps: inputs: testResultsFiles: '$(build.binariesDirectory)/test-results.xml' mergeTestResults: true - testRunTitle: '$(build.sourceBranchName)-macOS' - platform: macOS + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) condition: succeededOrFailed() diff --git a/.vsts/install_deps.sh b/.azure-pipelines/posix-deps.sh similarity index 51% rename from .vsts/install_deps.sh rename to .azure-pipelines/posix-deps.sh index b1fa576047e6..a57210756601 100755 --- a/.vsts/install_deps.sh +++ b/.azure-pipelines/posix-deps.sh @@ -1,4 +1,4 @@ -sudo apt-get update || true +sudo apt-get update sudo apt-get -yq install \ build-essential \ @@ -17,3 +17,10 @@ sudo apt-get -yq install \ libffi-dev \ uuid-dev \ xvfb + +if [ ! -z "$1" ] +then + echo ##vso[task.prependpath]$PWD/multissl/openssl/$1 + echo ##vso[task.setvariable variable=OPENSSL_DIR]$PWD/multissl/openssl/$1 + python3 Tools/ssl/multissltests.py --steps=library --base-directory $PWD/multissl --openssl $1 --system Linux +fi diff --git a/.azure-pipelines/posix-steps.yml b/.azure-pipelines/posix-steps.yml new file mode 100644 index 000000000000..9fec9be8014f --- /dev/null +++ b/.azure-pipelines/posix-steps.yml @@ -0,0 +1,63 @@ +parameters: + coverage: false + +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- script: ./.azure-pipelines/posix-deps.sh $(openssl_version) + displayName: 'Install dependencies' + +- script: ./configure --with-pydebug + displayName: 'Configure CPython (debug)' + +- script: make -s -j4 + displayName: 'Build CPython' + +- ${{ if eq(parameters.coverage, 'true') }}: + - script: ./python -m venv venv && ./venv/bin/python -m pip install -U coverage + displayName: 'Set up virtual environment' + + - script: ./venv/bin/python -m test.pythoninfo + displayName: 'Display build info' + + - script: | + xvfb-run ./venv/bin/python -m coverage run --pylib -m test \ + --fail-env-changed \ + -uall,-cpu \ + --junit-xml=$(build.binariesDirectory)/test-results.xml" \ + -x test_multiprocessing_fork \ + -x test_multiprocessing_forkserver \ + -x test_multiprocessing_spawn \ + -x test_concurrent_futures + displayName: 'Tests with coverage' + + - script: ./venv/bin/python -m coverage xml + displayName: 'Generate coverage.xml' + + - script: source ./venv/bin/activate && bash <(curl -s https://codecov.io/bash) + displayName: 'Publish code coverage results' + + +- ${{ if ne(parameters.coverage, 'true') }}: + - script: make pythoninfo + displayName: 'Display build info' + + - script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" + displayName: 'Tests' + + +- script: ./python Tools/scripts/patchcheck.py --travis true + displayName: 'Run patchcheck.py' + condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) + + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) + condition: succeededOrFailed() diff --git a/.azure-pipelines/pr.yml b/.azure-pipelines/pr.yml new file mode 100644 index 000000000000..653f55b69b13 --- /dev/null +++ b/.azure-pipelines/pr.yml @@ -0,0 +1,86 @@ +jobs: +- job: Prebuild + displayName: Pre-build checks + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./prebuild-checks.yml + + +- job: Docs_PR + displayName: Docs PR + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['docs.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./docs-steps.yml + + +- job: macOS_PR_Tests + displayName: macOS PR Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + variables: + testRunTitle: '$(system.pullRequest.TargetBranch)-macos' + testRunPlatform: macos + + pool: + vmImage: xcode9-macos10.13 + + steps: + - template: ./macos-steps.yml + parameters: + targetBranch: $(System.PullRequest.TargetBranch) + + +- job: Ubuntu_PR_Tests + displayName: Ubuntu PR Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(system.pullRequest.TargetBranch)-linux' + testRunPlatform: linux + openssl_version: 1.1.0g + + steps: + - template: ./posix-steps.yml + parameters: + targetBranch: $(System.PullRequest.TargetBranch) + + +- job: Windows_PR_Tests + displayName: Windows PR Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: vs2017-win2017 + + strategy: + matrix: + win32: + arch: win32 + buildOpt: + testRunTitle: '$(System.PullRequest.TargetBranch)-win32' + testRunPlatform: win32 + win64: + arch: amd64 + buildOpt: '-p x64' + testRunTitle: '$(System.PullRequest.TargetBranch)-win64' + testRunPlatform: win64 + maxParallel: 2 + + steps: + - template: ./windows-steps.yml + parameters: + targetBranch: $(System.PullRequest.TargetBranch) diff --git a/.azure-pipelines/prebuild-checks.yml b/.azure-pipelines/prebuild-checks.yml new file mode 100644 index 000000000000..30ff642d1267 --- /dev/null +++ b/.azure-pipelines/prebuild-checks.yml @@ -0,0 +1,36 @@ +steps: +- checkout: self + fetchDepth: 5 + +- script: echo "##vso[task.setvariable variable=diffTarget]HEAD~1" + displayName: Set default diff target + +- script: | + git fetch -q origin $(System.PullRequest.TargetBranch) + echo "##vso[task.setvariable variable=diffTarget]HEAD \$(git merge-base HEAD FETCH_HEAD)" + displayName: Fetch comparison tree + condition: and(succeeded(), variables['System.PullRequest.TargetBranch']) + +- script: | + if ! git diff --name-only $(diffTarget) | grep -qE '(\.rst$|^Doc|^Misc)' + then + echo "No docs were updated: docs.run=false" + echo "##vso[task.setvariable variable=run;isOutput=true]false" + else + echo "Docs were updated: docs.run=true" + echo "##vso[task.setvariable variable=run;isOutput=true]true" + fi + displayName: Detect documentation changes + name: docs + +- script: | + if ! git diff --name-only $(diffTarget) | grep -qvE '(\.rst$|^Doc|^Misc)' + then + echo "Only docs were updated: tests.run=false" + echo "##vso[task.setvariable variable=run;isOutput=true]false" + else + echo "Code was updated: tests.run=true" + echo "##vso[task.setvariable variable=run;isOutput=true]true" + fi + displayName: Detect source changes + name: tests diff --git a/.azure-pipelines/windows-steps.yml b/.azure-pipelines/windows-steps.yml new file mode 100644 index 000000000000..d8d5f1753a07 --- /dev/null +++ b/.azure-pipelines/windows-steps.yml @@ -0,0 +1,32 @@ +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- powershell: | + # Relocate build outputs outside of source directory to make cleaning faster + Write-Host '##vso[task.setvariable variable=Py_IntDir]$(Build.BinariesDirectory)\obj' + # UNDONE: Do not build to a different directory because of broken tests + Write-Host '##vso[task.setvariable variable=Py_OutDir]$(Build.SourcesDirectory)\PCbuild' + Write-Host '##vso[task.setvariable variable=EXTERNAL_DIR]$(Build.BinariesDirectory)\externals' + displayName: Update build locations + +- script: PCbuild\build.bat -e $(buildOpt) + displayName: 'Build CPython' + +- script: python.bat -m test.pythoninfo + displayName: 'Display build info' + +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" + displayName: 'Tests' + env: + PREFIX: $(Py_OutDir)\$(arch) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' + mergeTestResults: true + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) + condition: succeededOrFailed() diff --git a/.vsts/docs-release.yml b/.vsts/docs-release.yml deleted file mode 100644 index e90428a42494..000000000000 --- a/.vsts/docs-release.yml +++ /dev/null @@ -1,43 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted Linux Preview - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: sudo apt-get update && sudo apt-get install -qy --force-yes texlive-full - displayName: 'Install LaTeX' - -- task: UsePythonVersion at 0 - displayName: 'Use Python 3.6 or later' - inputs: - versionSpec: '>=3.6' - -- script: python -m pip install sphinx blurb python-docs-theme - displayName: 'Install build dependencies' - -- script: make dist PYTHON=python SPHINXBUILD='python -m sphinx' BLURB='python -m blurb' - workingDirectory: '$(build.sourcesDirectory)/Doc' - displayName: 'Build documentation' - -- task: PublishBuildArtifacts at 1 - displayName: 'Publish build' - inputs: - PathToPublish: '$(build.sourcesDirectory)/Doc/build' - ArtifactName: build - publishLocation: Container - -- task: PublishBuildArtifacts at 1 - displayName: 'Publish dist' - inputs: - PathToPublish: '$(build.sourcesDirectory)/Doc/dist' - ArtifactName: dist - publishLocation: Container diff --git a/.vsts/docs.yml b/.vsts/docs.yml deleted file mode 100644 index 0be07b31dfcc..000000000000 --- a/.vsts/docs.yml +++ /dev/null @@ -1,58 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - include: - - Doc/* - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qE '(\.rst$|^Doc|^Misc)' - then - echo "No docs were updated, stopping build process." - echo "##vso[task.setvariable variable=NoDocs]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -- task: UsePythonVersion at 0 - displayName: 'Use Python 3.6 or later' - inputs: - versionSpec: '>=3.6' - condition: and(succeeded(), ne(variables['NoDocs'], 'true')) - -- script: python -m pip install sphinx~=1.6.1 blurb python-docs-theme - displayName: 'Install build dependencies' - condition: and(succeeded(), ne(variables['NoDocs'], 'true')) - -- script: make check suspicious html PYTHON=python - workingDirectory: '$(build.sourcesDirectory)/Doc' - displayName: 'Build documentation' - condition: and(succeeded(), ne(variables['NoDocs'], 'true')) - -- task: PublishBuildArtifacts at 1 - displayName: 'Publish build' - condition: and(and(succeeded(), ne(variables['Build.Reason'], 'PullRequest')), ne(variables['NoDocs'], 'true')) - inputs: - PathToPublish: '$(build.sourcesDirectory)/Doc/build' - ArtifactName: build - publishLocation: Container diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml deleted file mode 100644 index 417125ae3a46..000000000000 --- a/.vsts/linux-buildbot.yml +++ /dev/null @@ -1,59 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Copy-pasted from linux-deps.yml until template support arrives - OPENSSL: 1.1.0g - OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" - - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -#- template: linux-deps.yml - -- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) - displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: ./.vsts/install_deps.sh - displayName: 'Install dependencies' -- script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux - displayName: 'python multissltests.py' - -- script: ./configure --with-pydebug - displayName: 'Configure CPython (debug)' - -- script: make -s -j4 - displayName: 'Build CPython' - -- script: make pythoninfo - displayName: 'Display build info' - -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" - displayName: 'Tests' - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(build.binariesDirectory)/test-results.xml' - mergeTestResults: true - testRunTitle: '$(build.sourceBranchName)-linux' - platform: linux - condition: succeededOrFailed() diff --git a/.vsts/linux-coverage.yml b/.vsts/linux-coverage.yml deleted file mode 100644 index cc03e4258ab4..000000000000 --- a/.vsts/linux-coverage.yml +++ /dev/null @@ -1,75 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Copy-pasted from linux-deps.yml until template support arrives - OPENSSL: 1.1.0g - OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qvE '(\.rst$|^Doc|^Misc)' - then - echo "Only docs were updated, stopping build process." - echo "##vso[task.setvariable variable=DocOnly]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -#- template: linux-deps.yml - -- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) - displayName: 'Add $(OPENSSL_DIR) to PATH' -- script: ./.vsts/install_deps.sh - displayName: 'Install dependencies' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) -- script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux - displayName: 'python multissltests.py' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - - -- script: ./configure --with-pydebug - displayName: 'Configure CPython (debug)' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make -s -j4 - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./python -m venv venv && ./venv/bin/python -m pip install -U coverage - displayName: 'Set up virtual environment' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./venv/bin/python -m test.pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: xvfb-run ./venv/bin/python -m coverage run --pylib -m test --fail-env-changed -uall,-cpu -x test_multiprocessing_fork -x test_multiprocessing_forkserver -x test_multiprocessing_spawn -x test_concurrent_futures - displayName: 'Tests with coverage' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: source ./venv/bin/activate && bash <(curl -s https://codecov.io/bash) - displayName: 'Publish code coverage results' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml deleted file mode 100644 index d11a4f06e4e1..000000000000 --- a/.vsts/linux-pr.yml +++ /dev/null @@ -1,84 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Copy-pasted from linux-deps.yml until template support arrives - OPENSSL: 1.1.0g - OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qvE '(\.rst$|^Doc|^Misc)' - then - echo "Only docs were updated, stopping build process." - echo "##vso[task.setvariable variable=DocOnly]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -#- template: linux-deps.yml - -- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) - displayName: 'Add $(OPENSSL_DIR) to PATH' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./.vsts/install_deps.sh - displayName: 'Install dependencies' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: python3 Tools/ssl/multissltests.py --steps=library --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) --system Linux - displayName: 'python multissltests.py' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - - -- script: ./configure --with-pydebug - displayName: 'Configure CPython (debug)' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make -s -j4 - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -# Run patchcheck and fail if anything is discovered -- script: ./python Tools/scripts/patchcheck.py --travis true - displayName: 'Run patchcheck.py' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" - displayName: 'Tests' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(build.binariesDirectory)/test-results.xml' - mergeTestResults: true - testRunTitle: '$(system.pullRequest.targetBranch)-linux' - platform: linux - condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml deleted file mode 100644 index 69b619e47577..000000000000 --- a/.vsts/macos-pr.yml +++ /dev/null @@ -1,64 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted macOS - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - changes = $(git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD)) - echo "Files changed:" - echo "$changes" - if ! echo "$changes" | grep -qvE '(\.rst$)|(^Doc)|(^Misc)' - then - echo "Only docs were updated, stopping build process." - echo "##vso[task.setvariable variable=DocOnly]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl --prefix=/opt/python-vsts - displayName: 'Configure CPython (debug)' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make -s -j4 - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" - displayName: 'Tests' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(build.binariesDirectory)/test-results.xml' - mergeTestResults: true - testRunTitle: '$(system.pullRequest.targetBranch)-macOS' - platform: macOS - condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/windows-buildbot.yml b/.vsts/windows-buildbot.yml deleted file mode 100644 index 15aebeda4077..000000000000 --- a/.vsts/windows-buildbot.yml +++ /dev/null @@ -1,58 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted VS2017 - parallel: 2 - matrix: - amd64: - buildOpt: -p x64 - outDirSuffix: amd64 - win32: - buildOpt: - outDirSuffix: win32 - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Relocate build outputs outside of source directory to make cleaning faster - Py_IntDir: $(Build.BinariesDirectory)\obj - # UNDONE: Do not build to a different directory because of broken tests - Py_OutDir: $(Build.SourcesDirectory)\PCbuild - EXTERNAL_DIR: $(Build.BinariesDirectory)\externals - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: PCbuild\build.bat -e $(buildOpt) - displayName: 'Build CPython' - -- script: python.bat -m test.pythoninfo - displayName: 'Display build info' - -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" - displayName: 'Tests' - env: - PREFIX: $(Py_OutDir)\$(outDirSuffix) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' - mergeTestResults: true - testRunTitle: '$(Build.SourceBranchName)-$(outDirSuffix)' - platform: $(outDirSuffix) - condition: succeededOrFailed() diff --git a/.vsts/windows-pr.yml b/.vsts/windows-pr.yml deleted file mode 100644 index 7134120d6414..000000000000 --- a/.vsts/windows-pr.yml +++ /dev/null @@ -1,70 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted VS2017 - parallel: 2 - matrix: - amd64: - buildOpt: -p x64 - outDirSuffix: amd64 - win32: - buildOpt: - outDirSuffix: win32 - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Relocate build outputs outside of source directory to make cleaning faster - Py_IntDir: $(Build.BinariesDirectory)\obj - # UNDONE: Do not build to a different directory because of broken tests - Py_OutDir: $(Build.SourcesDirectory)\PCbuild - EXTERNAL_DIR: $(Build.BinariesDirectory)\externals - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- powershell: | - git fetch -q origin $(System.PullRequest.TargetBranch) - if (-not (git diff --name-only HEAD (git merge-base HEAD FETCH_HEAD) | sls -NotMatch '(\.rst$)|(^Doc)|(^Misc)')) { - Write-Host 'Only docs were updated. Skipping build' - Write-Host '##vso[task.setvariable variable=DocOnly]true' - } - displayName: Detect doc-only changes - condition: and(succeeded(), variables['System.PullRequest.TargetBranch']) - -- script: PCbuild\build.bat -e $(buildOpt) - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: python.bat -m test.pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" - displayName: 'Tests' - env: - PREFIX: $(Py_OutDir)\$(outDirSuffix) - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' - mergeTestResults: true - testRunTitle: '$(System.PullRequest.TargetBranch)-$(outDirSuffix)' - platform: $(outDirSuffix) - condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py index 99aa89bdc8c9..42cceafdc9fb 100644 --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -435,18 +435,29 @@ def test_flushes(self): # Test flush() with the various options, using all the # different levels in order to provide more variations. sync_opt = ['Z_NO_FLUSH', 'Z_SYNC_FLUSH', 'Z_FULL_FLUSH', - 'Z_PARTIAL_FLUSH', 'Z_BLOCK'] + 'Z_PARTIAL_FLUSH'] + + ver = tuple(int(v) for v in zlib.ZLIB_RUNTIME_VERSION.split('.')) + # Z_BLOCK has a known failure prior to 1.2.5.3 + if ver >= (1, 2, 5, 3): + sync_opt.append('Z_BLOCK') + sync_opt = [getattr(zlib, opt) for opt in sync_opt if hasattr(zlib, opt)] data = HAMLET_SCENE * 8 for sync in sync_opt: for level in range(10): - obj = zlib.compressobj( level ) - a = obj.compress( data[:3000] ) - b = obj.flush( sync ) - c = obj.compress( data[3000:] ) - d = obj.flush() + try: + obj = zlib.compressobj( level ) + a = obj.compress( data[:3000] ) + b = obj.flush( sync ) + c = obj.compress( data[3000:] ) + d = obj.flush() + except: + print("Error for flush mode={}, level={}" + .format(sync, level)) + raise self.assertEqual(zlib.decompress(b''.join([a,b,c,d])), data, ("Decompress failed: flush " "mode=%i, level=%i") % (sync, level)) From webhook-mailer at python.org Mon Sep 24 08:44:00 2018 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 24 Sep 2018 12:44:00 -0000 Subject: [Python-checkins] Update Azure Pipelines badge (GH-9530) Message-ID: https://github.com/python/cpython/commit/75dd688adbe685a029721c9fcbe5ba002d0cebd1 commit: 75dd688adbe685a029721c9fcbe5ba002d0cebd1 branch: 3.7 author: Steve Dower committer: GitHub date: 2018-09-24T08:43:56-04:00 summary: Update Azure Pipelines badge (GH-9530) files: M README.rst diff --git a/README.rst b/README.rst index 377aa3dfce6f..86de8f200ea8 100644 --- a/README.rst +++ b/README.rst @@ -9,16 +9,8 @@ This is Python version 3.7.0+ :alt: CPython build status on Appveyor :target: https://ci.appveyor.com/project/python/cpython/branch/master -.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Linux-Buildbot?branchName=3.7&label=Linux - :alt: CPython build status on VSTS (Linux) - :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=6&branchName=3.7 - -.. image:: https://dev.azure.com/python/cpython/_apis/build/status/macOS-Buildbot?branchName=3.7&label=macOS - :alt: CPython build status on VSTS (macOS) - :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=5&branchName=3.7 - -.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Windows-Buildbot?branchName=3.7&label=Windows - :alt: CPython build status on VSTS (Windows) +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Azure%20Pipelines%20CI?branchName=3.7 + :alt: CPython build status on Azure Pipelines :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=4&branchName=3.7 .. image:: https://codecov.io/gh/python/cpython/branch/master/graph/badge.svg From webhook-mailer at python.org Mon Sep 24 08:45:06 2018 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 24 Sep 2018 12:45:06 -0000 Subject: [Python-checkins] Update Azure Pipelines badge (GH-9531) Message-ID: https://github.com/python/cpython/commit/a3a639879883167802511d0c7ab6c1f06a08e0b4 commit: a3a639879883167802511d0c7ab6c1f06a08e0b4 branch: 3.6 author: Steve Dower committer: GitHub date: 2018-09-24T08:45:03-04:00 summary: Update Azure Pipelines badge (GH-9531) files: M README.rst diff --git a/README.rst b/README.rst index a2c76ac71dee..f00dccea9597 100644 --- a/README.rst +++ b/README.rst @@ -9,16 +9,8 @@ This is Python version 3.6.6+ :alt: CPython build status on Appveyor :target: https://ci.appveyor.com/project/python/cpython/branch/3.6 -.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Linux-Buildbot?branchName=3.6&label=Linux - :alt: CPython build status on VSTS (Linux) - :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=6&branchName=3.6 - -.. image:: https://dev.azure.com/python/cpython/_apis/build/status/macOS-Buildbot?branchName=3.6&label=macOS - :alt: CPython build status on VSTS (macOS) - :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=5&branchName=3.6 - -.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Windows-Buildbot?branchName=3.6&label=Windows - :alt: CPython build status on VSTS (Windows) +.. image:: https://dev.azure.com/python/cpython/_apis/build/status/Azure%20Pipelines%20CI?branchName=3.6 + :alt: CPython build status on Azure Pipelines :target: https://dev.azure.com/python/cpython/_build/latest?definitionId=4&branchName=3.6 .. image:: https://codecov.io/gh/python/cpython/branch/3.6/graph/badge.svg From webhook-mailer at python.org Mon Sep 24 08:53:38 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 24 Sep 2018 12:53:38 -0000 Subject: [Python-checkins] [3.6] bpo-34582: Update syntax of Azure Pipelines builds (GH-9521) (GH-9528) Message-ID: https://github.com/python/cpython/commit/73653062b1ef76b879c01a09163db2cff6548c3c commit: 73653062b1ef76b879c01a09163db2cff6548c3c branch: 3.6 author: Steve Dower committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-24T05:53:35-07:00 summary: [3.6] bpo-34582: Update syntax of Azure Pipelines builds (GH-9521) (GH-9528) https://bugs.python.org/issue34582 files: A .azure-pipelines/ci.yml A .azure-pipelines/docker-steps.yml A .azure-pipelines/docs-steps.yml A .azure-pipelines/macos-steps.yml A .azure-pipelines/posix-deps.sh A .azure-pipelines/posix-steps.yml A .azure-pipelines/pr.yml A .azure-pipelines/prebuild-checks.yml A .azure-pipelines/windows-steps.yml D .vsts/docs-release.yml D .vsts/docs.yml D .vsts/install_deps.sh D .vsts/linux-buildbot.yml D .vsts/linux-coverage.yml D .vsts/linux-pr.yml D .vsts/macos-buildbot.yml D .vsts/macos-pr.yml D .vsts/windows-buildbot.yml D .vsts/windows-pr.yml M Lib/test/test_zlib.py diff --git a/.azure-pipelines/ci.yml b/.azure-pipelines/ci.yml new file mode 100644 index 000000000000..f56254e38024 --- /dev/null +++ b/.azure-pipelines/ci.yml @@ -0,0 +1,136 @@ +variables: + manylinux: false + coverage: false + +jobs: +- job: Prebuild + displayName: Pre-build checks + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./prebuild-checks.yml + + +- job: Docs_PR + displayName: Docs PR + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['docs.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./docs-steps.yml + parameters: + upload: true + + +- job: macOS_CI_Tests + displayName: macOS CI Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + variables: + testRunTitle: '$(build.sourceBranchName)-macos' + testRunPlatform: macos + + pool: + vmImage: xcode9-macos10.13 + + steps: + - template: ./macos-steps.yml + + +- job: Ubuntu_CI_Tests + displayName: Ubuntu CI Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(build.sourceBranchName)-linux' + testRunPlatform: linux + openssl_version: 1.1.0g + + steps: + - template: ./posix-steps.yml + + +- job: ManyLinux1_CI_Tests + displayName: ManyLinux1 CI Tests + dependsOn: Prebuild + condition: | + and( + and( + succeeded(), + eq(variables['manylinux'], 'true') + ), + eq(dependencies.Prebuild.outputs['tests.run'], 'true') + ) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(build.sourceBranchName)-manylinux1' + testRunPlatform: manylinux1 + imageName: 'dockcross/manylinux-x64' + + steps: + - template: ./docker-steps.yml + + +- job: Ubuntu_Coverage_CI_Tests + displayName: Ubuntu CI Tests (coverage) + dependsOn: Prebuild + condition: | + and( + and( + succeeded(), + eq(variables['coverage'], 'true') + ), + eq(dependencies.Prebuild.outputs['tests.run'], 'true') + ) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(Build.SourceBranchName)-linux-coverage' + testRunPlatform: linux-coverage + openssl_version: 1.1.0g + + steps: + - template: ./posix-steps.yml + parameters: + coverage: true + + +- job: Windows_CI_Tests + displayName: Windows CI Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: vs2017-win2017 + + strategy: + matrix: + win32: + arch: win32 + buildOpt: + testRunTitle: '$(Build.SourceBranchName)-win32' + testRunPlatform: win32 + win64: + arch: amd64 + buildOpt: '-p x64' + testRunTitle: '$(Build.SourceBranchName)-win64' + testRunPlatform: win64 + maxParallel: 2 + + steps: + - template: ./windows-steps.yml diff --git a/.azure-pipelines/docker-steps.yml b/.azure-pipelines/docker-steps.yml new file mode 100644 index 000000000000..ba4dfd72dd8b --- /dev/null +++ b/.azure-pipelines/docker-steps.yml @@ -0,0 +1,76 @@ +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- ${{ if ne(parameters.targetBranch, '') }}: + - script: | + git fetch -q origin ${{ parameters.targetbranch }} + if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qvE '(\.rst$|^Doc|^Misc)' + then + echo "Only docs were updated, stopping build process." + echo "##vso[task.setvariable variable=DocOnly]true" + exit + fi + displayName: Detect doc-only changes + +- task: docker at 0 + displayName: 'Configure CPython (debug)' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: './configure --with-pydebug' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: docker at 0 + displayName: 'Build CPython' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: 'make -s -j4' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: docker at 0 + displayName: 'Display build info' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: 'make pythoninfo' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: docker at 0 + displayName: 'Tests' + inputs: + action: 'Run an image' + imageName: $(imageName) + volumes: | + $(build.sourcesDirectory):/src + $(build.binariesDirectory):/build + workDir: '/src' + containerCommand: 'make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=/build/test-results.xml"' + detached: false + condition: and(succeeded(), ne(variables['DocOnly'], 'true')) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) + condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.azure-pipelines/docs-steps.yml b/.azure-pipelines/docs-steps.yml new file mode 100644 index 000000000000..c0404aebdcc5 --- /dev/null +++ b/.azure-pipelines/docs-steps.yml @@ -0,0 +1,46 @@ +parameters: + latex: false + upload: false + +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- task: UsePythonVersion at 0 + displayName: 'Use Python 3.6 or later' + inputs: + versionSpec: '>=3.6' + +- script: python -m pip install sphinx~=1.6.1 blurb python-docs-theme + displayName: 'Install build dependencies' + +- ${{ if ne(parameters.latex, 'true') }}: + - script: make check suspicious html PYTHON=python + workingDirectory: '$(build.sourcesDirectory)/Doc' + displayName: 'Build documentation' + +- ${{ if eq(parameters.latex, 'true') }}: + - script: sudo apt-get update && sudo apt-get install -qy --force-yes texlive-full + displayName: 'Install LaTeX' + + - script: make dist PYTHON=python SPHINXBUILD='python -m sphinx' BLURB='python -m blurb' + workingDirectory: '$(build.sourcesDirectory)/Doc' + displayName: 'Build documentation' + +- ${{ if eq(parameters.upload, 'true') }}: + - task: PublishBuildArtifacts at 1 + displayName: 'Publish docs' + + inputs: + PathToPublish: '$(build.sourcesDirectory)/Doc/build' + ArtifactName: docs + publishLocation: Container + + - ${{ if eq(parameters.latex, 'true') }}: + - task: PublishBuildArtifacts at 1 + displayName: 'Publish dist' + inputs: + PathToPublish: '$(build.sourcesDirectory)/Doc/dist' + ArtifactName: docs_dist + publishLocation: Container diff --git a/.vsts/macos-buildbot.yml b/.azure-pipelines/macos-steps.yml similarity index 57% rename from .vsts/macos-buildbot.yml rename to .azure-pipelines/macos-steps.yml index d9c190c46131..647081689454 100644 --- a/.vsts/macos-buildbot.yml +++ b/.azure-pipelines/macos-steps.yml @@ -1,30 +1,9 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted macOS - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -#variables: - steps: - checkout: self clean: true fetchDepth: 5 -- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl --prefix=/opt/python-vsts +- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl --prefix=/opt/python-azdev displayName: 'Configure CPython (debug)' - script: make -s -j4 @@ -41,6 +20,6 @@ steps: inputs: testResultsFiles: '$(build.binariesDirectory)/test-results.xml' mergeTestResults: true - testRunTitle: '$(build.sourceBranchName)-macOS' - platform: macOS + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) condition: succeededOrFailed() diff --git a/.vsts/install_deps.sh b/.azure-pipelines/posix-deps.sh similarity index 51% rename from .vsts/install_deps.sh rename to .azure-pipelines/posix-deps.sh index b1fa576047e6..a57210756601 100755 --- a/.vsts/install_deps.sh +++ b/.azure-pipelines/posix-deps.sh @@ -1,4 +1,4 @@ -sudo apt-get update || true +sudo apt-get update sudo apt-get -yq install \ build-essential \ @@ -17,3 +17,10 @@ sudo apt-get -yq install \ libffi-dev \ uuid-dev \ xvfb + +if [ ! -z "$1" ] +then + echo ##vso[task.prependpath]$PWD/multissl/openssl/$1 + echo ##vso[task.setvariable variable=OPENSSL_DIR]$PWD/multissl/openssl/$1 + python3 Tools/ssl/multissltests.py --steps=library --base-directory $PWD/multissl --openssl $1 --system Linux +fi diff --git a/.azure-pipelines/posix-steps.yml b/.azure-pipelines/posix-steps.yml new file mode 100644 index 000000000000..9fec9be8014f --- /dev/null +++ b/.azure-pipelines/posix-steps.yml @@ -0,0 +1,63 @@ +parameters: + coverage: false + +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- script: ./.azure-pipelines/posix-deps.sh $(openssl_version) + displayName: 'Install dependencies' + +- script: ./configure --with-pydebug + displayName: 'Configure CPython (debug)' + +- script: make -s -j4 + displayName: 'Build CPython' + +- ${{ if eq(parameters.coverage, 'true') }}: + - script: ./python -m venv venv && ./venv/bin/python -m pip install -U coverage + displayName: 'Set up virtual environment' + + - script: ./venv/bin/python -m test.pythoninfo + displayName: 'Display build info' + + - script: | + xvfb-run ./venv/bin/python -m coverage run --pylib -m test \ + --fail-env-changed \ + -uall,-cpu \ + --junit-xml=$(build.binariesDirectory)/test-results.xml" \ + -x test_multiprocessing_fork \ + -x test_multiprocessing_forkserver \ + -x test_multiprocessing_spawn \ + -x test_concurrent_futures + displayName: 'Tests with coverage' + + - script: ./venv/bin/python -m coverage xml + displayName: 'Generate coverage.xml' + + - script: source ./venv/bin/activate && bash <(curl -s https://codecov.io/bash) + displayName: 'Publish code coverage results' + + +- ${{ if ne(parameters.coverage, 'true') }}: + - script: make pythoninfo + displayName: 'Display build info' + + - script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" + displayName: 'Tests' + + +- script: ./python Tools/scripts/patchcheck.py --travis true + displayName: 'Run patchcheck.py' + condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) + + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(build.binariesDirectory)/test-results.xml' + mergeTestResults: true + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) + condition: succeededOrFailed() diff --git a/.azure-pipelines/pr.yml b/.azure-pipelines/pr.yml new file mode 100644 index 000000000000..653f55b69b13 --- /dev/null +++ b/.azure-pipelines/pr.yml @@ -0,0 +1,86 @@ +jobs: +- job: Prebuild + displayName: Pre-build checks + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./prebuild-checks.yml + + +- job: Docs_PR + displayName: Docs PR + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['docs.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + steps: + - template: ./docs-steps.yml + + +- job: macOS_PR_Tests + displayName: macOS PR Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + variables: + testRunTitle: '$(system.pullRequest.TargetBranch)-macos' + testRunPlatform: macos + + pool: + vmImage: xcode9-macos10.13 + + steps: + - template: ./macos-steps.yml + parameters: + targetBranch: $(System.PullRequest.TargetBranch) + + +- job: Ubuntu_PR_Tests + displayName: Ubuntu PR Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: ubuntu-16.04 + + variables: + testRunTitle: '$(system.pullRequest.TargetBranch)-linux' + testRunPlatform: linux + openssl_version: 1.1.0g + + steps: + - template: ./posix-steps.yml + parameters: + targetBranch: $(System.PullRequest.TargetBranch) + + +- job: Windows_PR_Tests + displayName: Windows PR Tests + dependsOn: Prebuild + condition: and(succeeded(), eq(dependencies.Prebuild.outputs['tests.run'], 'true')) + + pool: + vmImage: vs2017-win2017 + + strategy: + matrix: + win32: + arch: win32 + buildOpt: + testRunTitle: '$(System.PullRequest.TargetBranch)-win32' + testRunPlatform: win32 + win64: + arch: amd64 + buildOpt: '-p x64' + testRunTitle: '$(System.PullRequest.TargetBranch)-win64' + testRunPlatform: win64 + maxParallel: 2 + + steps: + - template: ./windows-steps.yml + parameters: + targetBranch: $(System.PullRequest.TargetBranch) diff --git a/.azure-pipelines/prebuild-checks.yml b/.azure-pipelines/prebuild-checks.yml new file mode 100644 index 000000000000..30ff642d1267 --- /dev/null +++ b/.azure-pipelines/prebuild-checks.yml @@ -0,0 +1,36 @@ +steps: +- checkout: self + fetchDepth: 5 + +- script: echo "##vso[task.setvariable variable=diffTarget]HEAD~1" + displayName: Set default diff target + +- script: | + git fetch -q origin $(System.PullRequest.TargetBranch) + echo "##vso[task.setvariable variable=diffTarget]HEAD \$(git merge-base HEAD FETCH_HEAD)" + displayName: Fetch comparison tree + condition: and(succeeded(), variables['System.PullRequest.TargetBranch']) + +- script: | + if ! git diff --name-only $(diffTarget) | grep -qE '(\.rst$|^Doc|^Misc)' + then + echo "No docs were updated: docs.run=false" + echo "##vso[task.setvariable variable=run;isOutput=true]false" + else + echo "Docs were updated: docs.run=true" + echo "##vso[task.setvariable variable=run;isOutput=true]true" + fi + displayName: Detect documentation changes + name: docs + +- script: | + if ! git diff --name-only $(diffTarget) | grep -qvE '(\.rst$|^Doc|^Misc)' + then + echo "Only docs were updated: tests.run=false" + echo "##vso[task.setvariable variable=run;isOutput=true]false" + else + echo "Code was updated: tests.run=true" + echo "##vso[task.setvariable variable=run;isOutput=true]true" + fi + displayName: Detect source changes + name: tests diff --git a/.azure-pipelines/windows-steps.yml b/.azure-pipelines/windows-steps.yml new file mode 100644 index 000000000000..d8d5f1753a07 --- /dev/null +++ b/.azure-pipelines/windows-steps.yml @@ -0,0 +1,32 @@ +steps: +- checkout: self + clean: true + fetchDepth: 5 + +- powershell: | + # Relocate build outputs outside of source directory to make cleaning faster + Write-Host '##vso[task.setvariable variable=Py_IntDir]$(Build.BinariesDirectory)\obj' + # UNDONE: Do not build to a different directory because of broken tests + Write-Host '##vso[task.setvariable variable=Py_OutDir]$(Build.SourcesDirectory)\PCbuild' + Write-Host '##vso[task.setvariable variable=EXTERNAL_DIR]$(Build.BinariesDirectory)\externals' + displayName: Update build locations + +- script: PCbuild\build.bat -e $(buildOpt) + displayName: 'Build CPython' + +- script: python.bat -m test.pythoninfo + displayName: 'Display build info' + +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" + displayName: 'Tests' + env: + PREFIX: $(Py_OutDir)\$(arch) + +- task: PublishTestResults at 2 + displayName: 'Publish Test Results' + inputs: + testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' + mergeTestResults: true + testRunTitle: $(testRunTitle) + platform: $(testRunPlatform) + condition: succeededOrFailed() diff --git a/.vsts/docs-release.yml b/.vsts/docs-release.yml deleted file mode 100644 index e90428a42494..000000000000 --- a/.vsts/docs-release.yml +++ /dev/null @@ -1,43 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted Linux Preview - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: sudo apt-get update && sudo apt-get install -qy --force-yes texlive-full - displayName: 'Install LaTeX' - -- task: UsePythonVersion at 0 - displayName: 'Use Python 3.6 or later' - inputs: - versionSpec: '>=3.6' - -- script: python -m pip install sphinx blurb python-docs-theme - displayName: 'Install build dependencies' - -- script: make dist PYTHON=python SPHINXBUILD='python -m sphinx' BLURB='python -m blurb' - workingDirectory: '$(build.sourcesDirectory)/Doc' - displayName: 'Build documentation' - -- task: PublishBuildArtifacts at 1 - displayName: 'Publish build' - inputs: - PathToPublish: '$(build.sourcesDirectory)/Doc/build' - ArtifactName: build - publishLocation: Container - -- task: PublishBuildArtifacts at 1 - displayName: 'Publish dist' - inputs: - PathToPublish: '$(build.sourcesDirectory)/Doc/dist' - ArtifactName: dist - publishLocation: Container diff --git a/.vsts/docs.yml b/.vsts/docs.yml deleted file mode 100644 index 0be07b31dfcc..000000000000 --- a/.vsts/docs.yml +++ /dev/null @@ -1,58 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - include: - - Doc/* - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qE '(\.rst$|^Doc|^Misc)' - then - echo "No docs were updated, stopping build process." - echo "##vso[task.setvariable variable=NoDocs]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -- task: UsePythonVersion at 0 - displayName: 'Use Python 3.6 or later' - inputs: - versionSpec: '>=3.6' - condition: and(succeeded(), ne(variables['NoDocs'], 'true')) - -- script: python -m pip install sphinx~=1.6.1 blurb python-docs-theme - displayName: 'Install build dependencies' - condition: and(succeeded(), ne(variables['NoDocs'], 'true')) - -- script: make check suspicious html PYTHON=python - workingDirectory: '$(build.sourcesDirectory)/Doc' - displayName: 'Build documentation' - condition: and(succeeded(), ne(variables['NoDocs'], 'true')) - -- task: PublishBuildArtifacts at 1 - displayName: 'Publish build' - condition: and(and(succeeded(), ne(variables['Build.Reason'], 'PullRequest')), ne(variables['NoDocs'], 'true')) - inputs: - PathToPublish: '$(build.sourcesDirectory)/Doc/build' - ArtifactName: build - publishLocation: Container diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml deleted file mode 100644 index bc5530cbaedc..000000000000 --- a/.vsts/linux-buildbot.yml +++ /dev/null @@ -1,50 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -#variables: - - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: ./.vsts/install_deps.sh - displayName: 'Install dependencies' - -- script: ./configure --with-pydebug - displayName: 'Configure CPython (debug)' - -- script: make -s -j4 - displayName: 'Build CPython' - -- script: make pythoninfo - displayName: 'Display build info' - -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" - displayName: 'Tests' - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(build.binariesDirectory)/test-results.xml' - mergeTestResults: true - testRunTitle: '$(build.sourceBranchName)-linux' - platform: linux - condition: succeededOrFailed() diff --git a/.vsts/linux-coverage.yml b/.vsts/linux-coverage.yml deleted file mode 100644 index 4eb75efc8040..000000000000 --- a/.vsts/linux-coverage.yml +++ /dev/null @@ -1,64 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qvE '(\.rst$|^Doc|^Misc)' - then - echo "Only docs were updated, stopping build process." - echo "##vso[task.setvariable variable=DocOnly]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -- script: ./.vsts/install_deps.sh - displayName: 'Install dependencies' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./configure --with-pydebug - displayName: 'Configure CPython (debug)' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make -s -j4 - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./python -m venv venv && ./venv/bin/python -m pip install -U coverage - displayName: 'Set up virtual environment' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./venv/bin/python -m test.pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: xvfb-run./venv/bin/python -m coverage run --pylib -m test -uall,-cpu -x test_multiprocessing_fork -x test_multiprocessing_forkserver -x test_multiprocessing_spawn -x test_concurrent_futures - displayName: 'Tests with coverage' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: source ./venv/bin/activate && bash <(curl -s https://codecov.io/bash) - displayName: 'Publish code coverage results' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml deleted file mode 100644 index cf810052f8e9..000000000000 --- a/.vsts/linux-pr.yml +++ /dev/null @@ -1,70 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: 'Hosted Ubuntu 1604' - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - if ! git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD) | grep -qvE '(\.rst$|^Doc|^Misc)' - then - echo "Only docs were updated, stopping build process." - echo "##vso[task.setvariable variable=DocOnly]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -- script: ./.vsts/install_deps.sh - displayName: 'Install dependencies' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: ./configure --with-pydebug - displayName: 'Configure CPython (debug)' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make -s -j4 - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -# Run patchcheck and fail if anything is discovered -- script: ./python Tools/scripts/patchcheck.py --travis true - displayName: 'Run patchcheck.py' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" - displayName: 'Tests' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(build.binariesDirectory)/test-results.xml' - mergeTestResults: true - testRunTitle: '$(system.pullRequest.targetBranch)-linux' - platform: linux - condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml deleted file mode 100644 index 69b619e47577..000000000000 --- a/.vsts/macos-pr.yml +++ /dev/null @@ -1,64 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted macOS - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -#variables: - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: | - git fetch -q origin $(system.pullRequest.targetBranch) - changes = $(git diff --name-only HEAD $(git merge-base HEAD FETCH_HEAD)) - echo "Files changed:" - echo "$changes" - if ! echo "$changes" | grep -qvE '(\.rst$)|(^Doc)|(^Misc)' - then - echo "Only docs were updated, stopping build process." - echo "##vso[task.setvariable variable=DocOnly]true" - exit - fi - displayName: Detect doc-only changes - condition: and(succeeded(), variables['system.pullRequest.targetBranch']) - -- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl --prefix=/opt/python-vsts - displayName: 'Configure CPython (debug)' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make -s -j4 - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: make buildbottest TESTOPTS="-j4 -uall,-cpu --junit-xml=$(build.binariesDirectory)/test-results.xml" - displayName: 'Tests' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(build.binariesDirectory)/test-results.xml' - mergeTestResults: true - testRunTitle: '$(system.pullRequest.targetBranch)-macOS' - platform: macOS - condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/.vsts/windows-buildbot.yml b/.vsts/windows-buildbot.yml deleted file mode 100644 index 15aebeda4077..000000000000 --- a/.vsts/windows-buildbot.yml +++ /dev/null @@ -1,58 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted VS2017 - parallel: 2 - matrix: - amd64: - buildOpt: -p x64 - outDirSuffix: amd64 - win32: - buildOpt: - outDirSuffix: win32 - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Relocate build outputs outside of source directory to make cleaning faster - Py_IntDir: $(Build.BinariesDirectory)\obj - # UNDONE: Do not build to a different directory because of broken tests - Py_OutDir: $(Build.SourcesDirectory)\PCbuild - EXTERNAL_DIR: $(Build.BinariesDirectory)\externals - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- script: PCbuild\build.bat -e $(buildOpt) - displayName: 'Build CPython' - -- script: python.bat -m test.pythoninfo - displayName: 'Display build info' - -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" - displayName: 'Tests' - env: - PREFIX: $(Py_OutDir)\$(outDirSuffix) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' - mergeTestResults: true - testRunTitle: '$(Build.SourceBranchName)-$(outDirSuffix)' - platform: $(outDirSuffix) - condition: succeededOrFailed() diff --git a/.vsts/windows-pr.yml b/.vsts/windows-pr.yml deleted file mode 100644 index 7134120d6414..000000000000 --- a/.vsts/windows-pr.yml +++ /dev/null @@ -1,70 +0,0 @@ -# Current docs for the syntax of this file are at: -# https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md - -name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) - -queue: - name: Hosted VS2017 - parallel: 2 - matrix: - amd64: - buildOpt: -p x64 - outDirSuffix: amd64 - win32: - buildOpt: - outDirSuffix: win32 - -trigger: - branches: - include: - - master - - 3.7 - - 3.6 - paths: - exclude: - - Doc/* - - Tools/* - -variables: - # Relocate build outputs outside of source directory to make cleaning faster - Py_IntDir: $(Build.BinariesDirectory)\obj - # UNDONE: Do not build to a different directory because of broken tests - Py_OutDir: $(Build.SourcesDirectory)\PCbuild - EXTERNAL_DIR: $(Build.BinariesDirectory)\externals - -steps: -- checkout: self - clean: true - fetchDepth: 5 - -- powershell: | - git fetch -q origin $(System.PullRequest.TargetBranch) - if (-not (git diff --name-only HEAD (git merge-base HEAD FETCH_HEAD) | sls -NotMatch '(\.rst$)|(^Doc)|(^Misc)')) { - Write-Host 'Only docs were updated. Skipping build' - Write-Host '##vso[task.setvariable variable=DocOnly]true' - } - displayName: Detect doc-only changes - condition: and(succeeded(), variables['System.PullRequest.TargetBranch']) - -- script: PCbuild\build.bat -e $(buildOpt) - displayName: 'Build CPython' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: python.bat -m test.pythoninfo - displayName: 'Display build info' - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 --junit-xml="$(Build.BinariesDirectory)\test-results.xml" - displayName: 'Tests' - env: - PREFIX: $(Py_OutDir)\$(outDirSuffix) - condition: and(succeeded(), ne(variables['DocOnly'], 'true')) - -- task: PublishTestResults at 2 - displayName: 'Publish Test Results' - inputs: - testResultsFiles: '$(Build.BinariesDirectory)\test-results.xml' - mergeTestResults: true - testRunTitle: '$(System.PullRequest.TargetBranch)-$(outDirSuffix)' - platform: $(outDirSuffix) - condition: and(succeededOrFailed(), ne(variables['DocOnly'], 'true')) diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py index 5b58c77630b6..b7170b4ff525 100644 --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -435,18 +435,29 @@ def test_flushes(self): # Test flush() with the various options, using all the # different levels in order to provide more variations. sync_opt = ['Z_NO_FLUSH', 'Z_SYNC_FLUSH', 'Z_FULL_FLUSH', - 'Z_PARTIAL_FLUSH', 'Z_BLOCK'] + 'Z_PARTIAL_FLUSH'] + + ver = tuple(int(v) for v in zlib.ZLIB_RUNTIME_VERSION.split('.')) + # Z_BLOCK has a known failure prior to 1.2.5.3 + if ver >= (1, 2, 5, 3): + sync_opt.append('Z_BLOCK') + sync_opt = [getattr(zlib, opt) for opt in sync_opt if hasattr(zlib, opt)] data = HAMLET_SCENE * 8 for sync in sync_opt: for level in range(10): - obj = zlib.compressobj( level ) - a = obj.compress( data[:3000] ) - b = obj.flush( sync ) - c = obj.compress( data[3000:] ) - d = obj.flush() + try: + obj = zlib.compressobj( level ) + a = obj.compress( data[:3000] ) + b = obj.flush( sync ) + c = obj.compress( data[3000:] ) + d = obj.flush() + except: + print("Error for flush mode={}, level={}" + .format(sync, level)) + raise self.assertEqual(zlib.decompress(b''.join([a,b,c,d])), data, ("Decompress failed: flush " "mode=%i, level=%i") % (sync, level)) From webhook-mailer at python.org Mon Sep 24 09:03:06 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 24 Sep 2018 13:03:06 -0000 Subject: [Python-checkins] bpo-34582: Fixes Python version used for patch check (GH-9532) Message-ID: https://github.com/python/cpython/commit/1b77f929f8d18e9a97335a8ec5e838e5be506568 commit: 1b77f929f8d18e9a97335a8ec5e838e5be506568 branch: master author: Steve Dower committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-24T06:03:02-07:00 summary: bpo-34582: Fixes Python version used for patch check (GH-9532) https://bugs.python.org/issue34582 files: M .azure-pipelines/posix-steps.yml diff --git a/.azure-pipelines/posix-steps.yml b/.azure-pipelines/posix-steps.yml index 429381a7e242..9fec9be8014f 100644 --- a/.azure-pipelines/posix-steps.yml +++ b/.azure-pipelines/posix-steps.yml @@ -48,7 +48,7 @@ steps: displayName: 'Tests' -- script: python Tools/scripts/patchcheck.py --travis true +- script: ./python Tools/scripts/patchcheck.py --travis true displayName: 'Run patchcheck.py' condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) From webhook-mailer at python.org Mon Sep 24 11:06:34 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 24 Sep 2018 15:06:34 -0000 Subject: [Python-checkins] bpo-34783: Add test_cmd_line_script.test_nonexisting_script() (GH-9535) Message-ID: https://github.com/python/cpython/commit/a46467ff198c42c8f34768c7be4b4562f6f44736 commit: a46467ff198c42c8f34768c7be4b4562f6f44736 branch: master author: Victor Stinner committer: GitHub date: 2018-09-24T08:06:29-07:00 summary: bpo-34783: Add test_cmd_line_script.test_nonexisting_script() (GH-9535) Make sure that "./python script.py" does not crash if the script file doesn't exist. files: M Lib/test/test_cmd_line_script.py diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index a9c1309ad6d3..2595ca98c7ad 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -630,6 +630,25 @@ def test_consistent_sys_path_for_module_execution(self): traceback_lines = stderr.decode().splitlines() self.assertIn("No module named script_pkg", traceback_lines[-1]) + def test_nonexisting_script(self): + # bpo-34783: "./python script.py" must not crash + # if the script file doesn't exist. + script = 'nonexistingscript.py' + self.assertFalse(os.path.exists(script)) + # Only test the base name, since the error message can use + # a relative path, whereas sys.executable can be an asolution path. + program = os.path.basename(sys.executable) + + proc = spawn_python(script, text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + out, err = proc.communicate() + # "./python" must be in the error message: + # "./python: can't open file (...)" + self.assertIn(program, err) + self.assertNotEqual(proc.returncode, 0) + + def test_main(): support.run_unittest(CmdLineTest) support.reap_children() From webhook-mailer at python.org Mon Sep 24 11:28:30 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 24 Sep 2018 15:28:30 -0000 Subject: [Python-checkins] bpo-34783: Add test_cmd_line_script.test_nonexisting_script() (GH-9535) Message-ID: https://github.com/python/cpython/commit/7a26222d7caa507fc46119ba9246f163502eb1fd commit: 7a26222d7caa507fc46119ba9246f163502eb1fd branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-24T08:28:23-07:00 summary: bpo-34783: Add test_cmd_line_script.test_nonexisting_script() (GH-9535) Make sure that "./python script.py" does not crash if the script file doesn't exist. (cherry picked from commit a46467ff198c42c8f34768c7be4b4562f6f44736) Co-authored-by: Victor Stinner files: M Lib/test/test_cmd_line_script.py diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index a9c1309ad6d3..2595ca98c7ad 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -630,6 +630,25 @@ def test_consistent_sys_path_for_module_execution(self): traceback_lines = stderr.decode().splitlines() self.assertIn("No module named script_pkg", traceback_lines[-1]) + def test_nonexisting_script(self): + # bpo-34783: "./python script.py" must not crash + # if the script file doesn't exist. + script = 'nonexistingscript.py' + self.assertFalse(os.path.exists(script)) + # Only test the base name, since the error message can use + # a relative path, whereas sys.executable can be an asolution path. + program = os.path.basename(sys.executable) + + proc = spawn_python(script, text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + out, err = proc.communicate() + # "./python" must be in the error message: + # "./python: can't open file (...)" + self.assertIn(program, err) + self.assertNotEqual(proc.returncode, 0) + + def test_main(): support.run_unittest(CmdLineTest) support.reap_children() From webhook-mailer at python.org Mon Sep 24 12:33:56 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 24 Sep 2018 16:33:56 -0000 Subject: [Python-checkins] [3.7]Documentation minor update related to 3.6 - 3.7 migration (GH-9501) Message-ID: https://github.com/python/cpython/commit/881ddffbff4514564c19e2ca9044373a4b076759 commit: 881ddffbff4514564c19e2ca9044373a4b076759 branch: 3.7 author: Christophe Nanteuil <35002064+christopheNan at users.noreply.github.com> committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-24T09:33:44-07:00 summary: [3.7]Documentation minor update related to 3.6 - 3.7 migration (GH-9501) Change version from 3.6 to 3.7 in the documentation when it addresses newcomers. original request from python/python-docs-fr#273 files: M Doc/tutorial/interpreter.rst M Doc/using/cmdline.rst M Doc/using/windows.rst diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst index a2766e8810a5..6812f6489136 100644 --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -24,11 +24,11 @@ Python guru or system administrator. (E.g., :file:`/usr/local/python` is a popular alternative location.) On Windows machines, the Python installation is usually placed in -:file:`C:\\Python36`, though you can change this when you're running the +:file:`C:\\Python37`, though you can change this when you're running the installer. To add this directory to your path, you can type the following command into the command prompt in a DOS box:: - set path=%path%;C:\python36 + set path=%path%;C:\python37 Typing an end-of-file character (:kbd:`Control-D` on Unix, :kbd:`Control-Z` on Windows) at the primary prompt causes the interpreter to exit with a zero exit diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 70e8d1afb42a..f25e02b48190 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -182,13 +182,13 @@ Generic options .. code-block:: none - Python 3.6.0b2+ + Python 3.7.0b2+ When given twice, print more information about the build, like: .. code-block:: none - Python 3.6.0b2+ (3.6:84a3c5003510+, Oct 26 2016, 02:33:55) + Python 3.7.0b2+ (3.7:0c076caaa8, Sep 22 2018, 12:04:24) [GCC 6.2.0 20161005] .. versionadded:: 3.6 diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index e93fa0a78d1f..3ed39e64ad9b 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -193,13 +193,13 @@ of available options is shown below. For example, to silently install a default, system-wide Python installation, you could use the following command (from an elevated command prompt):: - python-3.6.0.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0 + python-3.7.0.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0 To allow users to easily install a personal copy of Python without the test suite, you could provide a shortcut with the following command. This will display a simplified initial page and disallow customization:: - python-3.6.0.exe InstallAllUsers=0 Include_launcher=0 Include_test=0 + python-3.7.0.exe InstallAllUsers=0 Include_launcher=0 Include_test=0 SimpleInstall=1 SimpleInstallDescription="Just for me, no test suite." (Note that omitting the launcher also omits file associations, and is only @@ -236,13 +236,13 @@ where a large number of installations are going to be performed it is very useful to have a locally cached copy. Execute the following command from Command Prompt to download all possible -required files. Remember to substitute ``python-3.6.0.exe`` for the actual +required files. Remember to substitute ``python-3.7.0.exe`` for the actual name of your installer, and to create layouts in their own directories to avoid collisions between files with the same name. :: - python-3.6.0.exe /layout [optional target directory] + python-3.7.0.exe /layout [optional target directory] You may also specify the ``/quiet`` option to hide the progress display. @@ -349,7 +349,7 @@ To temporarily set environment variables, open Command Prompt and use the .. code-block:: doscon - C:\>set PATH=C:\Program Files\Python 3.6;%PATH% + C:\>set PATH=C:\Program Files\Python 3.7;%PATH% C:\>set PYTHONPATH=%PYTHONPATH%;C:\My_python_lib C:\>python @@ -422,7 +422,7 @@ of your Python installation, delimited by a semicolon from other entries. An example variable could look like this (assuming the first two entries already existed):: - C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files\Python 3.6 + C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files\Python 3.7 .. _launcher: From webhook-mailer at python.org Mon Sep 24 13:21:15 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 24 Sep 2018 17:21:15 -0000 Subject: [Python-checkins] bpo-34791: xml package obeys ignore env flags (GH-9544) Message-ID: https://github.com/python/cpython/commit/223e501fb9c2b6ae21b96054e20c4c31d94a5d96 commit: 223e501fb9c2b6ae21b96054e20c4c31d94a5d96 branch: master author: Christian Heimes committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-24T10:21:12-07:00 summary: bpo-34791: xml package obeys ignore env flags (GH-9544) The xml.sax and xml.dom.domreg modules now obey sys.flags.ignore_environment. Signed-off-by: Christian Heimes https://bugs.python.org/issue34791 files: A Misc/NEWS.d/next/Security/2018-09-24-18-49-25.bpo-34791.78GmIG.rst M Lib/xml/dom/domreg.py M Lib/xml/sax/__init__.py diff --git a/Lib/xml/dom/domreg.py b/Lib/xml/dom/domreg.py index 8c3d901acb0b..69c17eebb265 100644 --- a/Lib/xml/dom/domreg.py +++ b/Lib/xml/dom/domreg.py @@ -6,6 +6,8 @@ # should be published by posting to xml-sig at python.org, and are # subsequently recorded in this file. +import sys + well_known_implementations = { 'minidom':'xml.dom.minidom', '4DOM': 'xml.dom.DOMImplementation', @@ -55,7 +57,7 @@ def getDOMImplementation(name=None, features=()): return mod.getDOMImplementation() elif name: return registered[name]() - elif "PYTHON_DOM" in os.environ: + elif not sys.flags.ignore_environment and "PYTHON_DOM" in os.environ: return getDOMImplementation(name = os.environ["PYTHON_DOM"]) # User did not specify a name, try implementations in arbitrary diff --git a/Lib/xml/sax/__init__.py b/Lib/xml/sax/__init__.py index ef67ae67a6bd..13f6cf58d0d2 100644 --- a/Lib/xml/sax/__init__.py +++ b/Lib/xml/sax/__init__.py @@ -58,7 +58,7 @@ def parseString(string, handler, errorHandler=ErrorHandler()): import xml.sax.expatreader import os, sys -if "PY_SAX_PARSER" in os.environ: +if not sys.flags.ignore_environment and "PY_SAX_PARSER" in os.environ: default_parser_list = os.environ["PY_SAX_PARSER"].split(",") del os diff --git a/Misc/NEWS.d/next/Security/2018-09-24-18-49-25.bpo-34791.78GmIG.rst b/Misc/NEWS.d/next/Security/2018-09-24-18-49-25.bpo-34791.78GmIG.rst new file mode 100644 index 000000000000..afb59f8cb0eb --- /dev/null +++ b/Misc/NEWS.d/next/Security/2018-09-24-18-49-25.bpo-34791.78GmIG.rst @@ -0,0 +1,3 @@ +The xml.sax and xml.dom.domreg no longer use environment variables to +override parser implementations when sys.flags.ignore_environment is set by +-E or -I arguments. From webhook-mailer at python.org Mon Sep 24 17:12:54 2018 From: webhook-mailer at python.org (Guido van Rossum) Date: Mon, 24 Sep 2018 21:12:54 -0000 Subject: [Python-checkins] bpo-34683: Make SyntaxError column offsets consistently 1-indexed (gh-9338) Message-ID: https://github.com/python/cpython/commit/025eb98dc0c1dc27404df6c544fc2944e0fa9f3a commit: 025eb98dc0c1dc27404df6c544fc2944e0fa9f3a branch: master author: Ammar Askar committer: Guido van Rossum date: 2018-09-24T14:12:49-07:00 summary: bpo-34683: Make SyntaxError column offsets consistently 1-indexed (gh-9338) Also point to start of tokens in parsing errors. Fixes bpo-34683 files: A Misc/NEWS.d/next/Core and Builtins/2018-09-15-19-32-34.bpo-34683.msCiQE.rst M Lib/test/test_exceptions.py M Lib/test/test_future.py M Lib/test/test_global.py M Lib/test/test_support.py M Lib/test/test_symtable.py M Parser/parsetok.c M Python/ast.c M Python/compile.c M Python/future.c M Python/symtable.c diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py index 2a9ec706467f..535324880089 100644 --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -187,6 +187,45 @@ def check(src, lineno, offset): check('def spam():\n print(1)\n print(2)', 3, 10) check('Python = "Python" +', 1, 20) check('Python = "\u1e54\xfd\u0163\u0125\xf2\xf1" +', 1, 20) + check('x = "a', 1, 7) + check('lambda x: x = 2', 1, 1) + + # Errors thrown by compile.c + check('class foo:return 1', 1, 11) + check('def f():\n continue', 2, 3) + check('def f():\n break', 2, 3) + check('try:\n pass\nexcept:\n pass\nexcept ValueError:\n pass', 2, 3) + + # Errors thrown by tokenizer.c + check('(0x+1)', 1, 3) + check('x = 0xI', 1, 6) + check('0010 + 2', 1, 4) + check('x = 32e-+4', 1, 8) + check('x = 0o9', 1, 6) + + # Errors thrown by symtable.c + check('x = [(yield i) for i in range(3)]', 1, 6) + check('def f():\n from _ import *', 1, 1) + check('def f(x, x):\n pass', 1, 1) + check('def f(x):\n nonlocal x', 2, 3) + check('def f(x):\n x = 1\n global x', 3, 3) + check('nonlocal x', 1, 1) + check('def f():\n global x\n nonlocal x', 2, 3) + + # Errors thrown by ast.c + check('for 1 in []: pass', 1, 5) + check('def f(*):\n pass', 1, 7) + check('[*x for x in xs]', 1, 2) + check('def f():\n x, y: int', 2, 3) + check('(yield i) = 2', 1, 1) + check('foo(x for x in range(10), 100)', 1, 5) + check('foo(1=2)', 1, 5) + + # Errors thrown by future.c + check('from __future__ import doesnt_exist', 1, 1) + check('from __future__ import braces', 1, 1) + check('x=1\nfrom __future__ import division', 2, 1) + @cpython_only def testSettingException(self): diff --git a/Lib/test/test_future.py b/Lib/test/test_future.py index 61cd63479d85..660701b4b3d8 100644 --- a/Lib/test/test_future.py +++ b/Lib/test/test_future.py @@ -15,7 +15,7 @@ def get_error_location(msg): class FutureTest(unittest.TestCase): - def check_syntax_error(self, err, basename, lineno, offset=0): + def check_syntax_error(self, err, basename, lineno, offset=1): self.assertIn('%s.py, line %d' % (basename, lineno), str(err)) self.assertEqual(os.path.basename(err.filename), basename + '.py') self.assertEqual(err.lineno, lineno) @@ -68,12 +68,12 @@ def test_badfuture8(self): def test_badfuture9(self): with self.assertRaises(SyntaxError) as cm: from test import badsyntax_future9 - self.check_syntax_error(cm.exception, "badsyntax_future9", 3, 0) + self.check_syntax_error(cm.exception, "badsyntax_future9", 3) def test_badfuture10(self): with self.assertRaises(SyntaxError) as cm: from test import badsyntax_future10 - self.check_syntax_error(cm.exception, "badsyntax_future10", 3, 0) + self.check_syntax_error(cm.exception, "badsyntax_future10", 3) def test_parserhack(self): # test that the parser.c::future_hack function works as expected diff --git a/Lib/test/test_global.py b/Lib/test/test_global.py index 9eeccf12506f..8159602be98e 100644 --- a/Lib/test/test_global.py +++ b/Lib/test/test_global.py @@ -24,7 +24,7 @@ def wrong1(): global a global b """ - check_syntax_error(self, prog_text_1, lineno=4, offset=4) + check_syntax_error(self, prog_text_1, lineno=4, offset=5) def test2(self): prog_text_2 = """\ @@ -32,7 +32,7 @@ def wrong2(): print(x) global x """ - check_syntax_error(self, prog_text_2, lineno=3, offset=4) + check_syntax_error(self, prog_text_2, lineno=3, offset=5) def test3(self): prog_text_3 = """\ @@ -41,7 +41,7 @@ def wrong3(): x = 2 global x """ - check_syntax_error(self, prog_text_3, lineno=4, offset=4) + check_syntax_error(self, prog_text_3, lineno=4, offset=5) def test4(self): prog_text_4 = """\ diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py index 7870e940a46e..171e28aaa802 100644 --- a/Lib/test/test_support.py +++ b/Lib/test/test_support.py @@ -286,7 +286,7 @@ def test_make_bad_fd(self): self.assertEqual(cm.exception.errno, errno.EBADF) def test_check_syntax_error(self): - support.check_syntax_error(self, "def class", lineno=1, offset=9) + support.check_syntax_error(self, "def class", lineno=1, offset=5) with self.assertRaises(AssertionError): support.check_syntax_error(self, "x=1") diff --git a/Lib/test/test_symtable.py b/Lib/test/test_symtable.py index dfaee173ef72..2cd735bdc508 100644 --- a/Lib/test/test_symtable.py +++ b/Lib/test/test_symtable.py @@ -169,7 +169,7 @@ def checkfilename(brokencode, offset): else: self.fail("no SyntaxError for %r" % (brokencode,)) checkfilename("def f(x): foo)(", 14) # parse-time - checkfilename("def f(x): global x", 10) # symtable-build-time + checkfilename("def f(x): global x", 11) # symtable-build-time symtable.symtable("pass", b"spam", "exec") with self.assertWarns(DeprecationWarning), \ self.assertRaises(TypeError): diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-15-19-32-34.bpo-34683.msCiQE.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-15-19-32-34.bpo-34683.msCiQE.rst new file mode 100644 index 000000000000..43bf61c5b86b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-15-19-32-34.bpo-34683.msCiQE.rst @@ -0,0 +1 @@ +Fixed a bug where some SyntaxError error pointed to locations that were off-by-one. diff --git a/Parser/parsetok.c b/Parser/parsetok.c index a1580e6751e4..fc878d89d563 100644 --- a/Parser/parsetok.c +++ b/Parser/parsetok.c @@ -187,6 +187,7 @@ parsetok(struct tok_state *tok, grammar *g, int start, perrdetail *err_ret, parser_state *ps; node *n; int started = 0; + int col_offset; if ((ps = PyParser_New(g, start)) == NULL) { err_ret->error = E_NOMEM; @@ -203,7 +204,7 @@ parsetok(struct tok_state *tok, grammar *g, int start, perrdetail *err_ret, int type; size_t len; char *str; - int col_offset; + col_offset = -1; type = PyTokenizer_Get(tok, &a, &b); if (type == ERRORTOKEN) { @@ -321,7 +322,10 @@ parsetok(struct tok_state *tok, grammar *g, int start, perrdetail *err_ret, if (tok->buf != NULL) { size_t len; assert(tok->cur - tok->buf < INT_MAX); - err_ret->offset = (int)(tok->cur - tok->buf); + /* if we've managed to parse a token, point the offset to its start, + * else use the current reading position of the tokenizer + */ + err_ret->offset = col_offset != -1 ? col_offset + 1 : ((int)(tok->cur - tok->buf)); len = tok->inp - tok->buf; err_ret->text = (char *) PyObject_MALLOC(len + 1); if (err_ret->text != NULL) { diff --git a/Python/ast.c b/Python/ast.c index b93eb88dae7f..b2fcb219752d 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -683,7 +683,7 @@ ast_error(struct compiling *c, const node *n, const char *errmsg) Py_INCREF(Py_None); loc = Py_None; } - tmp = Py_BuildValue("(OiiN)", c->c_filename, LINENO(n), n->n_col_offset, loc); + tmp = Py_BuildValue("(OiiN)", c->c_filename, LINENO(n), n->n_col_offset + 1, loc); if (!tmp) return 0; errstr = PyUnicode_FromString(errmsg); diff --git a/Python/compile.c b/Python/compile.c index 707da79ab662..3a45804580ed 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -4830,7 +4830,7 @@ compiler_error(struct compiler *c, const char *errstr) loc = Py_None; } u = Py_BuildValue("(OiiO)", c->c_filename, c->u->u_lineno, - c->u->u_col_offset, loc); + c->u->u_col_offset + 1, loc); if (!u) goto exit; v = Py_BuildValue("(zO)", errstr, u); diff --git a/Python/future.c b/Python/future.c index 4ea6827723bf..1663a38a6fdb 100644 --- a/Python/future.c +++ b/Python/future.c @@ -48,12 +48,12 @@ future_check_features(PyFutureFeatures *ff, stmt_ty s, PyObject *filename) } else if (strcmp(feature, "braces") == 0) { PyErr_SetString(PyExc_SyntaxError, "not a chance"); - PyErr_SyntaxLocationObject(filename, s->lineno, s->col_offset); + PyErr_SyntaxLocationObject(filename, s->lineno, s->col_offset + 1); return 0; } else { PyErr_Format(PyExc_SyntaxError, UNDEFINED_FUTURE_FEATURE, feature); - PyErr_SyntaxLocationObject(filename, s->lineno, s->col_offset); + PyErr_SyntaxLocationObject(filename, s->lineno, s->col_offset + 1); return 0; } } diff --git a/Python/symtable.c b/Python/symtable.c index 3e8c6f5dae30..e7216147a8a4 100644 --- a/Python/symtable.c +++ b/Python/symtable.c @@ -390,7 +390,7 @@ error_at_directive(PySTEntryObject *ste, PyObject *name) if (PyUnicode_Compare(PyTuple_GET_ITEM(data, 0), name) == 0) { PyErr_SyntaxLocationObject(ste->ste_table->st_filename, PyLong_AsLong(PyTuple_GET_ITEM(data, 1)), - PyLong_AsLong(PyTuple_GET_ITEM(data, 2))); + PyLong_AsLong(PyTuple_GET_ITEM(data, 2)) + 1); return 0; } @@ -990,7 +990,7 @@ symtable_add_def(struct symtable *st, PyObject *name, int flag) PyErr_Format(PyExc_SyntaxError, DUPLICATE_ARGUMENT, name); PyErr_SyntaxLocationObject(st->st_filename, st->st_cur->ste_lineno, - st->st_cur->ste_col_offset); + st->st_cur->ste_col_offset + 1); goto error; } val |= flag; @@ -1179,7 +1179,7 @@ symtable_visit_stmt(struct symtable *st, stmt_ty s) e_name->v.Name.id); PyErr_SyntaxLocationObject(st->st_filename, s->lineno, - s->col_offset); + s->col_offset + 1); VISIT_QUIT(st, 0); } if (s->v.AnnAssign.simple && @@ -1274,7 +1274,7 @@ symtable_visit_stmt(struct symtable *st, stmt_ty s) msg, name); PyErr_SyntaxLocationObject(st->st_filename, s->lineno, - s->col_offset); + s->col_offset + 1); VISIT_QUIT(st, 0); } if (!symtable_add_def(st, name, DEF_GLOBAL)) @@ -1306,7 +1306,7 @@ symtable_visit_stmt(struct symtable *st, stmt_ty s) PyErr_Format(PyExc_SyntaxError, msg, name); PyErr_SyntaxLocationObject(st->st_filename, s->lineno, - s->col_offset); + s->col_offset + 1); VISIT_QUIT(st, 0); } if (!symtable_add_def(st, name, DEF_NONLOCAL)) @@ -1645,7 +1645,7 @@ symtable_visit_alias(struct symtable *st, alias_ty a) int lineno = st->st_cur->ste_lineno; int col_offset = st->st_cur->ste_col_offset; PyErr_SetString(PyExc_SyntaxError, IMPORT_STAR_WARNING); - PyErr_SyntaxLocationObject(st->st_filename, lineno, col_offset); + PyErr_SyntaxLocationObject(st->st_filename, lineno, col_offset + 1); Py_DECREF(store_name); return 0; } @@ -1736,7 +1736,7 @@ symtable_handle_comprehension(struct symtable *st, expr_ty e, "'yield' inside generator expression"); PyErr_SyntaxLocationObject(st->st_filename, st->st_cur->ste_lineno, - st->st_cur->ste_col_offset); + st->st_cur->ste_col_offset + 1); symtable_exit_block(st, (void *)e); return 0; } From webhook-mailer at python.org Mon Sep 24 19:15:44 2018 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Mon, 24 Sep 2018 23:15:44 -0000 Subject: [Python-checkins] bpo-34162: Add entry for idlelib/NEWS.txt already included for 3.6/3.7 (GH-9549) Message-ID: https://github.com/python/cpython/commit/5a606674edbf4a94370ee55b0906dd9bec83abb0 commit: 5a606674edbf4a94370ee55b0906dd9bec83abb0 branch: master author: Terry Jan Reedy committer: GitHub date: 2018-09-24T19:15:40-04:00 summary: bpo-34162: Add entry for idlelib/NEWS.txt already included for 3.6/3.7 (GH-9549) files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 6b08408da610..273f7c49e5f4 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,11 @@ Released on 2019-10-20? ====================================== +bpo-33975: Avoid small type when running htests. +Since part of the purpose of human-viewed tests is to determine that +widgets look right, it is important that they look the same for testing +as when running IDLE. + bpo-33905: Add test for idlelib.stackview.StackBrowser. bpo-33924: Change mainmenu.menudefs key 'windows' to 'window'. From webhook-mailer at python.org Mon Sep 24 19:43:56 2018 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Mon, 24 Sep 2018 23:43:56 -0000 Subject: [Python-checkins] [2.7] bpo-34162: Update 2.7 idlelib/NEWS.txt to 2018-9-24 (GH-9550) Message-ID: https://github.com/python/cpython/commit/65fa1264a308a94e24afa76f41a68708a346738f commit: 65fa1264a308a94e24afa76f41a68708a346738f branch: 2.7 author: Terry Jan Reedy committer: GitHub date: 2018-09-24T19:43:54-04:00 summary: [2.7] bpo-34162: Update 2.7 idlelib/NEWS.txt to 2018-9-24 (GH-9550) files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 35e2a7ecce2b..257e28833e98 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -1,6 +1,19 @@ +Since 2.7.13, only severe bugs are fixed on the 2.7 branch. + +What's New in IDLE 2.7.16? +========================== +*Release date: 2019-01-01?* + +bpo-34275: Make calltips always visible on Mac. +Patch by Kevin Walzer. + +bpo-34120: Fix freezing after closing some dialogs on Mac. +This is one of multiple regressions from using newer tcl/tk. + + What's New in IDLE 2.7.13? ========================== -*Release date: 2017-01-01?* +*Release date: 2016-12-17* - Issue #27854: Make Help => IDLE Help work again on Windows. Include idlelib/help.html in 2.7 Windows installer. @@ -35,7 +48,7 @@ What's New in IDLE 2.7.13? What's New in IDLE 2.7.12? ========================== -*Release date: 2015-06-25* +*Release date: 2016-06-25* - Issue #5124: Paste with text selected now replaces the selection on X11. This matches how paste works on Windows, Mac, most modern Linux apps, From webhook-mailer at python.org Mon Sep 24 20:11:49 2018 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 25 Sep 2018 00:11:49 -0000 Subject: [Python-checkins] bpo-34162: Update idlelib/NEWS.txt to 2018-9-24. (GH-9551) Message-ID: https://github.com/python/cpython/commit/16fba6231458cc1ff82db8695ee772b74131cc83 commit: 16fba6231458cc1ff82db8695ee772b74131cc83 branch: master author: Terry Jan Reedy committer: GitHub date: 2018-09-24T20:11:45-04:00 summary: bpo-34162: Update idlelib/NEWS.txt to 2018-9-24. (GH-9551) files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 273f7c49e5f4..04e56c5a4456 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,10 +3,23 @@ Released on 2019-10-20? ====================================== +bpo-34548: Use configured color theme for read-only text views. + +bpo-33839: Refactor ToolTip and CallTip classes; add documentation +and tests. + +bpo-34047: Fix mouse wheel scrolling direction on macOS. + +bpo-34275: Make calltips always visible on Mac. +Patch by Kevin Walzer. + +bpo-34120: Fix freezing after closing some dialogs on Mac. +This is one of multiple regressions from using newer tcl/tk. + bpo-33975: Avoid small type when running htests. Since part of the purpose of human-viewed tests is to determine that -widgets look right, it is important that they look the same for testing -as when running IDLE. +widgets look right, it is important that they look the same for +testing as when running IDLE. bpo-33905: Add test for idlelib.stackview.StackBrowser. From webhook-mailer at python.org Mon Sep 24 20:23:13 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 00:23:13 -0000 Subject: [Python-checkins] bpo-34162: Update idlelib/NEWS.txt to 2018-9-24. (GH-9551) Message-ID: https://github.com/python/cpython/commit/65cc60b3687d33bfeda60b29a49a0708d4932b7b commit: 65cc60b3687d33bfeda60b29a49a0708d4932b7b branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-24T17:23:07-07:00 summary: bpo-34162: Update idlelib/NEWS.txt to 2018-9-24. (GH-9551) (cherry picked from commit 16fba6231458cc1ff82db8695ee772b74131cc83) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index e6b2ac2b01ff..c8be6d03e126 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,10 +3,23 @@ Released on 2018-07-31? ====================================== +bpo-34548: Use configured color theme for read-only text views. + +bpo-33839: Refactor ToolTip and CallTip classes; add documentation +and tests. + +bpo-34047: Fix mouse wheel scrolling direction on macOS. + +bpo-34275: Make calltips always visible on Mac. +Patch by Kevin Walzer. + +bpo-34120: Fix freezing after closing some dialogs on Mac. +This is one of multiple regressions from using newer tcl/tk. + bpo-33975: Avoid small type when running htests. Since part of the purpose of human-viewed tests is to determine that -widgets look right, it is important that they look the same for testing -as when running IDLE. +widgets look right, it is important that they look the same for +testing as when running IDLE. bpo-33905: Add test for idlelib.stackview.StackBrowser. From webhook-mailer at python.org Mon Sep 24 20:35:29 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 00:35:29 -0000 Subject: [Python-checkins] bpo-34162: Update idlelib/NEWS.txt to 2018-9-24. (GH-9551) Message-ID: https://github.com/python/cpython/commit/1ab9dfcfbe99d576b41209ca8ddbcacb682e4495 commit: 1ab9dfcfbe99d576b41209ca8ddbcacb682e4495 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-24T17:35:21-07:00 summary: bpo-34162: Update idlelib/NEWS.txt to 2018-9-24. (GH-9551) (cherry picked from commit 16fba6231458cc1ff82db8695ee772b74131cc83) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index ef4b49f3ef04..085060d53b5c 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,10 +3,23 @@ Released on 2018-09-24? ====================================== +bpo-34548: Use configured color theme for read-only text views. + +bpo-33839: Refactor ToolTip and CallTip classes; add documentation +and tests. + +bpo-34047: Fix mouse wheel scrolling direction on macOS. + +bpo-34275: Make calltips always visible on Mac. +Patch by Kevin Walzer. + +bpo-34120: Fix freezing after closing some dialogs on Mac. +This is one of multiple regressions from using newer tcl/tk. + bpo-33975: Avoid small type when running htests. Since part of the purpose of human-viewed tests is to determine that -widgets look right, it is important that they look the same for testing -as when running IDLE. +widgets look right, it is important that they look the same for +testing as when running IDLE. bpo-33905: Add test for idlelib.stackview.StackBrowser. From webhook-mailer at python.org Mon Sep 24 23:30:35 2018 From: webhook-mailer at python.org (Xiang Zhang) Date: Tue, 25 Sep 2018 03:30:35 -0000 Subject: [Python-checkins] Fix wrong exception reference: BrokenThreadPool -> BrokenProcessPool (GH-9533) Message-ID: https://github.com/python/cpython/commit/b60b4683f6c995e9205f68439023c80a0b628f39 commit: b60b4683f6c995e9205f68439023c80a0b628f39 branch: master author: Joni K?h?r? committer: Xiang Zhang date: 2018-09-25T11:30:25+08:00 summary: Fix wrong exception reference: BrokenThreadPool -> BrokenProcessPool (GH-9533) files: M Doc/library/concurrent.futures.rst diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst index 6934acc7f88e..47ca6b38f84e 100644 --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -223,7 +223,7 @@ to a :class:`ProcessPoolExecutor` will result in deadlock. *initializer* is an optional callable that is called at the start of each worker process; *initargs* is a tuple of arguments passed to the initializer. Should *initializer* raise an exception, all currently - pending jobs will raise a :exc:`~concurrent.futures.thread.BrokenThreadPool`, + pending jobs will raise a :exc:`~concurrent.futures.process.BrokenProcessPool`, as well any attempt to submit more jobs to the pool. .. versionchanged:: 3.3 From webhook-mailer at python.org Mon Sep 24 23:35:49 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 03:35:49 -0000 Subject: [Python-checkins] Fix wrong exception reference: BrokenThreadPool -> BrokenProcessPool (GH-9533) Message-ID: https://github.com/python/cpython/commit/bbdf8723324e31675f298dd273733cc13e1518df commit: bbdf8723324e31675f298dd273733cc13e1518df branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-24T20:35:46-07:00 summary: Fix wrong exception reference: BrokenThreadPool -> BrokenProcessPool (GH-9533) (cherry picked from commit b60b4683f6c995e9205f68439023c80a0b628f39) Co-authored-by: Joni K?h?r? files: M Doc/library/concurrent.futures.rst diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst index 707d24dc2529..b1b086b442de 100644 --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -223,7 +223,7 @@ to a :class:`ProcessPoolExecutor` will result in deadlock. *initializer* is an optional callable that is called at the start of each worker process; *initargs* is a tuple of arguments passed to the initializer. Should *initializer* raise an exception, all currently - pending jobs will raise a :exc:`~concurrent.futures.thread.BrokenThreadPool`, + pending jobs will raise a :exc:`~concurrent.futures.process.BrokenProcessPool`, as well any attempt to submit more jobs to the pool. .. versionchanged:: 3.3 From webhook-mailer at python.org Tue Sep 25 00:25:28 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 04:25:28 -0000 Subject: [Python-checkins] bpo-34770: Fix a possible null pointer dereference in pyshellext.cpp (GH-9497) Message-ID: https://github.com/python/cpython/commit/f6c8007a29b95b3ea3ca687a9b4924769a696328 commit: f6c8007a29b95b3ea3ca687a9b4924769a696328 branch: master author: Zackery Spytz committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-24T21:25:23-07:00 summary: bpo-34770: Fix a possible null pointer dereference in pyshellext.cpp (GH-9497) The GlobalLock() call in UpdateDropDescription() was not checked for failure. https://bugs.python.org/issue34770 files: A Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst M PC/pyshellext.cpp diff --git a/Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst b/Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst new file mode 100644 index 000000000000..5e4ba8868e84 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst @@ -0,0 +1 @@ +Fix a possible null pointer dereference in pyshellext.cpp. diff --git a/PC/pyshellext.cpp b/PC/pyshellext.cpp index 04fe61e89618..019880264bee 100644 --- a/PC/pyshellext.cpp +++ b/PC/pyshellext.cpp @@ -172,6 +172,11 @@ class PyShellExt : public RuntimeClass< return E_FAIL; } auto dd = (DROPDESCRIPTION*)GlobalLock(medium.hGlobal); + if (!dd) { + OutputDebugString(L"PyShellExt::UpdateDropDescription - failed to lock DROPDESCRIPTION hGlobal"); + ReleaseStgMedium(&medium); + return E_FAIL; + } StringCchCopy(dd->szMessage, sizeof(dd->szMessage) / sizeof(dd->szMessage[0]), DRAG_MESSAGE); StringCchCopy(dd->szInsert, sizeof(dd->szInsert) / sizeof(dd->szInsert[0]), PathFindFileNameW(target)); dd->type = DROPIMAGE_MOVE; From webhook-mailer at python.org Tue Sep 25 00:44:16 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 04:44:16 -0000 Subject: [Python-checkins] bpo-34770: Fix a possible null pointer dereference in pyshellext.cpp (GH-9497) Message-ID: https://github.com/python/cpython/commit/db23206367e2bfbbdfb29b7699f25a14ba353ae7 commit: db23206367e2bfbbdfb29b7699f25a14ba353ae7 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-24T21:44:11-07:00 summary: bpo-34770: Fix a possible null pointer dereference in pyshellext.cpp (GH-9497) The GlobalLock() call in UpdateDropDescription() was not checked for failure. https://bugs.python.org/issue34770 (cherry picked from commit f6c8007a29b95b3ea3ca687a9b4924769a696328) Co-authored-by: Zackery Spytz files: A Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst M PC/pyshellext.cpp diff --git a/Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst b/Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst new file mode 100644 index 000000000000..5e4ba8868e84 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst @@ -0,0 +1 @@ +Fix a possible null pointer dereference in pyshellext.cpp. diff --git a/PC/pyshellext.cpp b/PC/pyshellext.cpp index 04fe61e89618..019880264bee 100644 --- a/PC/pyshellext.cpp +++ b/PC/pyshellext.cpp @@ -172,6 +172,11 @@ class PyShellExt : public RuntimeClass< return E_FAIL; } auto dd = (DROPDESCRIPTION*)GlobalLock(medium.hGlobal); + if (!dd) { + OutputDebugString(L"PyShellExt::UpdateDropDescription - failed to lock DROPDESCRIPTION hGlobal"); + ReleaseStgMedium(&medium); + return E_FAIL; + } StringCchCopy(dd->szMessage, sizeof(dd->szMessage) / sizeof(dd->szMessage[0]), DRAG_MESSAGE); StringCchCopy(dd->szInsert, sizeof(dd->szInsert) / sizeof(dd->szInsert[0]), PathFindFileNameW(target)); dd->type = DROPIMAGE_MOVE; From webhook-mailer at python.org Tue Sep 25 00:47:14 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 04:47:14 -0000 Subject: [Python-checkins] bpo-34770: Fix a possible null pointer dereference in pyshellext.cpp (GH-9497) Message-ID: https://github.com/python/cpython/commit/936d740ac0308fdcb87c11cbe87962d4fe06f5ab commit: 936d740ac0308fdcb87c11cbe87962d4fe06f5ab branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-24T21:47:11-07:00 summary: bpo-34770: Fix a possible null pointer dereference in pyshellext.cpp (GH-9497) The GlobalLock() call in UpdateDropDescription() was not checked for failure. https://bugs.python.org/issue34770 (cherry picked from commit f6c8007a29b95b3ea3ca687a9b4924769a696328) Co-authored-by: Zackery Spytz files: A Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst M PC/pyshellext.cpp diff --git a/Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst b/Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst new file mode 100644 index 000000000000..5e4ba8868e84 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-22-11-02-35.bpo-34770.4lEUOd.rst @@ -0,0 +1 @@ +Fix a possible null pointer dereference in pyshellext.cpp. diff --git a/PC/pyshellext.cpp b/PC/pyshellext.cpp index 04fe61e89618..019880264bee 100644 --- a/PC/pyshellext.cpp +++ b/PC/pyshellext.cpp @@ -172,6 +172,11 @@ class PyShellExt : public RuntimeClass< return E_FAIL; } auto dd = (DROPDESCRIPTION*)GlobalLock(medium.hGlobal); + if (!dd) { + OutputDebugString(L"PyShellExt::UpdateDropDescription - failed to lock DROPDESCRIPTION hGlobal"); + ReleaseStgMedium(&medium); + return E_FAIL; + } StringCchCopy(dd->szMessage, sizeof(dd->szMessage) / sizeof(dd->szMessage[0]), DRAG_MESSAGE); StringCchCopy(dd->szInsert, sizeof(dd->szInsert) / sizeof(dd->szInsert[0]), PathFindFileNameW(target)); dd->type = DROPIMAGE_MOVE; From webhook-mailer at python.org Tue Sep 25 03:24:57 2018 From: webhook-mailer at python.org (Vinay Sajip) Date: Tue, 25 Sep 2018 07:24:57 -0000 Subject: [Python-checkins] Improved the more elaborate multiprocessing example in the logging cookbook (GH-9326) Message-ID: https://github.com/python/cpython/commit/5b3cbcd4a041eeda935dd6d0c75f2d38111ed03d commit: 5b3cbcd4a041eeda935dd6d0c75f2d38111ed03d branch: master author: G?ry Ogam committer: Vinay Sajip date: 2018-09-25T08:24:52+01:00 summary: Improved the more elaborate multiprocessing example in the logging cookbook (GH-9326) files: M Doc/howto/logging-cookbook.rst diff --git a/Doc/howto/logging-cookbook.rst b/Doc/howto/logging-cookbook.rst index b1930a791fca..285aff7937c3 100644 --- a/Doc/howto/logging-cookbook.rst +++ b/Doc/howto/logging-cookbook.rst @@ -1456,12 +1456,18 @@ works:: which then get dispatched, by the logging system, to the handlers configured for those loggers. """ + def handle(self, record): - logger = logging.getLogger(record.name) - # The process name is transformed just to show that it's the listener - # doing the logging to files and console - record.processName = '%s (for %s)' % (current_process().name, record.processName) - logger.handle(record) + if record.name == "root": + logger = logging.getLogger() + else: + logger = logging.getLogger(record.name) + + if logger.isEnabledFor(record.levelno): + # The process name is transformed just to show that it's the listener + # doing the logging to files and console + record.processName = '%s (for %s)' % (current_process().name, record.processName) + logger.handle(record) def listener_process(q, stop_event, config): """ @@ -1526,22 +1532,16 @@ works:: # The main process gets a simple configuration which prints to the console. config_initial = { 'version': 1, - 'formatters': { - 'detailed': { - 'class': 'logging.Formatter', - 'format': '%(asctime)s %(name)-15s %(levelname)-8s %(processName)-10s %(message)s' - } - }, 'handlers': { 'console': { 'class': 'logging.StreamHandler', - 'level': 'INFO', - }, + 'level': 'INFO' + } }, 'root': { - 'level': 'DEBUG', - 'handlers': ['console'] - }, + 'handlers': ['console'], + 'level': 'DEBUG' + } } # The worker process configuration is just a QueueHandler attached to the # root logger, which allows all messages to be sent to the queue. @@ -1554,13 +1554,13 @@ works:: 'handlers': { 'queue': { 'class': 'logging.handlers.QueueHandler', - 'queue': q, - }, + 'queue': q + } }, 'root': { - 'level': 'DEBUG', - 'handlers': ['queue'] - }, + 'handlers': ['queue'], + 'level': 'DEBUG' + } } # The listener process configuration shows that the full flexibility of # logging configuration is available to dispatch events to handlers however @@ -1584,28 +1584,28 @@ works:: 'handlers': { 'console': { 'class': 'logging.StreamHandler', - 'level': 'INFO', 'formatter': 'simple', + 'level': 'INFO' }, 'file': { 'class': 'logging.FileHandler', 'filename': 'mplog.log', 'mode': 'w', - 'formatter': 'detailed', + 'formatter': 'detailed' }, 'foofile': { 'class': 'logging.FileHandler', 'filename': 'mplog-foo.log', 'mode': 'w', - 'formatter': 'detailed', + 'formatter': 'detailed' }, 'errors': { 'class': 'logging.FileHandler', 'filename': 'mplog-errors.log', 'mode': 'w', - 'level': 'ERROR', 'formatter': 'detailed', - }, + 'level': 'ERROR' + } }, 'loggers': { 'foo': { @@ -1613,9 +1613,9 @@ works:: } }, 'root': { - 'level': 'DEBUG', - 'handlers': ['console', 'file', 'errors'] - }, + 'handlers': ['console', 'file', 'errors'], + 'level': 'DEBUG' + } } # Log some initial events, just to show that logging in the parent works # normally. From solipsis at pitrou.net Tue Sep 25 05:09:08 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 25 Sep 2018 09:09:08 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=13 Message-ID: <20180925090908.1.9F93EB0089FC438C@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [0, 0, 7] memory blocks, sum=7 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [2, -1, 1] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/refloguch7Df', '--timeout', '7200'] From webhook-mailer at python.org Tue Sep 25 08:10:18 2018 From: webhook-mailer at python.org (Tal Einat) Date: Tue, 25 Sep 2018 12:10:18 -0000 Subject: [Python-checkins] bpo-1529353: IDLE: squeeze large output in the shell (GH-7626) Message-ID: https://github.com/python/cpython/commit/604e7b9931f9e7881a2941816e538f5f15930db8 commit: 604e7b9931f9e7881a2941816e538f5f15930db8 branch: master author: Tal Einat committer: GitHub date: 2018-09-25T15:10:14+03:00 summary: bpo-1529353: IDLE: squeeze large output in the shell (GH-7626) files: A Lib/idlelib/idle_test/test_squeezer.py A Lib/idlelib/squeezer.py A Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst M Lib/idlelib/config-main.def M Lib/idlelib/configdialog.py M Lib/idlelib/editor.py M Lib/idlelib/idle_test/htest.py M Lib/idlelib/idle_test/test_config.py M Lib/idlelib/idle_test/test_textview.py M Lib/idlelib/pyshell.py M Lib/idlelib/textview.py diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def index 16f4b0959cf1..06e3c5adb0e3 100644 --- a/Lib/idlelib/config-main.def +++ b/Lib/idlelib/config-main.def @@ -66,6 +66,9 @@ font-size= 10 font-bold= 0 encoding= none +[PyShell] +auto-squeeze-min-lines= 50 + [Indent] use-spaces= 1 num-spaces= 4 diff --git a/Lib/idlelib/configdialog.py b/Lib/idlelib/configdialog.py index e682ec0da320..229dc8987433 100644 --- a/Lib/idlelib/configdialog.py +++ b/Lib/idlelib/configdialog.py @@ -30,10 +30,12 @@ from idlelib.codecontext import CodeContext from idlelib.parenmatch import ParenMatch from idlelib.paragraph import FormatParagraph +from idlelib.squeezer import Squeezer changes = ConfigChanges() # Reload changed options in the following classes. -reloadables = (AutoComplete, CodeContext, ParenMatch, FormatParagraph) +reloadables = (AutoComplete, CodeContext, ParenMatch, FormatParagraph, + Squeezer) class ConfigDialog(Toplevel): @@ -1748,9 +1750,9 @@ def delete_custom_keys(self): self.customlist.SetMenu(item_list, item_list[0]) # Revert to default key set. self.keyset_source.set(idleConf.defaultCfg['main'] - .Get('Keys', 'default')) + .Get('Keys', 'default')) self.builtin_name.set(idleConf.defaultCfg['main'].Get('Keys', 'name') - or idleConf.default_keys()) + or idleConf.default_keys()) # User can't back out of these changes, they must be applied now. changes.save_all() self.cd.save_all_changed_extensions() @@ -1817,6 +1819,10 @@ def create_page_general(self): frame_context: Frame context_title: Label (*)context_int: Entry - context_lines + frame_shell: LabelFrame + frame_auto_squeeze_min_lines: Frame + auto_squeeze_min_lines_title: Label + (*)auto_squeeze_min_lines_int: Entry - auto_squeeze_min_lines frame_help: LabelFrame frame_helplist: Frame frame_helplist_buttons: Frame @@ -1842,6 +1848,9 @@ def create_page_general(self): self.paren_bell = tracers.add( BooleanVar(self), ('extensions', 'ParenMatch', 'bell')) + self.auto_squeeze_min_lines = tracers.add( + StringVar(self), ('main', 'PyShell', 'auto-squeeze-min-lines')) + self.autosave = tracers.add( IntVar(self), ('main', 'General', 'autosave')) self.format_width = tracers.add( @@ -1855,8 +1864,10 @@ def create_page_general(self): text=' Window Preferences') frame_editor = LabelFrame(self, borderwidth=2, relief=GROOVE, text=' Editor Preferences') + frame_shell = LabelFrame(self, borderwidth=2, relief=GROOVE, + text=' Shell Preferences') frame_help = LabelFrame(self, borderwidth=2, relief=GROOVE, - text=' Additional Help Sources ') + text=' Additional Help Sources ') # Frame_window. frame_run = Frame(frame_window, borderwidth=0) startup_title = Label(frame_run, text='At Startup') @@ -1918,6 +1929,13 @@ def create_page_general(self): self.context_int = Entry( frame_context, textvariable=self.context_lines, width=3) + # Frame_shell. + frame_auto_squeeze_min_lines = Frame(frame_shell, borderwidth=0) + auto_squeeze_min_lines_title = Label(frame_auto_squeeze_min_lines, + text='Auto-Squeeze Min. Lines:') + self.auto_squeeze_min_lines_int = Entry( + frame_auto_squeeze_min_lines, width=4, + textvariable=self.auto_squeeze_min_lines) # frame_help. frame_helplist = Frame(frame_help) @@ -1943,6 +1961,7 @@ def create_page_general(self): # Body. frame_window.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) frame_editor.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) + frame_shell.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) frame_help.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) # frame_run. frame_run.pack(side=TOP, padx=5, pady=0, fill=X) @@ -1983,6 +2002,11 @@ def create_page_general(self): context_title.pack(side=LEFT, anchor=W, padx=5, pady=5) self.context_int.pack(side=TOP, padx=5, pady=5) + # frame_auto_squeeze_min_lines + frame_auto_squeeze_min_lines.pack(side=TOP, padx=5, pady=0, fill=X) + auto_squeeze_min_lines_title.pack(side=LEFT, anchor=W, padx=5, pady=5) + self.auto_squeeze_min_lines_int.pack(side=TOP, padx=5, pady=5) + # frame_help. frame_helplist_buttons.pack(side=RIGHT, padx=5, pady=5, fill=Y) frame_helplist.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) @@ -2018,6 +2042,10 @@ def load_general_cfg(self): self.context_lines.set(idleConf.GetOption( 'extensions', 'CodeContext', 'maxlines', type='int')) + # Set variables for shell windows. + self.auto_squeeze_min_lines.set(idleConf.GetOption( + 'main', 'PyShell', 'auto-squeeze-min-lines', type='int')) + # Set additional help sources. self.user_helplist = idleConf.GetAllExtraHelpSourcesList() self.helplist.delete(0, 'end') @@ -2211,6 +2239,9 @@ def detach(self): CodeContext: Maxlines is the maximum number of code context lines to display when Code Context is turned on for an editor window. + +Shell Preferences: Auto-Squeeze Min. Lines is the minimum number of lines +of output to automatically "squeeze". ''' } diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py index 227a74deb82d..6689af64c429 100644 --- a/Lib/idlelib/editor.py +++ b/Lib/idlelib/editor.py @@ -2,9 +2,7 @@ import importlib.util import os import platform -import re import string -import sys import tokenize import traceback import webbrowser @@ -50,7 +48,6 @@ class EditorWindow(object): from idlelib.undo import UndoDelegator from idlelib.iomenu import IOBinding, encoding from idlelib import mainmenu - from tkinter import Toplevel, EventType from idlelib.statusbar import MultiStatusBar from idlelib.autocomplete import AutoComplete from idlelib.autoexpand import AutoExpand @@ -59,6 +56,7 @@ class EditorWindow(object): from idlelib.paragraph import FormatParagraph from idlelib.parenmatch import ParenMatch from idlelib.rstrip import Rstrip + from idlelib.squeezer import Squeezer from idlelib.zoomheight import ZoomHeight filesystemencoding = sys.getfilesystemencoding() # for file names @@ -319,6 +317,9 @@ def __init__(self, flist=None, filename=None, key=None, root=None): text.bind("<>", self.ZoomHeight(self).zoom_height_event) text.bind("<>", self.CodeContext(self).toggle_code_context_event) + squeezer = self.Squeezer(self) + text.bind("<>", + squeezer.squeeze_current_text_event) def _filename_to_unicode(self, filename): """Return filename as BMP unicode so diplayable in Tk.""" diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py index 03bee5170735..8c1c24d070cc 100644 --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -163,7 +163,7 @@ def _wrapper(parent): # htest # 'msg': "Click the 'Show GrepDialog' button.\n" "Test the various 'Find-in-files' functions.\n" "The results should be displayed in a new '*Output*' window.\n" - "'Right-click'->'Goto file/line' anywhere in the search results " + "'Right-click'->'Go to file/line' anywhere in the search results " "should open that file \nin a new EditorWindow." } diff --git a/Lib/idlelib/idle_test/test_config.py b/Lib/idlelib/idle_test/test_config.py index f3d9f21dd86c..8c9197284e07 100644 --- a/Lib/idlelib/idle_test/test_config.py +++ b/Lib/idlelib/idle_test/test_config.py @@ -356,11 +356,11 @@ def test_get_section_list(self): self.assertCountEqual( conf.GetSectionList('default', 'main'), - ['General', 'EditorWindow', 'Indent', 'Theme', + ['General', 'EditorWindow', 'PyShell', 'Indent', 'Theme', 'Keys', 'History', 'HelpFiles']) self.assertCountEqual( conf.GetSectionList('user', 'main'), - ['General', 'EditorWindow', 'Indent', 'Theme', + ['General', 'EditorWindow', 'PyShell', 'Indent', 'Theme', 'Keys', 'History', 'HelpFiles']) with self.assertRaises(config.InvalidConfigSet): @@ -452,7 +452,7 @@ def test_remove_key_bind_names(self): self.assertCountEqual( conf.RemoveKeyBindNames(conf.GetSectionList('default', 'extensions')), - ['AutoComplete', 'CodeContext', 'FormatParagraph', 'ParenMatch','ZzDummy']) + ['AutoComplete', 'CodeContext', 'FormatParagraph', 'ParenMatch', 'ZzDummy']) def test_get_extn_name_for_event(self): userextn.read_string(''' diff --git a/Lib/idlelib/idle_test/test_squeezer.py b/Lib/idlelib/idle_test/test_squeezer.py new file mode 100644 index 000000000000..ca8b674cc236 --- /dev/null +++ b/Lib/idlelib/idle_test/test_squeezer.py @@ -0,0 +1,509 @@ +from collections import namedtuple +from tkinter import Text, Tk +import unittest +from unittest.mock import Mock, NonCallableMagicMock, patch, sentinel, ANY +from test.support import requires + +from idlelib.config import idleConf +from idlelib.squeezer import count_lines_with_wrapping, ExpandingButton, \ + Squeezer +from idlelib import macosx +from idlelib.textview import view_text +from idlelib.tooltip import Hovertip +from idlelib.pyshell import PyShell + + +SENTINEL_VALUE = sentinel.SENTINEL_VALUE + + +def get_test_tk_root(test_instance): + """Helper for tests: Create a root Tk object.""" + requires('gui') + root = Tk() + root.withdraw() + + def cleanup_root(): + root.update_idletasks() + root.destroy() + test_instance.addCleanup(cleanup_root) + + return root + + +class CountLinesTest(unittest.TestCase): + """Tests for the count_lines_with_wrapping function.""" + def check(self, expected, text, linewidth, tabwidth): + return self.assertEqual( + expected, + count_lines_with_wrapping(text, linewidth, tabwidth), + ) + + def test_count_empty(self): + """Test with an empty string.""" + self.assertEqual(count_lines_with_wrapping(""), 0) + + def test_count_begins_with_empty_line(self): + """Test with a string which begins with a newline.""" + self.assertEqual(count_lines_with_wrapping("\ntext"), 2) + + def test_count_ends_with_empty_line(self): + """Test with a string which ends with a newline.""" + self.assertEqual(count_lines_with_wrapping("text\n"), 1) + + def test_count_several_lines(self): + """Test with several lines of text.""" + self.assertEqual(count_lines_with_wrapping("1\n2\n3\n"), 3) + + def test_tab_width(self): + """Test with various tab widths and line widths.""" + self.check(expected=1, text='\t' * 1, linewidth=8, tabwidth=4) + self.check(expected=1, text='\t' * 2, linewidth=8, tabwidth=4) + self.check(expected=2, text='\t' * 3, linewidth=8, tabwidth=4) + self.check(expected=2, text='\t' * 4, linewidth=8, tabwidth=4) + self.check(expected=3, text='\t' * 5, linewidth=8, tabwidth=4) + + # test longer lines and various tab widths + self.check(expected=4, text='\t' * 10, linewidth=12, tabwidth=4) + self.check(expected=10, text='\t' * 10, linewidth=12, tabwidth=8) + self.check(expected=2, text='\t' * 4, linewidth=10, tabwidth=3) + + # test tabwidth=1 + self.check(expected=2, text='\t' * 9, linewidth=5, tabwidth=1) + self.check(expected=2, text='\t' * 10, linewidth=5, tabwidth=1) + self.check(expected=3, text='\t' * 11, linewidth=5, tabwidth=1) + + # test for off-by-one errors + self.check(expected=2, text='\t' * 6, linewidth=12, tabwidth=4) + self.check(expected=3, text='\t' * 6, linewidth=11, tabwidth=4) + self.check(expected=2, text='\t' * 6, linewidth=13, tabwidth=4) + + +class SqueezerTest(unittest.TestCase): + """Tests for the Squeezer class.""" + def make_mock_editor_window(self): + """Create a mock EditorWindow instance.""" + editwin = NonCallableMagicMock() + # isinstance(editwin, PyShell) must be true for Squeezer to enable + # auto-squeezing; in practice this will always be true + editwin.__class__ = PyShell + return editwin + + def make_squeezer_instance(self, editor_window=None): + """Create an actual Squeezer instance with a mock EditorWindow.""" + if editor_window is None: + editor_window = self.make_mock_editor_window() + return Squeezer(editor_window) + + def test_count_lines(self): + """Test Squeezer.count_lines() with various inputs. + + This checks that Squeezer.count_lines() calls the + count_lines_with_wrapping() function with the appropriate parameters. + """ + for tabwidth, linewidth in [(4, 80), (1, 79), (8, 80), (3, 120)]: + self._test_count_lines_helper(linewidth=linewidth, + tabwidth=tabwidth) + + def _prepare_mock_editwin_for_count_lines(self, editwin, + linewidth, tabwidth): + """Prepare a mock EditorWindow object for Squeezer.count_lines.""" + CHAR_WIDTH = 10 + BORDER_WIDTH = 2 + PADDING_WIDTH = 1 + + # Prepare all the required functionality on the mock EditorWindow object + # so that the calculations in Squeezer.count_lines() can run. + editwin.get_tk_tabwidth.return_value = tabwidth + editwin.text.winfo_width.return_value = \ + linewidth * CHAR_WIDTH + 2 * (BORDER_WIDTH + PADDING_WIDTH) + text_opts = { + 'border': BORDER_WIDTH, + 'padx': PADDING_WIDTH, + 'font': None, + } + editwin.text.cget = lambda opt: text_opts[opt] + + # monkey-path tkinter.font.Font with a mock object, so that + # Font.measure('0') returns CHAR_WIDTH + mock_font = Mock() + def measure(char): + if char == '0': + return CHAR_WIDTH + raise ValueError("measure should only be called on '0'!") + mock_font.return_value.measure = measure + patcher = patch('idlelib.squeezer.Font', mock_font) + patcher.start() + self.addCleanup(patcher.stop) + + def _test_count_lines_helper(self, linewidth, tabwidth): + """Helper for test_count_lines.""" + editwin = self.make_mock_editor_window() + self._prepare_mock_editwin_for_count_lines(editwin, linewidth, tabwidth) + squeezer = self.make_squeezer_instance(editwin) + + mock_count_lines = Mock(return_value=SENTINEL_VALUE) + text = 'TEXT' + with patch('idlelib.squeezer.count_lines_with_wrapping', + mock_count_lines): + self.assertIs(squeezer.count_lines(text), SENTINEL_VALUE) + mock_count_lines.assert_called_with(text, linewidth, tabwidth) + + def test_init(self): + """Test the creation of Squeezer instances.""" + editwin = self.make_mock_editor_window() + squeezer = self.make_squeezer_instance(editwin) + self.assertIs(squeezer.editwin, editwin) + self.assertEqual(squeezer.expandingbuttons, []) + + def test_write_no_tags(self): + """Test Squeezer's overriding of the EditorWindow's write() method.""" + editwin = self.make_mock_editor_window() + for text in ['', 'TEXT', 'LONG TEXT' * 1000, 'MANY_LINES\n' * 100]: + editwin.write = orig_write = Mock(return_value=SENTINEL_VALUE) + squeezer = self.make_squeezer_instance(editwin) + + self.assertEqual(squeezer.editwin.write(text, ()), SENTINEL_VALUE) + self.assertEqual(orig_write.call_count, 1) + orig_write.assert_called_with(text, ()) + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_write_not_stdout(self): + """Test Squeezer's overriding of the EditorWindow's write() method.""" + for text in ['', 'TEXT', 'LONG TEXT' * 1000, 'MANY_LINES\n' * 100]: + editwin = self.make_mock_editor_window() + editwin.write.return_value = SENTINEL_VALUE + orig_write = editwin.write + squeezer = self.make_squeezer_instance(editwin) + + self.assertEqual(squeezer.editwin.write(text, "stderr"), + SENTINEL_VALUE) + self.assertEqual(orig_write.call_count, 1) + orig_write.assert_called_with(text, "stderr") + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_write_stdout(self): + """Test Squeezer's overriding of the EditorWindow's write() method.""" + editwin = self.make_mock_editor_window() + self._prepare_mock_editwin_for_count_lines(editwin, + linewidth=80, tabwidth=8) + + for text in ['', 'TEXT']: + editwin.write = orig_write = Mock(return_value=SENTINEL_VALUE) + squeezer = self.make_squeezer_instance(editwin) + squeezer.auto_squeeze_min_lines = 50 + + self.assertEqual(squeezer.editwin.write(text, "stdout"), + SENTINEL_VALUE) + self.assertEqual(orig_write.call_count, 1) + orig_write.assert_called_with(text, "stdout") + self.assertEqual(len(squeezer.expandingbuttons), 0) + + for text in ['LONG TEXT' * 1000, 'MANY_LINES\n' * 100]: + editwin.write = orig_write = Mock(return_value=SENTINEL_VALUE) + squeezer = self.make_squeezer_instance(editwin) + squeezer.auto_squeeze_min_lines = 50 + + self.assertEqual(squeezer.editwin.write(text, "stdout"), None) + self.assertEqual(orig_write.call_count, 0) + self.assertEqual(len(squeezer.expandingbuttons), 1) + + def test_auto_squeeze(self): + """Test that the auto-squeezing creates an ExpandingButton properly.""" + root = get_test_tk_root(self) + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.auto_squeeze_min_lines = 5 + squeezer.count_lines = Mock(return_value=6) + + editwin.write('TEXT\n'*6, "stdout") + self.assertEqual(text_widget.get('1.0', 'end'), '\n') + self.assertEqual(len(squeezer.expandingbuttons), 1) + + def test_squeeze_current_text_event(self): + """Test the squeeze_current_text event.""" + root = get_test_tk_root(self) + + # squeezing text should work for both stdout and stderr + for tag_name in ["stdout", "stderr"]: + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = editwin.per.bottom = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.count_lines = Mock(return_value=6) + + # prepare some text in the Text widget + text_widget.insert("1.0", "SOME\nTEXT\n", tag_name) + text_widget.mark_set("insert", "1.0") + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + + self.assertEqual(len(squeezer.expandingbuttons), 0) + + # test squeezing the current text + retval = squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(retval, "break") + self.assertEqual(text_widget.get('1.0', 'end'), '\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 1) + self.assertEqual(squeezer.expandingbuttons[0].s, 'SOME\nTEXT') + + # test that expanding the squeezed text works and afterwards the + # Text widget contains the original text + squeezer.expandingbuttons[0].expand(event=Mock()) + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_squeeze_current_text_event_no_allowed_tags(self): + """Test that the event doesn't squeeze text without a relevant tag.""" + root = get_test_tk_root(self) + + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = editwin.per.bottom = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.count_lines = Mock(return_value=6) + + # prepare some text in the Text widget + text_widget.insert("1.0", "SOME\nTEXT\n", "TAG") + text_widget.mark_set("insert", "1.0") + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + + self.assertEqual(len(squeezer.expandingbuttons), 0) + + # test squeezing the current text + retval = squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(retval, "break") + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_squeeze_text_before_existing_squeezed_text(self): + """Test squeezing text before existing squeezed text.""" + root = get_test_tk_root(self) + + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = editwin.per.bottom = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.count_lines = Mock(return_value=6) + + # prepare some text in the Text widget and squeeze it + text_widget.insert("1.0", "SOME\nTEXT\n", "stdout") + text_widget.mark_set("insert", "1.0") + squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(len(squeezer.expandingbuttons), 1) + + # test squeezing the current text + text_widget.insert("1.0", "MORE\nSTUFF\n", "stdout") + text_widget.mark_set("insert", "1.0") + retval = squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(retval, "break") + self.assertEqual(text_widget.get('1.0', 'end'), '\n\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 2) + self.assertTrue(text_widget.compare( + squeezer.expandingbuttons[0], + '<', + squeezer.expandingbuttons[1], + )) + + GetOptionSignature = namedtuple('GetOptionSignature', + 'configType section option default type warn_on_default raw') + @classmethod + def _make_sig(cls, configType, section, option, default=sentinel.NOT_GIVEN, + type=sentinel.NOT_GIVEN, + warn_on_default=sentinel.NOT_GIVEN, + raw=sentinel.NOT_GIVEN): + return cls.GetOptionSignature(configType, section, option, default, + type, warn_on_default, raw) + + @classmethod + def get_GetOption_signature(cls, mock_call_obj): + args, kwargs = mock_call_obj[-2:] + return cls._make_sig(*args, **kwargs) + + def test_reload(self): + """Test the reload() class-method.""" + self.assertIsInstance(Squeezer.auto_squeeze_min_lines, int) + idleConf.SetOption('main', 'PyShell', 'auto-squeeze-min-lines', '42') + Squeezer.reload() + self.assertEqual(Squeezer.auto_squeeze_min_lines, 42) + + +class ExpandingButtonTest(unittest.TestCase): + """Tests for the ExpandingButton class.""" + # In these tests the squeezer instance is a mock, but actual tkinter + # Text and Button instances are created. + def make_mock_squeezer(self): + """Helper for tests: Create a mock Squeezer object.""" + root = get_test_tk_root(self) + squeezer = Mock() + squeezer.editwin.text = Text(root) + + # Set default values for the configuration settings + squeezer.auto_squeeze_min_lines = 50 + return squeezer + + @patch('idlelib.squeezer.Hovertip', autospec=Hovertip) + def test_init(self, MockHovertip): + """Test the simplest creation of an ExpandingButton.""" + squeezer = self.make_mock_squeezer() + text_widget = squeezer.editwin.text + + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + self.assertEqual(expandingbutton.s, 'TEXT') + + # check that the underlying tkinter.Button is properly configured + self.assertEqual(expandingbutton.master, text_widget) + self.assertTrue('50 lines' in expandingbutton.cget('text')) + + # check that the text widget still contains no text + self.assertEqual(text_widget.get('1.0', 'end'), '\n') + + # check that the mouse events are bound + self.assertIn('', expandingbutton.bind()) + right_button_code = '' % ('2' if macosx.isAquaTk() else '3') + self.assertIn(right_button_code, expandingbutton.bind()) + + # check that ToolTip was called once, with appropriate values + self.assertEqual(MockHovertip.call_count, 1) + MockHovertip.assert_called_with(expandingbutton, ANY, hover_delay=ANY) + + # check that 'right-click' appears in the tooltip text + tooltip_text = MockHovertip.call_args[0][1] + self.assertIn('right-click', tooltip_text.lower()) + + def test_expand(self): + """Test the expand event.""" + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + + # insert the button into the text widget + # (this is normally done by the Squeezer class) + text_widget = expandingbutton.text + text_widget.window_create("1.0", window=expandingbutton) + + # set base_text to the text widget, so that changes are actually made + # to it (by ExpandingButton) and we can inspect these changes afterwards + expandingbutton.base_text = expandingbutton.text + + # trigger the expand event + retval = expandingbutton.expand(event=Mock()) + self.assertEqual(retval, None) + + # check that the text was inserted into the text widget + self.assertEqual(text_widget.get('1.0', 'end'), 'TEXT\n') + + # check that the 'TAGS' tag was set on the inserted text + text_end_index = text_widget.index('end-1c') + self.assertEqual(text_widget.get('1.0', text_end_index), 'TEXT') + self.assertEqual(text_widget.tag_nextrange('TAGS', '1.0'), + ('1.0', text_end_index)) + + # check that the button removed itself from squeezer.expandingbuttons + self.assertEqual(squeezer.expandingbuttons.remove.call_count, 1) + squeezer.expandingbuttons.remove.assert_called_with(expandingbutton) + + def test_expand_dangerous_oupput(self): + """Test that expanding very long output asks user for confirmation.""" + squeezer = self.make_mock_squeezer() + text = 'a' * 10**5 + expandingbutton = ExpandingButton(text, 'TAGS', 50, squeezer) + expandingbutton.set_is_dangerous() + self.assertTrue(expandingbutton.is_dangerous) + + # insert the button into the text widget + # (this is normally done by the Squeezer class) + text_widget = expandingbutton.text + text_widget.window_create("1.0", window=expandingbutton) + + # set base_text to the text widget, so that changes are actually made + # to it (by ExpandingButton) and we can inspect these changes afterwards + expandingbutton.base_text = expandingbutton.text + + # patch the message box module to always return False + with patch('idlelib.squeezer.tkMessageBox') as mock_msgbox: + mock_msgbox.askokcancel.return_value = False + mock_msgbox.askyesno.return_value = False + + # trigger the expand event + retval = expandingbutton.expand(event=Mock()) + + # check that the event chain was broken and no text was inserted + self.assertEqual(retval, 'break') + self.assertEqual(expandingbutton.text.get('1.0', 'end-1c'), '') + + # patch the message box module to always return True + with patch('idlelib.squeezer.tkMessageBox') as mock_msgbox: + mock_msgbox.askokcancel.return_value = True + mock_msgbox.askyesno.return_value = True + + # trigger the expand event + retval = expandingbutton.expand(event=Mock()) + + # check that the event chain wasn't broken and the text was inserted + self.assertEqual(retval, None) + self.assertEqual(expandingbutton.text.get('1.0', 'end-1c'), text) + + def test_copy(self): + """Test the copy event.""" + # testing with the actual clipboard proved problematic, so this test + # replaces the clipboard manipulation functions with mocks and checks + # that they are called appropriately + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + expandingbutton.clipboard_clear = Mock() + expandingbutton.clipboard_append = Mock() + + # trigger the copy event + retval = expandingbutton.copy(event=Mock()) + self.assertEqual(retval, None) + + # check that the expanding button called clipboard_clear() and + # clipboard_append('TEXT') once each + self.assertEqual(expandingbutton.clipboard_clear.call_count, 1) + self.assertEqual(expandingbutton.clipboard_append.call_count, 1) + expandingbutton.clipboard_append.assert_called_with('TEXT') + + def test_view(self): + """Test the view event.""" + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + expandingbutton.selection_own = Mock() + + with patch('idlelib.squeezer.view_text', autospec=view_text)\ + as mock_view_text: + # trigger the view event + expandingbutton.view(event=Mock()) + + # check that the expanding button called view_text + self.assertEqual(mock_view_text.call_count, 1) + + # check that the proper text was passed + self.assertEqual(mock_view_text.call_args[0][2], 'TEXT') + + def test_rmenu(self): + """Test the context menu.""" + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + with patch('tkinter.Menu') as mock_Menu: + mock_menu = Mock() + mock_Menu.return_value = mock_menu + mock_event = Mock() + mock_event.x = 10 + mock_event.y = 10 + expandingbutton.context_menu_event(event=mock_event) + self.assertEqual(mock_menu.add_command.call_count, + len(expandingbutton.rmenu_specs)) + for label, *data in expandingbutton.rmenu_specs: + mock_menu.add_command.assert_any_call(label=label, command=ANY) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/Lib/idlelib/idle_test/test_textview.py b/Lib/idlelib/idle_test/test_textview.py index 0d11e41e0fb4..6f0c1930518a 100644 --- a/Lib/idlelib/idle_test/test_textview.py +++ b/Lib/idlelib/idle_test/test_textview.py @@ -73,7 +73,6 @@ class TextFrameTest(unittest.TestCase): @classmethod def setUpClass(cls): - "By itself, this tests that file parsed without exception." cls.root = root = Tk() root.withdraw() cls.frame = tv.TextFrame(root, 'test text') @@ -126,11 +125,15 @@ def test_bad_file(self): def test_bad_encoding(self): p = os.path fn = p.abspath(p.join(p.dirname(__file__), '..', 'CREDITS.txt')) - tv.showerror.title = None view = tv.view_file(root, 'Title', fn, 'ascii', modal=False) self.assertIsNone(view) self.assertEqual(tv.showerror.title, 'Unicode Decode Error') + def test_nowrap(self): + view = tv.view_text(root, 'Title', 'test', modal=False, wrap='none') + text_widget = view.viewframe.textframe.text + self.assertEqual(text_widget.cget('wrap'), 'none') + # Call ViewWindow with _utest=True. class ButtonClickTest(unittest.TestCase): diff --git a/Lib/idlelib/pyshell.py b/Lib/idlelib/pyshell.py index 52c11e30dbd5..5458c59dbd7e 100755 --- a/Lib/idlelib/pyshell.py +++ b/Lib/idlelib/pyshell.py @@ -856,6 +856,10 @@ class PyShell(OutputWindow): ("help", "_Help"), ] + # Extend right-click context menu + rmenu_specs = OutputWindow.rmenu_specs + [ + ("Squeeze", "<>"), + ] # New classes from idlelib.history import History diff --git a/Lib/idlelib/squeezer.py b/Lib/idlelib/squeezer.py new file mode 100644 index 000000000000..f5aac813a159 --- /dev/null +++ b/Lib/idlelib/squeezer.py @@ -0,0 +1,355 @@ +"""An IDLE extension to avoid having very long texts printed in the shell. + +A common problem in IDLE's interactive shell is printing of large amounts of +text into the shell. This makes looking at the previous history difficult. +Worse, this can cause IDLE to become very slow, even to the point of being +completely unusable. + +This extension will automatically replace long texts with a small button. +Double-cliking this button will remove it and insert the original text instead. +Middle-clicking will copy the text to the clipboard. Right-clicking will open +the text in a separate viewing window. + +Additionally, any output can be manually "squeezed" by the user. This includes +output written to the standard error stream ("stderr"), such as exception +messages and their tracebacks. +""" +import re + +import tkinter as tk +from tkinter.font import Font +import tkinter.messagebox as tkMessageBox + +from idlelib.config import idleConf +from idlelib.textview import view_text +from idlelib.tooltip import Hovertip +from idlelib import macosx + + +def count_lines_with_wrapping(s, linewidth=80, tabwidth=8): + """Count the number of lines in a given string. + + Lines are counted as if the string was wrapped so that lines are never over + linewidth characters long. + + Tabs are considered tabwidth characters long. + """ + pos = 0 + linecount = 1 + current_column = 0 + + for m in re.finditer(r"[\t\n]", s): + # process the normal chars up to tab or newline + numchars = m.start() - pos + pos += numchars + current_column += numchars + + # deal with tab or newline + if s[pos] == '\n': + linecount += 1 + current_column = 0 + else: + assert s[pos] == '\t' + current_column += tabwidth - (current_column % tabwidth) + + # if a tab passes the end of the line, consider the entire tab as + # being on the next line + if current_column > linewidth: + linecount += 1 + current_column = tabwidth + + pos += 1 # after the tab or newline + + # avoid divmod(-1, linewidth) + if current_column > 0: + # If the length was exactly linewidth, divmod would give (1,0), + # even though a new line hadn't yet been started. The same is true + # if length is any exact multiple of linewidth. Therefore, subtract + # 1 before doing divmod, and later add 1 to the column to + # compensate. + lines, column = divmod(current_column - 1, linewidth) + linecount += lines + current_column = column + 1 + + # process remaining chars (no more tabs or newlines) + current_column += len(s) - pos + # avoid divmod(-1, linewidth) + if current_column > 0: + linecount += (current_column - 1) // linewidth + else: + # the text ended with a newline; don't count an extra line after it + linecount -= 1 + + return linecount + + +class ExpandingButton(tk.Button): + """Class for the "squeezed" text buttons used by Squeezer + + These buttons are displayed inside a Tk Text widget in place of text. A + user can then use the button to replace it with the original text, copy + the original text to the clipboard or view the original text in a separate + window. + + Each button is tied to a Squeezer instance, and it knows to update the + Squeezer instance when it is expanded (and therefore removed). + """ + def __init__(self, s, tags, numoflines, squeezer): + self.s = s + self.tags = tags + self.numoflines = numoflines + self.squeezer = squeezer + self.editwin = editwin = squeezer.editwin + self.text = text = editwin.text + + # the base Text widget of the PyShell object, used to change text + # before the iomark + self.base_text = editwin.per.bottom + + button_text = "Squeezed text (%d lines)." % self.numoflines + tk.Button.__init__(self, text, text=button_text, + background="#FFFFC0", activebackground="#FFFFE0") + + button_tooltip_text = ( + "Double-click to expand, right-click for more options." + ) + Hovertip(self, button_tooltip_text, hover_delay=80) + + self.bind("", self.expand) + if macosx.isAquaTk(): + # AquaTk defines <2> as the right button, not <3>. + self.bind("", self.context_menu_event) + else: + self.bind("", self.context_menu_event) + self.selection_handle( + lambda offset, length: s[int(offset):int(offset) + int(length)]) + + self.is_dangerous = None + self.after_idle(self.set_is_dangerous) + + def set_is_dangerous(self): + dangerous_line_len = 50 * self.text.winfo_width() + self.is_dangerous = ( + self.numoflines > 1000 or + len(self.s) > 50000 or + any( + len(line_match.group(0)) >= dangerous_line_len + for line_match in re.finditer(r'[^\n]+', self.s) + ) + ) + + def expand(self, event=None): + """expand event handler + + This inserts the original text in place of the button in the Text + widget, removes the button and updates the Squeezer instance. + + If the original text is dangerously long, i.e. expanding it could + cause a performance degradation, ask the user for confirmation. + """ + if self.is_dangerous is None: + self.set_is_dangerous() + if self.is_dangerous: + confirm = tkMessageBox.askokcancel( + title="Expand huge output?", + message="\n\n".join([ + "The squeezed output is very long: %d lines, %d chars.", + "Expanding it could make IDLE slow or unresponsive.", + "It is recommended to view or copy the output instead.", + "Really expand?" + ]) % (self.numoflines, len(self.s)), + default=tkMessageBox.CANCEL, + parent=self.text) + if not confirm: + return "break" + + self.base_text.insert(self.text.index(self), self.s, self.tags) + self.base_text.delete(self) + self.squeezer.expandingbuttons.remove(self) + + def copy(self, event=None): + """copy event handler + + Copy the original text to the clipboard. + """ + self.clipboard_clear() + self.clipboard_append(self.s) + + def view(self, event=None): + """view event handler + + View the original text in a separate text viewer window. + """ + view_text(self.text, "Squeezed Output Viewer", self.s, + modal=False, wrap='none') + + rmenu_specs = ( + # item structure: (label, method_name) + ('copy', 'copy'), + ('view', 'view'), + ) + + def context_menu_event(self, event): + self.text.mark_set("insert", "@%d,%d" % (event.x, event.y)) + rmenu = tk.Menu(self.text, tearoff=0) + for label, method_name in self.rmenu_specs: + rmenu.add_command(label=label, command=getattr(self, method_name)) + rmenu.tk_popup(event.x_root, event.y_root) + return "break" + + +class Squeezer: + """Replace long outputs in the shell with a simple button. + + This avoids IDLE's shell slowing down considerably, and even becoming + completely unresponsive, when very long outputs are written. + """ + @classmethod + def reload(cls): + """Load class variables from config.""" + cls.auto_squeeze_min_lines = idleConf.GetOption( + "main", "PyShell", "auto-squeeze-min-lines", + type="int", default=50, + ) + + def __init__(self, editwin): + """Initialize settings for Squeezer. + + editwin is the shell's Editor window. + self.text is the editor window text widget. + self.base_test is the actual editor window Tk text widget, rather than + EditorWindow's wrapper. + self.expandingbuttons is the list of all buttons representing + "squeezed" output. + """ + self.editwin = editwin + self.text = text = editwin.text + + # Get the base Text widget of the PyShell object, used to change text + # before the iomark. PyShell deliberately disables changing text before + # the iomark via its 'text' attribute, which is actually a wrapper for + # the actual Text widget. Squeezer, however, needs to make such changes. + self.base_text = editwin.per.bottom + + self.expandingbuttons = [] + from idlelib.pyshell import PyShell # done here to avoid import cycle + if isinstance(editwin, PyShell): + # If we get a PyShell instance, replace its write method with a + # wrapper, which inserts an ExpandingButton instead of a long text. + def mywrite(s, tags=(), write=editwin.write): + # only auto-squeeze text which has just the "stdout" tag + if tags != "stdout": + return write(s, tags) + + # only auto-squeeze text with at least the minimum + # configured number of lines + numoflines = self.count_lines(s) + if numoflines < self.auto_squeeze_min_lines: + return write(s, tags) + + # create an ExpandingButton instance + expandingbutton = ExpandingButton(s, tags, numoflines, + self) + + # insert the ExpandingButton into the Text widget + text.mark_gravity("iomark", tk.RIGHT) + text.window_create("iomark", window=expandingbutton, + padx=3, pady=5) + text.see("iomark") + text.update() + text.mark_gravity("iomark", tk.LEFT) + + # add the ExpandingButton to the Squeezer's list + self.expandingbuttons.append(expandingbutton) + + editwin.write = mywrite + + def count_lines(self, s): + """Count the number of lines in a given text. + + Before calculation, the tab width and line length of the text are + fetched, so that up-to-date values are used. + + Lines are counted as if the string was wrapped so that lines are never + over linewidth characters long. + + Tabs are considered tabwidth characters long. + """ + # Tab width is configurable + tabwidth = self.editwin.get_tk_tabwidth() + + # Get the Text widget's size + linewidth = self.editwin.text.winfo_width() + # Deduct the border and padding + linewidth -= 2*sum([int(self.editwin.text.cget(opt)) + for opt in ('border', 'padx')]) + + # Get the Text widget's font + font = Font(self.editwin.text, name=self.editwin.text.cget('font')) + # Divide the size of the Text widget by the font's width. + # According to Tk8.5 docs, the Text widget's width is set + # according to the width of its font's '0' (zero) character, + # so we will use this as an approximation. + # see: http://www.tcl.tk/man/tcl8.5/TkCmd/text.htm#M-width + linewidth //= font.measure('0') + + return count_lines_with_wrapping(s, linewidth, tabwidth) + + def squeeze_current_text_event(self, event): + """squeeze-current-text event handler + + Squeeze the block of text inside which contains the "insert" cursor. + + If the insert cursor is not in a squeezable block of text, give the + user a small warning and do nothing. + """ + # set tag_name to the first valid tag found on the "insert" cursor + tag_names = self.text.tag_names(tk.INSERT) + for tag_name in ("stdout", "stderr"): + if tag_name in tag_names: + break + else: + # the insert cursor doesn't have a "stdout" or "stderr" tag + self.text.bell() + return "break" + + # find the range to squeeze + start, end = self.text.tag_prevrange(tag_name, tk.INSERT + "+1c") + s = self.text.get(start, end) + + # if the last char is a newline, remove it from the range + if len(s) > 0 and s[-1] == '\n': + end = self.text.index("%s-1c" % end) + s = s[:-1] + + # delete the text + self.base_text.delete(start, end) + + # prepare an ExpandingButton + numoflines = self.count_lines(s) + expandingbutton = ExpandingButton(s, tag_name, numoflines, self) + + # insert the ExpandingButton to the Text + self.text.window_create(start, window=expandingbutton, + padx=3, pady=5) + + # insert the ExpandingButton to the list of ExpandingButtons, while + # keeping the list ordered according to the position of the buttons in + # the Text widget + i = len(self.expandingbuttons) + while i > 0 and self.text.compare(self.expandingbuttons[i-1], + ">", expandingbutton): + i -= 1 + self.expandingbuttons.insert(i, expandingbutton) + + return "break" + + +Squeezer.reload() + + +if __name__ == "__main__": + from unittest import main + main('idlelib.idle_test.test_squeezer', verbosity=2, exit=False) + + # Add htest. diff --git a/Lib/idlelib/textview.py b/Lib/idlelib/textview.py index 464e6ac6b94e..4867a80db1ab 100644 --- a/Lib/idlelib/textview.py +++ b/Lib/idlelib/textview.py @@ -1,17 +1,37 @@ """Simple text browser for IDLE """ -from tkinter import Toplevel, Text +from tkinter import Toplevel, Text, TclError,\ + HORIZONTAL, VERTICAL, N, S, E, W from tkinter.ttk import Frame, Scrollbar, Button from tkinter.messagebox import showerror from idlelib.colorizer import color_config +class AutoHiddenScrollbar(Scrollbar): + """A scrollbar that is automatically hidden when not needed. + + Only the grid geometry manager is supported. + """ + def set(self, lo, hi): + if float(lo) > 0.0 or float(hi) < 1.0: + self.grid() + else: + self.grid_remove() + super().set(lo, hi) + + def pack(self, **kwargs): + raise TclError(f'{self.__class__.__name__} does not support "pack"') + + def place(self, **kwargs): + raise TclError(f'{self.__class__.__name__} does not support "place"') + + class TextFrame(Frame): "Display text with scrollbar." - def __init__(self, parent, rawtext): + def __init__(self, parent, rawtext, wrap='word'): """Create a frame for Textview. parent - parent widget for this frame @@ -21,27 +41,39 @@ def __init__(self, parent, rawtext): self['relief'] = 'sunken' self['height'] = 700 - self.text = text = Text(self, wrap='word', highlightthickness=0) + self.text = text = Text(self, wrap=wrap, highlightthickness=0) color_config(text) - self.scroll = scroll = Scrollbar(self, orient='vertical', - takefocus=False, command=text.yview) - text['yscrollcommand'] = scroll.set + text.grid(row=0, column=0, sticky=N+S+E+W) + self.grid_rowconfigure(0, weight=1) + self.grid_columnconfigure(0, weight=1) text.insert(0.0, rawtext) text['state'] = 'disabled' text.focus_set() - scroll.pack(side='right', fill='y') - text.pack(side='left', expand=True, fill='both') + # vertical scrollbar + self.yscroll = yscroll = AutoHiddenScrollbar(self, orient=VERTICAL, + takefocus=False, + command=text.yview) + text['yscrollcommand'] = yscroll.set + yscroll.grid(row=0, column=1, sticky=N+S) + + if wrap == 'none': + # horizontal scrollbar + self.xscroll = xscroll = AutoHiddenScrollbar(self, orient=HORIZONTAL, + takefocus=False, + command=text.xview) + text['xscrollcommand'] = xscroll.set + xscroll.grid(row=1, column=0, sticky=E+W) class ViewFrame(Frame): "Display TextFrame and Close button." - def __init__(self, parent, text): + def __init__(self, parent, text, wrap='word'): super().__init__(parent) self.parent = parent self.bind('', self.ok) self.bind('', self.ok) - self.textframe = TextFrame(self, text) + self.textframe = TextFrame(self, text, wrap=wrap) self.button_ok = button_ok = Button( self, text='Close', command=self.ok, takefocus=False) self.textframe.pack(side='top', expand=True, fill='both') @@ -55,7 +87,7 @@ def ok(self, event=None): class ViewWindow(Toplevel): "A simple text viewer dialog for IDLE." - def __init__(self, parent, title, text, modal=True, + def __init__(self, parent, title, text, modal=True, wrap='word', *, _htest=False, _utest=False): """Show the given text in a scrollable window with a 'close' button. @@ -65,6 +97,7 @@ def __init__(self, parent, title, text, modal=True, parent - parent of this dialog title - string which is title of popup dialog text - text to display in dialog + wrap - type of text wrapping to use ('word', 'char' or 'none') _htest - bool; change box location when running htest. _utest - bool; don't wait_window when running unittest. """ @@ -76,7 +109,7 @@ def __init__(self, parent, title, text, modal=True, self.geometry(f'=750x500+{x}+{y}') self.title(title) - self.viewframe = ViewFrame(self, text) + self.viewframe = ViewFrame(self, text, wrap=wrap) self.protocol("WM_DELETE_WINDOW", self.ok) self.button_ok = button_ok = Button(self, text='Close', command=self.ok, takefocus=False) @@ -96,20 +129,22 @@ def ok(self, event=None): self.destroy() -def view_text(parent, title, text, modal=True, _utest=False): +def view_text(parent, title, text, modal=True, wrap='word', _utest=False): """Create text viewer for given text. parent - parent of this dialog title - string which is the title of popup dialog text - text to display in this dialog + wrap - type of text wrapping to use ('word', 'char' or 'none') modal - controls if users can interact with other windows while this dialog is displayed _utest - bool; controls wait_window on unittest """ - return ViewWindow(parent, title, text, modal, _utest=_utest) + return ViewWindow(parent, title, text, modal, wrap=wrap, _utest=_utest) -def view_file(parent, title, filename, encoding, modal=True, _utest=False): +def view_file(parent, title, filename, encoding, modal=True, wrap='word', + _utest=False): """Create text viewer for text in filename. Return error message if file cannot be read. Otherwise calls view_text @@ -127,7 +162,8 @@ def view_file(parent, title, filename, encoding, modal=True, _utest=False): message=str(err), parent=parent) else: - return view_text(parent, title, contents, modal, _utest=_utest) + return view_text(parent, title, contents, modal, wrap=wrap, + _utest=_utest) return None diff --git a/Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst b/Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst new file mode 100644 index 000000000000..cae4af8f2e23 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst @@ -0,0 +1,3 @@ +Enable "squeezing" of long outputs in the shell, to avoid performance +degradation and to clean up the history without losing it. Squeezed outputs +may be copied, viewed in a separate window, and "unsqueezed". From webhook-mailer at python.org Tue Sep 25 08:38:50 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 12:38:50 -0000 Subject: [Python-checkins] bpo-1529353: IDLE: squeeze large output in the shell (GH-7626) Message-ID: https://github.com/python/cpython/commit/321f28c5f4b7361fa1b6330697b28481b4565ec8 commit: 321f28c5f4b7361fa1b6330697b28481b4565ec8 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T05:38:45-07:00 summary: bpo-1529353: IDLE: squeeze large output in the shell (GH-7626) (cherry picked from commit 604e7b9931f9e7881a2941816e538f5f15930db8) Co-authored-by: Tal Einat files: A Lib/idlelib/idle_test/test_squeezer.py A Lib/idlelib/squeezer.py A Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst M Lib/idlelib/config-main.def M Lib/idlelib/configdialog.py M Lib/idlelib/editor.py M Lib/idlelib/idle_test/htest.py M Lib/idlelib/idle_test/test_config.py M Lib/idlelib/idle_test/test_textview.py M Lib/idlelib/pyshell.py M Lib/idlelib/textview.py diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def index 16f4b0959cf1..06e3c5adb0e3 100644 --- a/Lib/idlelib/config-main.def +++ b/Lib/idlelib/config-main.def @@ -66,6 +66,9 @@ font-size= 10 font-bold= 0 encoding= none +[PyShell] +auto-squeeze-min-lines= 50 + [Indent] use-spaces= 1 num-spaces= 4 diff --git a/Lib/idlelib/configdialog.py b/Lib/idlelib/configdialog.py index e682ec0da320..229dc8987433 100644 --- a/Lib/idlelib/configdialog.py +++ b/Lib/idlelib/configdialog.py @@ -30,10 +30,12 @@ from idlelib.codecontext import CodeContext from idlelib.parenmatch import ParenMatch from idlelib.paragraph import FormatParagraph +from idlelib.squeezer import Squeezer changes = ConfigChanges() # Reload changed options in the following classes. -reloadables = (AutoComplete, CodeContext, ParenMatch, FormatParagraph) +reloadables = (AutoComplete, CodeContext, ParenMatch, FormatParagraph, + Squeezer) class ConfigDialog(Toplevel): @@ -1748,9 +1750,9 @@ def delete_custom_keys(self): self.customlist.SetMenu(item_list, item_list[0]) # Revert to default key set. self.keyset_source.set(idleConf.defaultCfg['main'] - .Get('Keys', 'default')) + .Get('Keys', 'default')) self.builtin_name.set(idleConf.defaultCfg['main'].Get('Keys', 'name') - or idleConf.default_keys()) + or idleConf.default_keys()) # User can't back out of these changes, they must be applied now. changes.save_all() self.cd.save_all_changed_extensions() @@ -1817,6 +1819,10 @@ def create_page_general(self): frame_context: Frame context_title: Label (*)context_int: Entry - context_lines + frame_shell: LabelFrame + frame_auto_squeeze_min_lines: Frame + auto_squeeze_min_lines_title: Label + (*)auto_squeeze_min_lines_int: Entry - auto_squeeze_min_lines frame_help: LabelFrame frame_helplist: Frame frame_helplist_buttons: Frame @@ -1842,6 +1848,9 @@ def create_page_general(self): self.paren_bell = tracers.add( BooleanVar(self), ('extensions', 'ParenMatch', 'bell')) + self.auto_squeeze_min_lines = tracers.add( + StringVar(self), ('main', 'PyShell', 'auto-squeeze-min-lines')) + self.autosave = tracers.add( IntVar(self), ('main', 'General', 'autosave')) self.format_width = tracers.add( @@ -1855,8 +1864,10 @@ def create_page_general(self): text=' Window Preferences') frame_editor = LabelFrame(self, borderwidth=2, relief=GROOVE, text=' Editor Preferences') + frame_shell = LabelFrame(self, borderwidth=2, relief=GROOVE, + text=' Shell Preferences') frame_help = LabelFrame(self, borderwidth=2, relief=GROOVE, - text=' Additional Help Sources ') + text=' Additional Help Sources ') # Frame_window. frame_run = Frame(frame_window, borderwidth=0) startup_title = Label(frame_run, text='At Startup') @@ -1918,6 +1929,13 @@ def create_page_general(self): self.context_int = Entry( frame_context, textvariable=self.context_lines, width=3) + # Frame_shell. + frame_auto_squeeze_min_lines = Frame(frame_shell, borderwidth=0) + auto_squeeze_min_lines_title = Label(frame_auto_squeeze_min_lines, + text='Auto-Squeeze Min. Lines:') + self.auto_squeeze_min_lines_int = Entry( + frame_auto_squeeze_min_lines, width=4, + textvariable=self.auto_squeeze_min_lines) # frame_help. frame_helplist = Frame(frame_help) @@ -1943,6 +1961,7 @@ def create_page_general(self): # Body. frame_window.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) frame_editor.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) + frame_shell.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) frame_help.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) # frame_run. frame_run.pack(side=TOP, padx=5, pady=0, fill=X) @@ -1983,6 +2002,11 @@ def create_page_general(self): context_title.pack(side=LEFT, anchor=W, padx=5, pady=5) self.context_int.pack(side=TOP, padx=5, pady=5) + # frame_auto_squeeze_min_lines + frame_auto_squeeze_min_lines.pack(side=TOP, padx=5, pady=0, fill=X) + auto_squeeze_min_lines_title.pack(side=LEFT, anchor=W, padx=5, pady=5) + self.auto_squeeze_min_lines_int.pack(side=TOP, padx=5, pady=5) + # frame_help. frame_helplist_buttons.pack(side=RIGHT, padx=5, pady=5, fill=Y) frame_helplist.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) @@ -2018,6 +2042,10 @@ def load_general_cfg(self): self.context_lines.set(idleConf.GetOption( 'extensions', 'CodeContext', 'maxlines', type='int')) + # Set variables for shell windows. + self.auto_squeeze_min_lines.set(idleConf.GetOption( + 'main', 'PyShell', 'auto-squeeze-min-lines', type='int')) + # Set additional help sources. self.user_helplist = idleConf.GetAllExtraHelpSourcesList() self.helplist.delete(0, 'end') @@ -2211,6 +2239,9 @@ def detach(self): CodeContext: Maxlines is the maximum number of code context lines to display when Code Context is turned on for an editor window. + +Shell Preferences: Auto-Squeeze Min. Lines is the minimum number of lines +of output to automatically "squeeze". ''' } diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py index 227a74deb82d..6689af64c429 100644 --- a/Lib/idlelib/editor.py +++ b/Lib/idlelib/editor.py @@ -2,9 +2,7 @@ import importlib.util import os import platform -import re import string -import sys import tokenize import traceback import webbrowser @@ -50,7 +48,6 @@ class EditorWindow(object): from idlelib.undo import UndoDelegator from idlelib.iomenu import IOBinding, encoding from idlelib import mainmenu - from tkinter import Toplevel, EventType from idlelib.statusbar import MultiStatusBar from idlelib.autocomplete import AutoComplete from idlelib.autoexpand import AutoExpand @@ -59,6 +56,7 @@ class EditorWindow(object): from idlelib.paragraph import FormatParagraph from idlelib.parenmatch import ParenMatch from idlelib.rstrip import Rstrip + from idlelib.squeezer import Squeezer from idlelib.zoomheight import ZoomHeight filesystemencoding = sys.getfilesystemencoding() # for file names @@ -319,6 +317,9 @@ def __init__(self, flist=None, filename=None, key=None, root=None): text.bind("<>", self.ZoomHeight(self).zoom_height_event) text.bind("<>", self.CodeContext(self).toggle_code_context_event) + squeezer = self.Squeezer(self) + text.bind("<>", + squeezer.squeeze_current_text_event) def _filename_to_unicode(self, filename): """Return filename as BMP unicode so diplayable in Tk.""" diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py index 03bee5170735..8c1c24d070cc 100644 --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -163,7 +163,7 @@ def _wrapper(parent): # htest # 'msg': "Click the 'Show GrepDialog' button.\n" "Test the various 'Find-in-files' functions.\n" "The results should be displayed in a new '*Output*' window.\n" - "'Right-click'->'Goto file/line' anywhere in the search results " + "'Right-click'->'Go to file/line' anywhere in the search results " "should open that file \nin a new EditorWindow." } diff --git a/Lib/idlelib/idle_test/test_config.py b/Lib/idlelib/idle_test/test_config.py index f3d9f21dd86c..8c9197284e07 100644 --- a/Lib/idlelib/idle_test/test_config.py +++ b/Lib/idlelib/idle_test/test_config.py @@ -356,11 +356,11 @@ def test_get_section_list(self): self.assertCountEqual( conf.GetSectionList('default', 'main'), - ['General', 'EditorWindow', 'Indent', 'Theme', + ['General', 'EditorWindow', 'PyShell', 'Indent', 'Theme', 'Keys', 'History', 'HelpFiles']) self.assertCountEqual( conf.GetSectionList('user', 'main'), - ['General', 'EditorWindow', 'Indent', 'Theme', + ['General', 'EditorWindow', 'PyShell', 'Indent', 'Theme', 'Keys', 'History', 'HelpFiles']) with self.assertRaises(config.InvalidConfigSet): @@ -452,7 +452,7 @@ def test_remove_key_bind_names(self): self.assertCountEqual( conf.RemoveKeyBindNames(conf.GetSectionList('default', 'extensions')), - ['AutoComplete', 'CodeContext', 'FormatParagraph', 'ParenMatch','ZzDummy']) + ['AutoComplete', 'CodeContext', 'FormatParagraph', 'ParenMatch', 'ZzDummy']) def test_get_extn_name_for_event(self): userextn.read_string(''' diff --git a/Lib/idlelib/idle_test/test_squeezer.py b/Lib/idlelib/idle_test/test_squeezer.py new file mode 100644 index 000000000000..ca8b674cc236 --- /dev/null +++ b/Lib/idlelib/idle_test/test_squeezer.py @@ -0,0 +1,509 @@ +from collections import namedtuple +from tkinter import Text, Tk +import unittest +from unittest.mock import Mock, NonCallableMagicMock, patch, sentinel, ANY +from test.support import requires + +from idlelib.config import idleConf +from idlelib.squeezer import count_lines_with_wrapping, ExpandingButton, \ + Squeezer +from idlelib import macosx +from idlelib.textview import view_text +from idlelib.tooltip import Hovertip +from idlelib.pyshell import PyShell + + +SENTINEL_VALUE = sentinel.SENTINEL_VALUE + + +def get_test_tk_root(test_instance): + """Helper for tests: Create a root Tk object.""" + requires('gui') + root = Tk() + root.withdraw() + + def cleanup_root(): + root.update_idletasks() + root.destroy() + test_instance.addCleanup(cleanup_root) + + return root + + +class CountLinesTest(unittest.TestCase): + """Tests for the count_lines_with_wrapping function.""" + def check(self, expected, text, linewidth, tabwidth): + return self.assertEqual( + expected, + count_lines_with_wrapping(text, linewidth, tabwidth), + ) + + def test_count_empty(self): + """Test with an empty string.""" + self.assertEqual(count_lines_with_wrapping(""), 0) + + def test_count_begins_with_empty_line(self): + """Test with a string which begins with a newline.""" + self.assertEqual(count_lines_with_wrapping("\ntext"), 2) + + def test_count_ends_with_empty_line(self): + """Test with a string which ends with a newline.""" + self.assertEqual(count_lines_with_wrapping("text\n"), 1) + + def test_count_several_lines(self): + """Test with several lines of text.""" + self.assertEqual(count_lines_with_wrapping("1\n2\n3\n"), 3) + + def test_tab_width(self): + """Test with various tab widths and line widths.""" + self.check(expected=1, text='\t' * 1, linewidth=8, tabwidth=4) + self.check(expected=1, text='\t' * 2, linewidth=8, tabwidth=4) + self.check(expected=2, text='\t' * 3, linewidth=8, tabwidth=4) + self.check(expected=2, text='\t' * 4, linewidth=8, tabwidth=4) + self.check(expected=3, text='\t' * 5, linewidth=8, tabwidth=4) + + # test longer lines and various tab widths + self.check(expected=4, text='\t' * 10, linewidth=12, tabwidth=4) + self.check(expected=10, text='\t' * 10, linewidth=12, tabwidth=8) + self.check(expected=2, text='\t' * 4, linewidth=10, tabwidth=3) + + # test tabwidth=1 + self.check(expected=2, text='\t' * 9, linewidth=5, tabwidth=1) + self.check(expected=2, text='\t' * 10, linewidth=5, tabwidth=1) + self.check(expected=3, text='\t' * 11, linewidth=5, tabwidth=1) + + # test for off-by-one errors + self.check(expected=2, text='\t' * 6, linewidth=12, tabwidth=4) + self.check(expected=3, text='\t' * 6, linewidth=11, tabwidth=4) + self.check(expected=2, text='\t' * 6, linewidth=13, tabwidth=4) + + +class SqueezerTest(unittest.TestCase): + """Tests for the Squeezer class.""" + def make_mock_editor_window(self): + """Create a mock EditorWindow instance.""" + editwin = NonCallableMagicMock() + # isinstance(editwin, PyShell) must be true for Squeezer to enable + # auto-squeezing; in practice this will always be true + editwin.__class__ = PyShell + return editwin + + def make_squeezer_instance(self, editor_window=None): + """Create an actual Squeezer instance with a mock EditorWindow.""" + if editor_window is None: + editor_window = self.make_mock_editor_window() + return Squeezer(editor_window) + + def test_count_lines(self): + """Test Squeezer.count_lines() with various inputs. + + This checks that Squeezer.count_lines() calls the + count_lines_with_wrapping() function with the appropriate parameters. + """ + for tabwidth, linewidth in [(4, 80), (1, 79), (8, 80), (3, 120)]: + self._test_count_lines_helper(linewidth=linewidth, + tabwidth=tabwidth) + + def _prepare_mock_editwin_for_count_lines(self, editwin, + linewidth, tabwidth): + """Prepare a mock EditorWindow object for Squeezer.count_lines.""" + CHAR_WIDTH = 10 + BORDER_WIDTH = 2 + PADDING_WIDTH = 1 + + # Prepare all the required functionality on the mock EditorWindow object + # so that the calculations in Squeezer.count_lines() can run. + editwin.get_tk_tabwidth.return_value = tabwidth + editwin.text.winfo_width.return_value = \ + linewidth * CHAR_WIDTH + 2 * (BORDER_WIDTH + PADDING_WIDTH) + text_opts = { + 'border': BORDER_WIDTH, + 'padx': PADDING_WIDTH, + 'font': None, + } + editwin.text.cget = lambda opt: text_opts[opt] + + # monkey-path tkinter.font.Font with a mock object, so that + # Font.measure('0') returns CHAR_WIDTH + mock_font = Mock() + def measure(char): + if char == '0': + return CHAR_WIDTH + raise ValueError("measure should only be called on '0'!") + mock_font.return_value.measure = measure + patcher = patch('idlelib.squeezer.Font', mock_font) + patcher.start() + self.addCleanup(patcher.stop) + + def _test_count_lines_helper(self, linewidth, tabwidth): + """Helper for test_count_lines.""" + editwin = self.make_mock_editor_window() + self._prepare_mock_editwin_for_count_lines(editwin, linewidth, tabwidth) + squeezer = self.make_squeezer_instance(editwin) + + mock_count_lines = Mock(return_value=SENTINEL_VALUE) + text = 'TEXT' + with patch('idlelib.squeezer.count_lines_with_wrapping', + mock_count_lines): + self.assertIs(squeezer.count_lines(text), SENTINEL_VALUE) + mock_count_lines.assert_called_with(text, linewidth, tabwidth) + + def test_init(self): + """Test the creation of Squeezer instances.""" + editwin = self.make_mock_editor_window() + squeezer = self.make_squeezer_instance(editwin) + self.assertIs(squeezer.editwin, editwin) + self.assertEqual(squeezer.expandingbuttons, []) + + def test_write_no_tags(self): + """Test Squeezer's overriding of the EditorWindow's write() method.""" + editwin = self.make_mock_editor_window() + for text in ['', 'TEXT', 'LONG TEXT' * 1000, 'MANY_LINES\n' * 100]: + editwin.write = orig_write = Mock(return_value=SENTINEL_VALUE) + squeezer = self.make_squeezer_instance(editwin) + + self.assertEqual(squeezer.editwin.write(text, ()), SENTINEL_VALUE) + self.assertEqual(orig_write.call_count, 1) + orig_write.assert_called_with(text, ()) + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_write_not_stdout(self): + """Test Squeezer's overriding of the EditorWindow's write() method.""" + for text in ['', 'TEXT', 'LONG TEXT' * 1000, 'MANY_LINES\n' * 100]: + editwin = self.make_mock_editor_window() + editwin.write.return_value = SENTINEL_VALUE + orig_write = editwin.write + squeezer = self.make_squeezer_instance(editwin) + + self.assertEqual(squeezer.editwin.write(text, "stderr"), + SENTINEL_VALUE) + self.assertEqual(orig_write.call_count, 1) + orig_write.assert_called_with(text, "stderr") + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_write_stdout(self): + """Test Squeezer's overriding of the EditorWindow's write() method.""" + editwin = self.make_mock_editor_window() + self._prepare_mock_editwin_for_count_lines(editwin, + linewidth=80, tabwidth=8) + + for text in ['', 'TEXT']: + editwin.write = orig_write = Mock(return_value=SENTINEL_VALUE) + squeezer = self.make_squeezer_instance(editwin) + squeezer.auto_squeeze_min_lines = 50 + + self.assertEqual(squeezer.editwin.write(text, "stdout"), + SENTINEL_VALUE) + self.assertEqual(orig_write.call_count, 1) + orig_write.assert_called_with(text, "stdout") + self.assertEqual(len(squeezer.expandingbuttons), 0) + + for text in ['LONG TEXT' * 1000, 'MANY_LINES\n' * 100]: + editwin.write = orig_write = Mock(return_value=SENTINEL_VALUE) + squeezer = self.make_squeezer_instance(editwin) + squeezer.auto_squeeze_min_lines = 50 + + self.assertEqual(squeezer.editwin.write(text, "stdout"), None) + self.assertEqual(orig_write.call_count, 0) + self.assertEqual(len(squeezer.expandingbuttons), 1) + + def test_auto_squeeze(self): + """Test that the auto-squeezing creates an ExpandingButton properly.""" + root = get_test_tk_root(self) + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.auto_squeeze_min_lines = 5 + squeezer.count_lines = Mock(return_value=6) + + editwin.write('TEXT\n'*6, "stdout") + self.assertEqual(text_widget.get('1.0', 'end'), '\n') + self.assertEqual(len(squeezer.expandingbuttons), 1) + + def test_squeeze_current_text_event(self): + """Test the squeeze_current_text event.""" + root = get_test_tk_root(self) + + # squeezing text should work for both stdout and stderr + for tag_name in ["stdout", "stderr"]: + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = editwin.per.bottom = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.count_lines = Mock(return_value=6) + + # prepare some text in the Text widget + text_widget.insert("1.0", "SOME\nTEXT\n", tag_name) + text_widget.mark_set("insert", "1.0") + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + + self.assertEqual(len(squeezer.expandingbuttons), 0) + + # test squeezing the current text + retval = squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(retval, "break") + self.assertEqual(text_widget.get('1.0', 'end'), '\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 1) + self.assertEqual(squeezer.expandingbuttons[0].s, 'SOME\nTEXT') + + # test that expanding the squeezed text works and afterwards the + # Text widget contains the original text + squeezer.expandingbuttons[0].expand(event=Mock()) + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_squeeze_current_text_event_no_allowed_tags(self): + """Test that the event doesn't squeeze text without a relevant tag.""" + root = get_test_tk_root(self) + + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = editwin.per.bottom = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.count_lines = Mock(return_value=6) + + # prepare some text in the Text widget + text_widget.insert("1.0", "SOME\nTEXT\n", "TAG") + text_widget.mark_set("insert", "1.0") + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + + self.assertEqual(len(squeezer.expandingbuttons), 0) + + # test squeezing the current text + retval = squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(retval, "break") + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_squeeze_text_before_existing_squeezed_text(self): + """Test squeezing text before existing squeezed text.""" + root = get_test_tk_root(self) + + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = editwin.per.bottom = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.count_lines = Mock(return_value=6) + + # prepare some text in the Text widget and squeeze it + text_widget.insert("1.0", "SOME\nTEXT\n", "stdout") + text_widget.mark_set("insert", "1.0") + squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(len(squeezer.expandingbuttons), 1) + + # test squeezing the current text + text_widget.insert("1.0", "MORE\nSTUFF\n", "stdout") + text_widget.mark_set("insert", "1.0") + retval = squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(retval, "break") + self.assertEqual(text_widget.get('1.0', 'end'), '\n\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 2) + self.assertTrue(text_widget.compare( + squeezer.expandingbuttons[0], + '<', + squeezer.expandingbuttons[1], + )) + + GetOptionSignature = namedtuple('GetOptionSignature', + 'configType section option default type warn_on_default raw') + @classmethod + def _make_sig(cls, configType, section, option, default=sentinel.NOT_GIVEN, + type=sentinel.NOT_GIVEN, + warn_on_default=sentinel.NOT_GIVEN, + raw=sentinel.NOT_GIVEN): + return cls.GetOptionSignature(configType, section, option, default, + type, warn_on_default, raw) + + @classmethod + def get_GetOption_signature(cls, mock_call_obj): + args, kwargs = mock_call_obj[-2:] + return cls._make_sig(*args, **kwargs) + + def test_reload(self): + """Test the reload() class-method.""" + self.assertIsInstance(Squeezer.auto_squeeze_min_lines, int) + idleConf.SetOption('main', 'PyShell', 'auto-squeeze-min-lines', '42') + Squeezer.reload() + self.assertEqual(Squeezer.auto_squeeze_min_lines, 42) + + +class ExpandingButtonTest(unittest.TestCase): + """Tests for the ExpandingButton class.""" + # In these tests the squeezer instance is a mock, but actual tkinter + # Text and Button instances are created. + def make_mock_squeezer(self): + """Helper for tests: Create a mock Squeezer object.""" + root = get_test_tk_root(self) + squeezer = Mock() + squeezer.editwin.text = Text(root) + + # Set default values for the configuration settings + squeezer.auto_squeeze_min_lines = 50 + return squeezer + + @patch('idlelib.squeezer.Hovertip', autospec=Hovertip) + def test_init(self, MockHovertip): + """Test the simplest creation of an ExpandingButton.""" + squeezer = self.make_mock_squeezer() + text_widget = squeezer.editwin.text + + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + self.assertEqual(expandingbutton.s, 'TEXT') + + # check that the underlying tkinter.Button is properly configured + self.assertEqual(expandingbutton.master, text_widget) + self.assertTrue('50 lines' in expandingbutton.cget('text')) + + # check that the text widget still contains no text + self.assertEqual(text_widget.get('1.0', 'end'), '\n') + + # check that the mouse events are bound + self.assertIn('', expandingbutton.bind()) + right_button_code = '' % ('2' if macosx.isAquaTk() else '3') + self.assertIn(right_button_code, expandingbutton.bind()) + + # check that ToolTip was called once, with appropriate values + self.assertEqual(MockHovertip.call_count, 1) + MockHovertip.assert_called_with(expandingbutton, ANY, hover_delay=ANY) + + # check that 'right-click' appears in the tooltip text + tooltip_text = MockHovertip.call_args[0][1] + self.assertIn('right-click', tooltip_text.lower()) + + def test_expand(self): + """Test the expand event.""" + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + + # insert the button into the text widget + # (this is normally done by the Squeezer class) + text_widget = expandingbutton.text + text_widget.window_create("1.0", window=expandingbutton) + + # set base_text to the text widget, so that changes are actually made + # to it (by ExpandingButton) and we can inspect these changes afterwards + expandingbutton.base_text = expandingbutton.text + + # trigger the expand event + retval = expandingbutton.expand(event=Mock()) + self.assertEqual(retval, None) + + # check that the text was inserted into the text widget + self.assertEqual(text_widget.get('1.0', 'end'), 'TEXT\n') + + # check that the 'TAGS' tag was set on the inserted text + text_end_index = text_widget.index('end-1c') + self.assertEqual(text_widget.get('1.0', text_end_index), 'TEXT') + self.assertEqual(text_widget.tag_nextrange('TAGS', '1.0'), + ('1.0', text_end_index)) + + # check that the button removed itself from squeezer.expandingbuttons + self.assertEqual(squeezer.expandingbuttons.remove.call_count, 1) + squeezer.expandingbuttons.remove.assert_called_with(expandingbutton) + + def test_expand_dangerous_oupput(self): + """Test that expanding very long output asks user for confirmation.""" + squeezer = self.make_mock_squeezer() + text = 'a' * 10**5 + expandingbutton = ExpandingButton(text, 'TAGS', 50, squeezer) + expandingbutton.set_is_dangerous() + self.assertTrue(expandingbutton.is_dangerous) + + # insert the button into the text widget + # (this is normally done by the Squeezer class) + text_widget = expandingbutton.text + text_widget.window_create("1.0", window=expandingbutton) + + # set base_text to the text widget, so that changes are actually made + # to it (by ExpandingButton) and we can inspect these changes afterwards + expandingbutton.base_text = expandingbutton.text + + # patch the message box module to always return False + with patch('idlelib.squeezer.tkMessageBox') as mock_msgbox: + mock_msgbox.askokcancel.return_value = False + mock_msgbox.askyesno.return_value = False + + # trigger the expand event + retval = expandingbutton.expand(event=Mock()) + + # check that the event chain was broken and no text was inserted + self.assertEqual(retval, 'break') + self.assertEqual(expandingbutton.text.get('1.0', 'end-1c'), '') + + # patch the message box module to always return True + with patch('idlelib.squeezer.tkMessageBox') as mock_msgbox: + mock_msgbox.askokcancel.return_value = True + mock_msgbox.askyesno.return_value = True + + # trigger the expand event + retval = expandingbutton.expand(event=Mock()) + + # check that the event chain wasn't broken and the text was inserted + self.assertEqual(retval, None) + self.assertEqual(expandingbutton.text.get('1.0', 'end-1c'), text) + + def test_copy(self): + """Test the copy event.""" + # testing with the actual clipboard proved problematic, so this test + # replaces the clipboard manipulation functions with mocks and checks + # that they are called appropriately + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + expandingbutton.clipboard_clear = Mock() + expandingbutton.clipboard_append = Mock() + + # trigger the copy event + retval = expandingbutton.copy(event=Mock()) + self.assertEqual(retval, None) + + # check that the expanding button called clipboard_clear() and + # clipboard_append('TEXT') once each + self.assertEqual(expandingbutton.clipboard_clear.call_count, 1) + self.assertEqual(expandingbutton.clipboard_append.call_count, 1) + expandingbutton.clipboard_append.assert_called_with('TEXT') + + def test_view(self): + """Test the view event.""" + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + expandingbutton.selection_own = Mock() + + with patch('idlelib.squeezer.view_text', autospec=view_text)\ + as mock_view_text: + # trigger the view event + expandingbutton.view(event=Mock()) + + # check that the expanding button called view_text + self.assertEqual(mock_view_text.call_count, 1) + + # check that the proper text was passed + self.assertEqual(mock_view_text.call_args[0][2], 'TEXT') + + def test_rmenu(self): + """Test the context menu.""" + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + with patch('tkinter.Menu') as mock_Menu: + mock_menu = Mock() + mock_Menu.return_value = mock_menu + mock_event = Mock() + mock_event.x = 10 + mock_event.y = 10 + expandingbutton.context_menu_event(event=mock_event) + self.assertEqual(mock_menu.add_command.call_count, + len(expandingbutton.rmenu_specs)) + for label, *data in expandingbutton.rmenu_specs: + mock_menu.add_command.assert_any_call(label=label, command=ANY) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/Lib/idlelib/idle_test/test_textview.py b/Lib/idlelib/idle_test/test_textview.py index 0d11e41e0fb4..6f0c1930518a 100644 --- a/Lib/idlelib/idle_test/test_textview.py +++ b/Lib/idlelib/idle_test/test_textview.py @@ -73,7 +73,6 @@ class TextFrameTest(unittest.TestCase): @classmethod def setUpClass(cls): - "By itself, this tests that file parsed without exception." cls.root = root = Tk() root.withdraw() cls.frame = tv.TextFrame(root, 'test text') @@ -126,11 +125,15 @@ def test_bad_file(self): def test_bad_encoding(self): p = os.path fn = p.abspath(p.join(p.dirname(__file__), '..', 'CREDITS.txt')) - tv.showerror.title = None view = tv.view_file(root, 'Title', fn, 'ascii', modal=False) self.assertIsNone(view) self.assertEqual(tv.showerror.title, 'Unicode Decode Error') + def test_nowrap(self): + view = tv.view_text(root, 'Title', 'test', modal=False, wrap='none') + text_widget = view.viewframe.textframe.text + self.assertEqual(text_widget.cget('wrap'), 'none') + # Call ViewWindow with _utest=True. class ButtonClickTest(unittest.TestCase): diff --git a/Lib/idlelib/pyshell.py b/Lib/idlelib/pyshell.py index 52c11e30dbd5..5458c59dbd7e 100755 --- a/Lib/idlelib/pyshell.py +++ b/Lib/idlelib/pyshell.py @@ -856,6 +856,10 @@ class PyShell(OutputWindow): ("help", "_Help"), ] + # Extend right-click context menu + rmenu_specs = OutputWindow.rmenu_specs + [ + ("Squeeze", "<>"), + ] # New classes from idlelib.history import History diff --git a/Lib/idlelib/squeezer.py b/Lib/idlelib/squeezer.py new file mode 100644 index 000000000000..f5aac813a159 --- /dev/null +++ b/Lib/idlelib/squeezer.py @@ -0,0 +1,355 @@ +"""An IDLE extension to avoid having very long texts printed in the shell. + +A common problem in IDLE's interactive shell is printing of large amounts of +text into the shell. This makes looking at the previous history difficult. +Worse, this can cause IDLE to become very slow, even to the point of being +completely unusable. + +This extension will automatically replace long texts with a small button. +Double-cliking this button will remove it and insert the original text instead. +Middle-clicking will copy the text to the clipboard. Right-clicking will open +the text in a separate viewing window. + +Additionally, any output can be manually "squeezed" by the user. This includes +output written to the standard error stream ("stderr"), such as exception +messages and their tracebacks. +""" +import re + +import tkinter as tk +from tkinter.font import Font +import tkinter.messagebox as tkMessageBox + +from idlelib.config import idleConf +from idlelib.textview import view_text +from idlelib.tooltip import Hovertip +from idlelib import macosx + + +def count_lines_with_wrapping(s, linewidth=80, tabwidth=8): + """Count the number of lines in a given string. + + Lines are counted as if the string was wrapped so that lines are never over + linewidth characters long. + + Tabs are considered tabwidth characters long. + """ + pos = 0 + linecount = 1 + current_column = 0 + + for m in re.finditer(r"[\t\n]", s): + # process the normal chars up to tab or newline + numchars = m.start() - pos + pos += numchars + current_column += numchars + + # deal with tab or newline + if s[pos] == '\n': + linecount += 1 + current_column = 0 + else: + assert s[pos] == '\t' + current_column += tabwidth - (current_column % tabwidth) + + # if a tab passes the end of the line, consider the entire tab as + # being on the next line + if current_column > linewidth: + linecount += 1 + current_column = tabwidth + + pos += 1 # after the tab or newline + + # avoid divmod(-1, linewidth) + if current_column > 0: + # If the length was exactly linewidth, divmod would give (1,0), + # even though a new line hadn't yet been started. The same is true + # if length is any exact multiple of linewidth. Therefore, subtract + # 1 before doing divmod, and later add 1 to the column to + # compensate. + lines, column = divmod(current_column - 1, linewidth) + linecount += lines + current_column = column + 1 + + # process remaining chars (no more tabs or newlines) + current_column += len(s) - pos + # avoid divmod(-1, linewidth) + if current_column > 0: + linecount += (current_column - 1) // linewidth + else: + # the text ended with a newline; don't count an extra line after it + linecount -= 1 + + return linecount + + +class ExpandingButton(tk.Button): + """Class for the "squeezed" text buttons used by Squeezer + + These buttons are displayed inside a Tk Text widget in place of text. A + user can then use the button to replace it with the original text, copy + the original text to the clipboard or view the original text in a separate + window. + + Each button is tied to a Squeezer instance, and it knows to update the + Squeezer instance when it is expanded (and therefore removed). + """ + def __init__(self, s, tags, numoflines, squeezer): + self.s = s + self.tags = tags + self.numoflines = numoflines + self.squeezer = squeezer + self.editwin = editwin = squeezer.editwin + self.text = text = editwin.text + + # the base Text widget of the PyShell object, used to change text + # before the iomark + self.base_text = editwin.per.bottom + + button_text = "Squeezed text (%d lines)." % self.numoflines + tk.Button.__init__(self, text, text=button_text, + background="#FFFFC0", activebackground="#FFFFE0") + + button_tooltip_text = ( + "Double-click to expand, right-click for more options." + ) + Hovertip(self, button_tooltip_text, hover_delay=80) + + self.bind("", self.expand) + if macosx.isAquaTk(): + # AquaTk defines <2> as the right button, not <3>. + self.bind("", self.context_menu_event) + else: + self.bind("", self.context_menu_event) + self.selection_handle( + lambda offset, length: s[int(offset):int(offset) + int(length)]) + + self.is_dangerous = None + self.after_idle(self.set_is_dangerous) + + def set_is_dangerous(self): + dangerous_line_len = 50 * self.text.winfo_width() + self.is_dangerous = ( + self.numoflines > 1000 or + len(self.s) > 50000 or + any( + len(line_match.group(0)) >= dangerous_line_len + for line_match in re.finditer(r'[^\n]+', self.s) + ) + ) + + def expand(self, event=None): + """expand event handler + + This inserts the original text in place of the button in the Text + widget, removes the button and updates the Squeezer instance. + + If the original text is dangerously long, i.e. expanding it could + cause a performance degradation, ask the user for confirmation. + """ + if self.is_dangerous is None: + self.set_is_dangerous() + if self.is_dangerous: + confirm = tkMessageBox.askokcancel( + title="Expand huge output?", + message="\n\n".join([ + "The squeezed output is very long: %d lines, %d chars.", + "Expanding it could make IDLE slow or unresponsive.", + "It is recommended to view or copy the output instead.", + "Really expand?" + ]) % (self.numoflines, len(self.s)), + default=tkMessageBox.CANCEL, + parent=self.text) + if not confirm: + return "break" + + self.base_text.insert(self.text.index(self), self.s, self.tags) + self.base_text.delete(self) + self.squeezer.expandingbuttons.remove(self) + + def copy(self, event=None): + """copy event handler + + Copy the original text to the clipboard. + """ + self.clipboard_clear() + self.clipboard_append(self.s) + + def view(self, event=None): + """view event handler + + View the original text in a separate text viewer window. + """ + view_text(self.text, "Squeezed Output Viewer", self.s, + modal=False, wrap='none') + + rmenu_specs = ( + # item structure: (label, method_name) + ('copy', 'copy'), + ('view', 'view'), + ) + + def context_menu_event(self, event): + self.text.mark_set("insert", "@%d,%d" % (event.x, event.y)) + rmenu = tk.Menu(self.text, tearoff=0) + for label, method_name in self.rmenu_specs: + rmenu.add_command(label=label, command=getattr(self, method_name)) + rmenu.tk_popup(event.x_root, event.y_root) + return "break" + + +class Squeezer: + """Replace long outputs in the shell with a simple button. + + This avoids IDLE's shell slowing down considerably, and even becoming + completely unresponsive, when very long outputs are written. + """ + @classmethod + def reload(cls): + """Load class variables from config.""" + cls.auto_squeeze_min_lines = idleConf.GetOption( + "main", "PyShell", "auto-squeeze-min-lines", + type="int", default=50, + ) + + def __init__(self, editwin): + """Initialize settings for Squeezer. + + editwin is the shell's Editor window. + self.text is the editor window text widget. + self.base_test is the actual editor window Tk text widget, rather than + EditorWindow's wrapper. + self.expandingbuttons is the list of all buttons representing + "squeezed" output. + """ + self.editwin = editwin + self.text = text = editwin.text + + # Get the base Text widget of the PyShell object, used to change text + # before the iomark. PyShell deliberately disables changing text before + # the iomark via its 'text' attribute, which is actually a wrapper for + # the actual Text widget. Squeezer, however, needs to make such changes. + self.base_text = editwin.per.bottom + + self.expandingbuttons = [] + from idlelib.pyshell import PyShell # done here to avoid import cycle + if isinstance(editwin, PyShell): + # If we get a PyShell instance, replace its write method with a + # wrapper, which inserts an ExpandingButton instead of a long text. + def mywrite(s, tags=(), write=editwin.write): + # only auto-squeeze text which has just the "stdout" tag + if tags != "stdout": + return write(s, tags) + + # only auto-squeeze text with at least the minimum + # configured number of lines + numoflines = self.count_lines(s) + if numoflines < self.auto_squeeze_min_lines: + return write(s, tags) + + # create an ExpandingButton instance + expandingbutton = ExpandingButton(s, tags, numoflines, + self) + + # insert the ExpandingButton into the Text widget + text.mark_gravity("iomark", tk.RIGHT) + text.window_create("iomark", window=expandingbutton, + padx=3, pady=5) + text.see("iomark") + text.update() + text.mark_gravity("iomark", tk.LEFT) + + # add the ExpandingButton to the Squeezer's list + self.expandingbuttons.append(expandingbutton) + + editwin.write = mywrite + + def count_lines(self, s): + """Count the number of lines in a given text. + + Before calculation, the tab width and line length of the text are + fetched, so that up-to-date values are used. + + Lines are counted as if the string was wrapped so that lines are never + over linewidth characters long. + + Tabs are considered tabwidth characters long. + """ + # Tab width is configurable + tabwidth = self.editwin.get_tk_tabwidth() + + # Get the Text widget's size + linewidth = self.editwin.text.winfo_width() + # Deduct the border and padding + linewidth -= 2*sum([int(self.editwin.text.cget(opt)) + for opt in ('border', 'padx')]) + + # Get the Text widget's font + font = Font(self.editwin.text, name=self.editwin.text.cget('font')) + # Divide the size of the Text widget by the font's width. + # According to Tk8.5 docs, the Text widget's width is set + # according to the width of its font's '0' (zero) character, + # so we will use this as an approximation. + # see: http://www.tcl.tk/man/tcl8.5/TkCmd/text.htm#M-width + linewidth //= font.measure('0') + + return count_lines_with_wrapping(s, linewidth, tabwidth) + + def squeeze_current_text_event(self, event): + """squeeze-current-text event handler + + Squeeze the block of text inside which contains the "insert" cursor. + + If the insert cursor is not in a squeezable block of text, give the + user a small warning and do nothing. + """ + # set tag_name to the first valid tag found on the "insert" cursor + tag_names = self.text.tag_names(tk.INSERT) + for tag_name in ("stdout", "stderr"): + if tag_name in tag_names: + break + else: + # the insert cursor doesn't have a "stdout" or "stderr" tag + self.text.bell() + return "break" + + # find the range to squeeze + start, end = self.text.tag_prevrange(tag_name, tk.INSERT + "+1c") + s = self.text.get(start, end) + + # if the last char is a newline, remove it from the range + if len(s) > 0 and s[-1] == '\n': + end = self.text.index("%s-1c" % end) + s = s[:-1] + + # delete the text + self.base_text.delete(start, end) + + # prepare an ExpandingButton + numoflines = self.count_lines(s) + expandingbutton = ExpandingButton(s, tag_name, numoflines, self) + + # insert the ExpandingButton to the Text + self.text.window_create(start, window=expandingbutton, + padx=3, pady=5) + + # insert the ExpandingButton to the list of ExpandingButtons, while + # keeping the list ordered according to the position of the buttons in + # the Text widget + i = len(self.expandingbuttons) + while i > 0 and self.text.compare(self.expandingbuttons[i-1], + ">", expandingbutton): + i -= 1 + self.expandingbuttons.insert(i, expandingbutton) + + return "break" + + +Squeezer.reload() + + +if __name__ == "__main__": + from unittest import main + main('idlelib.idle_test.test_squeezer', verbosity=2, exit=False) + + # Add htest. diff --git a/Lib/idlelib/textview.py b/Lib/idlelib/textview.py index 464e6ac6b94e..4867a80db1ab 100644 --- a/Lib/idlelib/textview.py +++ b/Lib/idlelib/textview.py @@ -1,17 +1,37 @@ """Simple text browser for IDLE """ -from tkinter import Toplevel, Text +from tkinter import Toplevel, Text, TclError,\ + HORIZONTAL, VERTICAL, N, S, E, W from tkinter.ttk import Frame, Scrollbar, Button from tkinter.messagebox import showerror from idlelib.colorizer import color_config +class AutoHiddenScrollbar(Scrollbar): + """A scrollbar that is automatically hidden when not needed. + + Only the grid geometry manager is supported. + """ + def set(self, lo, hi): + if float(lo) > 0.0 or float(hi) < 1.0: + self.grid() + else: + self.grid_remove() + super().set(lo, hi) + + def pack(self, **kwargs): + raise TclError(f'{self.__class__.__name__} does not support "pack"') + + def place(self, **kwargs): + raise TclError(f'{self.__class__.__name__} does not support "place"') + + class TextFrame(Frame): "Display text with scrollbar." - def __init__(self, parent, rawtext): + def __init__(self, parent, rawtext, wrap='word'): """Create a frame for Textview. parent - parent widget for this frame @@ -21,27 +41,39 @@ def __init__(self, parent, rawtext): self['relief'] = 'sunken' self['height'] = 700 - self.text = text = Text(self, wrap='word', highlightthickness=0) + self.text = text = Text(self, wrap=wrap, highlightthickness=0) color_config(text) - self.scroll = scroll = Scrollbar(self, orient='vertical', - takefocus=False, command=text.yview) - text['yscrollcommand'] = scroll.set + text.grid(row=0, column=0, sticky=N+S+E+W) + self.grid_rowconfigure(0, weight=1) + self.grid_columnconfigure(0, weight=1) text.insert(0.0, rawtext) text['state'] = 'disabled' text.focus_set() - scroll.pack(side='right', fill='y') - text.pack(side='left', expand=True, fill='both') + # vertical scrollbar + self.yscroll = yscroll = AutoHiddenScrollbar(self, orient=VERTICAL, + takefocus=False, + command=text.yview) + text['yscrollcommand'] = yscroll.set + yscroll.grid(row=0, column=1, sticky=N+S) + + if wrap == 'none': + # horizontal scrollbar + self.xscroll = xscroll = AutoHiddenScrollbar(self, orient=HORIZONTAL, + takefocus=False, + command=text.xview) + text['xscrollcommand'] = xscroll.set + xscroll.grid(row=1, column=0, sticky=E+W) class ViewFrame(Frame): "Display TextFrame and Close button." - def __init__(self, parent, text): + def __init__(self, parent, text, wrap='word'): super().__init__(parent) self.parent = parent self.bind('', self.ok) self.bind('', self.ok) - self.textframe = TextFrame(self, text) + self.textframe = TextFrame(self, text, wrap=wrap) self.button_ok = button_ok = Button( self, text='Close', command=self.ok, takefocus=False) self.textframe.pack(side='top', expand=True, fill='both') @@ -55,7 +87,7 @@ def ok(self, event=None): class ViewWindow(Toplevel): "A simple text viewer dialog for IDLE." - def __init__(self, parent, title, text, modal=True, + def __init__(self, parent, title, text, modal=True, wrap='word', *, _htest=False, _utest=False): """Show the given text in a scrollable window with a 'close' button. @@ -65,6 +97,7 @@ def __init__(self, parent, title, text, modal=True, parent - parent of this dialog title - string which is title of popup dialog text - text to display in dialog + wrap - type of text wrapping to use ('word', 'char' or 'none') _htest - bool; change box location when running htest. _utest - bool; don't wait_window when running unittest. """ @@ -76,7 +109,7 @@ def __init__(self, parent, title, text, modal=True, self.geometry(f'=750x500+{x}+{y}') self.title(title) - self.viewframe = ViewFrame(self, text) + self.viewframe = ViewFrame(self, text, wrap=wrap) self.protocol("WM_DELETE_WINDOW", self.ok) self.button_ok = button_ok = Button(self, text='Close', command=self.ok, takefocus=False) @@ -96,20 +129,22 @@ def ok(self, event=None): self.destroy() -def view_text(parent, title, text, modal=True, _utest=False): +def view_text(parent, title, text, modal=True, wrap='word', _utest=False): """Create text viewer for given text. parent - parent of this dialog title - string which is the title of popup dialog text - text to display in this dialog + wrap - type of text wrapping to use ('word', 'char' or 'none') modal - controls if users can interact with other windows while this dialog is displayed _utest - bool; controls wait_window on unittest """ - return ViewWindow(parent, title, text, modal, _utest=_utest) + return ViewWindow(parent, title, text, modal, wrap=wrap, _utest=_utest) -def view_file(parent, title, filename, encoding, modal=True, _utest=False): +def view_file(parent, title, filename, encoding, modal=True, wrap='word', + _utest=False): """Create text viewer for text in filename. Return error message if file cannot be read. Otherwise calls view_text @@ -127,7 +162,8 @@ def view_file(parent, title, filename, encoding, modal=True, _utest=False): message=str(err), parent=parent) else: - return view_text(parent, title, contents, modal, _utest=_utest) + return view_text(parent, title, contents, modal, wrap=wrap, + _utest=_utest) return None diff --git a/Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst b/Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst new file mode 100644 index 000000000000..cae4af8f2e23 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst @@ -0,0 +1,3 @@ +Enable "squeezing" of long outputs in the shell, to avoid performance +degradation and to clean up the history without losing it. Squeezed outputs +may be copied, viewed in a separate window, and "unsqueezed". From webhook-mailer at python.org Tue Sep 25 08:39:17 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 12:39:17 -0000 Subject: [Python-checkins] bpo-1529353: IDLE: squeeze large output in the shell (GH-7626) Message-ID: https://github.com/python/cpython/commit/0b3e1208033aa1eb5452afe9387f86f299ef24e5 commit: 0b3e1208033aa1eb5452afe9387f86f299ef24e5 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T05:39:12-07:00 summary: bpo-1529353: IDLE: squeeze large output in the shell (GH-7626) (cherry picked from commit 604e7b9931f9e7881a2941816e538f5f15930db8) Co-authored-by: Tal Einat files: A Lib/idlelib/idle_test/test_squeezer.py A Lib/idlelib/squeezer.py A Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst M Lib/idlelib/config-main.def M Lib/idlelib/configdialog.py M Lib/idlelib/editor.py M Lib/idlelib/idle_test/htest.py M Lib/idlelib/idle_test/test_config.py M Lib/idlelib/idle_test/test_textview.py M Lib/idlelib/pyshell.py M Lib/idlelib/textview.py diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def index 16f4b0959cf1..06e3c5adb0e3 100644 --- a/Lib/idlelib/config-main.def +++ b/Lib/idlelib/config-main.def @@ -66,6 +66,9 @@ font-size= 10 font-bold= 0 encoding= none +[PyShell] +auto-squeeze-min-lines= 50 + [Indent] use-spaces= 1 num-spaces= 4 diff --git a/Lib/idlelib/configdialog.py b/Lib/idlelib/configdialog.py index e682ec0da320..229dc8987433 100644 --- a/Lib/idlelib/configdialog.py +++ b/Lib/idlelib/configdialog.py @@ -30,10 +30,12 @@ from idlelib.codecontext import CodeContext from idlelib.parenmatch import ParenMatch from idlelib.paragraph import FormatParagraph +from idlelib.squeezer import Squeezer changes = ConfigChanges() # Reload changed options in the following classes. -reloadables = (AutoComplete, CodeContext, ParenMatch, FormatParagraph) +reloadables = (AutoComplete, CodeContext, ParenMatch, FormatParagraph, + Squeezer) class ConfigDialog(Toplevel): @@ -1748,9 +1750,9 @@ def delete_custom_keys(self): self.customlist.SetMenu(item_list, item_list[0]) # Revert to default key set. self.keyset_source.set(idleConf.defaultCfg['main'] - .Get('Keys', 'default')) + .Get('Keys', 'default')) self.builtin_name.set(idleConf.defaultCfg['main'].Get('Keys', 'name') - or idleConf.default_keys()) + or idleConf.default_keys()) # User can't back out of these changes, they must be applied now. changes.save_all() self.cd.save_all_changed_extensions() @@ -1817,6 +1819,10 @@ def create_page_general(self): frame_context: Frame context_title: Label (*)context_int: Entry - context_lines + frame_shell: LabelFrame + frame_auto_squeeze_min_lines: Frame + auto_squeeze_min_lines_title: Label + (*)auto_squeeze_min_lines_int: Entry - auto_squeeze_min_lines frame_help: LabelFrame frame_helplist: Frame frame_helplist_buttons: Frame @@ -1842,6 +1848,9 @@ def create_page_general(self): self.paren_bell = tracers.add( BooleanVar(self), ('extensions', 'ParenMatch', 'bell')) + self.auto_squeeze_min_lines = tracers.add( + StringVar(self), ('main', 'PyShell', 'auto-squeeze-min-lines')) + self.autosave = tracers.add( IntVar(self), ('main', 'General', 'autosave')) self.format_width = tracers.add( @@ -1855,8 +1864,10 @@ def create_page_general(self): text=' Window Preferences') frame_editor = LabelFrame(self, borderwidth=2, relief=GROOVE, text=' Editor Preferences') + frame_shell = LabelFrame(self, borderwidth=2, relief=GROOVE, + text=' Shell Preferences') frame_help = LabelFrame(self, borderwidth=2, relief=GROOVE, - text=' Additional Help Sources ') + text=' Additional Help Sources ') # Frame_window. frame_run = Frame(frame_window, borderwidth=0) startup_title = Label(frame_run, text='At Startup') @@ -1918,6 +1929,13 @@ def create_page_general(self): self.context_int = Entry( frame_context, textvariable=self.context_lines, width=3) + # Frame_shell. + frame_auto_squeeze_min_lines = Frame(frame_shell, borderwidth=0) + auto_squeeze_min_lines_title = Label(frame_auto_squeeze_min_lines, + text='Auto-Squeeze Min. Lines:') + self.auto_squeeze_min_lines_int = Entry( + frame_auto_squeeze_min_lines, width=4, + textvariable=self.auto_squeeze_min_lines) # frame_help. frame_helplist = Frame(frame_help) @@ -1943,6 +1961,7 @@ def create_page_general(self): # Body. frame_window.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) frame_editor.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) + frame_shell.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) frame_help.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) # frame_run. frame_run.pack(side=TOP, padx=5, pady=0, fill=X) @@ -1983,6 +2002,11 @@ def create_page_general(self): context_title.pack(side=LEFT, anchor=W, padx=5, pady=5) self.context_int.pack(side=TOP, padx=5, pady=5) + # frame_auto_squeeze_min_lines + frame_auto_squeeze_min_lines.pack(side=TOP, padx=5, pady=0, fill=X) + auto_squeeze_min_lines_title.pack(side=LEFT, anchor=W, padx=5, pady=5) + self.auto_squeeze_min_lines_int.pack(side=TOP, padx=5, pady=5) + # frame_help. frame_helplist_buttons.pack(side=RIGHT, padx=5, pady=5, fill=Y) frame_helplist.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH) @@ -2018,6 +2042,10 @@ def load_general_cfg(self): self.context_lines.set(idleConf.GetOption( 'extensions', 'CodeContext', 'maxlines', type='int')) + # Set variables for shell windows. + self.auto_squeeze_min_lines.set(idleConf.GetOption( + 'main', 'PyShell', 'auto-squeeze-min-lines', type='int')) + # Set additional help sources. self.user_helplist = idleConf.GetAllExtraHelpSourcesList() self.helplist.delete(0, 'end') @@ -2211,6 +2239,9 @@ def detach(self): CodeContext: Maxlines is the maximum number of code context lines to display when Code Context is turned on for an editor window. + +Shell Preferences: Auto-Squeeze Min. Lines is the minimum number of lines +of output to automatically "squeeze". ''' } diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py index 227a74deb82d..6689af64c429 100644 --- a/Lib/idlelib/editor.py +++ b/Lib/idlelib/editor.py @@ -2,9 +2,7 @@ import importlib.util import os import platform -import re import string -import sys import tokenize import traceback import webbrowser @@ -50,7 +48,6 @@ class EditorWindow(object): from idlelib.undo import UndoDelegator from idlelib.iomenu import IOBinding, encoding from idlelib import mainmenu - from tkinter import Toplevel, EventType from idlelib.statusbar import MultiStatusBar from idlelib.autocomplete import AutoComplete from idlelib.autoexpand import AutoExpand @@ -59,6 +56,7 @@ class EditorWindow(object): from idlelib.paragraph import FormatParagraph from idlelib.parenmatch import ParenMatch from idlelib.rstrip import Rstrip + from idlelib.squeezer import Squeezer from idlelib.zoomheight import ZoomHeight filesystemencoding = sys.getfilesystemencoding() # for file names @@ -319,6 +317,9 @@ def __init__(self, flist=None, filename=None, key=None, root=None): text.bind("<>", self.ZoomHeight(self).zoom_height_event) text.bind("<>", self.CodeContext(self).toggle_code_context_event) + squeezer = self.Squeezer(self) + text.bind("<>", + squeezer.squeeze_current_text_event) def _filename_to_unicode(self, filename): """Return filename as BMP unicode so diplayable in Tk.""" diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py index 03bee5170735..8c1c24d070cc 100644 --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -163,7 +163,7 @@ def _wrapper(parent): # htest # 'msg': "Click the 'Show GrepDialog' button.\n" "Test the various 'Find-in-files' functions.\n" "The results should be displayed in a new '*Output*' window.\n" - "'Right-click'->'Goto file/line' anywhere in the search results " + "'Right-click'->'Go to file/line' anywhere in the search results " "should open that file \nin a new EditorWindow." } diff --git a/Lib/idlelib/idle_test/test_config.py b/Lib/idlelib/idle_test/test_config.py index f3d9f21dd86c..8c9197284e07 100644 --- a/Lib/idlelib/idle_test/test_config.py +++ b/Lib/idlelib/idle_test/test_config.py @@ -356,11 +356,11 @@ def test_get_section_list(self): self.assertCountEqual( conf.GetSectionList('default', 'main'), - ['General', 'EditorWindow', 'Indent', 'Theme', + ['General', 'EditorWindow', 'PyShell', 'Indent', 'Theme', 'Keys', 'History', 'HelpFiles']) self.assertCountEqual( conf.GetSectionList('user', 'main'), - ['General', 'EditorWindow', 'Indent', 'Theme', + ['General', 'EditorWindow', 'PyShell', 'Indent', 'Theme', 'Keys', 'History', 'HelpFiles']) with self.assertRaises(config.InvalidConfigSet): @@ -452,7 +452,7 @@ def test_remove_key_bind_names(self): self.assertCountEqual( conf.RemoveKeyBindNames(conf.GetSectionList('default', 'extensions')), - ['AutoComplete', 'CodeContext', 'FormatParagraph', 'ParenMatch','ZzDummy']) + ['AutoComplete', 'CodeContext', 'FormatParagraph', 'ParenMatch', 'ZzDummy']) def test_get_extn_name_for_event(self): userextn.read_string(''' diff --git a/Lib/idlelib/idle_test/test_squeezer.py b/Lib/idlelib/idle_test/test_squeezer.py new file mode 100644 index 000000000000..ca8b674cc236 --- /dev/null +++ b/Lib/idlelib/idle_test/test_squeezer.py @@ -0,0 +1,509 @@ +from collections import namedtuple +from tkinter import Text, Tk +import unittest +from unittest.mock import Mock, NonCallableMagicMock, patch, sentinel, ANY +from test.support import requires + +from idlelib.config import idleConf +from idlelib.squeezer import count_lines_with_wrapping, ExpandingButton, \ + Squeezer +from idlelib import macosx +from idlelib.textview import view_text +from idlelib.tooltip import Hovertip +from idlelib.pyshell import PyShell + + +SENTINEL_VALUE = sentinel.SENTINEL_VALUE + + +def get_test_tk_root(test_instance): + """Helper for tests: Create a root Tk object.""" + requires('gui') + root = Tk() + root.withdraw() + + def cleanup_root(): + root.update_idletasks() + root.destroy() + test_instance.addCleanup(cleanup_root) + + return root + + +class CountLinesTest(unittest.TestCase): + """Tests for the count_lines_with_wrapping function.""" + def check(self, expected, text, linewidth, tabwidth): + return self.assertEqual( + expected, + count_lines_with_wrapping(text, linewidth, tabwidth), + ) + + def test_count_empty(self): + """Test with an empty string.""" + self.assertEqual(count_lines_with_wrapping(""), 0) + + def test_count_begins_with_empty_line(self): + """Test with a string which begins with a newline.""" + self.assertEqual(count_lines_with_wrapping("\ntext"), 2) + + def test_count_ends_with_empty_line(self): + """Test with a string which ends with a newline.""" + self.assertEqual(count_lines_with_wrapping("text\n"), 1) + + def test_count_several_lines(self): + """Test with several lines of text.""" + self.assertEqual(count_lines_with_wrapping("1\n2\n3\n"), 3) + + def test_tab_width(self): + """Test with various tab widths and line widths.""" + self.check(expected=1, text='\t' * 1, linewidth=8, tabwidth=4) + self.check(expected=1, text='\t' * 2, linewidth=8, tabwidth=4) + self.check(expected=2, text='\t' * 3, linewidth=8, tabwidth=4) + self.check(expected=2, text='\t' * 4, linewidth=8, tabwidth=4) + self.check(expected=3, text='\t' * 5, linewidth=8, tabwidth=4) + + # test longer lines and various tab widths + self.check(expected=4, text='\t' * 10, linewidth=12, tabwidth=4) + self.check(expected=10, text='\t' * 10, linewidth=12, tabwidth=8) + self.check(expected=2, text='\t' * 4, linewidth=10, tabwidth=3) + + # test tabwidth=1 + self.check(expected=2, text='\t' * 9, linewidth=5, tabwidth=1) + self.check(expected=2, text='\t' * 10, linewidth=5, tabwidth=1) + self.check(expected=3, text='\t' * 11, linewidth=5, tabwidth=1) + + # test for off-by-one errors + self.check(expected=2, text='\t' * 6, linewidth=12, tabwidth=4) + self.check(expected=3, text='\t' * 6, linewidth=11, tabwidth=4) + self.check(expected=2, text='\t' * 6, linewidth=13, tabwidth=4) + + +class SqueezerTest(unittest.TestCase): + """Tests for the Squeezer class.""" + def make_mock_editor_window(self): + """Create a mock EditorWindow instance.""" + editwin = NonCallableMagicMock() + # isinstance(editwin, PyShell) must be true for Squeezer to enable + # auto-squeezing; in practice this will always be true + editwin.__class__ = PyShell + return editwin + + def make_squeezer_instance(self, editor_window=None): + """Create an actual Squeezer instance with a mock EditorWindow.""" + if editor_window is None: + editor_window = self.make_mock_editor_window() + return Squeezer(editor_window) + + def test_count_lines(self): + """Test Squeezer.count_lines() with various inputs. + + This checks that Squeezer.count_lines() calls the + count_lines_with_wrapping() function with the appropriate parameters. + """ + for tabwidth, linewidth in [(4, 80), (1, 79), (8, 80), (3, 120)]: + self._test_count_lines_helper(linewidth=linewidth, + tabwidth=tabwidth) + + def _prepare_mock_editwin_for_count_lines(self, editwin, + linewidth, tabwidth): + """Prepare a mock EditorWindow object for Squeezer.count_lines.""" + CHAR_WIDTH = 10 + BORDER_WIDTH = 2 + PADDING_WIDTH = 1 + + # Prepare all the required functionality on the mock EditorWindow object + # so that the calculations in Squeezer.count_lines() can run. + editwin.get_tk_tabwidth.return_value = tabwidth + editwin.text.winfo_width.return_value = \ + linewidth * CHAR_WIDTH + 2 * (BORDER_WIDTH + PADDING_WIDTH) + text_opts = { + 'border': BORDER_WIDTH, + 'padx': PADDING_WIDTH, + 'font': None, + } + editwin.text.cget = lambda opt: text_opts[opt] + + # monkey-path tkinter.font.Font with a mock object, so that + # Font.measure('0') returns CHAR_WIDTH + mock_font = Mock() + def measure(char): + if char == '0': + return CHAR_WIDTH + raise ValueError("measure should only be called on '0'!") + mock_font.return_value.measure = measure + patcher = patch('idlelib.squeezer.Font', mock_font) + patcher.start() + self.addCleanup(patcher.stop) + + def _test_count_lines_helper(self, linewidth, tabwidth): + """Helper for test_count_lines.""" + editwin = self.make_mock_editor_window() + self._prepare_mock_editwin_for_count_lines(editwin, linewidth, tabwidth) + squeezer = self.make_squeezer_instance(editwin) + + mock_count_lines = Mock(return_value=SENTINEL_VALUE) + text = 'TEXT' + with patch('idlelib.squeezer.count_lines_with_wrapping', + mock_count_lines): + self.assertIs(squeezer.count_lines(text), SENTINEL_VALUE) + mock_count_lines.assert_called_with(text, linewidth, tabwidth) + + def test_init(self): + """Test the creation of Squeezer instances.""" + editwin = self.make_mock_editor_window() + squeezer = self.make_squeezer_instance(editwin) + self.assertIs(squeezer.editwin, editwin) + self.assertEqual(squeezer.expandingbuttons, []) + + def test_write_no_tags(self): + """Test Squeezer's overriding of the EditorWindow's write() method.""" + editwin = self.make_mock_editor_window() + for text in ['', 'TEXT', 'LONG TEXT' * 1000, 'MANY_LINES\n' * 100]: + editwin.write = orig_write = Mock(return_value=SENTINEL_VALUE) + squeezer = self.make_squeezer_instance(editwin) + + self.assertEqual(squeezer.editwin.write(text, ()), SENTINEL_VALUE) + self.assertEqual(orig_write.call_count, 1) + orig_write.assert_called_with(text, ()) + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_write_not_stdout(self): + """Test Squeezer's overriding of the EditorWindow's write() method.""" + for text in ['', 'TEXT', 'LONG TEXT' * 1000, 'MANY_LINES\n' * 100]: + editwin = self.make_mock_editor_window() + editwin.write.return_value = SENTINEL_VALUE + orig_write = editwin.write + squeezer = self.make_squeezer_instance(editwin) + + self.assertEqual(squeezer.editwin.write(text, "stderr"), + SENTINEL_VALUE) + self.assertEqual(orig_write.call_count, 1) + orig_write.assert_called_with(text, "stderr") + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_write_stdout(self): + """Test Squeezer's overriding of the EditorWindow's write() method.""" + editwin = self.make_mock_editor_window() + self._prepare_mock_editwin_for_count_lines(editwin, + linewidth=80, tabwidth=8) + + for text in ['', 'TEXT']: + editwin.write = orig_write = Mock(return_value=SENTINEL_VALUE) + squeezer = self.make_squeezer_instance(editwin) + squeezer.auto_squeeze_min_lines = 50 + + self.assertEqual(squeezer.editwin.write(text, "stdout"), + SENTINEL_VALUE) + self.assertEqual(orig_write.call_count, 1) + orig_write.assert_called_with(text, "stdout") + self.assertEqual(len(squeezer.expandingbuttons), 0) + + for text in ['LONG TEXT' * 1000, 'MANY_LINES\n' * 100]: + editwin.write = orig_write = Mock(return_value=SENTINEL_VALUE) + squeezer = self.make_squeezer_instance(editwin) + squeezer.auto_squeeze_min_lines = 50 + + self.assertEqual(squeezer.editwin.write(text, "stdout"), None) + self.assertEqual(orig_write.call_count, 0) + self.assertEqual(len(squeezer.expandingbuttons), 1) + + def test_auto_squeeze(self): + """Test that the auto-squeezing creates an ExpandingButton properly.""" + root = get_test_tk_root(self) + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.auto_squeeze_min_lines = 5 + squeezer.count_lines = Mock(return_value=6) + + editwin.write('TEXT\n'*6, "stdout") + self.assertEqual(text_widget.get('1.0', 'end'), '\n') + self.assertEqual(len(squeezer.expandingbuttons), 1) + + def test_squeeze_current_text_event(self): + """Test the squeeze_current_text event.""" + root = get_test_tk_root(self) + + # squeezing text should work for both stdout and stderr + for tag_name in ["stdout", "stderr"]: + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = editwin.per.bottom = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.count_lines = Mock(return_value=6) + + # prepare some text in the Text widget + text_widget.insert("1.0", "SOME\nTEXT\n", tag_name) + text_widget.mark_set("insert", "1.0") + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + + self.assertEqual(len(squeezer.expandingbuttons), 0) + + # test squeezing the current text + retval = squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(retval, "break") + self.assertEqual(text_widget.get('1.0', 'end'), '\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 1) + self.assertEqual(squeezer.expandingbuttons[0].s, 'SOME\nTEXT') + + # test that expanding the squeezed text works and afterwards the + # Text widget contains the original text + squeezer.expandingbuttons[0].expand(event=Mock()) + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_squeeze_current_text_event_no_allowed_tags(self): + """Test that the event doesn't squeeze text without a relevant tag.""" + root = get_test_tk_root(self) + + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = editwin.per.bottom = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.count_lines = Mock(return_value=6) + + # prepare some text in the Text widget + text_widget.insert("1.0", "SOME\nTEXT\n", "TAG") + text_widget.mark_set("insert", "1.0") + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + + self.assertEqual(len(squeezer.expandingbuttons), 0) + + # test squeezing the current text + retval = squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(retval, "break") + self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 0) + + def test_squeeze_text_before_existing_squeezed_text(self): + """Test squeezing text before existing squeezed text.""" + root = get_test_tk_root(self) + + text_widget = Text(root) + text_widget.mark_set("iomark", "1.0") + + editwin = self.make_mock_editor_window() + editwin.text = editwin.per.bottom = text_widget + squeezer = self.make_squeezer_instance(editwin) + squeezer.count_lines = Mock(return_value=6) + + # prepare some text in the Text widget and squeeze it + text_widget.insert("1.0", "SOME\nTEXT\n", "stdout") + text_widget.mark_set("insert", "1.0") + squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(len(squeezer.expandingbuttons), 1) + + # test squeezing the current text + text_widget.insert("1.0", "MORE\nSTUFF\n", "stdout") + text_widget.mark_set("insert", "1.0") + retval = squeezer.squeeze_current_text_event(event=Mock()) + self.assertEqual(retval, "break") + self.assertEqual(text_widget.get('1.0', 'end'), '\n\n\n') + self.assertEqual(len(squeezer.expandingbuttons), 2) + self.assertTrue(text_widget.compare( + squeezer.expandingbuttons[0], + '<', + squeezer.expandingbuttons[1], + )) + + GetOptionSignature = namedtuple('GetOptionSignature', + 'configType section option default type warn_on_default raw') + @classmethod + def _make_sig(cls, configType, section, option, default=sentinel.NOT_GIVEN, + type=sentinel.NOT_GIVEN, + warn_on_default=sentinel.NOT_GIVEN, + raw=sentinel.NOT_GIVEN): + return cls.GetOptionSignature(configType, section, option, default, + type, warn_on_default, raw) + + @classmethod + def get_GetOption_signature(cls, mock_call_obj): + args, kwargs = mock_call_obj[-2:] + return cls._make_sig(*args, **kwargs) + + def test_reload(self): + """Test the reload() class-method.""" + self.assertIsInstance(Squeezer.auto_squeeze_min_lines, int) + idleConf.SetOption('main', 'PyShell', 'auto-squeeze-min-lines', '42') + Squeezer.reload() + self.assertEqual(Squeezer.auto_squeeze_min_lines, 42) + + +class ExpandingButtonTest(unittest.TestCase): + """Tests for the ExpandingButton class.""" + # In these tests the squeezer instance is a mock, but actual tkinter + # Text and Button instances are created. + def make_mock_squeezer(self): + """Helper for tests: Create a mock Squeezer object.""" + root = get_test_tk_root(self) + squeezer = Mock() + squeezer.editwin.text = Text(root) + + # Set default values for the configuration settings + squeezer.auto_squeeze_min_lines = 50 + return squeezer + + @patch('idlelib.squeezer.Hovertip', autospec=Hovertip) + def test_init(self, MockHovertip): + """Test the simplest creation of an ExpandingButton.""" + squeezer = self.make_mock_squeezer() + text_widget = squeezer.editwin.text + + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + self.assertEqual(expandingbutton.s, 'TEXT') + + # check that the underlying tkinter.Button is properly configured + self.assertEqual(expandingbutton.master, text_widget) + self.assertTrue('50 lines' in expandingbutton.cget('text')) + + # check that the text widget still contains no text + self.assertEqual(text_widget.get('1.0', 'end'), '\n') + + # check that the mouse events are bound + self.assertIn('', expandingbutton.bind()) + right_button_code = '' % ('2' if macosx.isAquaTk() else '3') + self.assertIn(right_button_code, expandingbutton.bind()) + + # check that ToolTip was called once, with appropriate values + self.assertEqual(MockHovertip.call_count, 1) + MockHovertip.assert_called_with(expandingbutton, ANY, hover_delay=ANY) + + # check that 'right-click' appears in the tooltip text + tooltip_text = MockHovertip.call_args[0][1] + self.assertIn('right-click', tooltip_text.lower()) + + def test_expand(self): + """Test the expand event.""" + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + + # insert the button into the text widget + # (this is normally done by the Squeezer class) + text_widget = expandingbutton.text + text_widget.window_create("1.0", window=expandingbutton) + + # set base_text to the text widget, so that changes are actually made + # to it (by ExpandingButton) and we can inspect these changes afterwards + expandingbutton.base_text = expandingbutton.text + + # trigger the expand event + retval = expandingbutton.expand(event=Mock()) + self.assertEqual(retval, None) + + # check that the text was inserted into the text widget + self.assertEqual(text_widget.get('1.0', 'end'), 'TEXT\n') + + # check that the 'TAGS' tag was set on the inserted text + text_end_index = text_widget.index('end-1c') + self.assertEqual(text_widget.get('1.0', text_end_index), 'TEXT') + self.assertEqual(text_widget.tag_nextrange('TAGS', '1.0'), + ('1.0', text_end_index)) + + # check that the button removed itself from squeezer.expandingbuttons + self.assertEqual(squeezer.expandingbuttons.remove.call_count, 1) + squeezer.expandingbuttons.remove.assert_called_with(expandingbutton) + + def test_expand_dangerous_oupput(self): + """Test that expanding very long output asks user for confirmation.""" + squeezer = self.make_mock_squeezer() + text = 'a' * 10**5 + expandingbutton = ExpandingButton(text, 'TAGS', 50, squeezer) + expandingbutton.set_is_dangerous() + self.assertTrue(expandingbutton.is_dangerous) + + # insert the button into the text widget + # (this is normally done by the Squeezer class) + text_widget = expandingbutton.text + text_widget.window_create("1.0", window=expandingbutton) + + # set base_text to the text widget, so that changes are actually made + # to it (by ExpandingButton) and we can inspect these changes afterwards + expandingbutton.base_text = expandingbutton.text + + # patch the message box module to always return False + with patch('idlelib.squeezer.tkMessageBox') as mock_msgbox: + mock_msgbox.askokcancel.return_value = False + mock_msgbox.askyesno.return_value = False + + # trigger the expand event + retval = expandingbutton.expand(event=Mock()) + + # check that the event chain was broken and no text was inserted + self.assertEqual(retval, 'break') + self.assertEqual(expandingbutton.text.get('1.0', 'end-1c'), '') + + # patch the message box module to always return True + with patch('idlelib.squeezer.tkMessageBox') as mock_msgbox: + mock_msgbox.askokcancel.return_value = True + mock_msgbox.askyesno.return_value = True + + # trigger the expand event + retval = expandingbutton.expand(event=Mock()) + + # check that the event chain wasn't broken and the text was inserted + self.assertEqual(retval, None) + self.assertEqual(expandingbutton.text.get('1.0', 'end-1c'), text) + + def test_copy(self): + """Test the copy event.""" + # testing with the actual clipboard proved problematic, so this test + # replaces the clipboard manipulation functions with mocks and checks + # that they are called appropriately + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + expandingbutton.clipboard_clear = Mock() + expandingbutton.clipboard_append = Mock() + + # trigger the copy event + retval = expandingbutton.copy(event=Mock()) + self.assertEqual(retval, None) + + # check that the expanding button called clipboard_clear() and + # clipboard_append('TEXT') once each + self.assertEqual(expandingbutton.clipboard_clear.call_count, 1) + self.assertEqual(expandingbutton.clipboard_append.call_count, 1) + expandingbutton.clipboard_append.assert_called_with('TEXT') + + def test_view(self): + """Test the view event.""" + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + expandingbutton.selection_own = Mock() + + with patch('idlelib.squeezer.view_text', autospec=view_text)\ + as mock_view_text: + # trigger the view event + expandingbutton.view(event=Mock()) + + # check that the expanding button called view_text + self.assertEqual(mock_view_text.call_count, 1) + + # check that the proper text was passed + self.assertEqual(mock_view_text.call_args[0][2], 'TEXT') + + def test_rmenu(self): + """Test the context menu.""" + squeezer = self.make_mock_squeezer() + expandingbutton = ExpandingButton('TEXT', 'TAGS', 50, squeezer) + with patch('tkinter.Menu') as mock_Menu: + mock_menu = Mock() + mock_Menu.return_value = mock_menu + mock_event = Mock() + mock_event.x = 10 + mock_event.y = 10 + expandingbutton.context_menu_event(event=mock_event) + self.assertEqual(mock_menu.add_command.call_count, + len(expandingbutton.rmenu_specs)) + for label, *data in expandingbutton.rmenu_specs: + mock_menu.add_command.assert_any_call(label=label, command=ANY) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/Lib/idlelib/idle_test/test_textview.py b/Lib/idlelib/idle_test/test_textview.py index 0d11e41e0fb4..6f0c1930518a 100644 --- a/Lib/idlelib/idle_test/test_textview.py +++ b/Lib/idlelib/idle_test/test_textview.py @@ -73,7 +73,6 @@ class TextFrameTest(unittest.TestCase): @classmethod def setUpClass(cls): - "By itself, this tests that file parsed without exception." cls.root = root = Tk() root.withdraw() cls.frame = tv.TextFrame(root, 'test text') @@ -126,11 +125,15 @@ def test_bad_file(self): def test_bad_encoding(self): p = os.path fn = p.abspath(p.join(p.dirname(__file__), '..', 'CREDITS.txt')) - tv.showerror.title = None view = tv.view_file(root, 'Title', fn, 'ascii', modal=False) self.assertIsNone(view) self.assertEqual(tv.showerror.title, 'Unicode Decode Error') + def test_nowrap(self): + view = tv.view_text(root, 'Title', 'test', modal=False, wrap='none') + text_widget = view.viewframe.textframe.text + self.assertEqual(text_widget.cget('wrap'), 'none') + # Call ViewWindow with _utest=True. class ButtonClickTest(unittest.TestCase): diff --git a/Lib/idlelib/pyshell.py b/Lib/idlelib/pyshell.py index 52c11e30dbd5..5458c59dbd7e 100755 --- a/Lib/idlelib/pyshell.py +++ b/Lib/idlelib/pyshell.py @@ -856,6 +856,10 @@ class PyShell(OutputWindow): ("help", "_Help"), ] + # Extend right-click context menu + rmenu_specs = OutputWindow.rmenu_specs + [ + ("Squeeze", "<>"), + ] # New classes from idlelib.history import History diff --git a/Lib/idlelib/squeezer.py b/Lib/idlelib/squeezer.py new file mode 100644 index 000000000000..f5aac813a159 --- /dev/null +++ b/Lib/idlelib/squeezer.py @@ -0,0 +1,355 @@ +"""An IDLE extension to avoid having very long texts printed in the shell. + +A common problem in IDLE's interactive shell is printing of large amounts of +text into the shell. This makes looking at the previous history difficult. +Worse, this can cause IDLE to become very slow, even to the point of being +completely unusable. + +This extension will automatically replace long texts with a small button. +Double-cliking this button will remove it and insert the original text instead. +Middle-clicking will copy the text to the clipboard. Right-clicking will open +the text in a separate viewing window. + +Additionally, any output can be manually "squeezed" by the user. This includes +output written to the standard error stream ("stderr"), such as exception +messages and their tracebacks. +""" +import re + +import tkinter as tk +from tkinter.font import Font +import tkinter.messagebox as tkMessageBox + +from idlelib.config import idleConf +from idlelib.textview import view_text +from idlelib.tooltip import Hovertip +from idlelib import macosx + + +def count_lines_with_wrapping(s, linewidth=80, tabwidth=8): + """Count the number of lines in a given string. + + Lines are counted as if the string was wrapped so that lines are never over + linewidth characters long. + + Tabs are considered tabwidth characters long. + """ + pos = 0 + linecount = 1 + current_column = 0 + + for m in re.finditer(r"[\t\n]", s): + # process the normal chars up to tab or newline + numchars = m.start() - pos + pos += numchars + current_column += numchars + + # deal with tab or newline + if s[pos] == '\n': + linecount += 1 + current_column = 0 + else: + assert s[pos] == '\t' + current_column += tabwidth - (current_column % tabwidth) + + # if a tab passes the end of the line, consider the entire tab as + # being on the next line + if current_column > linewidth: + linecount += 1 + current_column = tabwidth + + pos += 1 # after the tab or newline + + # avoid divmod(-1, linewidth) + if current_column > 0: + # If the length was exactly linewidth, divmod would give (1,0), + # even though a new line hadn't yet been started. The same is true + # if length is any exact multiple of linewidth. Therefore, subtract + # 1 before doing divmod, and later add 1 to the column to + # compensate. + lines, column = divmod(current_column - 1, linewidth) + linecount += lines + current_column = column + 1 + + # process remaining chars (no more tabs or newlines) + current_column += len(s) - pos + # avoid divmod(-1, linewidth) + if current_column > 0: + linecount += (current_column - 1) // linewidth + else: + # the text ended with a newline; don't count an extra line after it + linecount -= 1 + + return linecount + + +class ExpandingButton(tk.Button): + """Class for the "squeezed" text buttons used by Squeezer + + These buttons are displayed inside a Tk Text widget in place of text. A + user can then use the button to replace it with the original text, copy + the original text to the clipboard or view the original text in a separate + window. + + Each button is tied to a Squeezer instance, and it knows to update the + Squeezer instance when it is expanded (and therefore removed). + """ + def __init__(self, s, tags, numoflines, squeezer): + self.s = s + self.tags = tags + self.numoflines = numoflines + self.squeezer = squeezer + self.editwin = editwin = squeezer.editwin + self.text = text = editwin.text + + # the base Text widget of the PyShell object, used to change text + # before the iomark + self.base_text = editwin.per.bottom + + button_text = "Squeezed text (%d lines)." % self.numoflines + tk.Button.__init__(self, text, text=button_text, + background="#FFFFC0", activebackground="#FFFFE0") + + button_tooltip_text = ( + "Double-click to expand, right-click for more options." + ) + Hovertip(self, button_tooltip_text, hover_delay=80) + + self.bind("", self.expand) + if macosx.isAquaTk(): + # AquaTk defines <2> as the right button, not <3>. + self.bind("", self.context_menu_event) + else: + self.bind("", self.context_menu_event) + self.selection_handle( + lambda offset, length: s[int(offset):int(offset) + int(length)]) + + self.is_dangerous = None + self.after_idle(self.set_is_dangerous) + + def set_is_dangerous(self): + dangerous_line_len = 50 * self.text.winfo_width() + self.is_dangerous = ( + self.numoflines > 1000 or + len(self.s) > 50000 or + any( + len(line_match.group(0)) >= dangerous_line_len + for line_match in re.finditer(r'[^\n]+', self.s) + ) + ) + + def expand(self, event=None): + """expand event handler + + This inserts the original text in place of the button in the Text + widget, removes the button and updates the Squeezer instance. + + If the original text is dangerously long, i.e. expanding it could + cause a performance degradation, ask the user for confirmation. + """ + if self.is_dangerous is None: + self.set_is_dangerous() + if self.is_dangerous: + confirm = tkMessageBox.askokcancel( + title="Expand huge output?", + message="\n\n".join([ + "The squeezed output is very long: %d lines, %d chars.", + "Expanding it could make IDLE slow or unresponsive.", + "It is recommended to view or copy the output instead.", + "Really expand?" + ]) % (self.numoflines, len(self.s)), + default=tkMessageBox.CANCEL, + parent=self.text) + if not confirm: + return "break" + + self.base_text.insert(self.text.index(self), self.s, self.tags) + self.base_text.delete(self) + self.squeezer.expandingbuttons.remove(self) + + def copy(self, event=None): + """copy event handler + + Copy the original text to the clipboard. + """ + self.clipboard_clear() + self.clipboard_append(self.s) + + def view(self, event=None): + """view event handler + + View the original text in a separate text viewer window. + """ + view_text(self.text, "Squeezed Output Viewer", self.s, + modal=False, wrap='none') + + rmenu_specs = ( + # item structure: (label, method_name) + ('copy', 'copy'), + ('view', 'view'), + ) + + def context_menu_event(self, event): + self.text.mark_set("insert", "@%d,%d" % (event.x, event.y)) + rmenu = tk.Menu(self.text, tearoff=0) + for label, method_name in self.rmenu_specs: + rmenu.add_command(label=label, command=getattr(self, method_name)) + rmenu.tk_popup(event.x_root, event.y_root) + return "break" + + +class Squeezer: + """Replace long outputs in the shell with a simple button. + + This avoids IDLE's shell slowing down considerably, and even becoming + completely unresponsive, when very long outputs are written. + """ + @classmethod + def reload(cls): + """Load class variables from config.""" + cls.auto_squeeze_min_lines = idleConf.GetOption( + "main", "PyShell", "auto-squeeze-min-lines", + type="int", default=50, + ) + + def __init__(self, editwin): + """Initialize settings for Squeezer. + + editwin is the shell's Editor window. + self.text is the editor window text widget. + self.base_test is the actual editor window Tk text widget, rather than + EditorWindow's wrapper. + self.expandingbuttons is the list of all buttons representing + "squeezed" output. + """ + self.editwin = editwin + self.text = text = editwin.text + + # Get the base Text widget of the PyShell object, used to change text + # before the iomark. PyShell deliberately disables changing text before + # the iomark via its 'text' attribute, which is actually a wrapper for + # the actual Text widget. Squeezer, however, needs to make such changes. + self.base_text = editwin.per.bottom + + self.expandingbuttons = [] + from idlelib.pyshell import PyShell # done here to avoid import cycle + if isinstance(editwin, PyShell): + # If we get a PyShell instance, replace its write method with a + # wrapper, which inserts an ExpandingButton instead of a long text. + def mywrite(s, tags=(), write=editwin.write): + # only auto-squeeze text which has just the "stdout" tag + if tags != "stdout": + return write(s, tags) + + # only auto-squeeze text with at least the minimum + # configured number of lines + numoflines = self.count_lines(s) + if numoflines < self.auto_squeeze_min_lines: + return write(s, tags) + + # create an ExpandingButton instance + expandingbutton = ExpandingButton(s, tags, numoflines, + self) + + # insert the ExpandingButton into the Text widget + text.mark_gravity("iomark", tk.RIGHT) + text.window_create("iomark", window=expandingbutton, + padx=3, pady=5) + text.see("iomark") + text.update() + text.mark_gravity("iomark", tk.LEFT) + + # add the ExpandingButton to the Squeezer's list + self.expandingbuttons.append(expandingbutton) + + editwin.write = mywrite + + def count_lines(self, s): + """Count the number of lines in a given text. + + Before calculation, the tab width and line length of the text are + fetched, so that up-to-date values are used. + + Lines are counted as if the string was wrapped so that lines are never + over linewidth characters long. + + Tabs are considered tabwidth characters long. + """ + # Tab width is configurable + tabwidth = self.editwin.get_tk_tabwidth() + + # Get the Text widget's size + linewidth = self.editwin.text.winfo_width() + # Deduct the border and padding + linewidth -= 2*sum([int(self.editwin.text.cget(opt)) + for opt in ('border', 'padx')]) + + # Get the Text widget's font + font = Font(self.editwin.text, name=self.editwin.text.cget('font')) + # Divide the size of the Text widget by the font's width. + # According to Tk8.5 docs, the Text widget's width is set + # according to the width of its font's '0' (zero) character, + # so we will use this as an approximation. + # see: http://www.tcl.tk/man/tcl8.5/TkCmd/text.htm#M-width + linewidth //= font.measure('0') + + return count_lines_with_wrapping(s, linewidth, tabwidth) + + def squeeze_current_text_event(self, event): + """squeeze-current-text event handler + + Squeeze the block of text inside which contains the "insert" cursor. + + If the insert cursor is not in a squeezable block of text, give the + user a small warning and do nothing. + """ + # set tag_name to the first valid tag found on the "insert" cursor + tag_names = self.text.tag_names(tk.INSERT) + for tag_name in ("stdout", "stderr"): + if tag_name in tag_names: + break + else: + # the insert cursor doesn't have a "stdout" or "stderr" tag + self.text.bell() + return "break" + + # find the range to squeeze + start, end = self.text.tag_prevrange(tag_name, tk.INSERT + "+1c") + s = self.text.get(start, end) + + # if the last char is a newline, remove it from the range + if len(s) > 0 and s[-1] == '\n': + end = self.text.index("%s-1c" % end) + s = s[:-1] + + # delete the text + self.base_text.delete(start, end) + + # prepare an ExpandingButton + numoflines = self.count_lines(s) + expandingbutton = ExpandingButton(s, tag_name, numoflines, self) + + # insert the ExpandingButton to the Text + self.text.window_create(start, window=expandingbutton, + padx=3, pady=5) + + # insert the ExpandingButton to the list of ExpandingButtons, while + # keeping the list ordered according to the position of the buttons in + # the Text widget + i = len(self.expandingbuttons) + while i > 0 and self.text.compare(self.expandingbuttons[i-1], + ">", expandingbutton): + i -= 1 + self.expandingbuttons.insert(i, expandingbutton) + + return "break" + + +Squeezer.reload() + + +if __name__ == "__main__": + from unittest import main + main('idlelib.idle_test.test_squeezer', verbosity=2, exit=False) + + # Add htest. diff --git a/Lib/idlelib/textview.py b/Lib/idlelib/textview.py index 464e6ac6b94e..4867a80db1ab 100644 --- a/Lib/idlelib/textview.py +++ b/Lib/idlelib/textview.py @@ -1,17 +1,37 @@ """Simple text browser for IDLE """ -from tkinter import Toplevel, Text +from tkinter import Toplevel, Text, TclError,\ + HORIZONTAL, VERTICAL, N, S, E, W from tkinter.ttk import Frame, Scrollbar, Button from tkinter.messagebox import showerror from idlelib.colorizer import color_config +class AutoHiddenScrollbar(Scrollbar): + """A scrollbar that is automatically hidden when not needed. + + Only the grid geometry manager is supported. + """ + def set(self, lo, hi): + if float(lo) > 0.0 or float(hi) < 1.0: + self.grid() + else: + self.grid_remove() + super().set(lo, hi) + + def pack(self, **kwargs): + raise TclError(f'{self.__class__.__name__} does not support "pack"') + + def place(self, **kwargs): + raise TclError(f'{self.__class__.__name__} does not support "place"') + + class TextFrame(Frame): "Display text with scrollbar." - def __init__(self, parent, rawtext): + def __init__(self, parent, rawtext, wrap='word'): """Create a frame for Textview. parent - parent widget for this frame @@ -21,27 +41,39 @@ def __init__(self, parent, rawtext): self['relief'] = 'sunken' self['height'] = 700 - self.text = text = Text(self, wrap='word', highlightthickness=0) + self.text = text = Text(self, wrap=wrap, highlightthickness=0) color_config(text) - self.scroll = scroll = Scrollbar(self, orient='vertical', - takefocus=False, command=text.yview) - text['yscrollcommand'] = scroll.set + text.grid(row=0, column=0, sticky=N+S+E+W) + self.grid_rowconfigure(0, weight=1) + self.grid_columnconfigure(0, weight=1) text.insert(0.0, rawtext) text['state'] = 'disabled' text.focus_set() - scroll.pack(side='right', fill='y') - text.pack(side='left', expand=True, fill='both') + # vertical scrollbar + self.yscroll = yscroll = AutoHiddenScrollbar(self, orient=VERTICAL, + takefocus=False, + command=text.yview) + text['yscrollcommand'] = yscroll.set + yscroll.grid(row=0, column=1, sticky=N+S) + + if wrap == 'none': + # horizontal scrollbar + self.xscroll = xscroll = AutoHiddenScrollbar(self, orient=HORIZONTAL, + takefocus=False, + command=text.xview) + text['xscrollcommand'] = xscroll.set + xscroll.grid(row=1, column=0, sticky=E+W) class ViewFrame(Frame): "Display TextFrame and Close button." - def __init__(self, parent, text): + def __init__(self, parent, text, wrap='word'): super().__init__(parent) self.parent = parent self.bind('', self.ok) self.bind('', self.ok) - self.textframe = TextFrame(self, text) + self.textframe = TextFrame(self, text, wrap=wrap) self.button_ok = button_ok = Button( self, text='Close', command=self.ok, takefocus=False) self.textframe.pack(side='top', expand=True, fill='both') @@ -55,7 +87,7 @@ def ok(self, event=None): class ViewWindow(Toplevel): "A simple text viewer dialog for IDLE." - def __init__(self, parent, title, text, modal=True, + def __init__(self, parent, title, text, modal=True, wrap='word', *, _htest=False, _utest=False): """Show the given text in a scrollable window with a 'close' button. @@ -65,6 +97,7 @@ def __init__(self, parent, title, text, modal=True, parent - parent of this dialog title - string which is title of popup dialog text - text to display in dialog + wrap - type of text wrapping to use ('word', 'char' or 'none') _htest - bool; change box location when running htest. _utest - bool; don't wait_window when running unittest. """ @@ -76,7 +109,7 @@ def __init__(self, parent, title, text, modal=True, self.geometry(f'=750x500+{x}+{y}') self.title(title) - self.viewframe = ViewFrame(self, text) + self.viewframe = ViewFrame(self, text, wrap=wrap) self.protocol("WM_DELETE_WINDOW", self.ok) self.button_ok = button_ok = Button(self, text='Close', command=self.ok, takefocus=False) @@ -96,20 +129,22 @@ def ok(self, event=None): self.destroy() -def view_text(parent, title, text, modal=True, _utest=False): +def view_text(parent, title, text, modal=True, wrap='word', _utest=False): """Create text viewer for given text. parent - parent of this dialog title - string which is the title of popup dialog text - text to display in this dialog + wrap - type of text wrapping to use ('word', 'char' or 'none') modal - controls if users can interact with other windows while this dialog is displayed _utest - bool; controls wait_window on unittest """ - return ViewWindow(parent, title, text, modal, _utest=_utest) + return ViewWindow(parent, title, text, modal, wrap=wrap, _utest=_utest) -def view_file(parent, title, filename, encoding, modal=True, _utest=False): +def view_file(parent, title, filename, encoding, modal=True, wrap='word', + _utest=False): """Create text viewer for text in filename. Return error message if file cannot be read. Otherwise calls view_text @@ -127,7 +162,8 @@ def view_file(parent, title, filename, encoding, modal=True, _utest=False): message=str(err), parent=parent) else: - return view_text(parent, title, contents, modal, _utest=_utest) + return view_text(parent, title, contents, modal, wrap=wrap, + _utest=_utest) return None diff --git a/Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst b/Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst new file mode 100644 index 000000000000..cae4af8f2e23 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2018-08-13-16-31-24.bpo-1529353.wXfQJk.rst @@ -0,0 +1,3 @@ +Enable "squeezing" of long outputs in the shell, to avoid performance +degradation and to clean up the history without losing it. Squeezed outputs +may be copied, viewed in a separate window, and "unsqueezed". From webhook-mailer at python.org Tue Sep 25 10:57:42 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 14:57:42 -0000 Subject: [Python-checkins] bpo-32557: allow shutil.disk_usage to take a file path on Windows also (GH-9372) Message-ID: https://github.com/python/cpython/commit/c8c0249c9e8f61ab7670119a5a5278354df27bbb commit: c8c0249c9e8f61ab7670119a5a5278354df27bbb branch: master author: Joe Pamer committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-25T07:57:36-07:00 summary: bpo-32557: allow shutil.disk_usage to take a file path on Windows also (GH-9372) https://bugs.python.org/issue32557 files: A Misc/NEWS.d/next/Windows/2018-09-25-10-39-27.bpo-32557.Rs1bf9.rst M Doc/library/shutil.rst M Lib/test/test_shutil.py M Modules/posixmodule.c diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index c692cf43b6d7..d8826277a46c 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -343,11 +343,14 @@ Directory and files operations Return disk usage statistics about the given path as a :term:`named tuple` with the attributes *total*, *used* and *free*, which are the amount of - total, used and free space, in bytes. On Windows, *path* must be a - directory; on Unix, it can be a file or directory. + total, used and free space, in bytes. *path* may be a file or a + directory. .. versionadded:: 3.3 + .. versionchanged:: 3.8 + On Windows, *path* can now be a file or directory. + Availability: Unix, Windows. .. function:: chown(path, user=None, group=None) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index a169c3684f2d..9db6aec19206 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1363,6 +1363,7 @@ def _boo(filename, extract_dir, extra): "disk_usage not available on this platform") def test_disk_usage(self): usage = shutil.disk_usage(os.path.dirname(__file__)) + self.assertEqual(usage, shutil.disk_usage(__file__)) self.assertGreater(usage.total, 0) self.assertGreater(usage.used, 0) self.assertGreaterEqual(usage.free, 0) diff --git a/Misc/NEWS.d/next/Windows/2018-09-25-10-39-27.bpo-32557.Rs1bf9.rst b/Misc/NEWS.d/next/Windows/2018-09-25-10-39-27.bpo-32557.Rs1bf9.rst new file mode 100644 index 000000000000..d93c55ac62fe --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-25-10-39-27.bpo-32557.Rs1bf9.rst @@ -0,0 +1 @@ +Allow shutil.disk_usage to take a file path on Windows diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 400ed979821d..c7223ab52055 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -10079,13 +10079,38 @@ os__getdiskusage_impl(PyObject *module, path_t *path) { BOOL retval; ULARGE_INTEGER _, total, free; + DWORD err = 0; Py_BEGIN_ALLOW_THREADS retval = GetDiskFreeSpaceExW(path->wide, &_, &total, &free); Py_END_ALLOW_THREADS - if (retval == 0) - return PyErr_SetFromWindowsErr(0); + if (retval == 0) { + if (GetLastError() == ERROR_DIRECTORY) { + wchar_t *dir_path = NULL; + + dir_path = PyMem_New(wchar_t, path->length + 1); + if (dir_path == NULL) { + return PyErr_NoMemory(); + } + + wcscpy_s(dir_path, path->length + 1, path->wide); + + if (_dirnameW(dir_path) != -1) { + Py_BEGIN_ALLOW_THREADS + retval = GetDiskFreeSpaceExW(dir_path, &_, &total, &free); + Py_END_ALLOW_THREADS + } + /* Record the last error in case it's modified by PyMem_Free. */ + err = GetLastError(); + PyMem_Free(dir_path); + if (retval) { + goto success; + } + } + return PyErr_SetFromWindowsErr(err); + } +success: return Py_BuildValue("(LL)", total.QuadPart, free.QuadPart); } #endif /* MS_WINDOWS */ From webhook-mailer at python.org Tue Sep 25 11:27:21 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 25 Sep 2018 15:27:21 -0000 Subject: [Python-checkins] bpo-34687: Make asynico use ProactorEventLoop by default (GH-9538) Message-ID: https://github.com/python/cpython/commit/6ea29c5e90dde6c240bd8e0815614b52ac307ea1 commit: 6ea29c5e90dde6c240bd8e0815614b52ac307ea1 branch: master author: Victor Stinner committer: Yury Selivanov date: 2018-09-25T11:27:08-04:00 summary: bpo-34687: Make asynico use ProactorEventLoop by default (GH-9538) files: A Misc/NEWS.d/next/Library/2018-09-24-17-14-57.bpo-34687.Fku_8S.rst M Doc/library/asyncio-platforms.rst M Doc/library/asyncio-policy.rst M Doc/whatsnew/3.8.rst M Lib/asyncio/windows_events.py M Lib/test/test_asyncio/test_base_events.py M Lib/test/test_asyncio/test_streams.py diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst index f8ecb58d3a01..81d840e23277 100644 --- a/Doc/library/asyncio-platforms.rst +++ b/Doc/library/asyncio-platforms.rst @@ -23,6 +23,10 @@ All Platforms Windows ======= +.. versionchanged:: 3.8 + + On Windows, :class:`ProactorEventLoop` is now the default event loop. + All event loops on Windows do not support the following methods: * :meth:`loop.create_unix_connection` and @@ -67,16 +71,8 @@ Windows configuration. Subprocess Support on Windows ----------------------------- -:class:`SelectorEventLoop` on Windows does not support subproceses. -On Windows, :class:`ProactorEventLoop` should be used instead:: - - import asyncio - - asyncio.set_event_loop_policy( - asyncio.WindowsProactorEventLoopPolicy()) - - asyncio.run(your_code()) - +On Windows, the default event loop :class:`ProactorEventLoop` supports +subprocesses, whereas :class:`SelectorEventLoop` does not. The :meth:`policy.set_child_watcher() ` function is also diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst index 42f936da468e..560f8b3135e1 100644 --- a/Doc/library/asyncio-policy.rst +++ b/Doc/library/asyncio-policy.rst @@ -92,11 +92,23 @@ asyncio ships with the following built-in policies: .. class:: DefaultEventLoopPolicy The default asyncio policy. Uses :class:`SelectorEventLoop` - on both Unix and Windows platforms. + on Unix and :class:`ProactorEventLoop` on Windows. There is no need to install the default policy manually. asyncio is configured to use the default policy automatically. + .. versionchanged:: 3.8 + + On Windows, :class:`ProactorEventLoop` is now used by default. + + +.. class:: WindowsSelectorEventLoopPolicy + + An alternative event loop policy that uses the + :class:`SelectorEventLoop` event loop implementation. + + Availability: Windows. + .. class:: WindowsProactorEventLoopPolicy diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index e37a70f32d99..89764c8cd2bc 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -116,6 +116,11 @@ New Modules Improved Modules ================ +asyncio +------- + +On Windows, the default event loop is now :class:`~asyncio.ProactorEventLoop`. + os.path ------- diff --git a/Lib/asyncio/windows_events.py b/Lib/asyncio/windows_events.py index fdde8e9e0bf4..772ddf4dfebd 100644 --- a/Lib/asyncio/windows_events.py +++ b/Lib/asyncio/windows_events.py @@ -811,4 +811,4 @@ class WindowsProactorEventLoopPolicy(events.BaseDefaultEventLoopPolicy): _loop_factory = ProactorEventLoop -DefaultEventLoopPolicy = WindowsSelectorEventLoopPolicy +DefaultEventLoopPolicy = WindowsProactorEventLoopPolicy diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py index fe3c38371d0c..95f4f6b82d53 100644 --- a/Lib/test/test_asyncio/test_base_events.py +++ b/Lib/test/test_asyncio/test_base_events.py @@ -1014,7 +1014,7 @@ class BaseEventLoopWithSelectorTests(test_utils.TestCase): def setUp(self): super().setUp() - self.loop = asyncio.new_event_loop() + self.loop = asyncio.SelectorEventLoop() self.set_event_loop(self.loop) @mock.patch('socket.getnameinfo') diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py index d8e371510dea..c529e5208c99 100644 --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -816,7 +816,8 @@ def server(): addr = q.get() # Should not be stuck in an infinite loop. - with self.assertRaises((ConnectionResetError, BrokenPipeError)): + with self.assertRaises((ConnectionResetError, ConnectionAbortedError, + BrokenPipeError)): self.loop.run_until_complete(client(*addr)) # Clean up the thread. (Only on success; on failure, it may diff --git a/Misc/NEWS.d/next/Library/2018-09-24-17-14-57.bpo-34687.Fku_8S.rst b/Misc/NEWS.d/next/Library/2018-09-24-17-14-57.bpo-34687.Fku_8S.rst new file mode 100644 index 000000000000..0e203c4f2786 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-24-17-14-57.bpo-34687.Fku_8S.rst @@ -0,0 +1,2 @@ +On Windows, asyncio now uses ProactorEventLoop, instead of +SelectorEventLoop, by default. From webhook-mailer at python.org Tue Sep 25 11:30:21 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 25 Sep 2018 15:30:21 -0000 Subject: [Python-checkins] bpo-33937: Catch ENOMEM error in test_socket (#9557) Message-ID: https://github.com/python/cpython/commit/46f40be8b907854deb81c6132b7cb038e9e5202a commit: 46f40be8b907854deb81c6132b7cb038e9e5202a branch: master author: Victor Stinner committer: GitHub date: 2018-09-25T08:30:15-07:00 summary: bpo-33937: Catch ENOMEM error in test_socket (#9557) Fix test_socket.SendmsgSCTPStreamTest: catch ENOMEM error. testSendmsgTimeout() and testSendmsgDontWait() randomly fail on Travis CI with: "OSError: [Errno 12] Cannot allocate memory". files: M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index f4d58ebf7157..bd4fad1f6380 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -2616,9 +2616,18 @@ def testSendmsgTimeout(self): def _testSendmsgTimeout(self): try: self.cli_sock.settimeout(0.03) - with self.assertRaises(socket.timeout): + try: while True: self.sendmsgToServer([b"a"*512]) + except socket.timeout: + pass + except OSError as exc: + if exc.errno != errno.ENOMEM: + raise + # bpo-33937 the test randomly fails on Travis CI with + # "OSError: [Errno 12] Cannot allocate memory" + else: + self.fail("socket.timeout not raised") finally: self.misc_event.set() @@ -2641,8 +2650,10 @@ def _testSendmsgDontWait(self): with self.assertRaises(OSError) as cm: while True: self.sendmsgToServer([b"a"*512], [], socket.MSG_DONTWAIT) + # bpo-33937: catch also ENOMEM, the test randomly fails on Travis CI + # with "OSError: [Errno 12] Cannot allocate memory" self.assertIn(cm.exception.errno, - (errno.EAGAIN, errno.EWOULDBLOCK)) + (errno.EAGAIN, errno.EWOULDBLOCK, errno.ENOMEM)) finally: self.misc_event.set() From webhook-mailer at python.org Tue Sep 25 11:49:36 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 15:49:36 -0000 Subject: [Python-checkins] bpo-33937: Catch ENOMEM error in test_socket (GH-9557) Message-ID: https://github.com/python/cpython/commit/ef1173ab141ba5387598f8859ba96f98d20d743e commit: ef1173ab141ba5387598f8859ba96f98d20d743e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T08:49:28-07:00 summary: bpo-33937: Catch ENOMEM error in test_socket (GH-9557) Fix test_socket.SendmsgSCTPStreamTest: catch ENOMEM error. testSendmsgTimeout() and testSendmsgDontWait() randomly fail on Travis CI with: "OSError: [Errno 12] Cannot allocate memory". (cherry picked from commit 46f40be8b907854deb81c6132b7cb038e9e5202a) Co-authored-by: Victor Stinner files: M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index f4d58ebf7157..bd4fad1f6380 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -2616,9 +2616,18 @@ def testSendmsgTimeout(self): def _testSendmsgTimeout(self): try: self.cli_sock.settimeout(0.03) - with self.assertRaises(socket.timeout): + try: while True: self.sendmsgToServer([b"a"*512]) + except socket.timeout: + pass + except OSError as exc: + if exc.errno != errno.ENOMEM: + raise + # bpo-33937 the test randomly fails on Travis CI with + # "OSError: [Errno 12] Cannot allocate memory" + else: + self.fail("socket.timeout not raised") finally: self.misc_event.set() @@ -2641,8 +2650,10 @@ def _testSendmsgDontWait(self): with self.assertRaises(OSError) as cm: while True: self.sendmsgToServer([b"a"*512], [], socket.MSG_DONTWAIT) + # bpo-33937: catch also ENOMEM, the test randomly fails on Travis CI + # with "OSError: [Errno 12] Cannot allocate memory" self.assertIn(cm.exception.errno, - (errno.EAGAIN, errno.EWOULDBLOCK)) + (errno.EAGAIN, errno.EWOULDBLOCK, errno.ENOMEM)) finally: self.misc_event.set() From webhook-mailer at python.org Tue Sep 25 12:00:55 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 16:00:55 -0000 Subject: [Python-checkins] bpo-33937: Catch ENOMEM error in test_socket (GH-9557) Message-ID: https://github.com/python/cpython/commit/857b0028c0832c3159927489bc55f802bc5146cb commit: 857b0028c0832c3159927489bc55f802bc5146cb branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T09:00:51-07:00 summary: bpo-33937: Catch ENOMEM error in test_socket (GH-9557) Fix test_socket.SendmsgSCTPStreamTest: catch ENOMEM error. testSendmsgTimeout() and testSendmsgDontWait() randomly fail on Travis CI with: "OSError: [Errno 12] Cannot allocate memory". (cherry picked from commit 46f40be8b907854deb81c6132b7cb038e9e5202a) Co-authored-by: Victor Stinner files: M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index cd249ff9b444..f8559ac4eab9 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -2287,9 +2287,18 @@ def testSendmsgTimeout(self): def _testSendmsgTimeout(self): try: self.cli_sock.settimeout(0.03) - with self.assertRaises(socket.timeout): + try: while True: self.sendmsgToServer([b"a"*512]) + except socket.timeout: + pass + except OSError as exc: + if exc.errno != errno.ENOMEM: + raise + # bpo-33937 the test randomly fails on Travis CI with + # "OSError: [Errno 12] Cannot allocate memory" + else: + self.fail("socket.timeout not raised") finally: self.misc_event.set() @@ -2312,8 +2321,10 @@ def _testSendmsgDontWait(self): with self.assertRaises(OSError) as cm: while True: self.sendmsgToServer([b"a"*512], [], socket.MSG_DONTWAIT) + # bpo-33937: catch also ENOMEM, the test randomly fails on Travis CI + # with "OSError: [Errno 12] Cannot allocate memory" self.assertIn(cm.exception.errno, - (errno.EAGAIN, errno.EWOULDBLOCK)) + (errno.EAGAIN, errno.EWOULDBLOCK, errno.ENOMEM)) finally: self.misc_event.set() From webhook-mailer at python.org Tue Sep 25 12:09:00 2018 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 25 Sep 2018 16:09:00 -0000 Subject: [Python-checkins] bpo-1529353: IDLE: Squeezer What's New for 3.6.7 (#9567) Message-ID: https://github.com/python/cpython/commit/dac712d51667227ce3862fc61be6a8094b1066fa commit: dac712d51667227ce3862fc61be6a8094b1066fa branch: master author: Terry Jan Reedy committer: GitHub date: 2018-09-25T12:08:54-04:00 summary: bpo-1529353: IDLE: Squeezer What's New for 3.6.7 (#9567) files: M Doc/whatsnew/3.6.rst diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index b413b5a65d52..d5358b3efeae 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1177,13 +1177,22 @@ Editor code context option revised. Box displays all context lines up to maxlines. Clicking on a context line jumps the editor to that line. Context colors for custom themes is added to Highlights tab of Settings dialog. (Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, -:issue:`33768`, and :issue:`33679`) +:issue:`33768`, and :issue:`33679`.) On Windows, a new API call tells Windows that tk scales for DPI. On Windows 8.1+ or 10, with DPI compatibility properties of the Python binary unchanged, and a monitor resolution greater than 96 DPI, this should make text and lines sharper. It should otherwise have no effect. -(Contributed by Terry Jan Reedy in :issue:`33656`). +(Contributed by Terry Jan Reedy in :issue:`33656`.) + +New in 3.6.7: + +Output over N lines (50 by default) is squeezed down to a button. +N can be changed in the PyShell section of the General page of the +Settings dialog. Fewer, but possibly extra long, lines can be squeezed by +right clicking on the output. Squeezed output can be expanded in place +by double-clicking the button or into the clipboard or a separate window +by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) importlib From webhook-mailer at python.org Tue Sep 25 12:09:55 2018 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 25 Sep 2018 16:09:55 -0000 Subject: [Python-checkins] bpo-1529353: IDLE - Squeezer What's New for 3.7.1 (#9568) Message-ID: https://github.com/python/cpython/commit/ea718d377db3941ecfc86288a3465fce653cc682 commit: ea718d377db3941ecfc86288a3465fce653cc682 branch: master author: Terry Jan Reedy committer: GitHub date: 2018-09-25T12:09:43-04:00 summary: bpo-1529353: IDLE - Squeezer What's New for 3.7.1 (#9568) files: M Doc/whatsnew/3.7.rst diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index a2c5c284f255..32704593410d 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -998,13 +998,22 @@ Editor code context option revised. Box displays all context lines up to maxlines. Clicking on a context line jumps the editor to that line. Context colors for custom themes is added to Highlights tab of Settings dialog. (Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, -:issue:`33768`, and :issue:`33679`, +:issue:`33768`, and :issue:`33679`.) On Windows, a new API call tells Windows that tk scales for DPI. On Windows 8.1+ or 10, with DPI compatibility properties of the Python binary unchanged, and a monitor resolution greater than 96 DPI, this should make text and lines sharper. It should otherwise have no effect. -(Contributed by Terry Jan Reedy in :issue:`33656`). +(Contributed by Terry Jan Reedy in :issue:`33656`.) + +New in 3.7.1: + +Output over N lines (50 by default) is squeezed down to a button. +N can be changed in the PyShell section of the General page of the +Settings dialog. Fewer, but possibly extra long, lines can be squeezed by +right clicking on the output. Squeezed output can be expanded in place +by double-clicking the button or into the clipboard or a separate window +by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) The changes above have been backported to 3.6 maintenance releases. From webhook-mailer at python.org Tue Sep 25 12:15:07 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 16:15:07 -0000 Subject: [Python-checkins] bpo-1529353: IDLE: Squeezer What's New for 3.6.7 (GH-9567) Message-ID: https://github.com/python/cpython/commit/98c8236cc75529407e305f26de95d9e72a72a0eb commit: 98c8236cc75529407e305f26de95d9e72a72a0eb branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T09:15:03-07:00 summary: bpo-1529353: IDLE: Squeezer What's New for 3.6.7 (GH-9567) (cherry picked from commit dac712d51667227ce3862fc61be6a8094b1066fa) Co-authored-by: Terry Jan Reedy files: M Doc/whatsnew/3.6.rst diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 08d2d24294d0..96a8831cd6ff 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1176,13 +1176,22 @@ Editor code context option revised. Box displays all context lines up to maxlines. Clicking on a context line jumps the editor to that line. Context colors for custom themes is added to Highlights tab of Settings dialog. (Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, -:issue:`33768`, and :issue:`33679`) +:issue:`33768`, and :issue:`33679`.) On Windows, a new API call tells Windows that tk scales for DPI. On Windows 8.1+ or 10, with DPI compatibility properties of the Python binary unchanged, and a monitor resolution greater than 96 DPI, this should make text and lines sharper. It should otherwise have no effect. -(Contributed by Terry Jan Reedy in :issue:`33656`). +(Contributed by Terry Jan Reedy in :issue:`33656`.) + +New in 3.6.7: + +Output over N lines (50 by default) is squeezed down to a button. +N can be changed in the PyShell section of the General page of the +Settings dialog. Fewer, but possibly extra long, lines can be squeezed by +right clicking on the output. Squeezed output can be expanded in place +by double-clicking the button or into the clipboard or a separate window +by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) importlib From webhook-mailer at python.org Tue Sep 25 12:15:17 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 16:15:17 -0000 Subject: [Python-checkins] bpo-1529353: IDLE: Squeezer What's New for 3.6.7 (GH-9567) Message-ID: https://github.com/python/cpython/commit/92ad2612bef198f2e3f8f09bf552189e27afcc4e commit: 92ad2612bef198f2e3f8f09bf552189e27afcc4e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T09:15:14-07:00 summary: bpo-1529353: IDLE: Squeezer What's New for 3.6.7 (GH-9567) (cherry picked from commit dac712d51667227ce3862fc61be6a8094b1066fa) Co-authored-by: Terry Jan Reedy files: M Doc/whatsnew/3.6.rst diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index b413b5a65d52..d5358b3efeae 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1177,13 +1177,22 @@ Editor code context option revised. Box displays all context lines up to maxlines. Clicking on a context line jumps the editor to that line. Context colors for custom themes is added to Highlights tab of Settings dialog. (Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, -:issue:`33768`, and :issue:`33679`) +:issue:`33768`, and :issue:`33679`.) On Windows, a new API call tells Windows that tk scales for DPI. On Windows 8.1+ or 10, with DPI compatibility properties of the Python binary unchanged, and a monitor resolution greater than 96 DPI, this should make text and lines sharper. It should otherwise have no effect. -(Contributed by Terry Jan Reedy in :issue:`33656`). +(Contributed by Terry Jan Reedy in :issue:`33656`.) + +New in 3.6.7: + +Output over N lines (50 by default) is squeezed down to a button. +N can be changed in the PyShell section of the General page of the +Settings dialog. Fewer, but possibly extra long, lines can be squeezed by +right clicking on the output. Squeezed output can be expanded in place +by double-clicking the button or into the clipboard or a separate window +by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) importlib From webhook-mailer at python.org Tue Sep 25 12:18:02 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 16:18:02 -0000 Subject: [Python-checkins] bpo-1529353: IDLE - Squeezer What's New for 3.7.1 (GH-9568) Message-ID: https://github.com/python/cpython/commit/3637e68d7c92eda0b80e6ab3f58610e1cfb4f1d8 commit: 3637e68d7c92eda0b80e6ab3f58610e1cfb4f1d8 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T09:17:57-07:00 summary: bpo-1529353: IDLE - Squeezer What's New for 3.7.1 (GH-9568) (cherry picked from commit ea718d377db3941ecfc86288a3465fce653cc682) Co-authored-by: Terry Jan Reedy files: M Doc/whatsnew/3.7.rst diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index e988aecf9683..534b9a07a081 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -998,13 +998,22 @@ Editor code context option revised. Box displays all context lines up to maxlines. Clicking on a context line jumps the editor to that line. Context colors for custom themes is added to Highlights tab of Settings dialog. (Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, -:issue:`33768`, and :issue:`33679`, +:issue:`33768`, and :issue:`33679`.) On Windows, a new API call tells Windows that tk scales for DPI. On Windows 8.1+ or 10, with DPI compatibility properties of the Python binary unchanged, and a monitor resolution greater than 96 DPI, this should make text and lines sharper. It should otherwise have no effect. -(Contributed by Terry Jan Reedy in :issue:`33656`). +(Contributed by Terry Jan Reedy in :issue:`33656`.) + +New in 3.7.1: + +Output over N lines (50 by default) is squeezed down to a button. +N can be changed in the PyShell section of the General page of the +Settings dialog. Fewer, but possibly extra long, lines can be squeezed by +right clicking on the output. Squeezed output can be expanded in place +by double-clicking the button or into the clipboard or a separate window +by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) The changes above have been backported to 3.6 maintenance releases. From webhook-mailer at python.org Tue Sep 25 12:45:33 2018 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 25 Sep 2018 16:45:33 -0000 Subject: [Python-checkins] bpo-1529353: IDLE - Squeezer What's New for 3.8 (#9572) Message-ID: https://github.com/python/cpython/commit/fdcb5ae25c0b5c82a32955617d253810ef110cac commit: fdcb5ae25c0b5c82a32955617d253810ef110cac branch: master author: Terry Jan Reedy committer: GitHub date: 2018-09-25T12:45:27-04:00 summary: bpo-1529353: IDLE - Squeezer What's New for 3.8 (#9572) files: M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 89764c8cd2bc..7fbe98105d12 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -121,6 +121,20 @@ asyncio On Windows, the default event loop is now :class:`~asyncio.ProactorEventLoop`. + +idlelib and IDLE +---------------- + +Output over N lines (50 by default) is squeezed down to a button. +N can be changed in the PyShell section of the General page of the +Settings dialog. Fewer, but possibly extra long, lines can be squeezed by +right clicking on the output. Squeezed output can be expanded in place +by double-clicking the button or into the clipboard or a separate window +by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) + +The changes above have been backported to 3.7 maintenance releases. + + os.path ------- From webhook-mailer at python.org Tue Sep 25 13:41:32 2018 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 25 Sep 2018 17:41:32 -0000 Subject: [Python-checkins] bpo-34162: idlelib/NEWS.txt entry for squeezer (GH-9573) Message-ID: https://github.com/python/cpython/commit/22ef31d0b4b497eda5e356528c3e1d29584d6757 commit: 22ef31d0b4b497eda5e356528c3e1d29584d6757 branch: master author: Terry Jan Reedy committer: GitHub date: 2018-09-25T13:41:25-04:00 summary: bpo-34162: idlelib/NEWS.txt entry for squeezer (GH-9573) files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 04e56c5a4456..5b4e4f8c953b 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,13 @@ Released on 2019-10-20? ====================================== +bpo-1529353: Output over N lines (50 by default) is squeezed down to a button. +N can be changed in the PyShell section of the General page of the +Settings dialog. Fewer, but possibly extra long, lines can be squeezed by +right clicking on the output. Squeezed output can be expanded in place +by double-clicking the button or into the clipboard or a separate window +by right-clicking the button. + bpo-34548: Use configured color theme for read-only text views. bpo-33839: Refactor ToolTip and CallTip classes; add documentation From webhook-mailer at python.org Tue Sep 25 14:02:10 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 18:02:10 -0000 Subject: [Python-checkins] bpo-34162: idlelib/NEWS.txt entry for squeezer (GH-9573) Message-ID: https://github.com/python/cpython/commit/b2ae5502981ae7e8fc080f5155060235696afe22 commit: b2ae5502981ae7e8fc080f5155060235696afe22 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T11:02:06-07:00 summary: bpo-34162: idlelib/NEWS.txt entry for squeezer (GH-9573) (cherry picked from commit 22ef31d0b4b497eda5e356528c3e1d29584d6757) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index c8be6d03e126..3bfe01fa4fbb 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,13 @@ Released on 2018-07-31? ====================================== +bpo-1529353: Output over N lines (50 by default) is squeezed down to a button. +N can be changed in the PyShell section of the General page of the +Settings dialog. Fewer, but possibly extra long, lines can be squeezed by +right clicking on the output. Squeezed output can be expanded in place +by double-clicking the button or into the clipboard or a separate window +by right-clicking the button. + bpo-34548: Use configured color theme for read-only text views. bpo-33839: Refactor ToolTip and CallTip classes; add documentation From webhook-mailer at python.org Tue Sep 25 14:06:59 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 18:06:59 -0000 Subject: [Python-checkins] bpo-34162: idlelib/NEWS.txt entry for squeezer (GH-9573) Message-ID: https://github.com/python/cpython/commit/dc335ae77dfc1fb6a500eb1cd0baf23fcda45434 commit: dc335ae77dfc1fb6a500eb1cd0baf23fcda45434 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T11:06:56-07:00 summary: bpo-34162: idlelib/NEWS.txt entry for squeezer (GH-9573) (cherry picked from commit 22ef31d0b4b497eda5e356528c3e1d29584d6757) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 085060d53b5c..84880cc45b29 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,13 @@ Released on 2018-09-24? ====================================== +bpo-1529353: Output over N lines (50 by default) is squeezed down to a button. +N can be changed in the PyShell section of the General page of the +Settings dialog. Fewer, but possibly extra long, lines can be squeezed by +right clicking on the output. Squeezed output can be expanded in place +by double-clicking the button or into the clipboard or a separate window +by right-clicking the button. + bpo-34548: Use configured color theme for read-only text views. bpo-33839: Refactor ToolTip and CallTip classes; add documentation From webhook-mailer at python.org Tue Sep 25 14:51:24 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 25 Sep 2018 18:51:24 -0000 Subject: [Python-checkins] bpo-34790: [docs] Passing coroutines to asyncio.wait() can be confusing. (GH-9543) Message-ID: https://github.com/python/cpython/commit/996859a90df51f84eab47351702cb59c6db4428a commit: 996859a90df51f84eab47351702cb59c6db4428a branch: master author: Yury Selivanov committer: GitHub date: 2018-09-25T14:51:21-04:00 summary: bpo-34790: [docs] Passing coroutines to asyncio.wait() can be confusing. (GH-9543) files: A Misc/NEWS.d/next/Documentation/2018-09-24-12-47-08.bpo-34790.G2KXIH.rst M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index e995fb6391ad..bb693d7a6475 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -472,14 +472,20 @@ Waiting Primitives return_when=ALL_COMPLETED) Run :ref:`awaitable objects ` in the *aws* - sequence concurrently and block until the condition specified + set concurrently and block until the condition specified by *return_when*. If any awaitable in *aws* is a coroutine, it is automatically - scheduled as a Task. + scheduled as a Task. Passing coroutines objects to + ``wait()`` directly is deprecated as it leads to + :ref:`confusing behavior `. Returns two sets of Tasks/Futures: ``(done, pending)``. + Usage:: + + done, pending = await asyncio.wait(aws) + The *loop* argument is deprecated and scheduled for removal in Python 4.0. @@ -514,9 +520,35 @@ Waiting Primitives Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the futures when a timeout occurs. - Usage:: + .. _asyncio_example_wait_coroutine: + .. note:: - done, pending = await asyncio.wait(aws) + ``wait()`` schedules coroutines as Tasks automatically and later + returns those implicitly created Task objects in ``(done, pending)`` + sets. Therefore the following code won't work as expected:: + + async def foo(): + return 42 + + coro = foo() + done, pending = await asyncio.wait({coro}) + + if coro in done: + # This branch will never be run! + + Here is how the above snippet can be fixed:: + + async def foo(): + return 42 + + task = asyncio.create_task(foo()) + done, pending = await asyncio.wait({task}) + + if task in done: + # Everything will work as expected now. + + Passing coroutine objects to ``wait()`` directly is + deprecated. .. function:: as_completed(aws, \*, loop=None, timeout=None) diff --git a/Misc/NEWS.d/next/Documentation/2018-09-24-12-47-08.bpo-34790.G2KXIH.rst b/Misc/NEWS.d/next/Documentation/2018-09-24-12-47-08.bpo-34790.G2KXIH.rst new file mode 100644 index 000000000000..dc3de2c7d4c8 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-09-24-12-47-08.bpo-34790.G2KXIH.rst @@ -0,0 +1 @@ +Document how passing coroutines to asyncio.wait() can be confusing. From webhook-mailer at python.org Tue Sep 25 14:57:54 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 25 Sep 2018 18:57:54 -0000 Subject: [Python-checkins] bpo-34790: [docs] Passing coroutines to asyncio.wait() can be confusing. (GH-9543) Message-ID: https://github.com/python/cpython/commit/3cc9557d9f03086fa3b0ea166f8506087e6950e3 commit: 3cc9557d9f03086fa3b0ea166f8506087e6950e3 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T11:57:49-07:00 summary: bpo-34790: [docs] Passing coroutines to asyncio.wait() can be confusing. (GH-9543) (cherry picked from commit 996859a90df51f84eab47351702cb59c6db4428a) Co-authored-by: Yury Selivanov files: A Misc/NEWS.d/next/Documentation/2018-09-24-12-47-08.bpo-34790.G2KXIH.rst M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index bdb475a797db..7fbfd426dba6 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -466,14 +466,20 @@ Waiting Primitives return_when=ALL_COMPLETED) Run :ref:`awaitable objects ` in the *aws* - sequence concurrently and block until the condition specified + set concurrently and block until the condition specified by *return_when*. If any awaitable in *aws* is a coroutine, it is automatically - scheduled as a Task. + scheduled as a Task. Passing coroutines objects to + ``wait()`` directly is deprecated as it leads to + :ref:`confusing behavior `. Returns two sets of Tasks/Futures: ``(done, pending)``. + Usage:: + + done, pending = await asyncio.wait(aws) + The *loop* argument is deprecated and scheduled for removal in Python 4.0. @@ -508,9 +514,35 @@ Waiting Primitives Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the futures when a timeout occurs. - Usage:: + .. _asyncio_example_wait_coroutine: + .. note:: - done, pending = await asyncio.wait(aws) + ``wait()`` schedules coroutines as Tasks automatically and later + returns those implicitly created Task objects in ``(done, pending)`` + sets. Therefore the following code won't work as expected:: + + async def foo(): + return 42 + + coro = foo() + done, pending = await asyncio.wait({coro}) + + if coro in done: + # This branch will never be run! + + Here is how the above snippet can be fixed:: + + async def foo(): + return 42 + + task = asyncio.create_task(foo()) + done, pending = await asyncio.wait({task}) + + if task in done: + # Everything will work as expected now. + + Passing coroutine objects to ``wait()`` directly is + deprecated. .. function:: as_completed(aws, \*, loop=None, timeout=None) diff --git a/Misc/NEWS.d/next/Documentation/2018-09-24-12-47-08.bpo-34790.G2KXIH.rst b/Misc/NEWS.d/next/Documentation/2018-09-24-12-47-08.bpo-34790.G2KXIH.rst new file mode 100644 index 000000000000..dc3de2c7d4c8 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-09-24-12-47-08.bpo-34790.G2KXIH.rst @@ -0,0 +1 @@ +Document how passing coroutines to asyncio.wait() can be confusing. From webhook-mailer at python.org Tue Sep 25 15:15:51 2018 From: webhook-mailer at python.org (Barry Warsaw) Date: Tue, 25 Sep 2018 19:15:51 -0000 Subject: [Python-checkins] bpo-5950: Support reading zips with comments in zipimport (#9548) Message-ID: https://github.com/python/cpython/commit/5a5ce064b3baadcb79605c5a42ee3d0aee57cdfc commit: 5a5ce064b3baadcb79605c5a42ee3d0aee57cdfc branch: master author: Zackery Spytz committer: Barry Warsaw date: 2018-09-25T15:15:47-04:00 summary: bpo-5950: Support reading zips with comments in zipimport (#9548) * bpo-5950: Support reading zips with comments in zipimport files: A Misc/NEWS.d/next/Library/2018-09-24-14-21-58.bpo-5950.xH0ekQ.rst M Doc/library/zipimport.rst M Lib/test/test_zipimport.py M Lib/zipimport.py M Python/importlib_zipimport.h diff --git a/Doc/library/zipimport.rst b/Doc/library/zipimport.rst index eaae2bb04b74..4fbd0e863c53 100644 --- a/Doc/library/zipimport.rst +++ b/Doc/library/zipimport.rst @@ -28,7 +28,8 @@ Any files may be present in the ZIP archive, but only files :file:`.py` and corresponding :file:`.pyc` file, meaning that if a ZIP archive doesn't contain :file:`.pyc` files, importing may be rather slow. -ZIP archives with an archive comment are currently not supported. +.. versionchanged:: 3.8 + Previously, ZIP archives with an archive comment were not supported. .. seealso:: diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py index 63e567254cd9..e98b09021c5f 100644 --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -116,6 +116,9 @@ def makeZip(self, files, zipName=TEMP_ZIP, **kw): zinfo = ZipInfo(name, time.localtime(mtime)) zinfo.compress_type = self.compression z.writestr(zinfo, data) + comment = kw.get("comment", None) + if comment is not None: + z.comment = comment stuff = kw.get("stuff", None) if stuff is not None: @@ -665,6 +668,18 @@ def testBytesPath(self): with self.assertRaises(TypeError): zipimport.zipimporter(memoryview(os.fsencode(filename))) + def testComment(self): + files = {TESTMOD + ".py": (NOW, test_src)} + self.doTest(".py", files, TESTMOD, comment=b"comment") + + def testBeginningCruftAndComment(self): + files = {TESTMOD + ".py": (NOW, test_src)} + self.doTest(".py", files, TESTMOD, stuff=b"cruft" * 64, comment=b"hi") + + def testLargestPossibleComment(self): + files = {TESTMOD + ".py": (NOW, test_src)} + self.doTest(".py", files, TESTMOD, comment=b"c" * ((1 << 16) - 1)) + @support.requires_zlib class CompressedZipImportTestCase(UncompressedZipImportTestCase): diff --git a/Lib/zipimport.py b/Lib/zipimport.py index 4017340d1184..2c11f68f4946 100644 --- a/Lib/zipimport.py +++ b/Lib/zipimport.py @@ -38,6 +38,9 @@ class ZipImportError(ImportError): _module_type = type(sys) +END_CENTRAL_DIR_SIZE = 22 +STRING_END_ARCHIVE = b'PK\x05\x06' +MAX_COMMENT_LEN = (1 << 16) - 1 class zipimporter: """zipimporter(archivepath) -> zipimporter object @@ -354,16 +357,39 @@ def _read_directory(archive): with fp: try: - fp.seek(-22, 2) + fp.seek(-END_CENTRAL_DIR_SIZE, 2) header_position = fp.tell() - buffer = fp.read(22) + buffer = fp.read(END_CENTRAL_DIR_SIZE) except OSError: raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) - if len(buffer) != 22: + if len(buffer) != END_CENTRAL_DIR_SIZE: raise ZipImportError(f"can't read Zip file: {archive!r}", path=archive) - if buffer[:4] != b'PK\x05\x06': + if buffer[:4] != STRING_END_ARCHIVE: # Bad: End of Central Dir signature - raise ZipImportError(f'not a Zip file: {archive!r}', path=archive) + # Check if there's a comment. + try: + fp.seek(0, 2) + file_size = fp.tell() + except OSError: + raise ZipImportError(f"can't read Zip file: {archive!r}", + path=archive) + max_comment_start = max(file_size - MAX_COMMENT_LEN - + END_CENTRAL_DIR_SIZE, 0) + try: + fp.seek(max_comment_start) + data = fp.read() + except OSError: + raise ZipImportError(f"can't read Zip file: {archive!r}", + path=archive) + pos = data.rfind(STRING_END_ARCHIVE) + if pos < 0: + raise ZipImportError(f'not a Zip file: {archive!r}', + path=archive) + buffer = data[pos:pos+END_CENTRAL_DIR_SIZE] + if len(buffer) != END_CENTRAL_DIR_SIZE: + raise ZipImportError(f"corrupt Zip file: {archive!r}", + path=archive) + header_position = file_size - len(data) + pos header_size = _unpack_uint32(buffer[12:16]) header_offset = _unpack_uint32(buffer[16:20]) diff --git a/Misc/NEWS.d/next/Library/2018-09-24-14-21-58.bpo-5950.xH0ekQ.rst b/Misc/NEWS.d/next/Library/2018-09-24-14-21-58.bpo-5950.xH0ekQ.rst new file mode 100644 index 000000000000..f0f9c0b80170 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-24-14-21-58.bpo-5950.xH0ekQ.rst @@ -0,0 +1 @@ +Support reading zip files with archive comments in :mod:`zipimport`. diff --git a/Python/importlib_zipimport.h b/Python/importlib_zipimport.h index 5384154aaf57..5f1a046942cb 100644 --- a/Python/importlib_zipimport.h +++ b/Python/importlib_zipimport.h @@ -1,7 +1,7 @@ /* Auto-generated by Programs/_freeze_importlib.c */ const unsigned char _Py_M__zipimport[] = { 99,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0, - 0,64,0,0,0,115,62,1,0,0,100,0,90,0,100,1, + 0,64,0,0,0,115,74,1,0,0,100,0,90,0,100,1, 100,2,108,1,90,2,100,1,100,3,108,1,109,3,90,3, 109,4,90,4,1,0,100,1,100,2,108,5,90,6,100,1, 100,2,108,7,90,7,100,1,100,2,108,8,90,8,100,1, @@ -9,1012 +9,1040 @@ const unsigned char _Py_M__zipimport[] = { 100,2,108,11,90,11,100,4,100,5,103,2,90,12,101,2, 106,13,90,13,101,2,106,14,100,6,100,2,133,2,25,0, 90,15,71,0,100,7,100,4,132,0,100,4,101,16,131,3, - 90,17,105,0,90,18,101,19,101,10,131,1,90,20,71,0, - 100,8,100,5,132,0,100,5,131,2,90,21,101,13,100,9, - 23,0,100,10,100,10,102,3,101,13,100,11,23,0,100,12, - 100,10,102,3,100,13,100,14,102,4,90,22,100,15,100,16, - 132,0,90,23,100,17,100,18,132,0,90,24,100,19,100,20, - 132,0,90,25,100,21,100,22,132,0,90,26,100,23,90,27, - 100,12,97,28,100,24,100,25,132,0,90,29,100,26,100,27, - 132,0,90,30,100,28,100,29,132,0,90,31,100,30,100,31, - 132,0,90,32,101,19,101,32,106,33,131,1,90,34,100,32, - 100,33,132,0,90,35,100,34,100,35,132,0,90,36,100,36, - 100,37,132,0,90,37,100,38,100,39,132,0,90,38,100,40, - 100,41,132,0,90,39,71,0,100,42,100,43,132,0,100,43, - 131,2,90,40,100,2,83,0,41,44,97,80,2,0,0,122, - 105,112,105,109,112,111,114,116,32,112,114,111,118,105,100,101, - 115,32,115,117,112,112,111,114,116,32,102,111,114,32,105,109, - 112,111,114,116,105,110,103,32,80,121,116,104,111,110,32,109, - 111,100,117,108,101,115,32,102,114,111,109,32,90,105,112,32, - 97,114,99,104,105,118,101,115,46,10,10,84,104,105,115,32, - 109,111,100,117,108,101,32,101,120,112,111,114,116,115,32,116, - 104,114,101,101,32,111,98,106,101,99,116,115,58,10,45,32, - 122,105,112,105,109,112,111,114,116,101,114,58,32,97,32,99, - 108,97,115,115,59,32,105,116,115,32,99,111,110,115,116,114, - 117,99,116,111,114,32,116,97,107,101,115,32,97,32,112,97, - 116,104,32,116,111,32,97,32,90,105,112,32,97,114,99,104, - 105,118,101,46,10,45,32,90,105,112,73,109,112,111,114,116, - 69,114,114,111,114,58,32,101,120,99,101,112,116,105,111,110, - 32,114,97,105,115,101,100,32,98,121,32,122,105,112,105,109, - 112,111,114,116,101,114,32,111,98,106,101,99,116,115,46,32, - 73,116,39,115,32,97,10,32,32,115,117,98,99,108,97,115, - 115,32,111,102,32,73,109,112,111,114,116,69,114,114,111,114, - 44,32,115,111,32,105,116,32,99,97,110,32,98,101,32,99, - 97,117,103,104,116,32,97,115,32,73,109,112,111,114,116,69, - 114,114,111,114,44,32,116,111,111,46,10,45,32,95,122,105, - 112,95,100,105,114,101,99,116,111,114,121,95,99,97,99,104, - 101,58,32,97,32,100,105,99,116,44,32,109,97,112,112,105, - 110,103,32,97,114,99,104,105,118,101,32,112,97,116,104,115, - 32,116,111,32,122,105,112,32,100,105,114,101,99,116,111,114, - 121,10,32,32,105,110,102,111,32,100,105,99,116,115,44,32, - 97,115,32,117,115,101,100,32,105,110,32,122,105,112,105,109, - 112,111,114,116,101,114,46,95,102,105,108,101,115,46,10,10, - 73,116,32,105,115,32,117,115,117,97,108,108,121,32,110,111, - 116,32,110,101,101,100,101,100,32,116,111,32,117,115,101,32, - 116,104,101,32,122,105,112,105,109,112,111,114,116,32,109,111, - 100,117,108,101,32,101,120,112,108,105,99,105,116,108,121,59, - 32,105,116,32,105,115,10,117,115,101,100,32,98,121,32,116, - 104,101,32,98,117,105,108,116,105,110,32,105,109,112,111,114, - 116,32,109,101,99,104,97,110,105,115,109,32,102,111,114,32, - 115,121,115,46,112,97,116,104,32,105,116,101,109,115,32,116, - 104,97,116,32,97,114,101,32,112,97,116,104,115,10,116,111, - 32,90,105,112,32,97,114,99,104,105,118,101,115,46,10,233, - 0,0,0,0,78,41,2,218,14,95,117,110,112,97,99,107, - 95,117,105,110,116,49,54,218,14,95,117,110,112,97,99,107, - 95,117,105,110,116,51,50,218,14,90,105,112,73,109,112,111, - 114,116,69,114,114,111,114,218,11,122,105,112,105,109,112,111, - 114,116,101,114,233,1,0,0,0,99,0,0,0,0,0,0, - 0,0,0,0,0,0,1,0,0,0,64,0,0,0,115,12, - 0,0,0,101,0,90,1,100,0,90,2,100,1,83,0,41, - 2,114,3,0,0,0,78,41,3,218,8,95,95,110,97,109, - 101,95,95,218,10,95,95,109,111,100,117,108,101,95,95,218, - 12,95,95,113,117,97,108,110,97,109,101,95,95,169,0,114, - 9,0,0,0,114,9,0,0,0,250,18,60,102,114,111,122, - 101,110,32,122,105,112,105,109,112,111,114,116,62,114,3,0, - 0,0,33,0,0,0,115,2,0,0,0,8,1,99,0,0, - 0,0,0,0,0,0,0,0,0,0,3,0,0,0,64,0, - 0,0,115,108,0,0,0,101,0,90,1,100,0,90,2,100, - 1,90,3,100,2,100,3,132,0,90,4,100,25,100,5,100, - 6,132,1,90,5,100,26,100,7,100,8,132,1,90,6,100, - 9,100,10,132,0,90,7,100,11,100,12,132,0,90,8,100, - 13,100,14,132,0,90,9,100,15,100,16,132,0,90,10,100, - 17,100,18,132,0,90,11,100,19,100,20,132,0,90,12,100, - 21,100,22,132,0,90,13,100,23,100,24,132,0,90,14,100, - 4,83,0,41,27,114,4,0,0,0,97,255,1,0,0,122, - 105,112,105,109,112,111,114,116,101,114,40,97,114,99,104,105, - 118,101,112,97,116,104,41,32,45,62,32,122,105,112,105,109, - 112,111,114,116,101,114,32,111,98,106,101,99,116,10,10,32, - 32,32,32,67,114,101,97,116,101,32,97,32,110,101,119,32, - 122,105,112,105,109,112,111,114,116,101,114,32,105,110,115,116, - 97,110,99,101,46,32,39,97,114,99,104,105,118,101,112,97, - 116,104,39,32,109,117,115,116,32,98,101,32,97,32,112,97, - 116,104,32,116,111,10,32,32,32,32,97,32,122,105,112,102, - 105,108,101,44,32,111,114,32,116,111,32,97,32,115,112,101, - 99,105,102,105,99,32,112,97,116,104,32,105,110,115,105,100, - 101,32,97,32,122,105,112,102,105,108,101,46,32,70,111,114, - 32,101,120,97,109,112,108,101,44,32,105,116,32,99,97,110, - 32,98,101,10,32,32,32,32,39,47,116,109,112,47,109,121, - 105,109,112,111,114,116,46,122,105,112,39,44,32,111,114,32, - 39,47,116,109,112,47,109,121,105,109,112,111,114,116,46,122, - 105,112,47,109,121,100,105,114,101,99,116,111,114,121,39,44, - 32,105,102,32,109,121,100,105,114,101,99,116,111,114,121,32, - 105,115,32,97,10,32,32,32,32,118,97,108,105,100,32,100, - 105,114,101,99,116,111,114,121,32,105,110,115,105,100,101,32, - 116,104,101,32,97,114,99,104,105,118,101,46,10,10,32,32, - 32,32,39,90,105,112,73,109,112,111,114,116,69,114,114,111, - 114,32,105,115,32,114,97,105,115,101,100,32,105,102,32,39, - 97,114,99,104,105,118,101,112,97,116,104,39,32,100,111,101, - 115,110,39,116,32,112,111,105,110,116,32,116,111,32,97,32, - 118,97,108,105,100,32,90,105,112,10,32,32,32,32,97,114, - 99,104,105,118,101,46,10,10,32,32,32,32,84,104,101,32, - 39,97,114,99,104,105,118,101,39,32,97,116,116,114,105,98, - 117,116,101,32,111,102,32,122,105,112,105,109,112,111,114,116, - 101,114,32,111,98,106,101,99,116,115,32,99,111,110,116,97, - 105,110,115,32,116,104,101,32,110,97,109,101,32,111,102,32, - 116,104,101,10,32,32,32,32,122,105,112,102,105,108,101,32, - 116,97,114,103,101,116,101,100,46,10,32,32,32,32,99,2, - 0,0,0,0,0,0,0,8,0,0,0,9,0,0,0,67, - 0,0,0,115,36,1,0,0,116,0,124,1,116,1,131,2, - 115,28,100,1,100,0,108,2,125,2,124,2,160,3,124,1, - 161,1,125,1,124,1,115,44,116,4,100,2,124,1,100,3, - 141,2,130,1,116,5,114,60,124,1,160,6,116,5,116,7, - 161,2,125,1,103,0,125,3,122,14,116,8,160,9,124,1, - 161,1,125,4,87,0,110,72,4,0,116,10,116,11,102,2, - 107,10,114,150,1,0,1,0,1,0,116,8,160,12,124,1, - 161,1,92,2,125,5,125,6,124,5,124,1,107,2,114,132, - 116,4,100,4,124,1,100,3,141,2,130,1,124,5,125,1, - 124,3,160,13,124,6,161,1,1,0,89,0,113,64,88,0, - 124,4,106,14,100,5,64,0,100,6,107,3,114,182,116,4, - 100,4,124,1,100,3,141,2,130,1,113,182,113,64,122,12, - 116,15,124,1,25,0,125,7,87,0,110,36,4,0,116,16, - 107,10,114,230,1,0,1,0,1,0,116,17,124,1,131,1, - 125,7,124,7,116,15,124,1,60,0,89,0,110,2,88,0, - 124,7,124,0,95,18,124,1,124,0,95,19,116,8,106,20, - 124,3,100,0,100,0,100,7,133,3,25,0,142,0,124,0, - 95,21,124,0,106,21,144,1,114,32,124,0,4,0,106,21, - 116,7,55,0,2,0,95,21,100,0,83,0,41,8,78,114, - 0,0,0,0,122,21,97,114,99,104,105,118,101,32,112,97, - 116,104,32,105,115,32,101,109,112,116,121,41,1,218,4,112, - 97,116,104,122,14,110,111,116,32,97,32,90,105,112,32,102, - 105,108,101,105,0,240,0,0,105,0,128,0,0,233,255,255, - 255,255,41,22,218,10,105,115,105,110,115,116,97,110,99,101, - 218,3,115,116,114,218,2,111,115,90,8,102,115,100,101,99, - 111,100,101,114,3,0,0,0,218,12,97,108,116,95,112,97, - 116,104,95,115,101,112,218,7,114,101,112,108,97,99,101,218, - 8,112,97,116,104,95,115,101,112,218,19,95,98,111,111,116, - 115,116,114,97,112,95,101,120,116,101,114,110,97,108,90,10, - 95,112,97,116,104,95,115,116,97,116,218,7,79,83,69,114, - 114,111,114,218,10,86,97,108,117,101,69,114,114,111,114,90, - 11,95,112,97,116,104,95,115,112,108,105,116,218,6,97,112, - 112,101,110,100,90,7,115,116,95,109,111,100,101,218,20,95, - 122,105,112,95,100,105,114,101,99,116,111,114,121,95,99,97, - 99,104,101,218,8,75,101,121,69,114,114,111,114,218,15,95, - 114,101,97,100,95,100,105,114,101,99,116,111,114,121,218,6, - 95,102,105,108,101,115,218,7,97,114,99,104,105,118,101,218, - 10,95,112,97,116,104,95,106,111,105,110,218,6,112,114,101, - 102,105,120,41,8,218,4,115,101,108,102,114,11,0,0,0, - 114,15,0,0,0,114,29,0,0,0,90,2,115,116,90,7, - 100,105,114,110,97,109,101,90,8,98,97,115,101,110,97,109, - 101,218,5,102,105,108,101,115,114,9,0,0,0,114,9,0, - 0,0,114,10,0,0,0,218,8,95,95,105,110,105,116,95, - 95,60,0,0,0,115,58,0,0,0,0,1,10,1,8,1, - 10,1,4,1,12,1,4,1,12,2,4,2,2,1,14,1, - 18,3,14,1,8,1,12,1,4,1,16,3,14,2,12,1, - 4,2,2,1,12,1,14,1,8,1,14,1,6,1,6,2, - 22,1,8,1,122,20,122,105,112,105,109,112,111,114,116,101, - 114,46,95,95,105,110,105,116,95,95,78,99,3,0,0,0, - 0,0,0,0,5,0,0,0,4,0,0,0,67,0,0,0, - 115,78,0,0,0,116,0,124,0,124,1,131,2,125,3,124, - 3,100,1,107,9,114,26,124,0,103,0,102,2,83,0,116, - 1,124,0,124,1,131,2,125,4,116,2,124,0,124,4,131, - 2,114,70,100,1,124,0,106,3,155,0,116,4,155,0,124, - 4,155,0,157,3,103,1,102,2,83,0,100,1,103,0,102, - 2,83,0,41,2,97,239,1,0,0,102,105,110,100,95,108, - 111,97,100,101,114,40,102,117,108,108,110,97,109,101,44,32, - 112,97,116,104,61,78,111,110,101,41,32,45,62,32,115,101, - 108,102,44,32,115,116,114,32,111,114,32,78,111,110,101,46, - 10,10,32,32,32,32,32,32,32,32,83,101,97,114,99,104, - 32,102,111,114,32,97,32,109,111,100,117,108,101,32,115,112, - 101,99,105,102,105,101,100,32,98,121,32,39,102,117,108,108, - 110,97,109,101,39,46,32,39,102,117,108,108,110,97,109,101, - 39,32,109,117,115,116,32,98,101,32,116,104,101,10,32,32, - 32,32,32,32,32,32,102,117,108,108,121,32,113,117,97,108, - 105,102,105,101,100,32,40,100,111,116,116,101,100,41,32,109, - 111,100,117,108,101,32,110,97,109,101,46,32,73,116,32,114, - 101,116,117,114,110,115,32,116,104,101,32,122,105,112,105,109, - 112,111,114,116,101,114,10,32,32,32,32,32,32,32,32,105, - 110,115,116,97,110,99,101,32,105,116,115,101,108,102,32,105, - 102,32,116,104,101,32,109,111,100,117,108,101,32,119,97,115, - 32,102,111,117,110,100,44,32,97,32,115,116,114,105,110,103, - 32,99,111,110,116,97,105,110,105,110,103,32,116,104,101,10, - 32,32,32,32,32,32,32,32,102,117,108,108,32,112,97,116, - 104,32,110,97,109,101,32,105,102,32,105,116,39,115,32,112, - 111,115,115,105,98,108,121,32,97,32,112,111,114,116,105,111, - 110,32,111,102,32,97,32,110,97,109,101,115,112,97,99,101, - 32,112,97,99,107,97,103,101,44,10,32,32,32,32,32,32, - 32,32,111,114,32,78,111,110,101,32,111,116,104,101,114,119, - 105,115,101,46,32,84,104,101,32,111,112,116,105,111,110,97, - 108,32,39,112,97,116,104,39,32,97,114,103,117,109,101,110, - 116,32,105,115,32,105,103,110,111,114,101,100,32,45,45,32, - 105,116,39,115,10,32,32,32,32,32,32,32,32,116,104,101, - 114,101,32,102,111,114,32,99,111,109,112,97,116,105,98,105, - 108,105,116,121,32,119,105,116,104,32,116,104,101,32,105,109, - 112,111,114,116,101,114,32,112,114,111,116,111,99,111,108,46, - 10,32,32,32,32,32,32,32,32,78,41,5,218,16,95,103, - 101,116,95,109,111,100,117,108,101,95,105,110,102,111,218,16, - 95,103,101,116,95,109,111,100,117,108,101,95,112,97,116,104, - 218,7,95,105,115,95,100,105,114,114,27,0,0,0,114,18, - 0,0,0,41,5,114,30,0,0,0,218,8,102,117,108,108, - 110,97,109,101,114,11,0,0,0,218,2,109,105,218,7,109, - 111,100,112,97,116,104,114,9,0,0,0,114,9,0,0,0, - 114,10,0,0,0,218,11,102,105,110,100,95,108,111,97,100, - 101,114,106,0,0,0,115,14,0,0,0,0,10,10,1,8, - 2,8,7,10,1,10,4,24,2,122,23,122,105,112,105,109, - 112,111,114,116,101,114,46,102,105,110,100,95,108,111,97,100, - 101,114,99,3,0,0,0,0,0,0,0,3,0,0,0,4, - 0,0,0,67,0,0,0,115,16,0,0,0,124,0,160,0, - 124,1,124,2,161,2,100,1,25,0,83,0,41,2,97,139, - 1,0,0,102,105,110,100,95,109,111,100,117,108,101,40,102, - 117,108,108,110,97,109,101,44,32,112,97,116,104,61,78,111, - 110,101,41,32,45,62,32,115,101,108,102,32,111,114,32,78, - 111,110,101,46,10,10,32,32,32,32,32,32,32,32,83,101, - 97,114,99,104,32,102,111,114,32,97,32,109,111,100,117,108, - 101,32,115,112,101,99,105,102,105,101,100,32,98,121,32,39, - 102,117,108,108,110,97,109,101,39,46,32,39,102,117,108,108, - 110,97,109,101,39,32,109,117,115,116,32,98,101,32,116,104, - 101,10,32,32,32,32,32,32,32,32,102,117,108,108,121,32, - 113,117,97,108,105,102,105,101,100,32,40,100,111,116,116,101, - 100,41,32,109,111,100,117,108,101,32,110,97,109,101,46,32, - 73,116,32,114,101,116,117,114,110,115,32,116,104,101,32,122, - 105,112,105,109,112,111,114,116,101,114,10,32,32,32,32,32, - 32,32,32,105,110,115,116,97,110,99,101,32,105,116,115,101, - 108,102,32,105,102,32,116,104,101,32,109,111,100,117,108,101, - 32,119,97,115,32,102,111,117,110,100,44,32,111,114,32,78, - 111,110,101,32,105,102,32,105,116,32,119,97,115,110,39,116, - 46,10,32,32,32,32,32,32,32,32,84,104,101,32,111,112, - 116,105,111,110,97,108,32,39,112,97,116,104,39,32,97,114, - 103,117,109,101,110,116,32,105,115,32,105,103,110,111,114,101, - 100,32,45,45,32,105,116,39,115,32,116,104,101,114,101,32, - 102,111,114,32,99,111,109,112,97,116,105,98,105,108,105,116, - 121,10,32,32,32,32,32,32,32,32,119,105,116,104,32,116, - 104,101,32,105,109,112,111,114,116,101,114,32,112,114,111,116, - 111,99,111,108,46,10,32,32,32,32,32,32,32,32,114,0, - 0,0,0,41,1,114,39,0,0,0,41,3,114,30,0,0, - 0,114,36,0,0,0,114,11,0,0,0,114,9,0,0,0, - 114,9,0,0,0,114,10,0,0,0,218,11,102,105,110,100, - 95,109,111,100,117,108,101,138,0,0,0,115,2,0,0,0, - 0,9,122,23,122,105,112,105,109,112,111,114,116,101,114,46, - 102,105,110,100,95,109,111,100,117,108,101,99,2,0,0,0, - 0,0,0,0,5,0,0,0,3,0,0,0,67,0,0,0, - 115,20,0,0,0,116,0,124,0,124,1,131,2,92,3,125, - 2,125,3,125,4,124,2,83,0,41,1,122,163,103,101,116, - 95,99,111,100,101,40,102,117,108,108,110,97,109,101,41,32, - 45,62,32,99,111,100,101,32,111,98,106,101,99,116,46,10, - 10,32,32,32,32,32,32,32,32,82,101,116,117,114,110,32, - 116,104,101,32,99,111,100,101,32,111,98,106,101,99,116,32, - 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, - 100,32,109,111,100,117,108,101,46,32,82,97,105,115,101,32, - 90,105,112,73,109,112,111,114,116,69,114,114,111,114,10,32, - 32,32,32,32,32,32,32,105,102,32,116,104,101,32,109,111, - 100,117,108,101,32,99,111,117,108,100,110,39,116,32,98,101, - 32,102,111,117,110,100,46,10,32,32,32,32,32,32,32,32, - 41,1,218,16,95,103,101,116,95,109,111,100,117,108,101,95, - 99,111,100,101,41,5,114,30,0,0,0,114,36,0,0,0, - 218,4,99,111,100,101,218,9,105,115,112,97,99,107,97,103, - 101,114,38,0,0,0,114,9,0,0,0,114,9,0,0,0, - 114,10,0,0,0,218,8,103,101,116,95,99,111,100,101,150, - 0,0,0,115,4,0,0,0,0,6,16,1,122,20,122,105, - 112,105,109,112,111,114,116,101,114,46,103,101,116,95,99,111, - 100,101,99,2,0,0,0,0,0,0,0,4,0,0,0,8, - 0,0,0,67,0,0,0,115,118,0,0,0,116,0,114,16, - 124,1,160,1,116,0,116,2,161,2,125,1,124,1,125,2, - 124,1,160,3,124,0,106,4,116,2,23,0,161,1,114,58, - 124,1,116,5,124,0,106,4,116,2,23,0,131,1,100,1, - 133,2,25,0,125,2,122,14,124,0,106,6,124,2,25,0, - 125,3,87,0,110,32,4,0,116,7,107,10,114,104,1,0, - 1,0,1,0,116,8,100,2,100,3,124,2,131,3,130,1, - 89,0,110,2,88,0,116,9,124,0,106,4,124,3,131,2, - 83,0,41,4,122,154,103,101,116,95,100,97,116,97,40,112, - 97,116,104,110,97,109,101,41,32,45,62,32,115,116,114,105, - 110,103,32,119,105,116,104,32,102,105,108,101,32,100,97,116, - 97,46,10,10,32,32,32,32,32,32,32,32,82,101,116,117, - 114,110,32,116,104,101,32,100,97,116,97,32,97,115,115,111, - 99,105,97,116,101,100,32,119,105,116,104,32,39,112,97,116, - 104,110,97,109,101,39,46,32,82,97,105,115,101,32,79,83, - 69,114,114,111,114,32,105,102,10,32,32,32,32,32,32,32, - 32,116,104,101,32,102,105,108,101,32,119,97,115,110,39,116, - 32,102,111,117,110,100,46,10,32,32,32,32,32,32,32,32, - 78,114,0,0,0,0,218,0,41,10,114,16,0,0,0,114, - 17,0,0,0,114,18,0,0,0,218,10,115,116,97,114,116, - 115,119,105,116,104,114,27,0,0,0,218,3,108,101,110,114, - 26,0,0,0,114,24,0,0,0,114,20,0,0,0,218,9, - 95,103,101,116,95,100,97,116,97,41,4,114,30,0,0,0, - 218,8,112,97,116,104,110,97,109,101,90,3,107,101,121,218, - 9,116,111,99,95,101,110,116,114,121,114,9,0,0,0,114, - 9,0,0,0,114,10,0,0,0,218,8,103,101,116,95,100, - 97,116,97,160,0,0,0,115,20,0,0,0,0,6,4,1, - 12,2,4,1,16,1,22,2,2,1,14,1,14,1,18,1, - 122,20,122,105,112,105,109,112,111,114,116,101,114,46,103,101, - 116,95,100,97,116,97,99,2,0,0,0,0,0,0,0,5, - 0,0,0,3,0,0,0,67,0,0,0,115,20,0,0,0, - 116,0,124,0,124,1,131,2,92,3,125,2,125,3,125,4, - 124,4,83,0,41,1,122,106,103,101,116,95,102,105,108,101, - 110,97,109,101,40,102,117,108,108,110,97,109,101,41,32,45, - 62,32,102,105,108,101,110,97,109,101,32,115,116,114,105,110, - 103,46,10,10,32,32,32,32,32,32,32,32,82,101,116,117, - 114,110,32,116,104,101,32,102,105,108,101,110,97,109,101,32, - 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, - 100,32,109,111,100,117,108,101,46,10,32,32,32,32,32,32, - 32,32,41,1,114,41,0,0,0,41,5,114,30,0,0,0, - 114,36,0,0,0,114,42,0,0,0,114,43,0,0,0,114, - 38,0,0,0,114,9,0,0,0,114,9,0,0,0,114,10, - 0,0,0,218,12,103,101,116,95,102,105,108,101,110,97,109, - 101,181,0,0,0,115,4,0,0,0,0,7,16,1,122,24, - 122,105,112,105,109,112,111,114,116,101,114,46,103,101,116,95, - 102,105,108,101,110,97,109,101,99,2,0,0,0,0,0,0, - 0,6,0,0,0,8,0,0,0,67,0,0,0,115,128,0, - 0,0,116,0,124,0,124,1,131,2,125,2,124,2,100,1, - 107,8,114,36,116,1,100,2,124,1,155,2,157,2,124,1, - 100,3,141,2,130,1,116,2,124,0,124,1,131,2,125,3, - 124,2,114,64,116,3,160,4,124,3,100,4,161,2,125,4, - 110,10,124,3,155,0,100,5,157,2,125,4,122,14,124,0, - 106,5,124,4,25,0,125,5,87,0,110,22,4,0,116,6, - 107,10,114,110,1,0,1,0,1,0,89,0,100,1,83,0, - 88,0,116,7,124,0,106,8,124,5,131,2,160,9,161,0, - 83,0,41,6,122,253,103,101,116,95,115,111,117,114,99,101, - 40,102,117,108,108,110,97,109,101,41,32,45,62,32,115,111, - 117,114,99,101,32,115,116,114,105,110,103,46,10,10,32,32, - 32,32,32,32,32,32,82,101,116,117,114,110,32,116,104,101, - 32,115,111,117,114,99,101,32,99,111,100,101,32,102,111,114, - 32,116,104,101,32,115,112,101,99,105,102,105,101,100,32,109, - 111,100,117,108,101,46,32,82,97,105,115,101,32,90,105,112, - 73,109,112,111,114,116,69,114,114,111,114,10,32,32,32,32, - 32,32,32,32,105,102,32,116,104,101,32,109,111,100,117,108, - 101,32,99,111,117,108,100,110,39,116,32,98,101,32,102,111, - 117,110,100,44,32,114,101,116,117,114,110,32,78,111,110,101, - 32,105,102,32,116,104,101,32,97,114,99,104,105,118,101,32, - 100,111,101,115,10,32,32,32,32,32,32,32,32,99,111,110, - 116,97,105,110,32,116,104,101,32,109,111,100,117,108,101,44, - 32,98,117,116,32,104,97,115,32,110,111,32,115,111,117,114, - 99,101,32,102,111,114,32,105,116,46,10,32,32,32,32,32, - 32,32,32,78,122,18,99,97,110,39,116,32,102,105,110,100, - 32,109,111,100,117,108,101,32,41,1,218,4,110,97,109,101, - 122,11,95,95,105,110,105,116,95,95,46,112,121,122,3,46, - 112,121,41,10,114,33,0,0,0,114,3,0,0,0,114,34, - 0,0,0,114,19,0,0,0,114,28,0,0,0,114,26,0, - 0,0,114,24,0,0,0,114,48,0,0,0,114,27,0,0, - 0,218,6,100,101,99,111,100,101,41,6,114,30,0,0,0, - 114,36,0,0,0,114,37,0,0,0,114,11,0,0,0,218, - 8,102,117,108,108,112,97,116,104,114,50,0,0,0,114,9, - 0,0,0,114,9,0,0,0,114,10,0,0,0,218,10,103, - 101,116,95,115,111,117,114,99,101,192,0,0,0,115,24,0, - 0,0,0,7,10,1,8,1,18,2,10,1,4,1,14,2, - 10,2,2,1,14,1,14,2,8,1,122,22,122,105,112,105, - 109,112,111,114,116,101,114,46,103,101,116,95,115,111,117,114, - 99,101,99,2,0,0,0,0,0,0,0,3,0,0,0,4, - 0,0,0,67,0,0,0,115,40,0,0,0,116,0,124,0, - 124,1,131,2,125,2,124,2,100,1,107,8,114,36,116,1, - 100,2,124,1,155,2,157,2,124,1,100,3,141,2,130,1, - 124,2,83,0,41,4,122,171,105,115,95,112,97,99,107,97, - 103,101,40,102,117,108,108,110,97,109,101,41,32,45,62,32, - 98,111,111,108,46,10,10,32,32,32,32,32,32,32,32,82, - 101,116,117,114,110,32,84,114,117,101,32,105,102,32,116,104, - 101,32,109,111,100,117,108,101,32,115,112,101,99,105,102,105, - 101,100,32,98,121,32,102,117,108,108,110,97,109,101,32,105, - 115,32,97,32,112,97,99,107,97,103,101,46,10,32,32,32, - 32,32,32,32,32,82,97,105,115,101,32,90,105,112,73,109, - 112,111,114,116,69,114,114,111,114,32,105,102,32,116,104,101, - 32,109,111,100,117,108,101,32,99,111,117,108,100,110,39,116, - 32,98,101,32,102,111,117,110,100,46,10,32,32,32,32,32, - 32,32,32,78,122,18,99,97,110,39,116,32,102,105,110,100, - 32,109,111,100,117,108,101,32,41,1,114,53,0,0,0,41, - 2,114,33,0,0,0,114,3,0,0,0,41,3,114,30,0, - 0,0,114,36,0,0,0,114,37,0,0,0,114,9,0,0, - 0,114,9,0,0,0,114,10,0,0,0,218,10,105,115,95, - 112,97,99,107,97,103,101,218,0,0,0,115,8,0,0,0, - 0,6,10,1,8,1,18,1,122,22,122,105,112,105,109,112, - 111,114,116,101,114,46,105,115,95,112,97,99,107,97,103,101, - 99,2,0,0,0,0,0,0,0,8,0,0,0,8,0,0, - 0,67,0,0,0,115,248,0,0,0,116,0,124,0,124,1, - 131,2,92,3,125,2,125,3,125,4,116,1,106,2,160,3, - 124,1,161,1,125,5,124,5,100,1,107,8,115,46,116,4, - 124,5,116,5,131,2,115,64,116,5,124,1,131,1,125,5, - 124,5,116,1,106,2,124,1,60,0,124,0,124,5,95,6, - 122,84,124,3,114,108,116,7,124,0,124,1,131,2,125,6, - 116,8,160,9,124,0,106,10,124,6,161,2,125,7,124,7, - 103,1,124,5,95,11,116,12,124,5,100,2,131,2,115,124, - 116,13,124,5,95,13,116,8,160,14,124,5,106,15,124,1, - 124,4,161,3,1,0,116,16,124,2,124,5,106,15,131,2, - 1,0,87,0,110,22,1,0,1,0,1,0,116,1,106,2, - 124,1,61,0,130,0,89,0,110,2,88,0,122,14,116,1, - 106,2,124,1,25,0,125,5,87,0,110,36,4,0,116,17, - 107,10,114,228,1,0,1,0,1,0,116,18,100,3,124,1, - 155,2,100,4,157,3,131,1,130,1,89,0,110,2,88,0, - 116,19,160,20,100,5,124,1,124,4,161,3,1,0,124,5, - 83,0,41,6,122,245,108,111,97,100,95,109,111,100,117,108, - 101,40,102,117,108,108,110,97,109,101,41,32,45,62,32,109, - 111,100,117,108,101,46,10,10,32,32,32,32,32,32,32,32, - 76,111,97,100,32,116,104,101,32,109,111,100,117,108,101,32, - 115,112,101,99,105,102,105,101,100,32,98,121,32,39,102,117, - 108,108,110,97,109,101,39,46,32,39,102,117,108,108,110,97, - 109,101,39,32,109,117,115,116,32,98,101,32,116,104,101,10, - 32,32,32,32,32,32,32,32,102,117,108,108,121,32,113,117, - 97,108,105,102,105,101,100,32,40,100,111,116,116,101,100,41, - 32,109,111,100,117,108,101,32,110,97,109,101,46,32,73,116, - 32,114,101,116,117,114,110,115,32,116,104,101,32,105,109,112, - 111,114,116,101,100,10,32,32,32,32,32,32,32,32,109,111, - 100,117,108,101,44,32,111,114,32,114,97,105,115,101,115,32, - 90,105,112,73,109,112,111,114,116,69,114,114,111,114,32,105, - 102,32,105,116,32,119,97,115,110,39,116,32,102,111,117,110, - 100,46,10,32,32,32,32,32,32,32,32,78,218,12,95,95, - 98,117,105,108,116,105,110,115,95,95,122,14,76,111,97,100, - 101,100,32,109,111,100,117,108,101,32,122,25,32,110,111,116, - 32,102,111,117,110,100,32,105,110,32,115,121,115,46,109,111, - 100,117,108,101,115,122,30,105,109,112,111,114,116,32,123,125, - 32,35,32,108,111,97,100,101,100,32,102,114,111,109,32,90, - 105,112,32,123,125,41,21,114,41,0,0,0,218,3,115,121, - 115,218,7,109,111,100,117,108,101,115,218,3,103,101,116,114, - 13,0,0,0,218,12,95,109,111,100,117,108,101,95,116,121, - 112,101,218,10,95,95,108,111,97,100,101,114,95,95,114,34, - 0,0,0,114,19,0,0,0,114,28,0,0,0,114,27,0, - 0,0,90,8,95,95,112,97,116,104,95,95,218,7,104,97, - 115,97,116,116,114,114,58,0,0,0,90,14,95,102,105,120, - 95,117,112,95,109,111,100,117,108,101,218,8,95,95,100,105, - 99,116,95,95,218,4,101,120,101,99,114,24,0,0,0,218, - 11,73,109,112,111,114,116,69,114,114,111,114,218,10,95,98, - 111,111,116,115,116,114,97,112,218,16,95,118,101,114,98,111, - 115,101,95,109,101,115,115,97,103,101,41,8,114,30,0,0, - 0,114,36,0,0,0,114,42,0,0,0,114,43,0,0,0, - 114,38,0,0,0,90,3,109,111,100,114,11,0,0,0,114, - 55,0,0,0,114,9,0,0,0,114,9,0,0,0,114,10, - 0,0,0,218,11,108,111,97,100,95,109,111,100,117,108,101, - 231,0,0,0,115,48,0,0,0,0,7,16,1,12,1,18, - 1,8,1,10,1,6,2,2,1,4,3,10,1,14,1,8, - 2,10,1,6,1,16,1,16,1,6,1,8,1,8,2,2, - 1,14,1,14,1,22,1,14,1,122,23,122,105,112,105,109, - 112,111,114,116,101,114,46,108,111,97,100,95,109,111,100,117, - 108,101,99,2,0,0,0,0,0,0,0,3,0,0,0,8, - 0,0,0,67,0,0,0,115,88,0,0,0,122,20,124,0, - 160,0,124,1,161,1,115,18,87,0,100,1,83,0,87,0, - 110,22,4,0,116,1,107,10,114,42,1,0,1,0,1,0, - 89,0,100,1,83,0,88,0,116,2,106,3,115,78,100,2, - 100,3,108,4,109,5,125,2,1,0,124,2,160,6,116,2, - 161,1,1,0,100,4,116,2,95,3,116,2,124,0,124,1, - 131,2,83,0,41,5,122,204,82,101,116,117,114,110,32,116, - 104,101,32,82,101,115,111,117,114,99,101,82,101,97,100,101, - 114,32,102,111,114,32,97,32,112,97,99,107,97,103,101,32, - 105,110,32,97,32,122,105,112,32,102,105,108,101,46,10,10, - 32,32,32,32,32,32,32,32,73,102,32,39,102,117,108,108, - 110,97,109,101,39,32,105,115,32,97,32,112,97,99,107,97, - 103,101,32,119,105,116,104,105,110,32,116,104,101,32,122,105, - 112,32,102,105,108,101,44,32,114,101,116,117,114,110,32,116, - 104,101,10,32,32,32,32,32,32,32,32,39,82,101,115,111, - 117,114,99,101,82,101,97,100,101,114,39,32,111,98,106,101, - 99,116,32,102,111,114,32,116,104,101,32,112,97,99,107,97, - 103,101,46,32,32,79,116,104,101,114,119,105,115,101,32,114, - 101,116,117,114,110,32,78,111,110,101,46,10,32,32,32,32, - 32,32,32,32,78,114,0,0,0,0,41,1,218,14,82,101, - 115,111,117,114,99,101,82,101,97,100,101,114,84,41,7,114, - 57,0,0,0,114,3,0,0,0,218,24,95,90,105,112,73, - 109,112,111,114,116,82,101,115,111,117,114,99,101,82,101,97, - 100,101,114,218,11,95,114,101,103,105,115,116,101,114,101,100, - 90,13,105,109,112,111,114,116,108,105,98,46,97,98,99,114, - 71,0,0,0,90,8,114,101,103,105,115,116,101,114,41,3, - 114,30,0,0,0,114,36,0,0,0,114,71,0,0,0,114, - 9,0,0,0,114,9,0,0,0,114,10,0,0,0,218,19, - 103,101,116,95,114,101,115,111,117,114,99,101,95,114,101,97, - 100,101,114,13,1,0,0,115,20,0,0,0,0,6,2,1, - 10,1,10,1,14,1,8,1,6,1,12,1,10,1,6,1, - 122,31,122,105,112,105,109,112,111,114,116,101,114,46,103,101, - 116,95,114,101,115,111,117,114,99,101,95,114,101,97,100,101, - 114,99,1,0,0,0,0,0,0,0,1,0,0,0,5,0, - 0,0,67,0,0,0,115,24,0,0,0,100,1,124,0,106, - 0,155,0,116,1,155,0,124,0,106,2,155,0,100,2,157, - 5,83,0,41,3,78,122,21,60,122,105,112,105,109,112,111, - 114,116,101,114,32,111,98,106,101,99,116,32,34,122,2,34, - 62,41,3,114,27,0,0,0,114,18,0,0,0,114,29,0, - 0,0,41,1,114,30,0,0,0,114,9,0,0,0,114,9, - 0,0,0,114,10,0,0,0,218,8,95,95,114,101,112,114, - 95,95,31,1,0,0,115,2,0,0,0,0,1,122,20,122, - 105,112,105,109,112,111,114,116,101,114,46,95,95,114,101,112, - 114,95,95,41,1,78,41,1,78,41,15,114,6,0,0,0, - 114,7,0,0,0,114,8,0,0,0,218,7,95,95,100,111, - 99,95,95,114,32,0,0,0,114,39,0,0,0,114,40,0, - 0,0,114,44,0,0,0,114,51,0,0,0,114,52,0,0, - 0,114,56,0,0,0,114,57,0,0,0,114,70,0,0,0, - 114,74,0,0,0,114,75,0,0,0,114,9,0,0,0,114, - 9,0,0,0,114,9,0,0,0,114,10,0,0,0,114,4, - 0,0,0,42,0,0,0,115,24,0,0,0,8,13,4,5, - 8,46,10,32,10,12,8,10,8,21,8,11,8,26,8,13, - 8,38,8,18,122,12,95,95,105,110,105,116,95,95,46,112, - 121,99,84,122,11,95,95,105,110,105,116,95,95,46,112,121, - 70,41,3,122,4,46,112,121,99,84,70,41,3,122,3,46, - 112,121,70,70,99,2,0,0,0,0,0,0,0,2,0,0, - 0,4,0,0,0,67,0,0,0,115,20,0,0,0,124,0, - 106,0,124,1,160,1,100,1,161,1,100,2,25,0,23,0, - 83,0,41,3,78,218,1,46,233,2,0,0,0,41,2,114, - 29,0,0,0,218,10,114,112,97,114,116,105,116,105,111,110, - 41,2,114,30,0,0,0,114,36,0,0,0,114,9,0,0, - 0,114,9,0,0,0,114,10,0,0,0,114,34,0,0,0, - 49,1,0,0,115,2,0,0,0,0,1,114,34,0,0,0, - 99,2,0,0,0,0,0,0,0,3,0,0,0,2,0,0, - 0,67,0,0,0,115,18,0,0,0,124,1,116,0,23,0, - 125,2,124,2,124,0,106,1,107,6,83,0,41,1,78,41, - 2,114,18,0,0,0,114,26,0,0,0,41,3,114,30,0, - 0,0,114,11,0,0,0,90,7,100,105,114,112,97,116,104, - 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,114, - 35,0,0,0,53,1,0,0,115,4,0,0,0,0,4,8, - 2,114,35,0,0,0,99,2,0,0,0,0,0,0,0,7, - 0,0,0,4,0,0,0,67,0,0,0,115,56,0,0,0, - 116,0,124,0,124,1,131,2,125,2,116,1,68,0,93,36, - 92,3,125,3,125,4,125,5,124,2,124,3,23,0,125,6, - 124,6,124,0,106,2,107,6,114,14,124,5,2,0,1,0, - 83,0,113,14,100,0,83,0,41,1,78,41,3,114,34,0, - 0,0,218,16,95,122,105,112,95,115,101,97,114,99,104,111, - 114,100,101,114,114,26,0,0,0,41,7,114,30,0,0,0, - 114,36,0,0,0,114,11,0,0,0,218,6,115,117,102,102, - 105,120,218,10,105,115,98,121,116,101,99,111,100,101,114,43, - 0,0,0,114,55,0,0,0,114,9,0,0,0,114,9,0, - 0,0,114,10,0,0,0,114,33,0,0,0,62,1,0,0, - 115,12,0,0,0,0,1,10,1,14,1,8,1,10,1,10, - 1,114,33,0,0,0,99,1,0,0,0,0,0,0,0,23, - 0,0,0,9,0,0,0,67,0,0,0,115,20,4,0,0, - 122,16,116,0,160,1,124,0,100,1,161,2,125,1,87,0, - 110,38,4,0,116,2,107,10,114,54,1,0,1,0,1,0, - 116,3,100,2,124,0,155,2,157,2,124,0,100,3,141,2, - 130,1,89,0,110,2,88,0,124,1,144,3,143,190,1,0, - 122,34,124,1,160,4,100,4,100,5,161,2,1,0,124,1, - 160,5,161,0,125,2,124,1,160,6,100,6,161,1,125,3, - 87,0,110,38,4,0,116,2,107,10,114,136,1,0,1,0, - 1,0,116,3,100,7,124,0,155,2,157,2,124,0,100,3, - 141,2,130,1,89,0,110,2,88,0,116,7,124,3,131,1, - 100,6,107,3,114,168,116,3,100,7,124,0,155,2,157,2, - 124,0,100,3,141,2,130,1,124,3,100,0,100,8,133,2, - 25,0,100,9,107,3,114,202,116,3,100,10,124,0,155,2, - 157,2,124,0,100,3,141,2,130,1,116,8,124,3,100,11, - 100,12,133,2,25,0,131,1,125,4,116,8,124,3,100,12, - 100,13,133,2,25,0,131,1,125,5,124,2,124,4,107,0, - 144,1,114,6,116,3,100,14,124,0,155,2,157,2,124,0, - 100,3,141,2,130,1,124,2,124,5,107,0,144,1,114,34, - 116,3,100,15,124,0,155,2,157,2,124,0,100,3,141,2, - 130,1,124,2,124,4,56,0,125,2,124,2,124,5,24,0, - 125,6,124,6,100,16,107,0,144,1,114,78,116,3,100,17, - 124,0,155,2,157,2,124,0,100,3,141,2,130,1,105,0, - 125,7,100,16,125,8,122,14,124,1,160,4,124,2,161,1, - 1,0,87,0,110,40,4,0,116,2,107,10,144,1,114,140, - 1,0,1,0,1,0,116,3,100,7,124,0,155,2,157,2, - 124,0,100,3,141,2,130,1,89,0,110,2,88,0,124,1, - 160,6,100,18,161,1,125,3,116,7,124,3,131,1,100,8, - 107,0,144,1,114,174,116,9,100,19,131,1,130,1,124,3, - 100,0,100,8,133,2,25,0,100,20,107,3,144,1,114,196, - 144,3,113,248,116,7,124,3,131,1,100,18,107,3,144,1, - 114,218,116,9,100,19,131,1,130,1,116,10,124,3,100,21, - 100,22,133,2,25,0,131,1,125,9,116,10,124,3,100,22, - 100,11,133,2,25,0,131,1,125,10,116,10,124,3,100,11, - 100,23,133,2,25,0,131,1,125,11,116,10,124,3,100,23, - 100,12,133,2,25,0,131,1,125,12,116,8,124,3,100,12, - 100,13,133,2,25,0,131,1,125,13,116,8,124,3,100,13, - 100,24,133,2,25,0,131,1,125,14,116,8,124,3,100,24, - 100,25,133,2,25,0,131,1,125,15,116,10,124,3,100,25, - 100,26,133,2,25,0,131,1,125,16,116,10,124,3,100,26, - 100,27,133,2,25,0,131,1,125,17,116,10,124,3,100,27, - 100,28,133,2,25,0,131,1,125,18,116,8,124,3,100,29, - 100,18,133,2,25,0,131,1,125,19,124,16,124,17,23,0, - 124,18,23,0,125,4,124,19,124,5,107,4,144,2,114,178, - 116,3,100,30,124,0,155,2,157,2,124,0,100,3,141,2, - 130,1,124,19,124,6,55,0,125,19,122,14,124,1,160,6, - 124,16,161,1,125,20,87,0,110,40,4,0,116,2,107,10, - 144,2,114,240,1,0,1,0,1,0,116,3,100,7,124,0, - 155,2,157,2,124,0,100,3,141,2,130,1,89,0,110,2, - 88,0,116,7,124,20,131,1,124,16,107,3,144,3,114,18, - 116,3,100,7,124,0,155,2,157,2,124,0,100,3,141,2, - 130,1,122,50,116,7,124,1,160,6,124,4,124,16,24,0, - 161,1,131,1,124,4,124,16,24,0,107,3,144,3,114,66, - 116,3,100,7,124,0,155,2,157,2,124,0,100,3,141,2, - 130,1,87,0,110,40,4,0,116,2,107,10,144,3,114,108, - 1,0,1,0,1,0,116,3,100,7,124,0,155,2,157,2, - 124,0,100,3,141,2,130,1,89,0,110,2,88,0,124,9, - 100,31,64,0,144,3,114,130,124,20,160,11,161,0,125,20, - 110,54,122,14,124,20,160,11,100,32,161,1,125,20,87,0, - 110,38,4,0,116,12,107,10,144,3,114,182,1,0,1,0, - 1,0,124,20,160,11,100,33,161,1,160,13,116,14,161,1, - 125,20,89,0,110,2,88,0,124,20,160,15,100,34,116,16, - 161,2,125,20,116,17,160,18,124,0,124,20,161,2,125,21, - 124,21,124,10,124,14,124,15,124,19,124,11,124,12,124,13, - 102,8,125,22,124,22,124,7,124,20,60,0,124,8,100,35, - 55,0,125,8,144,1,113,142,87,0,53,0,81,0,82,0, - 88,0,116,19,160,20,100,36,124,8,124,0,161,3,1,0, - 124,7,83,0,41,37,78,218,2,114,98,122,21,99,97,110, - 39,116,32,111,112,101,110,32,90,105,112,32,102,105,108,101, - 58,32,41,1,114,11,0,0,0,105,234,255,255,255,114,78, - 0,0,0,233,22,0,0,0,122,21,99,97,110,39,116,32, - 114,101,97,100,32,90,105,112,32,102,105,108,101,58,32,233, - 4,0,0,0,115,4,0,0,0,80,75,5,6,122,16,110, - 111,116,32,97,32,90,105,112,32,102,105,108,101,58,32,233, - 12,0,0,0,233,16,0,0,0,233,20,0,0,0,122,28, - 98,97,100,32,99,101,110,116,114,97,108,32,100,105,114,101, - 99,116,111,114,121,32,115,105,122,101,58,32,122,30,98,97, - 100,32,99,101,110,116,114,97,108,32,100,105,114,101,99,116, - 111,114,121,32,111,102,102,115,101,116,58,32,114,0,0,0, - 0,122,38,98,97,100,32,99,101,110,116,114,97,108,32,100, - 105,114,101,99,116,111,114,121,32,115,105,122,101,32,111,114, - 32,111,102,102,115,101,116,58,32,233,46,0,0,0,122,27, + 90,17,105,0,90,18,101,19,101,10,131,1,90,20,100,8, + 90,21,100,9,90,22,100,10,90,23,71,0,100,11,100,5, + 132,0,100,5,131,2,90,24,101,13,100,12,23,0,100,13, + 100,13,102,3,101,13,100,14,23,0,100,15,100,13,102,3, + 100,16,100,17,102,4,90,25,100,18,100,19,132,0,90,26, + 100,20,100,21,132,0,90,27,100,22,100,23,132,0,90,28, + 100,24,100,25,132,0,90,29,100,26,90,30,100,15,97,31, + 100,27,100,28,132,0,90,32,100,29,100,30,132,0,90,33, + 100,31,100,32,132,0,90,34,100,33,100,34,132,0,90,35, + 101,19,101,35,106,36,131,1,90,37,100,35,100,36,132,0, + 90,38,100,37,100,38,132,0,90,39,100,39,100,40,132,0, + 90,40,100,41,100,42,132,0,90,41,100,43,100,44,132,0, + 90,42,71,0,100,45,100,46,132,0,100,46,131,2,90,43, + 100,2,83,0,41,47,97,80,2,0,0,122,105,112,105,109, + 112,111,114,116,32,112,114,111,118,105,100,101,115,32,115,117, + 112,112,111,114,116,32,102,111,114,32,105,109,112,111,114,116, + 105,110,103,32,80,121,116,104,111,110,32,109,111,100,117,108, + 101,115,32,102,114,111,109,32,90,105,112,32,97,114,99,104, + 105,118,101,115,46,10,10,84,104,105,115,32,109,111,100,117, + 108,101,32,101,120,112,111,114,116,115,32,116,104,114,101,101, + 32,111,98,106,101,99,116,115,58,10,45,32,122,105,112,105, + 109,112,111,114,116,101,114,58,32,97,32,99,108,97,115,115, + 59,32,105,116,115,32,99,111,110,115,116,114,117,99,116,111, + 114,32,116,97,107,101,115,32,97,32,112,97,116,104,32,116, + 111,32,97,32,90,105,112,32,97,114,99,104,105,118,101,46, + 10,45,32,90,105,112,73,109,112,111,114,116,69,114,114,111, + 114,58,32,101,120,99,101,112,116,105,111,110,32,114,97,105, + 115,101,100,32,98,121,32,122,105,112,105,109,112,111,114,116, + 101,114,32,111,98,106,101,99,116,115,46,32,73,116,39,115, + 32,97,10,32,32,115,117,98,99,108,97,115,115,32,111,102, + 32,73,109,112,111,114,116,69,114,114,111,114,44,32,115,111, + 32,105,116,32,99,97,110,32,98,101,32,99,97,117,103,104, + 116,32,97,115,32,73,109,112,111,114,116,69,114,114,111,114, + 44,32,116,111,111,46,10,45,32,95,122,105,112,95,100,105, + 114,101,99,116,111,114,121,95,99,97,99,104,101,58,32,97, + 32,100,105,99,116,44,32,109,97,112,112,105,110,103,32,97, + 114,99,104,105,118,101,32,112,97,116,104,115,32,116,111,32, + 122,105,112,32,100,105,114,101,99,116,111,114,121,10,32,32, + 105,110,102,111,32,100,105,99,116,115,44,32,97,115,32,117, + 115,101,100,32,105,110,32,122,105,112,105,109,112,111,114,116, + 101,114,46,95,102,105,108,101,115,46,10,10,73,116,32,105, + 115,32,117,115,117,97,108,108,121,32,110,111,116,32,110,101, + 101,100,101,100,32,116,111,32,117,115,101,32,116,104,101,32, + 122,105,112,105,109,112,111,114,116,32,109,111,100,117,108,101, + 32,101,120,112,108,105,99,105,116,108,121,59,32,105,116,32, + 105,115,10,117,115,101,100,32,98,121,32,116,104,101,32,98, + 117,105,108,116,105,110,32,105,109,112,111,114,116,32,109,101, + 99,104,97,110,105,115,109,32,102,111,114,32,115,121,115,46, + 112,97,116,104,32,105,116,101,109,115,32,116,104,97,116,32, + 97,114,101,32,112,97,116,104,115,10,116,111,32,90,105,112, + 32,97,114,99,104,105,118,101,115,46,10,233,0,0,0,0, + 78,41,2,218,14,95,117,110,112,97,99,107,95,117,105,110, + 116,49,54,218,14,95,117,110,112,97,99,107,95,117,105,110, + 116,51,50,218,14,90,105,112,73,109,112,111,114,116,69,114, + 114,111,114,218,11,122,105,112,105,109,112,111,114,116,101,114, + 233,1,0,0,0,99,0,0,0,0,0,0,0,0,0,0, + 0,0,1,0,0,0,64,0,0,0,115,12,0,0,0,101, + 0,90,1,100,0,90,2,100,1,83,0,41,2,114,3,0, + 0,0,78,41,3,218,8,95,95,110,97,109,101,95,95,218, + 10,95,95,109,111,100,117,108,101,95,95,218,12,95,95,113, + 117,97,108,110,97,109,101,95,95,169,0,114,9,0,0,0, + 114,9,0,0,0,250,18,60,102,114,111,122,101,110,32,122, + 105,112,105,109,112,111,114,116,62,114,3,0,0,0,33,0, + 0,0,115,2,0,0,0,8,1,233,22,0,0,0,115,4, + 0,0,0,80,75,5,6,105,255,255,0,0,99,0,0,0, + 0,0,0,0,0,0,0,0,0,3,0,0,0,64,0,0, + 0,115,108,0,0,0,101,0,90,1,100,0,90,2,100,1, + 90,3,100,2,100,3,132,0,90,4,100,25,100,5,100,6, + 132,1,90,5,100,26,100,7,100,8,132,1,90,6,100,9, + 100,10,132,0,90,7,100,11,100,12,132,0,90,8,100,13, + 100,14,132,0,90,9,100,15,100,16,132,0,90,10,100,17, + 100,18,132,0,90,11,100,19,100,20,132,0,90,12,100,21, + 100,22,132,0,90,13,100,23,100,24,132,0,90,14,100,4, + 83,0,41,27,114,4,0,0,0,97,255,1,0,0,122,105, + 112,105,109,112,111,114,116,101,114,40,97,114,99,104,105,118, + 101,112,97,116,104,41,32,45,62,32,122,105,112,105,109,112, + 111,114,116,101,114,32,111,98,106,101,99,116,10,10,32,32, + 32,32,67,114,101,97,116,101,32,97,32,110,101,119,32,122, + 105,112,105,109,112,111,114,116,101,114,32,105,110,115,116,97, + 110,99,101,46,32,39,97,114,99,104,105,118,101,112,97,116, + 104,39,32,109,117,115,116,32,98,101,32,97,32,112,97,116, + 104,32,116,111,10,32,32,32,32,97,32,122,105,112,102,105, + 108,101,44,32,111,114,32,116,111,32,97,32,115,112,101,99, + 105,102,105,99,32,112,97,116,104,32,105,110,115,105,100,101, + 32,97,32,122,105,112,102,105,108,101,46,32,70,111,114,32, + 101,120,97,109,112,108,101,44,32,105,116,32,99,97,110,32, + 98,101,10,32,32,32,32,39,47,116,109,112,47,109,121,105, + 109,112,111,114,116,46,122,105,112,39,44,32,111,114,32,39, + 47,116,109,112,47,109,121,105,109,112,111,114,116,46,122,105, + 112,47,109,121,100,105,114,101,99,116,111,114,121,39,44,32, + 105,102,32,109,121,100,105,114,101,99,116,111,114,121,32,105, + 115,32,97,10,32,32,32,32,118,97,108,105,100,32,100,105, + 114,101,99,116,111,114,121,32,105,110,115,105,100,101,32,116, + 104,101,32,97,114,99,104,105,118,101,46,10,10,32,32,32, + 32,39,90,105,112,73,109,112,111,114,116,69,114,114,111,114, + 32,105,115,32,114,97,105,115,101,100,32,105,102,32,39,97, + 114,99,104,105,118,101,112,97,116,104,39,32,100,111,101,115, + 110,39,116,32,112,111,105,110,116,32,116,111,32,97,32,118, + 97,108,105,100,32,90,105,112,10,32,32,32,32,97,114,99, + 104,105,118,101,46,10,10,32,32,32,32,84,104,101,32,39, + 97,114,99,104,105,118,101,39,32,97,116,116,114,105,98,117, + 116,101,32,111,102,32,122,105,112,105,109,112,111,114,116,101, + 114,32,111,98,106,101,99,116,115,32,99,111,110,116,97,105, + 110,115,32,116,104,101,32,110,97,109,101,32,111,102,32,116, + 104,101,10,32,32,32,32,122,105,112,102,105,108,101,32,116, + 97,114,103,101,116,101,100,46,10,32,32,32,32,99,2,0, + 0,0,0,0,0,0,8,0,0,0,9,0,0,0,67,0, + 0,0,115,36,1,0,0,116,0,124,1,116,1,131,2,115, + 28,100,1,100,0,108,2,125,2,124,2,160,3,124,1,161, + 1,125,1,124,1,115,44,116,4,100,2,124,1,100,3,141, + 2,130,1,116,5,114,60,124,1,160,6,116,5,116,7,161, + 2,125,1,103,0,125,3,122,14,116,8,160,9,124,1,161, + 1,125,4,87,0,110,72,4,0,116,10,116,11,102,2,107, + 10,114,150,1,0,1,0,1,0,116,8,160,12,124,1,161, + 1,92,2,125,5,125,6,124,5,124,1,107,2,114,132,116, + 4,100,4,124,1,100,3,141,2,130,1,124,5,125,1,124, + 3,160,13,124,6,161,1,1,0,89,0,113,64,88,0,124, + 4,106,14,100,5,64,0,100,6,107,3,114,182,116,4,100, + 4,124,1,100,3,141,2,130,1,113,182,113,64,122,12,116, + 15,124,1,25,0,125,7,87,0,110,36,4,0,116,16,107, + 10,114,230,1,0,1,0,1,0,116,17,124,1,131,1,125, + 7,124,7,116,15,124,1,60,0,89,0,110,2,88,0,124, + 7,124,0,95,18,124,1,124,0,95,19,116,8,106,20,124, + 3,100,0,100,0,100,7,133,3,25,0,142,0,124,0,95, + 21,124,0,106,21,144,1,114,32,124,0,4,0,106,21,116, + 7,55,0,2,0,95,21,100,0,83,0,41,8,78,114,0, + 0,0,0,122,21,97,114,99,104,105,118,101,32,112,97,116, + 104,32,105,115,32,101,109,112,116,121,41,1,218,4,112,97, + 116,104,122,14,110,111,116,32,97,32,90,105,112,32,102,105, + 108,101,105,0,240,0,0,105,0,128,0,0,233,255,255,255, + 255,41,22,218,10,105,115,105,110,115,116,97,110,99,101,218, + 3,115,116,114,218,2,111,115,90,8,102,115,100,101,99,111, + 100,101,114,3,0,0,0,218,12,97,108,116,95,112,97,116, + 104,95,115,101,112,218,7,114,101,112,108,97,99,101,218,8, + 112,97,116,104,95,115,101,112,218,19,95,98,111,111,116,115, + 116,114,97,112,95,101,120,116,101,114,110,97,108,90,10,95, + 112,97,116,104,95,115,116,97,116,218,7,79,83,69,114,114, + 111,114,218,10,86,97,108,117,101,69,114,114,111,114,90,11, + 95,112,97,116,104,95,115,112,108,105,116,218,6,97,112,112, + 101,110,100,90,7,115,116,95,109,111,100,101,218,20,95,122, + 105,112,95,100,105,114,101,99,116,111,114,121,95,99,97,99, + 104,101,218,8,75,101,121,69,114,114,111,114,218,15,95,114, + 101,97,100,95,100,105,114,101,99,116,111,114,121,218,6,95, + 102,105,108,101,115,218,7,97,114,99,104,105,118,101,218,10, + 95,112,97,116,104,95,106,111,105,110,218,6,112,114,101,102, + 105,120,41,8,218,4,115,101,108,102,114,12,0,0,0,114, + 16,0,0,0,114,30,0,0,0,90,2,115,116,90,7,100, + 105,114,110,97,109,101,90,8,98,97,115,101,110,97,109,101, + 218,5,102,105,108,101,115,114,9,0,0,0,114,9,0,0, + 0,114,10,0,0,0,218,8,95,95,105,110,105,116,95,95, + 63,0,0,0,115,58,0,0,0,0,1,10,1,8,1,10, + 1,4,1,12,1,4,1,12,2,4,2,2,1,14,1,18, + 3,14,1,8,1,12,1,4,1,16,3,14,2,12,1,4, + 2,2,1,12,1,14,1,8,1,14,1,6,1,6,2,22, + 1,8,1,122,20,122,105,112,105,109,112,111,114,116,101,114, + 46,95,95,105,110,105,116,95,95,78,99,3,0,0,0,0, + 0,0,0,5,0,0,0,4,0,0,0,67,0,0,0,115, + 78,0,0,0,116,0,124,0,124,1,131,2,125,3,124,3, + 100,1,107,9,114,26,124,0,103,0,102,2,83,0,116,1, + 124,0,124,1,131,2,125,4,116,2,124,0,124,4,131,2, + 114,70,100,1,124,0,106,3,155,0,116,4,155,0,124,4, + 155,0,157,3,103,1,102,2,83,0,100,1,103,0,102,2, + 83,0,41,2,97,239,1,0,0,102,105,110,100,95,108,111, + 97,100,101,114,40,102,117,108,108,110,97,109,101,44,32,112, + 97,116,104,61,78,111,110,101,41,32,45,62,32,115,101,108, + 102,44,32,115,116,114,32,111,114,32,78,111,110,101,46,10, + 10,32,32,32,32,32,32,32,32,83,101,97,114,99,104,32, + 102,111,114,32,97,32,109,111,100,117,108,101,32,115,112,101, + 99,105,102,105,101,100,32,98,121,32,39,102,117,108,108,110, + 97,109,101,39,46,32,39,102,117,108,108,110,97,109,101,39, + 32,109,117,115,116,32,98,101,32,116,104,101,10,32,32,32, + 32,32,32,32,32,102,117,108,108,121,32,113,117,97,108,105, + 102,105,101,100,32,40,100,111,116,116,101,100,41,32,109,111, + 100,117,108,101,32,110,97,109,101,46,32,73,116,32,114,101, + 116,117,114,110,115,32,116,104,101,32,122,105,112,105,109,112, + 111,114,116,101,114,10,32,32,32,32,32,32,32,32,105,110, + 115,116,97,110,99,101,32,105,116,115,101,108,102,32,105,102, + 32,116,104,101,32,109,111,100,117,108,101,32,119,97,115,32, + 102,111,117,110,100,44,32,97,32,115,116,114,105,110,103,32, + 99,111,110,116,97,105,110,105,110,103,32,116,104,101,10,32, + 32,32,32,32,32,32,32,102,117,108,108,32,112,97,116,104, + 32,110,97,109,101,32,105,102,32,105,116,39,115,32,112,111, + 115,115,105,98,108,121,32,97,32,112,111,114,116,105,111,110, + 32,111,102,32,97,32,110,97,109,101,115,112,97,99,101,32, + 112,97,99,107,97,103,101,44,10,32,32,32,32,32,32,32, + 32,111,114,32,78,111,110,101,32,111,116,104,101,114,119,105, + 115,101,46,32,84,104,101,32,111,112,116,105,111,110,97,108, + 32,39,112,97,116,104,39,32,97,114,103,117,109,101,110,116, + 32,105,115,32,105,103,110,111,114,101,100,32,45,45,32,105, + 116,39,115,10,32,32,32,32,32,32,32,32,116,104,101,114, + 101,32,102,111,114,32,99,111,109,112,97,116,105,98,105,108, + 105,116,121,32,119,105,116,104,32,116,104,101,32,105,109,112, + 111,114,116,101,114,32,112,114,111,116,111,99,111,108,46,10, + 32,32,32,32,32,32,32,32,78,41,5,218,16,95,103,101, + 116,95,109,111,100,117,108,101,95,105,110,102,111,218,16,95, + 103,101,116,95,109,111,100,117,108,101,95,112,97,116,104,218, + 7,95,105,115,95,100,105,114,114,28,0,0,0,114,19,0, + 0,0,41,5,114,31,0,0,0,218,8,102,117,108,108,110, + 97,109,101,114,12,0,0,0,218,2,109,105,218,7,109,111, + 100,112,97,116,104,114,9,0,0,0,114,9,0,0,0,114, + 10,0,0,0,218,11,102,105,110,100,95,108,111,97,100,101, + 114,109,0,0,0,115,14,0,0,0,0,10,10,1,8,2, + 8,7,10,1,10,4,24,2,122,23,122,105,112,105,109,112, + 111,114,116,101,114,46,102,105,110,100,95,108,111,97,100,101, + 114,99,3,0,0,0,0,0,0,0,3,0,0,0,4,0, + 0,0,67,0,0,0,115,16,0,0,0,124,0,160,0,124, + 1,124,2,161,2,100,1,25,0,83,0,41,2,97,139,1, + 0,0,102,105,110,100,95,109,111,100,117,108,101,40,102,117, + 108,108,110,97,109,101,44,32,112,97,116,104,61,78,111,110, + 101,41,32,45,62,32,115,101,108,102,32,111,114,32,78,111, + 110,101,46,10,10,32,32,32,32,32,32,32,32,83,101,97, + 114,99,104,32,102,111,114,32,97,32,109,111,100,117,108,101, + 32,115,112,101,99,105,102,105,101,100,32,98,121,32,39,102, + 117,108,108,110,97,109,101,39,46,32,39,102,117,108,108,110, + 97,109,101,39,32,109,117,115,116,32,98,101,32,116,104,101, + 10,32,32,32,32,32,32,32,32,102,117,108,108,121,32,113, + 117,97,108,105,102,105,101,100,32,40,100,111,116,116,101,100, + 41,32,109,111,100,117,108,101,32,110,97,109,101,46,32,73, + 116,32,114,101,116,117,114,110,115,32,116,104,101,32,122,105, + 112,105,109,112,111,114,116,101,114,10,32,32,32,32,32,32, + 32,32,105,110,115,116,97,110,99,101,32,105,116,115,101,108, + 102,32,105,102,32,116,104,101,32,109,111,100,117,108,101,32, + 119,97,115,32,102,111,117,110,100,44,32,111,114,32,78,111, + 110,101,32,105,102,32,105,116,32,119,97,115,110,39,116,46, + 10,32,32,32,32,32,32,32,32,84,104,101,32,111,112,116, + 105,111,110,97,108,32,39,112,97,116,104,39,32,97,114,103, + 117,109,101,110,116,32,105,115,32,105,103,110,111,114,101,100, + 32,45,45,32,105,116,39,115,32,116,104,101,114,101,32,102, + 111,114,32,99,111,109,112,97,116,105,98,105,108,105,116,121, + 10,32,32,32,32,32,32,32,32,119,105,116,104,32,116,104, + 101,32,105,109,112,111,114,116,101,114,32,112,114,111,116,111, + 99,111,108,46,10,32,32,32,32,32,32,32,32,114,0,0, + 0,0,41,1,114,40,0,0,0,41,3,114,31,0,0,0, + 114,37,0,0,0,114,12,0,0,0,114,9,0,0,0,114, + 9,0,0,0,114,10,0,0,0,218,11,102,105,110,100,95, + 109,111,100,117,108,101,141,0,0,0,115,2,0,0,0,0, + 9,122,23,122,105,112,105,109,112,111,114,116,101,114,46,102, + 105,110,100,95,109,111,100,117,108,101,99,2,0,0,0,0, + 0,0,0,5,0,0,0,3,0,0,0,67,0,0,0,115, + 20,0,0,0,116,0,124,0,124,1,131,2,92,3,125,2, + 125,3,125,4,124,2,83,0,41,1,122,163,103,101,116,95, + 99,111,100,101,40,102,117,108,108,110,97,109,101,41,32,45, + 62,32,99,111,100,101,32,111,98,106,101,99,116,46,10,10, + 32,32,32,32,32,32,32,32,82,101,116,117,114,110,32,116, + 104,101,32,99,111,100,101,32,111,98,106,101,99,116,32,102, + 111,114,32,116,104,101,32,115,112,101,99,105,102,105,101,100, + 32,109,111,100,117,108,101,46,32,82,97,105,115,101,32,90, + 105,112,73,109,112,111,114,116,69,114,114,111,114,10,32,32, + 32,32,32,32,32,32,105,102,32,116,104,101,32,109,111,100, + 117,108,101,32,99,111,117,108,100,110,39,116,32,98,101,32, + 102,111,117,110,100,46,10,32,32,32,32,32,32,32,32,41, + 1,218,16,95,103,101,116,95,109,111,100,117,108,101,95,99, + 111,100,101,41,5,114,31,0,0,0,114,37,0,0,0,218, + 4,99,111,100,101,218,9,105,115,112,97,99,107,97,103,101, + 114,39,0,0,0,114,9,0,0,0,114,9,0,0,0,114, + 10,0,0,0,218,8,103,101,116,95,99,111,100,101,153,0, + 0,0,115,4,0,0,0,0,6,16,1,122,20,122,105,112, + 105,109,112,111,114,116,101,114,46,103,101,116,95,99,111,100, + 101,99,2,0,0,0,0,0,0,0,4,0,0,0,8,0, + 0,0,67,0,0,0,115,118,0,0,0,116,0,114,16,124, + 1,160,1,116,0,116,2,161,2,125,1,124,1,125,2,124, + 1,160,3,124,0,106,4,116,2,23,0,161,1,114,58,124, + 1,116,5,124,0,106,4,116,2,23,0,131,1,100,1,133, + 2,25,0,125,2,122,14,124,0,106,6,124,2,25,0,125, + 3,87,0,110,32,4,0,116,7,107,10,114,104,1,0,1, + 0,1,0,116,8,100,2,100,3,124,2,131,3,130,1,89, + 0,110,2,88,0,116,9,124,0,106,4,124,3,131,2,83, + 0,41,4,122,154,103,101,116,95,100,97,116,97,40,112,97, + 116,104,110,97,109,101,41,32,45,62,32,115,116,114,105,110, + 103,32,119,105,116,104,32,102,105,108,101,32,100,97,116,97, + 46,10,10,32,32,32,32,32,32,32,32,82,101,116,117,114, + 110,32,116,104,101,32,100,97,116,97,32,97,115,115,111,99, + 105,97,116,101,100,32,119,105,116,104,32,39,112,97,116,104, + 110,97,109,101,39,46,32,82,97,105,115,101,32,79,83,69, + 114,114,111,114,32,105,102,10,32,32,32,32,32,32,32,32, + 116,104,101,32,102,105,108,101,32,119,97,115,110,39,116,32, + 102,111,117,110,100,46,10,32,32,32,32,32,32,32,32,78, + 114,0,0,0,0,218,0,41,10,114,17,0,0,0,114,18, + 0,0,0,114,19,0,0,0,218,10,115,116,97,114,116,115, + 119,105,116,104,114,28,0,0,0,218,3,108,101,110,114,27, + 0,0,0,114,25,0,0,0,114,21,0,0,0,218,9,95, + 103,101,116,95,100,97,116,97,41,4,114,31,0,0,0,218, + 8,112,97,116,104,110,97,109,101,90,3,107,101,121,218,9, + 116,111,99,95,101,110,116,114,121,114,9,0,0,0,114,9, + 0,0,0,114,10,0,0,0,218,8,103,101,116,95,100,97, + 116,97,163,0,0,0,115,20,0,0,0,0,6,4,1,12, + 2,4,1,16,1,22,2,2,1,14,1,14,1,18,1,122, + 20,122,105,112,105,109,112,111,114,116,101,114,46,103,101,116, + 95,100,97,116,97,99,2,0,0,0,0,0,0,0,5,0, + 0,0,3,0,0,0,67,0,0,0,115,20,0,0,0,116, + 0,124,0,124,1,131,2,92,3,125,2,125,3,125,4,124, + 4,83,0,41,1,122,106,103,101,116,95,102,105,108,101,110, + 97,109,101,40,102,117,108,108,110,97,109,101,41,32,45,62, + 32,102,105,108,101,110,97,109,101,32,115,116,114,105,110,103, + 46,10,10,32,32,32,32,32,32,32,32,82,101,116,117,114, + 110,32,116,104,101,32,102,105,108,101,110,97,109,101,32,102, + 111,114,32,116,104,101,32,115,112,101,99,105,102,105,101,100, + 32,109,111,100,117,108,101,46,10,32,32,32,32,32,32,32, + 32,41,1,114,42,0,0,0,41,5,114,31,0,0,0,114, + 37,0,0,0,114,43,0,0,0,114,44,0,0,0,114,39, + 0,0,0,114,9,0,0,0,114,9,0,0,0,114,10,0, + 0,0,218,12,103,101,116,95,102,105,108,101,110,97,109,101, + 184,0,0,0,115,4,0,0,0,0,7,16,1,122,24,122, + 105,112,105,109,112,111,114,116,101,114,46,103,101,116,95,102, + 105,108,101,110,97,109,101,99,2,0,0,0,0,0,0,0, + 6,0,0,0,8,0,0,0,67,0,0,0,115,128,0,0, + 0,116,0,124,0,124,1,131,2,125,2,124,2,100,1,107, + 8,114,36,116,1,100,2,124,1,155,2,157,2,124,1,100, + 3,141,2,130,1,116,2,124,0,124,1,131,2,125,3,124, + 2,114,64,116,3,160,4,124,3,100,4,161,2,125,4,110, + 10,124,3,155,0,100,5,157,2,125,4,122,14,124,0,106, + 5,124,4,25,0,125,5,87,0,110,22,4,0,116,6,107, + 10,114,110,1,0,1,0,1,0,89,0,100,1,83,0,88, + 0,116,7,124,0,106,8,124,5,131,2,160,9,161,0,83, + 0,41,6,122,253,103,101,116,95,115,111,117,114,99,101,40, + 102,117,108,108,110,97,109,101,41,32,45,62,32,115,111,117, + 114,99,101,32,115,116,114,105,110,103,46,10,10,32,32,32, + 32,32,32,32,32,82,101,116,117,114,110,32,116,104,101,32, + 115,111,117,114,99,101,32,99,111,100,101,32,102,111,114,32, + 116,104,101,32,115,112,101,99,105,102,105,101,100,32,109,111, + 100,117,108,101,46,32,82,97,105,115,101,32,90,105,112,73, + 109,112,111,114,116,69,114,114,111,114,10,32,32,32,32,32, + 32,32,32,105,102,32,116,104,101,32,109,111,100,117,108,101, + 32,99,111,117,108,100,110,39,116,32,98,101,32,102,111,117, + 110,100,44,32,114,101,116,117,114,110,32,78,111,110,101,32, + 105,102,32,116,104,101,32,97,114,99,104,105,118,101,32,100, + 111,101,115,10,32,32,32,32,32,32,32,32,99,111,110,116, + 97,105,110,32,116,104,101,32,109,111,100,117,108,101,44,32, + 98,117,116,32,104,97,115,32,110,111,32,115,111,117,114,99, + 101,32,102,111,114,32,105,116,46,10,32,32,32,32,32,32, + 32,32,78,122,18,99,97,110,39,116,32,102,105,110,100,32, + 109,111,100,117,108,101,32,41,1,218,4,110,97,109,101,122, + 11,95,95,105,110,105,116,95,95,46,112,121,122,3,46,112, + 121,41,10,114,34,0,0,0,114,3,0,0,0,114,35,0, + 0,0,114,20,0,0,0,114,29,0,0,0,114,27,0,0, + 0,114,25,0,0,0,114,49,0,0,0,114,28,0,0,0, + 218,6,100,101,99,111,100,101,41,6,114,31,0,0,0,114, + 37,0,0,0,114,38,0,0,0,114,12,0,0,0,218,8, + 102,117,108,108,112,97,116,104,114,51,0,0,0,114,9,0, + 0,0,114,9,0,0,0,114,10,0,0,0,218,10,103,101, + 116,95,115,111,117,114,99,101,195,0,0,0,115,24,0,0, + 0,0,7,10,1,8,1,18,2,10,1,4,1,14,2,10, + 2,2,1,14,1,14,2,8,1,122,22,122,105,112,105,109, + 112,111,114,116,101,114,46,103,101,116,95,115,111,117,114,99, + 101,99,2,0,0,0,0,0,0,0,3,0,0,0,4,0, + 0,0,67,0,0,0,115,40,0,0,0,116,0,124,0,124, + 1,131,2,125,2,124,2,100,1,107,8,114,36,116,1,100, + 2,124,1,155,2,157,2,124,1,100,3,141,2,130,1,124, + 2,83,0,41,4,122,171,105,115,95,112,97,99,107,97,103, + 101,40,102,117,108,108,110,97,109,101,41,32,45,62,32,98, + 111,111,108,46,10,10,32,32,32,32,32,32,32,32,82,101, + 116,117,114,110,32,84,114,117,101,32,105,102,32,116,104,101, + 32,109,111,100,117,108,101,32,115,112,101,99,105,102,105,101, + 100,32,98,121,32,102,117,108,108,110,97,109,101,32,105,115, + 32,97,32,112,97,99,107,97,103,101,46,10,32,32,32,32, + 32,32,32,32,82,97,105,115,101,32,90,105,112,73,109,112, + 111,114,116,69,114,114,111,114,32,105,102,32,116,104,101,32, + 109,111,100,117,108,101,32,99,111,117,108,100,110,39,116,32, + 98,101,32,102,111,117,110,100,46,10,32,32,32,32,32,32, + 32,32,78,122,18,99,97,110,39,116,32,102,105,110,100,32, + 109,111,100,117,108,101,32,41,1,114,54,0,0,0,41,2, + 114,34,0,0,0,114,3,0,0,0,41,3,114,31,0,0, + 0,114,37,0,0,0,114,38,0,0,0,114,9,0,0,0, + 114,9,0,0,0,114,10,0,0,0,218,10,105,115,95,112, + 97,99,107,97,103,101,221,0,0,0,115,8,0,0,0,0, + 6,10,1,8,1,18,1,122,22,122,105,112,105,109,112,111, + 114,116,101,114,46,105,115,95,112,97,99,107,97,103,101,99, + 2,0,0,0,0,0,0,0,8,0,0,0,8,0,0,0, + 67,0,0,0,115,248,0,0,0,116,0,124,0,124,1,131, + 2,92,3,125,2,125,3,125,4,116,1,106,2,160,3,124, + 1,161,1,125,5,124,5,100,1,107,8,115,46,116,4,124, + 5,116,5,131,2,115,64,116,5,124,1,131,1,125,5,124, + 5,116,1,106,2,124,1,60,0,124,0,124,5,95,6,122, + 84,124,3,114,108,116,7,124,0,124,1,131,2,125,6,116, + 8,160,9,124,0,106,10,124,6,161,2,125,7,124,7,103, + 1,124,5,95,11,116,12,124,5,100,2,131,2,115,124,116, + 13,124,5,95,13,116,8,160,14,124,5,106,15,124,1,124, + 4,161,3,1,0,116,16,124,2,124,5,106,15,131,2,1, + 0,87,0,110,22,1,0,1,0,1,0,116,1,106,2,124, + 1,61,0,130,0,89,0,110,2,88,0,122,14,116,1,106, + 2,124,1,25,0,125,5,87,0,110,36,4,0,116,17,107, + 10,114,228,1,0,1,0,1,0,116,18,100,3,124,1,155, + 2,100,4,157,3,131,1,130,1,89,0,110,2,88,0,116, + 19,160,20,100,5,124,1,124,4,161,3,1,0,124,5,83, + 0,41,6,122,245,108,111,97,100,95,109,111,100,117,108,101, + 40,102,117,108,108,110,97,109,101,41,32,45,62,32,109,111, + 100,117,108,101,46,10,10,32,32,32,32,32,32,32,32,76, + 111,97,100,32,116,104,101,32,109,111,100,117,108,101,32,115, + 112,101,99,105,102,105,101,100,32,98,121,32,39,102,117,108, + 108,110,97,109,101,39,46,32,39,102,117,108,108,110,97,109, + 101,39,32,109,117,115,116,32,98,101,32,116,104,101,10,32, + 32,32,32,32,32,32,32,102,117,108,108,121,32,113,117,97, + 108,105,102,105,101,100,32,40,100,111,116,116,101,100,41,32, + 109,111,100,117,108,101,32,110,97,109,101,46,32,73,116,32, + 114,101,116,117,114,110,115,32,116,104,101,32,105,109,112,111, + 114,116,101,100,10,32,32,32,32,32,32,32,32,109,111,100, + 117,108,101,44,32,111,114,32,114,97,105,115,101,115,32,90, + 105,112,73,109,112,111,114,116,69,114,114,111,114,32,105,102, + 32,105,116,32,119,97,115,110,39,116,32,102,111,117,110,100, + 46,10,32,32,32,32,32,32,32,32,78,218,12,95,95,98, + 117,105,108,116,105,110,115,95,95,122,14,76,111,97,100,101, + 100,32,109,111,100,117,108,101,32,122,25,32,110,111,116,32, + 102,111,117,110,100,32,105,110,32,115,121,115,46,109,111,100, + 117,108,101,115,122,30,105,109,112,111,114,116,32,123,125,32, + 35,32,108,111,97,100,101,100,32,102,114,111,109,32,90,105, + 112,32,123,125,41,21,114,42,0,0,0,218,3,115,121,115, + 218,7,109,111,100,117,108,101,115,218,3,103,101,116,114,14, + 0,0,0,218,12,95,109,111,100,117,108,101,95,116,121,112, + 101,218,10,95,95,108,111,97,100,101,114,95,95,114,35,0, + 0,0,114,20,0,0,0,114,29,0,0,0,114,28,0,0, + 0,90,8,95,95,112,97,116,104,95,95,218,7,104,97,115, + 97,116,116,114,114,59,0,0,0,90,14,95,102,105,120,95, + 117,112,95,109,111,100,117,108,101,218,8,95,95,100,105,99, + 116,95,95,218,4,101,120,101,99,114,25,0,0,0,218,11, + 73,109,112,111,114,116,69,114,114,111,114,218,10,95,98,111, + 111,116,115,116,114,97,112,218,16,95,118,101,114,98,111,115, + 101,95,109,101,115,115,97,103,101,41,8,114,31,0,0,0, + 114,37,0,0,0,114,43,0,0,0,114,44,0,0,0,114, + 39,0,0,0,90,3,109,111,100,114,12,0,0,0,114,56, + 0,0,0,114,9,0,0,0,114,9,0,0,0,114,10,0, + 0,0,218,11,108,111,97,100,95,109,111,100,117,108,101,234, + 0,0,0,115,48,0,0,0,0,7,16,1,12,1,18,1, + 8,1,10,1,6,2,2,1,4,3,10,1,14,1,8,2, + 10,1,6,1,16,1,16,1,6,1,8,1,8,2,2,1, + 14,1,14,1,22,1,14,1,122,23,122,105,112,105,109,112, + 111,114,116,101,114,46,108,111,97,100,95,109,111,100,117,108, + 101,99,2,0,0,0,0,0,0,0,3,0,0,0,8,0, + 0,0,67,0,0,0,115,88,0,0,0,122,20,124,0,160, + 0,124,1,161,1,115,18,87,0,100,1,83,0,87,0,110, + 22,4,0,116,1,107,10,114,42,1,0,1,0,1,0,89, + 0,100,1,83,0,88,0,116,2,106,3,115,78,100,2,100, + 3,108,4,109,5,125,2,1,0,124,2,160,6,116,2,161, + 1,1,0,100,4,116,2,95,3,116,2,124,0,124,1,131, + 2,83,0,41,5,122,204,82,101,116,117,114,110,32,116,104, + 101,32,82,101,115,111,117,114,99,101,82,101,97,100,101,114, + 32,102,111,114,32,97,32,112,97,99,107,97,103,101,32,105, + 110,32,97,32,122,105,112,32,102,105,108,101,46,10,10,32, + 32,32,32,32,32,32,32,73,102,32,39,102,117,108,108,110, + 97,109,101,39,32,105,115,32,97,32,112,97,99,107,97,103, + 101,32,119,105,116,104,105,110,32,116,104,101,32,122,105,112, + 32,102,105,108,101,44,32,114,101,116,117,114,110,32,116,104, + 101,10,32,32,32,32,32,32,32,32,39,82,101,115,111,117, + 114,99,101,82,101,97,100,101,114,39,32,111,98,106,101,99, + 116,32,102,111,114,32,116,104,101,32,112,97,99,107,97,103, + 101,46,32,32,79,116,104,101,114,119,105,115,101,32,114,101, + 116,117,114,110,32,78,111,110,101,46,10,32,32,32,32,32, + 32,32,32,78,114,0,0,0,0,41,1,218,14,82,101,115, + 111,117,114,99,101,82,101,97,100,101,114,84,41,7,114,58, + 0,0,0,114,3,0,0,0,218,24,95,90,105,112,73,109, + 112,111,114,116,82,101,115,111,117,114,99,101,82,101,97,100, + 101,114,218,11,95,114,101,103,105,115,116,101,114,101,100,90, + 13,105,109,112,111,114,116,108,105,98,46,97,98,99,114,72, + 0,0,0,90,8,114,101,103,105,115,116,101,114,41,3,114, + 31,0,0,0,114,37,0,0,0,114,72,0,0,0,114,9, + 0,0,0,114,9,0,0,0,114,10,0,0,0,218,19,103, + 101,116,95,114,101,115,111,117,114,99,101,95,114,101,97,100, + 101,114,16,1,0,0,115,20,0,0,0,0,6,2,1,10, + 1,10,1,14,1,8,1,6,1,12,1,10,1,6,1,122, + 31,122,105,112,105,109,112,111,114,116,101,114,46,103,101,116, + 95,114,101,115,111,117,114,99,101,95,114,101,97,100,101,114, + 99,1,0,0,0,0,0,0,0,1,0,0,0,5,0,0, + 0,67,0,0,0,115,24,0,0,0,100,1,124,0,106,0, + 155,0,116,1,155,0,124,0,106,2,155,0,100,2,157,5, + 83,0,41,3,78,122,21,60,122,105,112,105,109,112,111,114, + 116,101,114,32,111,98,106,101,99,116,32,34,122,2,34,62, + 41,3,114,28,0,0,0,114,19,0,0,0,114,30,0,0, + 0,41,1,114,31,0,0,0,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,218,8,95,95,114,101,112,114,95, + 95,34,1,0,0,115,2,0,0,0,0,1,122,20,122,105, + 112,105,109,112,111,114,116,101,114,46,95,95,114,101,112,114, + 95,95,41,1,78,41,1,78,41,15,114,6,0,0,0,114, + 7,0,0,0,114,8,0,0,0,218,7,95,95,100,111,99, + 95,95,114,33,0,0,0,114,40,0,0,0,114,41,0,0, + 0,114,45,0,0,0,114,52,0,0,0,114,53,0,0,0, + 114,57,0,0,0,114,58,0,0,0,114,71,0,0,0,114, + 75,0,0,0,114,76,0,0,0,114,9,0,0,0,114,9, + 0,0,0,114,9,0,0,0,114,10,0,0,0,114,4,0, + 0,0,45,0,0,0,115,24,0,0,0,8,13,4,5,8, + 46,10,32,10,12,8,10,8,21,8,11,8,26,8,13,8, + 38,8,18,122,12,95,95,105,110,105,116,95,95,46,112,121, + 99,84,122,11,95,95,105,110,105,116,95,95,46,112,121,70, + 41,3,122,4,46,112,121,99,84,70,41,3,122,3,46,112, + 121,70,70,99,2,0,0,0,0,0,0,0,2,0,0,0, + 4,0,0,0,67,0,0,0,115,20,0,0,0,124,0,106, + 0,124,1,160,1,100,1,161,1,100,2,25,0,23,0,83, + 0,41,3,78,218,1,46,233,2,0,0,0,41,2,114,30, + 0,0,0,218,10,114,112,97,114,116,105,116,105,111,110,41, + 2,114,31,0,0,0,114,37,0,0,0,114,9,0,0,0, + 114,9,0,0,0,114,10,0,0,0,114,35,0,0,0,52, + 1,0,0,115,2,0,0,0,0,1,114,35,0,0,0,99, + 2,0,0,0,0,0,0,0,3,0,0,0,2,0,0,0, + 67,0,0,0,115,18,0,0,0,124,1,116,0,23,0,125, + 2,124,2,124,0,106,1,107,6,83,0,41,1,78,41,2, + 114,19,0,0,0,114,27,0,0,0,41,3,114,31,0,0, + 0,114,12,0,0,0,90,7,100,105,114,112,97,116,104,114, + 9,0,0,0,114,9,0,0,0,114,10,0,0,0,114,36, + 0,0,0,56,1,0,0,115,4,0,0,0,0,4,8,2, + 114,36,0,0,0,99,2,0,0,0,0,0,0,0,7,0, + 0,0,4,0,0,0,67,0,0,0,115,56,0,0,0,116, + 0,124,0,124,1,131,2,125,2,116,1,68,0,93,36,92, + 3,125,3,125,4,125,5,124,2,124,3,23,0,125,6,124, + 6,124,0,106,2,107,6,114,14,124,5,2,0,1,0,83, + 0,113,14,100,0,83,0,41,1,78,41,3,114,35,0,0, + 0,218,16,95,122,105,112,95,115,101,97,114,99,104,111,114, + 100,101,114,114,27,0,0,0,41,7,114,31,0,0,0,114, + 37,0,0,0,114,12,0,0,0,218,6,115,117,102,102,105, + 120,218,10,105,115,98,121,116,101,99,111,100,101,114,44,0, + 0,0,114,56,0,0,0,114,9,0,0,0,114,9,0,0, + 0,114,10,0,0,0,114,34,0,0,0,65,1,0,0,115, + 12,0,0,0,0,1,10,1,14,1,8,1,10,1,10,1, + 114,34,0,0,0,99,1,0,0,0,0,0,0,0,26,0, + 0,0,9,0,0,0,67,0,0,0,115,254,4,0,0,122, + 16,116,0,160,1,124,0,100,1,161,2,125,1,87,0,110, + 38,4,0,116,2,107,10,114,54,1,0,1,0,1,0,116, + 3,100,2,124,0,155,2,157,2,124,0,100,3,141,2,130, + 1,89,0,110,2,88,0,124,1,144,4,143,168,1,0,122, + 36,124,1,160,4,116,5,11,0,100,4,161,2,1,0,124, + 1,160,6,161,0,125,2,124,1,160,7,116,5,161,1,125, + 3,87,0,110,38,4,0,116,2,107,10,114,138,1,0,1, + 0,1,0,116,3,100,5,124,0,155,2,157,2,124,0,100, + 3,141,2,130,1,89,0,110,2,88,0,116,8,124,3,131, + 1,116,5,107,3,114,170,116,3,100,5,124,0,155,2,157, + 2,124,0,100,3,141,2,130,1,124,3,100,0,100,6,133, + 2,25,0,116,9,107,3,144,1,114,180,122,24,124,1,160, + 4,100,7,100,4,161,2,1,0,124,1,160,6,161,0,125, + 4,87,0,110,38,4,0,116,2,107,10,114,250,1,0,1, + 0,1,0,116,3,100,5,124,0,155,2,157,2,124,0,100, + 3,141,2,130,1,89,0,110,2,88,0,116,10,124,4,116, + 11,24,0,116,5,24,0,100,7,131,2,125,5,122,22,124, + 1,160,4,124,5,161,1,1,0,124,1,160,7,161,0,125, + 6,87,0,110,40,4,0,116,2,107,10,144,1,114,76,1, + 0,1,0,1,0,116,3,100,5,124,0,155,2,157,2,124, + 0,100,3,141,2,130,1,89,0,110,2,88,0,124,6,160, + 12,116,9,161,1,125,7,124,7,100,7,107,0,144,1,114, + 116,116,3,100,8,124,0,155,2,157,2,124,0,100,3,141, + 2,130,1,124,6,124,7,124,7,116,5,23,0,133,2,25, + 0,125,3,116,8,124,3,131,1,116,5,107,3,144,1,114, + 164,116,3,100,9,124,0,155,2,157,2,124,0,100,3,141, + 2,130,1,124,4,116,8,124,6,131,1,24,0,124,7,23, + 0,125,2,116,13,124,3,100,10,100,11,133,2,25,0,131, + 1,125,8,116,13,124,3,100,11,100,12,133,2,25,0,131, + 1,125,9,124,2,124,8,107,0,144,1,114,240,116,3,100, + 13,124,0,155,2,157,2,124,0,100,3,141,2,130,1,124, + 2,124,9,107,0,144,2,114,12,116,3,100,14,124,0,155, + 2,157,2,124,0,100,3,141,2,130,1,124,2,124,8,56, + 0,125,2,124,2,124,9,24,0,125,10,124,10,100,7,107, + 0,144,2,114,56,116,3,100,15,124,0,155,2,157,2,124, + 0,100,3,141,2,130,1,105,0,125,11,100,7,125,12,122, + 14,124,1,160,4,124,2,161,1,1,0,87,0,110,40,4, + 0,116,2,107,10,144,2,114,118,1,0,1,0,1,0,116, + 3,100,5,124,0,155,2,157,2,124,0,100,3,141,2,130, + 1,89,0,110,2,88,0,124,1,160,7,100,16,161,1,125, + 3,116,8,124,3,131,1,100,6,107,0,144,2,114,152,116, + 14,100,17,131,1,130,1,124,3,100,0,100,6,133,2,25, + 0,100,18,107,3,144,2,114,174,144,4,113,226,116,8,124, + 3,131,1,100,16,107,3,144,2,114,196,116,14,100,17,131, + 1,130,1,116,15,124,3,100,19,100,20,133,2,25,0,131, + 1,125,13,116,15,124,3,100,20,100,10,133,2,25,0,131, + 1,125,14,116,15,124,3,100,10,100,21,133,2,25,0,131, + 1,125,15,116,15,124,3,100,21,100,11,133,2,25,0,131, + 1,125,16,116,13,124,3,100,11,100,12,133,2,25,0,131, + 1,125,17,116,13,124,3,100,12,100,22,133,2,25,0,131, + 1,125,18,116,13,124,3,100,22,100,23,133,2,25,0,131, + 1,125,4,116,15,124,3,100,23,100,24,133,2,25,0,131, + 1,125,19,116,15,124,3,100,24,100,25,133,2,25,0,131, + 1,125,20,116,15,124,3,100,25,100,26,133,2,25,0,131, + 1,125,21,116,13,124,3,100,27,100,16,133,2,25,0,131, + 1,125,22,124,19,124,20,23,0,124,21,23,0,125,8,124, + 22,124,9,107,4,144,3,114,156,116,3,100,28,124,0,155, + 2,157,2,124,0,100,3,141,2,130,1,124,22,124,10,55, + 0,125,22,122,14,124,1,160,7,124,19,161,1,125,23,87, + 0,110,40,4,0,116,2,107,10,144,3,114,218,1,0,1, + 0,1,0,116,3,100,5,124,0,155,2,157,2,124,0,100, + 3,141,2,130,1,89,0,110,2,88,0,116,8,124,23,131, + 1,124,19,107,3,144,3,114,252,116,3,100,5,124,0,155, + 2,157,2,124,0,100,3,141,2,130,1,122,50,116,8,124, + 1,160,7,124,8,124,19,24,0,161,1,131,1,124,8,124, + 19,24,0,107,3,144,4,114,44,116,3,100,5,124,0,155, + 2,157,2,124,0,100,3,141,2,130,1,87,0,110,40,4, + 0,116,2,107,10,144,4,114,86,1,0,1,0,1,0,116, + 3,100,5,124,0,155,2,157,2,124,0,100,3,141,2,130, + 1,89,0,110,2,88,0,124,13,100,29,64,0,144,4,114, + 108,124,23,160,16,161,0,125,23,110,54,122,14,124,23,160, + 16,100,30,161,1,125,23,87,0,110,38,4,0,116,17,107, + 10,144,4,114,160,1,0,1,0,1,0,124,23,160,16,100, + 31,161,1,160,18,116,19,161,1,125,23,89,0,110,2,88, + 0,124,23,160,20,100,32,116,21,161,2,125,23,116,22,160, + 23,124,0,124,23,161,2,125,24,124,24,124,14,124,18,124, + 4,124,22,124,15,124,16,124,17,102,8,125,25,124,25,124, + 11,124,23,60,0,124,12,100,33,55,0,125,12,144,2,113, + 120,87,0,53,0,81,0,82,0,88,0,116,24,160,25,100, + 34,124,12,124,0,161,3,1,0,124,11,83,0,41,35,78, + 218,2,114,98,122,21,99,97,110,39,116,32,111,112,101,110, + 32,90,105,112,32,102,105,108,101,58,32,41,1,114,12,0, + 0,0,114,79,0,0,0,122,21,99,97,110,39,116,32,114, + 101,97,100,32,90,105,112,32,102,105,108,101,58,32,233,4, + 0,0,0,114,0,0,0,0,122,16,110,111,116,32,97,32, + 90,105,112,32,102,105,108,101,58,32,122,18,99,111,114,114, + 117,112,116,32,90,105,112,32,102,105,108,101,58,32,233,12, + 0,0,0,233,16,0,0,0,233,20,0,0,0,122,28,98, + 97,100,32,99,101,110,116,114,97,108,32,100,105,114,101,99, + 116,111,114,121,32,115,105,122,101,58,32,122,30,98,97,100, + 32,99,101,110,116,114,97,108,32,100,105,114,101,99,116,111, + 114,121,32,111,102,102,115,101,116,58,32,122,38,98,97,100, + 32,99,101,110,116,114,97,108,32,100,105,114,101,99,116,111, + 114,121,32,115,105,122,101,32,111,114,32,111,102,102,115,101, + 116,58,32,233,46,0,0,0,122,27,69,79,70,32,114,101, + 97,100,32,119,104,101,114,101,32,110,111,116,32,101,120,112, + 101,99,116,101,100,115,4,0,0,0,80,75,1,2,233,8, + 0,0,0,233,10,0,0,0,233,14,0,0,0,233,24,0, + 0,0,233,28,0,0,0,233,30,0,0,0,233,32,0,0, + 0,233,34,0,0,0,233,42,0,0,0,122,25,98,97,100, + 32,108,111,99,97,108,32,104,101,97,100,101,114,32,111,102, + 102,115,101,116,58,32,105,0,8,0,0,218,5,97,115,99, + 105,105,90,6,108,97,116,105,110,49,250,1,47,114,5,0, + 0,0,122,33,122,105,112,105,109,112,111,114,116,58,32,102, + 111,117,110,100,32,123,125,32,110,97,109,101,115,32,105,110, + 32,123,33,114,125,41,26,218,3,95,105,111,218,4,111,112, + 101,110,114,21,0,0,0,114,3,0,0,0,218,4,115,101, + 101,107,218,20,69,78,68,95,67,69,78,84,82,65,76,95, + 68,73,82,95,83,73,90,69,90,4,116,101,108,108,218,4, + 114,101,97,100,114,48,0,0,0,218,18,83,84,82,73,78, + 71,95,69,78,68,95,65,82,67,72,73,86,69,218,3,109, + 97,120,218,15,77,65,88,95,67,79,77,77,69,78,84,95, + 76,69,78,218,5,114,102,105,110,100,114,2,0,0,0,218, + 8,69,79,70,69,114,114,111,114,114,1,0,0,0,114,55, + 0,0,0,218,18,85,110,105,99,111,100,101,68,101,99,111, + 100,101,69,114,114,111,114,218,9,116,114,97,110,115,108,97, + 116,101,218,11,99,112,52,51,55,95,116,97,98,108,101,114, + 18,0,0,0,114,19,0,0,0,114,20,0,0,0,114,29, + 0,0,0,114,69,0,0,0,114,70,0,0,0,41,26,114, + 28,0,0,0,218,2,102,112,90,15,104,101,97,100,101,114, + 95,112,111,115,105,116,105,111,110,218,6,98,117,102,102,101, + 114,218,9,102,105,108,101,95,115,105,122,101,90,17,109,97, + 120,95,99,111,109,109,101,110,116,95,115,116,97,114,116,218, + 4,100,97,116,97,90,3,112,111,115,218,11,104,101,97,100, + 101,114,95,115,105,122,101,90,13,104,101,97,100,101,114,95, + 111,102,102,115,101,116,90,10,97,114,99,95,111,102,102,115, + 101,116,114,32,0,0,0,218,5,99,111,117,110,116,218,5, + 102,108,97,103,115,218,8,99,111,109,112,114,101,115,115,218, + 4,116,105,109,101,218,4,100,97,116,101,218,3,99,114,99, + 218,9,100,97,116,97,95,115,105,122,101,218,9,110,97,109, + 101,95,115,105,122,101,218,10,101,120,116,114,97,95,115,105, + 122,101,90,12,99,111,109,109,101,110,116,95,115,105,122,101, + 218,11,102,105,108,101,95,111,102,102,115,101,116,114,54,0, + 0,0,114,12,0,0,0,218,1,116,114,9,0,0,0,114, + 9,0,0,0,114,10,0,0,0,114,26,0,0,0,96,1, + 0,0,115,212,0,0,0,0,1,2,1,16,1,14,1,24, + 2,8,1,2,1,14,1,8,1,14,1,14,1,24,1,12, + 1,18,1,18,3,2,1,12,1,12,1,14,1,10,1,2, + 255,12,2,8,1,2,255,2,1,2,255,4,2,2,1,10, + 1,12,1,16,1,10,1,2,255,12,2,10,1,10,1,10, + 1,2,255,6,2,16,1,14,1,10,1,2,255,6,2,16, + 2,16,1,16,1,10,1,18,1,10,1,18,1,8,1,8, + 1,10,1,18,2,4,2,4,1,2,1,14,1,16,1,24, + 2,10,1,14,1,8,2,18,1,4,1,14,1,8,1,16, + 1,16,1,16,1,16,1,16,1,16,1,16,1,16,1,16, + 1,16,1,16,1,12,1,10,1,18,1,8,2,2,1,14, + 1,16,1,24,1,14,1,18,4,2,1,28,1,22,1,16, + 1,24,2,10,2,10,3,2,1,14,1,16,1,22,2,12, + 1,12,1,20,1,8,1,22,1,14,1,114,26,0,0,0, + 117,190,1,0,0,0,1,2,3,4,5,6,7,8,9,10, + 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26, + 27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42, + 43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58, + 59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74, + 75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90, + 91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106, + 107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122, + 123,124,125,126,127,195,135,195,188,195,169,195,162,195,164,195, + 160,195,165,195,167,195,170,195,171,195,168,195,175,195,174,195, + 172,195,132,195,133,195,137,195,166,195,134,195,180,195,182,195, + 178,195,187,195,185,195,191,195,150,195,156,194,162,194,163,194, + 165,226,130,167,198,146,195,161,195,173,195,179,195,186,195,177, + 195,145,194,170,194,186,194,191,226,140,144,194,172,194,189,194, + 188,194,161,194,171,194,187,226,150,145,226,150,146,226,150,147, + 226,148,130,226,148,164,226,149,161,226,149,162,226,149,150,226, + 149,149,226,149,163,226,149,145,226,149,151,226,149,157,226,149, + 156,226,149,155,226,148,144,226,148,148,226,148,180,226,148,172, + 226,148,156,226,148,128,226,148,188,226,149,158,226,149,159,226, + 149,154,226,149,148,226,149,169,226,149,166,226,149,160,226,149, + 144,226,149,172,226,149,167,226,149,168,226,149,164,226,149,165, + 226,149,153,226,149,152,226,149,146,226,149,147,226,149,171,226, + 149,170,226,148,152,226,148,140,226,150,136,226,150,132,226,150, + 140,226,150,144,226,150,128,206,177,195,159,206,147,207,128,206, + 163,207,131,194,181,207,132,206,166,206,152,206,169,206,180,226, + 136,158,207,134,206,181,226,136,169,226,137,161,194,177,226,137, + 165,226,137,164,226,140,160,226,140,161,195,183,226,137,136,194, + 176,226,136,153,194,183,226,136,154,226,129,191,194,178,226,150, + 160,194,160,99,0,0,0,0,0,0,0,0,1,0,0,0, + 8,0,0,0,67,0,0,0,115,108,0,0,0,116,0,114, + 22,116,1,160,2,100,1,161,1,1,0,116,3,100,2,131, + 1,130,1,100,3,97,0,122,60,122,16,100,4,100,5,108, + 4,109,5,125,0,1,0,87,0,110,38,4,0,116,6,107, + 10,114,82,1,0,1,0,1,0,116,1,160,2,100,1,161, + 1,1,0,116,3,100,2,131,1,130,1,89,0,110,2,88, + 0,87,0,53,0,100,6,97,0,88,0,116,1,160,2,100, + 7,161,1,1,0,124,0,83,0,41,8,78,122,27,122,105, + 112,105,109,112,111,114,116,58,32,122,108,105,98,32,85,78, + 65,86,65,73,76,65,66,76,69,122,41,99,97,110,39,116, + 32,100,101,99,111,109,112,114,101,115,115,32,100,97,116,97, + 59,32,122,108,105,98,32,110,111,116,32,97,118,97,105,108, + 97,98,108,101,84,114,0,0,0,0,41,1,218,10,100,101, + 99,111,109,112,114,101,115,115,70,122,25,122,105,112,105,109, + 112,111,114,116,58,32,122,108,105,98,32,97,118,97,105,108, + 97,98,108,101,41,7,218,15,95,105,109,112,111,114,116,105, + 110,103,95,122,108,105,98,114,69,0,0,0,114,70,0,0, + 0,114,3,0,0,0,90,4,122,108,105,98,114,130,0,0, + 0,218,9,69,120,99,101,112,116,105,111,110,41,1,114,130, + 0,0,0,114,9,0,0,0,114,9,0,0,0,114,10,0, + 0,0,218,20,95,103,101,116,95,100,101,99,111,109,112,114, + 101,115,115,95,102,117,110,99,254,1,0,0,115,24,0,0, + 0,0,2,4,3,10,1,8,2,4,1,4,1,16,1,14, + 1,10,1,18,2,6,2,10,1,114,133,0,0,0,99,2, + 0,0,0,0,0,0,0,17,0,0,0,9,0,0,0,67, + 0,0,0,115,130,1,0,0,124,1,92,8,125,2,125,3, + 125,4,125,5,125,6,125,7,125,8,125,9,124,4,100,1, + 107,0,114,36,116,0,100,2,131,1,130,1,116,1,160,2, + 124,0,100,3,161,2,144,1,143,8,125,10,122,14,124,10, + 160,3,124,6,161,1,1,0,87,0,110,38,4,0,116,4, + 107,10,114,104,1,0,1,0,1,0,116,0,100,4,124,0, + 155,2,157,2,124,0,100,5,141,2,130,1,89,0,110,2, + 88,0,124,10,160,5,100,6,161,1,125,11,116,6,124,11, + 131,1,100,6,107,3,114,136,116,7,100,7,131,1,130,1, + 124,11,100,0,100,8,133,2,25,0,100,9,107,3,114,170, + 116,0,100,10,124,0,155,2,157,2,124,0,100,5,141,2, + 130,1,116,8,124,11,100,11,100,12,133,2,25,0,131,1, + 125,12,116,8,124,11,100,12,100,6,133,2,25,0,131,1, + 125,13,100,6,124,12,23,0,124,13,23,0,125,14,124,6, + 124,14,55,0,125,6,122,14,124,10,160,3,124,6,161,1, + 1,0,87,0,110,40,4,0,116,4,107,10,144,1,114,20, + 1,0,1,0,1,0,116,0,100,4,124,0,155,2,157,2, + 124,0,100,5,141,2,130,1,89,0,110,2,88,0,124,10, + 160,5,124,4,161,1,125,15,116,6,124,15,131,1,124,4, + 107,3,144,1,114,54,116,4,100,13,131,1,130,1,87,0, + 53,0,81,0,82,0,88,0,124,3,100,1,107,2,144,1, + 114,78,124,15,83,0,122,10,116,9,131,0,125,16,87,0, + 110,30,4,0,116,10,107,10,144,1,114,118,1,0,1,0, + 1,0,116,0,100,14,131,1,130,1,89,0,110,2,88,0, + 124,16,124,15,100,15,131,2,83,0,41,16,78,114,0,0, + 0,0,122,18,110,101,103,97,116,105,118,101,32,100,97,116, + 97,32,115,105,122,101,114,84,0,0,0,122,21,99,97,110, + 39,116,32,114,101,97,100,32,90,105,112,32,102,105,108,101, + 58,32,41,1,114,12,0,0,0,114,95,0,0,0,122,27, 69,79,70,32,114,101,97,100,32,119,104,101,114,101,32,110, - 111,116,32,101,120,112,101,99,116,101,100,115,4,0,0,0, - 80,75,1,2,233,8,0,0,0,233,10,0,0,0,233,14, - 0,0,0,233,24,0,0,0,233,28,0,0,0,233,30,0, - 0,0,233,32,0,0,0,233,34,0,0,0,233,42,0,0, - 0,122,25,98,97,100,32,108,111,99,97,108,32,104,101,97, - 100,101,114,32,111,102,102,115,101,116,58,32,105,0,8,0, - 0,218,5,97,115,99,105,105,90,6,108,97,116,105,110,49, - 250,1,47,114,5,0,0,0,122,33,122,105,112,105,109,112, - 111,114,116,58,32,102,111,117,110,100,32,123,125,32,110,97, - 109,101,115,32,105,110,32,123,33,114,125,41,21,218,3,95, - 105,111,218,4,111,112,101,110,114,20,0,0,0,114,3,0, - 0,0,218,4,115,101,101,107,90,4,116,101,108,108,218,4, - 114,101,97,100,114,47,0,0,0,114,2,0,0,0,218,8, - 69,79,70,69,114,114,111,114,114,1,0,0,0,114,54,0, - 0,0,218,18,85,110,105,99,111,100,101,68,101,99,111,100, - 101,69,114,114,111,114,218,9,116,114,97,110,115,108,97,116, - 101,218,11,99,112,52,51,55,95,116,97,98,108,101,114,17, - 0,0,0,114,18,0,0,0,114,19,0,0,0,114,28,0, - 0,0,114,68,0,0,0,114,69,0,0,0,41,23,114,27, - 0,0,0,218,2,102,112,90,15,104,101,97,100,101,114,95, - 112,111,115,105,116,105,111,110,218,6,98,117,102,102,101,114, - 218,11,104,101,97,100,101,114,95,115,105,122,101,90,13,104, - 101,97,100,101,114,95,111,102,102,115,101,116,90,10,97,114, - 99,95,111,102,102,115,101,116,114,31,0,0,0,218,5,99, - 111,117,110,116,218,5,102,108,97,103,115,218,8,99,111,109, - 112,114,101,115,115,218,4,116,105,109,101,218,4,100,97,116, - 101,218,3,99,114,99,218,9,100,97,116,97,95,115,105,122, - 101,218,9,102,105,108,101,95,115,105,122,101,218,9,110,97, - 109,101,95,115,105,122,101,218,10,101,120,116,114,97,95,115, - 105,122,101,90,12,99,111,109,109,101,110,116,95,115,105,122, - 101,218,11,102,105,108,101,95,111,102,102,115,101,116,114,53, - 0,0,0,114,11,0,0,0,218,1,116,114,9,0,0,0, - 114,9,0,0,0,114,10,0,0,0,114,25,0,0,0,93, - 1,0,0,115,154,0,0,0,0,1,2,1,16,1,14,1, - 24,2,8,1,2,1,12,1,8,1,14,1,14,1,24,1, - 12,1,18,1,16,2,18,2,16,1,16,1,10,1,18,1, - 10,1,18,1,8,1,8,1,10,1,18,2,4,2,4,1, - 2,1,14,1,16,1,24,2,10,1,14,1,8,2,18,1, - 4,1,14,1,8,1,16,1,16,1,16,1,16,1,16,1, - 16,1,16,1,16,1,16,1,16,1,16,1,12,1,10,1, - 18,1,8,2,2,1,14,1,16,1,24,1,14,1,18,4, - 2,1,28,1,22,1,16,1,24,2,10,2,10,3,2,1, - 14,1,16,1,22,2,12,1,12,1,20,1,8,1,22,1, - 14,1,114,25,0,0,0,117,190,1,0,0,0,1,2,3, - 4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19, - 20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35, - 36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51, - 52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67, - 68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83, - 84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99, - 100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115, - 116,117,118,119,120,121,122,123,124,125,126,127,195,135,195,188, - 195,169,195,162,195,164,195,160,195,165,195,167,195,170,195,171, - 195,168,195,175,195,174,195,172,195,132,195,133,195,137,195,166, - 195,134,195,180,195,182,195,178,195,187,195,185,195,191,195,150, - 195,156,194,162,194,163,194,165,226,130,167,198,146,195,161,195, - 173,195,179,195,186,195,177,195,145,194,170,194,186,194,191,226, - 140,144,194,172,194,189,194,188,194,161,194,171,194,187,226,150, - 145,226,150,146,226,150,147,226,148,130,226,148,164,226,149,161, - 226,149,162,226,149,150,226,149,149,226,149,163,226,149,145,226, - 149,151,226,149,157,226,149,156,226,149,155,226,148,144,226,148, - 148,226,148,180,226,148,172,226,148,156,226,148,128,226,148,188, - 226,149,158,226,149,159,226,149,154,226,149,148,226,149,169,226, - 149,166,226,149,160,226,149,144,226,149,172,226,149,167,226,149, - 168,226,149,164,226,149,165,226,149,153,226,149,152,226,149,146, - 226,149,147,226,149,171,226,149,170,226,148,152,226,148,140,226, - 150,136,226,150,132,226,150,140,226,150,144,226,150,128,206,177, - 195,159,206,147,207,128,206,163,207,131,194,181,207,132,206,166, - 206,152,206,169,206,180,226,136,158,207,134,206,181,226,136,169, - 226,137,161,194,177,226,137,165,226,137,164,226,140,160,226,140, - 161,195,183,226,137,136,194,176,226,136,153,194,183,226,136,154, - 226,129,191,194,178,226,150,160,194,160,99,0,0,0,0,0, - 0,0,0,1,0,0,0,8,0,0,0,67,0,0,0,115, - 108,0,0,0,116,0,114,22,116,1,160,2,100,1,161,1, - 1,0,116,3,100,2,131,1,130,1,100,3,97,0,122,60, - 122,16,100,4,100,5,108,4,109,5,125,0,1,0,87,0, - 110,38,4,0,116,6,107,10,114,82,1,0,1,0,1,0, - 116,1,160,2,100,1,161,1,1,0,116,3,100,2,131,1, - 130,1,89,0,110,2,88,0,87,0,53,0,100,6,97,0, - 88,0,116,1,160,2,100,7,161,1,1,0,124,0,83,0, - 41,8,78,122,27,122,105,112,105,109,112,111,114,116,58,32, - 122,108,105,98,32,85,78,65,86,65,73,76,65,66,76,69, - 122,41,99,97,110,39,116,32,100,101,99,111,109,112,114,101, - 115,115,32,100,97,116,97,59,32,122,108,105,98,32,110,111, - 116,32,97,118,97,105,108,97,98,108,101,84,114,0,0,0, - 0,41,1,218,10,100,101,99,111,109,112,114,101,115,115,70, - 122,25,122,105,112,105,109,112,111,114,116,58,32,122,108,105, - 98,32,97,118,97,105,108,97,98,108,101,41,7,218,15,95, - 105,109,112,111,114,116,105,110,103,95,122,108,105,98,114,68, - 0,0,0,114,69,0,0,0,114,3,0,0,0,90,4,122, - 108,105,98,114,124,0,0,0,218,9,69,120,99,101,112,116, - 105,111,110,41,1,114,124,0,0,0,114,9,0,0,0,114, - 9,0,0,0,114,10,0,0,0,218,20,95,103,101,116,95, - 100,101,99,111,109,112,114,101,115,115,95,102,117,110,99,228, - 1,0,0,115,24,0,0,0,0,2,4,3,10,1,8,2, - 4,1,4,1,16,1,14,1,10,1,18,2,6,2,10,1, - 114,127,0,0,0,99,2,0,0,0,0,0,0,0,17,0, - 0,0,9,0,0,0,67,0,0,0,115,130,1,0,0,124, - 1,92,8,125,2,125,3,125,4,125,5,125,6,125,7,125, - 8,125,9,124,4,100,1,107,0,114,36,116,0,100,2,131, - 1,130,1,116,1,160,2,124,0,100,3,161,2,144,1,143, - 8,125,10,122,14,124,10,160,3,124,6,161,1,1,0,87, - 0,110,38,4,0,116,4,107,10,114,104,1,0,1,0,1, - 0,116,0,100,4,124,0,155,2,157,2,124,0,100,5,141, - 2,130,1,89,0,110,2,88,0,124,10,160,5,100,6,161, - 1,125,11,116,6,124,11,131,1,100,6,107,3,114,136,116, - 7,100,7,131,1,130,1,124,11,100,0,100,8,133,2,25, - 0,100,9,107,3,114,170,116,0,100,10,124,0,155,2,157, - 2,124,0,100,5,141,2,130,1,116,8,124,11,100,11,100, - 12,133,2,25,0,131,1,125,12,116,8,124,11,100,12,100, - 6,133,2,25,0,131,1,125,13,100,6,124,12,23,0,124, - 13,23,0,125,14,124,6,124,14,55,0,125,6,122,14,124, - 10,160,3,124,6,161,1,1,0,87,0,110,40,4,0,116, - 4,107,10,144,1,114,20,1,0,1,0,1,0,116,0,100, - 4,124,0,155,2,157,2,124,0,100,5,141,2,130,1,89, - 0,110,2,88,0,124,10,160,5,124,4,161,1,125,15,116, - 6,124,15,131,1,124,4,107,3,144,1,114,54,116,4,100, - 13,131,1,130,1,87,0,53,0,81,0,82,0,88,0,124, - 3,100,1,107,2,144,1,114,78,124,15,83,0,122,10,116, - 9,131,0,125,16,87,0,110,30,4,0,116,10,107,10,144, - 1,114,118,1,0,1,0,1,0,116,0,100,14,131,1,130, - 1,89,0,110,2,88,0,124,16,124,15,100,15,131,2,83, - 0,41,16,78,114,0,0,0,0,122,18,110,101,103,97,116, - 105,118,101,32,100,97,116,97,32,115,105,122,101,114,83,0, - 0,0,122,21,99,97,110,39,116,32,114,101,97,100,32,90, - 105,112,32,102,105,108,101,58,32,41,1,114,11,0,0,0, - 114,95,0,0,0,122,27,69,79,70,32,114,101,97,100,32, - 119,104,101,114,101,32,110,111,116,32,101,120,112,101,99,116, - 101,100,114,85,0,0,0,115,4,0,0,0,80,75,3,4, - 122,23,98,97,100,32,108,111,99,97,108,32,102,105,108,101, - 32,104,101,97,100,101,114,58,32,233,26,0,0,0,114,94, - 0,0,0,122,26,122,105,112,105,109,112,111,114,116,58,32, - 99,97,110,39,116,32,114,101,97,100,32,100,97,116,97,122, - 41,99,97,110,39,116,32,100,101,99,111,109,112,114,101,115, - 115,32,100,97,116,97,59,32,122,108,105,98,32,110,111,116, - 32,97,118,97,105,108,97,98,108,101,105,241,255,255,255,41, - 11,114,3,0,0,0,114,101,0,0,0,114,102,0,0,0, - 114,103,0,0,0,114,20,0,0,0,114,104,0,0,0,114, - 47,0,0,0,114,105,0,0,0,114,1,0,0,0,114,127, - 0,0,0,114,126,0,0,0,41,17,114,27,0,0,0,114, - 50,0,0,0,90,8,100,97,116,97,112,97,116,104,114,114, - 0,0,0,114,118,0,0,0,114,119,0,0,0,114,122,0, - 0,0,114,115,0,0,0,114,116,0,0,0,114,117,0,0, - 0,114,109,0,0,0,114,110,0,0,0,114,120,0,0,0, - 114,121,0,0,0,114,111,0,0,0,90,8,114,97,119,95, - 100,97,116,97,114,124,0,0,0,114,9,0,0,0,114,9, - 0,0,0,114,10,0,0,0,114,48,0,0,0,249,1,0, - 0,115,62,0,0,0,0,1,20,1,8,1,8,2,16,2, - 2,1,14,1,14,1,24,1,10,1,12,1,8,2,16,2, - 18,2,16,1,16,1,12,1,8,1,2,1,14,1,16,1, - 24,1,10,1,14,1,18,2,10,2,4,3,2,1,10,1, - 16,1,14,1,114,48,0,0,0,99,2,0,0,0,0,0, - 0,0,2,0,0,0,3,0,0,0,67,0,0,0,115,16, - 0,0,0,116,0,124,0,124,1,24,0,131,1,100,1,107, - 1,83,0,41,2,78,114,5,0,0,0,41,1,218,3,97, - 98,115,41,2,90,2,116,49,90,2,116,50,114,9,0,0, - 0,114,9,0,0,0,114,10,0,0,0,218,9,95,101,113, - 95,109,116,105,109,101,39,2,0,0,115,2,0,0,0,0, - 2,114,130,0,0,0,99,3,0,0,0,0,0,0,0,5, - 0,0,0,5,0,0,0,67,0,0,0,115,206,0,0,0, - 116,0,124,1,131,1,100,1,107,0,114,20,116,1,100,2, - 131,1,130,1,124,1,100,0,100,3,133,2,25,0,116,2, - 106,3,107,3,114,54,116,4,160,5,100,4,124,0,161,2, - 1,0,100,0,83,0,116,6,124,1,100,3,100,5,133,2, - 25,0,131,1,125,3,124,3,100,6,107,3,114,112,116,7, - 106,8,100,7,107,3,114,110,124,3,100,8,107,3,115,106, - 116,7,106,8,100,9,107,2,114,110,100,0,83,0,110,46, - 124,2,100,6,107,3,114,158,116,9,116,6,124,1,100,5, - 100,10,133,2,25,0,131,1,124,2,131,2,115,158,116,4, - 160,5,100,11,124,0,161,2,1,0,100,0,83,0,116,10, - 160,11,124,1,100,1,100,0,133,2,25,0,161,1,125,4, - 116,12,124,4,116,13,131,2,115,202,116,14,100,12,124,0, - 155,2,100,13,157,3,131,1,130,1,124,4,83,0,41,14, - 78,114,87,0,0,0,122,12,98,97,100,32,112,121,99,32, - 100,97,116,97,114,85,0,0,0,122,18,123,33,114,125,32, - 104,97,115,32,98,97,100,32,109,97,103,105,99,114,90,0, - 0,0,114,0,0,0,0,90,5,110,101,118,101,114,114,5, - 0,0,0,90,6,97,108,119,97,121,115,114,86,0,0,0, - 122,18,123,33,114,125,32,104,97,115,32,98,97,100,32,109, - 116,105,109,101,122,16,99,111,109,112,105,108,101,100,32,109, - 111,100,117,108,101,32,122,21,32,105,115,32,110,111,116,32, - 97,32,99,111,100,101,32,111,98,106,101,99,116,41,15,114, - 47,0,0,0,114,3,0,0,0,114,19,0,0,0,90,12, - 77,65,71,73,67,95,78,85,77,66,69,82,114,68,0,0, - 0,114,69,0,0,0,114,2,0,0,0,218,4,95,105,109, - 112,90,21,99,104,101,99,107,95,104,97,115,104,95,98,97, - 115,101,100,95,112,121,99,115,114,130,0,0,0,218,7,109, - 97,114,115,104,97,108,90,5,108,111,97,100,115,114,13,0, - 0,0,218,10,95,99,111,100,101,95,116,121,112,101,218,9, - 84,121,112,101,69,114,114,111,114,41,5,114,49,0,0,0, - 218,4,100,97,116,97,218,5,109,116,105,109,101,114,113,0, - 0,0,114,42,0,0,0,114,9,0,0,0,114,9,0,0, - 0,114,10,0,0,0,218,15,95,117,110,109,97,114,115,104, - 97,108,95,99,111,100,101,47,2,0,0,115,40,0,0,0, - 0,1,12,1,8,2,18,1,12,1,4,2,16,1,8,5, - 10,1,6,255,2,1,8,255,2,2,6,1,30,1,12,1, - 4,4,18,1,10,1,16,1,114,137,0,0,0,99,1,0, - 0,0,0,0,0,0,1,0,0,0,4,0,0,0,67,0, - 0,0,115,28,0,0,0,124,0,160,0,100,1,100,2,161, - 2,125,0,124,0,160,0,100,3,100,2,161,2,125,0,124, - 0,83,0,41,4,78,115,2,0,0,0,13,10,243,1,0, - 0,0,10,243,1,0,0,0,13,41,1,114,17,0,0,0, - 41,1,218,6,115,111,117,114,99,101,114,9,0,0,0,114, - 9,0,0,0,114,10,0,0,0,218,23,95,110,111,114,109, - 97,108,105,122,101,95,108,105,110,101,95,101,110,100,105,110, - 103,115,80,2,0,0,115,6,0,0,0,0,1,12,1,12, - 1,114,141,0,0,0,99,2,0,0,0,0,0,0,0,2, - 0,0,0,6,0,0,0,67,0,0,0,115,24,0,0,0, - 116,0,124,1,131,1,125,1,116,1,124,1,124,0,100,1, - 100,2,100,3,141,4,83,0,41,4,78,114,66,0,0,0, - 84,41,1,90,12,100,111,110,116,95,105,110,104,101,114,105, - 116,41,2,114,141,0,0,0,218,7,99,111,109,112,105,108, - 101,41,2,114,49,0,0,0,114,140,0,0,0,114,9,0, - 0,0,114,9,0,0,0,114,10,0,0,0,218,15,95,99, - 111,109,112,105,108,101,95,115,111,117,114,99,101,87,2,0, - 0,115,4,0,0,0,0,1,8,1,114,143,0,0,0,99, - 2,0,0,0,0,0,0,0,2,0,0,0,11,0,0,0, - 67,0,0,0,115,68,0,0,0,116,0,160,1,124,0,100, - 1,63,0,100,2,23,0,124,0,100,3,63,0,100,4,64, - 0,124,0,100,5,64,0,124,1,100,6,63,0,124,1,100, - 3,63,0,100,7,64,0,124,1,100,5,64,0,100,8,20, - 0,100,9,100,9,100,9,102,9,161,1,83,0,41,10,78, - 233,9,0,0,0,105,188,7,0,0,233,5,0,0,0,233, - 15,0,0,0,233,31,0,0,0,233,11,0,0,0,233,63, - 0,0,0,114,78,0,0,0,114,12,0,0,0,41,2,114, - 115,0,0,0,90,6,109,107,116,105,109,101,41,2,218,1, - 100,114,123,0,0,0,114,9,0,0,0,114,9,0,0,0, - 114,10,0,0,0,218,14,95,112,97,114,115,101,95,100,111, - 115,116,105,109,101,93,2,0,0,115,24,0,0,0,0,1, - 4,1,10,1,10,1,6,1,6,1,10,1,10,1,2,0, - 2,0,2,250,2,255,114,151,0,0,0,99,2,0,0,0, - 0,0,0,0,5,0,0,0,10,0,0,0,67,0,0,0, - 115,104,0,0,0,122,70,124,1,100,1,100,0,133,2,25, - 0,100,2,107,6,115,22,116,0,130,1,124,1,100,0,100, - 1,133,2,25,0,125,1,124,0,106,1,124,1,25,0,125, - 2,124,2,100,3,25,0,125,3,124,2,100,4,25,0,125, - 4,116,2,124,4,124,3,131,2,87,0,83,0,4,0,116, - 3,116,4,116,5,102,3,107,10,114,98,1,0,1,0,1, - 0,89,0,100,5,83,0,88,0,100,0,83,0,41,6,78, - 114,12,0,0,0,41,2,218,1,99,218,1,111,114,145,0, - 0,0,233,6,0,0,0,114,0,0,0,0,41,6,218,14, - 65,115,115,101,114,116,105,111,110,69,114,114,111,114,114,26, - 0,0,0,114,151,0,0,0,114,24,0,0,0,218,10,73, - 110,100,101,120,69,114,114,111,114,114,134,0,0,0,41,5, - 114,30,0,0,0,114,11,0,0,0,114,50,0,0,0,114, - 115,0,0,0,114,116,0,0,0,114,9,0,0,0,114,9, - 0,0,0,114,10,0,0,0,218,20,95,103,101,116,95,109, - 116,105,109,101,95,111,102,95,115,111,117,114,99,101,106,2, - 0,0,115,18,0,0,0,0,1,2,2,20,1,12,1,10, - 3,8,1,8,1,12,1,20,1,114,157,0,0,0,99,2, - 0,0,0,0,0,0,0,12,0,0,0,9,0,0,0,67, - 0,0,0,115,204,0,0,0,116,0,124,0,124,1,131,2, - 125,2,116,1,68,0,93,166,92,3,125,3,125,4,125,5, - 124,2,124,3,23,0,125,6,116,2,106,3,100,1,124,0, - 106,4,116,5,124,6,100,2,100,3,141,5,1,0,122,14, - 124,0,106,6,124,6,25,0,125,7,87,0,110,20,4,0, - 116,7,107,10,114,88,1,0,1,0,1,0,89,0,113,14, - 88,0,124,7,100,4,25,0,125,8,116,8,124,0,106,4, - 124,7,131,2,125,9,124,4,114,138,116,9,124,0,124,6, - 131,2,125,10,116,10,124,8,124,9,124,10,131,3,125,11, - 110,10,116,11,124,8,124,9,131,2,125,11,124,11,100,0, - 107,8,114,158,113,14,124,7,100,4,25,0,125,8,124,11, - 124,5,124,8,102,3,2,0,1,0,83,0,113,14,116,12, - 100,5,124,1,155,2,157,2,124,1,100,6,141,2,130,1, - 100,0,83,0,41,7,78,122,13,116,114,121,105,110,103,32, - 123,125,123,125,123,125,114,78,0,0,0,41,1,90,9,118, - 101,114,98,111,115,105,116,121,114,0,0,0,0,122,18,99, - 97,110,39,116,32,102,105,110,100,32,109,111,100,117,108,101, - 32,41,1,114,53,0,0,0,41,13,114,34,0,0,0,114, - 80,0,0,0,114,68,0,0,0,114,69,0,0,0,114,27, - 0,0,0,114,18,0,0,0,114,26,0,0,0,114,24,0, - 0,0,114,48,0,0,0,114,157,0,0,0,114,137,0,0, - 0,114,143,0,0,0,114,3,0,0,0,41,12,114,30,0, - 0,0,114,36,0,0,0,114,11,0,0,0,114,81,0,0, - 0,114,82,0,0,0,114,43,0,0,0,114,55,0,0,0, - 114,50,0,0,0,114,38,0,0,0,114,135,0,0,0,114, - 136,0,0,0,114,42,0,0,0,114,9,0,0,0,114,9, - 0,0,0,114,10,0,0,0,114,41,0,0,0,122,2,0, - 0,115,38,0,0,0,0,1,10,1,14,1,8,1,22,1, - 2,1,14,1,14,1,6,2,8,1,12,1,4,1,10,1, - 14,2,10,1,8,3,2,1,8,1,16,2,114,41,0,0, - 0,99,0,0,0,0,0,0,0,0,0,0,0,0,2,0, - 0,0,64,0,0,0,115,60,0,0,0,101,0,90,1,100, - 0,90,2,100,1,90,3,100,2,90,4,100,3,100,4,132, - 0,90,5,100,5,100,6,132,0,90,6,100,7,100,8,132, - 0,90,7,100,9,100,10,132,0,90,8,100,11,100,12,132, - 0,90,9,100,13,83,0,41,14,114,72,0,0,0,122,165, - 80,114,105,118,97,116,101,32,99,108,97,115,115,32,117,115, - 101,100,32,116,111,32,115,117,112,112,111,114,116,32,90,105, - 112,73,109,112,111,114,116,46,103,101,116,95,114,101,115,111, - 117,114,99,101,95,114,101,97,100,101,114,40,41,46,10,10, - 32,32,32,32,84,104,105,115,32,99,108,97,115,115,32,105, - 115,32,97,108,108,111,119,101,100,32,116,111,32,114,101,102, - 101,114,101,110,99,101,32,97,108,108,32,116,104,101,32,105, - 110,110,97,114,100,115,32,97,110,100,32,112,114,105,118,97, - 116,101,32,112,97,114,116,115,32,111,102,10,32,32,32,32, - 116,104,101,32,122,105,112,105,109,112,111,114,116,101,114,46, - 10,32,32,32,32,70,99,3,0,0,0,0,0,0,0,3, - 0,0,0,2,0,0,0,67,0,0,0,115,16,0,0,0, - 124,1,124,0,95,0,124,2,124,0,95,1,100,0,83,0, - 41,1,78,41,2,114,4,0,0,0,114,36,0,0,0,41, - 3,114,30,0,0,0,114,4,0,0,0,114,36,0,0,0, - 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,114, - 32,0,0,0,157,2,0,0,115,4,0,0,0,0,1,6, - 1,122,33,95,90,105,112,73,109,112,111,114,116,82,101,115, - 111,117,114,99,101,82,101,97,100,101,114,46,95,95,105,110, - 105,116,95,95,99,2,0,0,0,0,0,0,0,5,0,0, - 0,8,0,0,0,67,0,0,0,115,92,0,0,0,124,0, - 106,0,160,1,100,1,100,2,161,2,125,2,124,2,155,0, - 100,2,124,1,155,0,157,3,125,3,100,3,100,4,108,2, - 109,3,125,4,1,0,122,18,124,4,124,0,106,4,160,5, - 124,3,161,1,131,1,87,0,83,0,4,0,116,6,107,10, - 114,86,1,0,1,0,1,0,116,7,124,3,131,1,130,1, - 89,0,110,2,88,0,100,0,83,0,41,5,78,114,77,0, - 0,0,114,100,0,0,0,114,0,0,0,0,41,1,218,7, - 66,121,116,101,115,73,79,41,8,114,36,0,0,0,114,17, - 0,0,0,90,2,105,111,114,158,0,0,0,114,4,0,0, - 0,114,51,0,0,0,114,20,0,0,0,218,17,70,105,108, - 101,78,111,116,70,111,117,110,100,69,114,114,111,114,41,5, - 114,30,0,0,0,218,8,114,101,115,111,117,114,99,101,218, - 16,102,117,108,108,110,97,109,101,95,97,115,95,112,97,116, - 104,114,11,0,0,0,114,158,0,0,0,114,9,0,0,0, - 114,9,0,0,0,114,10,0,0,0,218,13,111,112,101,110, - 95,114,101,115,111,117,114,99,101,161,2,0,0,115,14,0, - 0,0,0,1,14,1,14,1,12,1,2,1,18,1,14,1, - 122,38,95,90,105,112,73,109,112,111,114,116,82,101,115,111, - 117,114,99,101,82,101,97,100,101,114,46,111,112,101,110,95, - 114,101,115,111,117,114,99,101,99,2,0,0,0,0,0,0, - 0,2,0,0,0,1,0,0,0,67,0,0,0,115,8,0, - 0,0,116,0,130,1,100,0,83,0,41,1,78,41,1,114, - 159,0,0,0,41,2,114,30,0,0,0,114,160,0,0,0, - 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,218, - 13,114,101,115,111,117,114,99,101,95,112,97,116,104,170,2, - 0,0,115,2,0,0,0,0,4,122,38,95,90,105,112,73, + 111,116,32,101,120,112,101,99,116,101,100,114,85,0,0,0, + 115,4,0,0,0,80,75,3,4,122,23,98,97,100,32,108, + 111,99,97,108,32,102,105,108,101,32,104,101,97,100,101,114, + 58,32,233,26,0,0,0,114,94,0,0,0,122,26,122,105, + 112,105,109,112,111,114,116,58,32,99,97,110,39,116,32,114, + 101,97,100,32,100,97,116,97,122,41,99,97,110,39,116,32, + 100,101,99,111,109,112,114,101,115,115,32,100,97,116,97,59, + 32,122,108,105,98,32,110,111,116,32,97,118,97,105,108,97, + 98,108,101,105,241,255,255,255,41,11,114,3,0,0,0,114, + 101,0,0,0,114,102,0,0,0,114,103,0,0,0,114,21, + 0,0,0,114,105,0,0,0,114,48,0,0,0,114,110,0, + 0,0,114,1,0,0,0,114,133,0,0,0,114,132,0,0, + 0,41,17,114,28,0,0,0,114,51,0,0,0,90,8,100, + 97,116,97,112,97,116,104,114,121,0,0,0,114,125,0,0, + 0,114,116,0,0,0,114,128,0,0,0,114,122,0,0,0, + 114,123,0,0,0,114,124,0,0,0,114,114,0,0,0,114, + 115,0,0,0,114,126,0,0,0,114,127,0,0,0,114,118, + 0,0,0,90,8,114,97,119,95,100,97,116,97,114,130,0, + 0,0,114,9,0,0,0,114,9,0,0,0,114,10,0,0, + 0,114,49,0,0,0,19,2,0,0,115,62,0,0,0,0, + 1,20,1,8,1,8,2,16,2,2,1,14,1,14,1,24, + 1,10,1,12,1,8,2,16,2,18,2,16,1,16,1,12, + 1,8,1,2,1,14,1,16,1,24,1,10,1,14,1,18, + 2,10,2,4,3,2,1,10,1,16,1,14,1,114,49,0, + 0,0,99,2,0,0,0,0,0,0,0,2,0,0,0,3, + 0,0,0,67,0,0,0,115,16,0,0,0,116,0,124,0, + 124,1,24,0,131,1,100,1,107,1,83,0,41,2,78,114, + 5,0,0,0,41,1,218,3,97,98,115,41,2,90,2,116, + 49,90,2,116,50,114,9,0,0,0,114,9,0,0,0,114, + 10,0,0,0,218,9,95,101,113,95,109,116,105,109,101,65, + 2,0,0,115,2,0,0,0,0,2,114,136,0,0,0,99, + 3,0,0,0,0,0,0,0,5,0,0,0,5,0,0,0, + 67,0,0,0,115,206,0,0,0,116,0,124,1,131,1,100, + 1,107,0,114,20,116,1,100,2,131,1,130,1,124,1,100, + 0,100,3,133,2,25,0,116,2,106,3,107,3,114,54,116, + 4,160,5,100,4,124,0,161,2,1,0,100,0,83,0,116, + 6,124,1,100,3,100,5,133,2,25,0,131,1,125,3,124, + 3,100,6,107,3,114,112,116,7,106,8,100,7,107,3,114, + 110,124,3,100,8,107,3,115,106,116,7,106,8,100,9,107, + 2,114,110,100,0,83,0,110,46,124,2,100,6,107,3,114, + 158,116,9,116,6,124,1,100,5,100,10,133,2,25,0,131, + 1,124,2,131,2,115,158,116,4,160,5,100,11,124,0,161, + 2,1,0,100,0,83,0,116,10,160,11,124,1,100,1,100, + 0,133,2,25,0,161,1,125,4,116,12,124,4,116,13,131, + 2,115,202,116,14,100,12,124,0,155,2,100,13,157,3,131, + 1,130,1,124,4,83,0,41,14,78,114,87,0,0,0,122, + 12,98,97,100,32,112,121,99,32,100,97,116,97,114,85,0, + 0,0,122,18,123,33,114,125,32,104,97,115,32,98,97,100, + 32,109,97,103,105,99,114,90,0,0,0,114,0,0,0,0, + 90,5,110,101,118,101,114,114,5,0,0,0,90,6,97,108, + 119,97,121,115,114,86,0,0,0,122,18,123,33,114,125,32, + 104,97,115,32,98,97,100,32,109,116,105,109,101,122,16,99, + 111,109,112,105,108,101,100,32,109,111,100,117,108,101,32,122, + 21,32,105,115,32,110,111,116,32,97,32,99,111,100,101,32, + 111,98,106,101,99,116,41,15,114,48,0,0,0,114,3,0, + 0,0,114,20,0,0,0,90,12,77,65,71,73,67,95,78, + 85,77,66,69,82,114,69,0,0,0,114,70,0,0,0,114, + 2,0,0,0,218,4,95,105,109,112,90,21,99,104,101,99, + 107,95,104,97,115,104,95,98,97,115,101,100,95,112,121,99, + 115,114,136,0,0,0,218,7,109,97,114,115,104,97,108,90, + 5,108,111,97,100,115,114,14,0,0,0,218,10,95,99,111, + 100,101,95,116,121,112,101,218,9,84,121,112,101,69,114,114, + 111,114,41,5,114,50,0,0,0,114,117,0,0,0,218,5, + 109,116,105,109,101,114,120,0,0,0,114,43,0,0,0,114, + 9,0,0,0,114,9,0,0,0,114,10,0,0,0,218,15, + 95,117,110,109,97,114,115,104,97,108,95,99,111,100,101,73, + 2,0,0,115,40,0,0,0,0,1,12,1,8,2,18,1, + 12,1,4,2,16,1,8,5,10,1,6,255,2,1,8,255, + 2,2,6,1,30,1,12,1,4,4,18,1,10,1,16,1, + 114,142,0,0,0,99,1,0,0,0,0,0,0,0,1,0, + 0,0,4,0,0,0,67,0,0,0,115,28,0,0,0,124, + 0,160,0,100,1,100,2,161,2,125,0,124,0,160,0,100, + 3,100,2,161,2,125,0,124,0,83,0,41,4,78,115,2, + 0,0,0,13,10,243,1,0,0,0,10,243,1,0,0,0, + 13,41,1,114,18,0,0,0,41,1,218,6,115,111,117,114, + 99,101,114,9,0,0,0,114,9,0,0,0,114,10,0,0, + 0,218,23,95,110,111,114,109,97,108,105,122,101,95,108,105, + 110,101,95,101,110,100,105,110,103,115,106,2,0,0,115,6, + 0,0,0,0,1,12,1,12,1,114,146,0,0,0,99,2, + 0,0,0,0,0,0,0,2,0,0,0,6,0,0,0,67, + 0,0,0,115,24,0,0,0,116,0,124,1,131,1,125,1, + 116,1,124,1,124,0,100,1,100,2,100,3,141,4,83,0, + 41,4,78,114,67,0,0,0,84,41,1,90,12,100,111,110, + 116,95,105,110,104,101,114,105,116,41,2,114,146,0,0,0, + 218,7,99,111,109,112,105,108,101,41,2,114,50,0,0,0, + 114,145,0,0,0,114,9,0,0,0,114,9,0,0,0,114, + 10,0,0,0,218,15,95,99,111,109,112,105,108,101,95,115, + 111,117,114,99,101,113,2,0,0,115,4,0,0,0,0,1, + 8,1,114,148,0,0,0,99,2,0,0,0,0,0,0,0, + 2,0,0,0,11,0,0,0,67,0,0,0,115,68,0,0, + 0,116,0,160,1,124,0,100,1,63,0,100,2,23,0,124, + 0,100,3,63,0,100,4,64,0,124,0,100,5,64,0,124, + 1,100,6,63,0,124,1,100,3,63,0,100,7,64,0,124, + 1,100,5,64,0,100,8,20,0,100,9,100,9,100,9,102, + 9,161,1,83,0,41,10,78,233,9,0,0,0,105,188,7, + 0,0,233,5,0,0,0,233,15,0,0,0,233,31,0,0, + 0,233,11,0,0,0,233,63,0,0,0,114,79,0,0,0, + 114,13,0,0,0,41,2,114,122,0,0,0,90,6,109,107, + 116,105,109,101,41,2,218,1,100,114,129,0,0,0,114,9, + 0,0,0,114,9,0,0,0,114,10,0,0,0,218,14,95, + 112,97,114,115,101,95,100,111,115,116,105,109,101,119,2,0, + 0,115,24,0,0,0,0,1,4,1,10,1,10,1,6,1, + 6,1,10,1,10,1,2,0,2,0,2,250,2,255,114,156, + 0,0,0,99,2,0,0,0,0,0,0,0,5,0,0,0, + 10,0,0,0,67,0,0,0,115,104,0,0,0,122,70,124, + 1,100,1,100,0,133,2,25,0,100,2,107,6,115,22,116, + 0,130,1,124,1,100,0,100,1,133,2,25,0,125,1,124, + 0,106,1,124,1,25,0,125,2,124,2,100,3,25,0,125, + 3,124,2,100,4,25,0,125,4,116,2,124,4,124,3,131, + 2,87,0,83,0,4,0,116,3,116,4,116,5,102,3,107, + 10,114,98,1,0,1,0,1,0,89,0,100,5,83,0,88, + 0,100,0,83,0,41,6,78,114,13,0,0,0,41,2,218, + 1,99,218,1,111,114,150,0,0,0,233,6,0,0,0,114, + 0,0,0,0,41,6,218,14,65,115,115,101,114,116,105,111, + 110,69,114,114,111,114,114,27,0,0,0,114,156,0,0,0, + 114,25,0,0,0,218,10,73,110,100,101,120,69,114,114,111, + 114,114,140,0,0,0,41,5,114,31,0,0,0,114,12,0, + 0,0,114,51,0,0,0,114,122,0,0,0,114,123,0,0, + 0,114,9,0,0,0,114,9,0,0,0,114,10,0,0,0, + 218,20,95,103,101,116,95,109,116,105,109,101,95,111,102,95, + 115,111,117,114,99,101,132,2,0,0,115,18,0,0,0,0, + 1,2,2,20,1,12,1,10,3,8,1,8,1,12,1,20, + 1,114,162,0,0,0,99,2,0,0,0,0,0,0,0,12, + 0,0,0,9,0,0,0,67,0,0,0,115,204,0,0,0, + 116,0,124,0,124,1,131,2,125,2,116,1,68,0,93,166, + 92,3,125,3,125,4,125,5,124,2,124,3,23,0,125,6, + 116,2,106,3,100,1,124,0,106,4,116,5,124,6,100,2, + 100,3,141,5,1,0,122,14,124,0,106,6,124,6,25,0, + 125,7,87,0,110,20,4,0,116,7,107,10,114,88,1,0, + 1,0,1,0,89,0,113,14,88,0,124,7,100,4,25,0, + 125,8,116,8,124,0,106,4,124,7,131,2,125,9,124,4, + 114,138,116,9,124,0,124,6,131,2,125,10,116,10,124,8, + 124,9,124,10,131,3,125,11,110,10,116,11,124,8,124,9, + 131,2,125,11,124,11,100,0,107,8,114,158,113,14,124,7, + 100,4,25,0,125,8,124,11,124,5,124,8,102,3,2,0, + 1,0,83,0,113,14,116,12,100,5,124,1,155,2,157,2, + 124,1,100,6,141,2,130,1,100,0,83,0,41,7,78,122, + 13,116,114,121,105,110,103,32,123,125,123,125,123,125,114,79, + 0,0,0,41,1,90,9,118,101,114,98,111,115,105,116,121, + 114,0,0,0,0,122,18,99,97,110,39,116,32,102,105,110, + 100,32,109,111,100,117,108,101,32,41,1,114,54,0,0,0, + 41,13,114,35,0,0,0,114,81,0,0,0,114,69,0,0, + 0,114,70,0,0,0,114,28,0,0,0,114,19,0,0,0, + 114,27,0,0,0,114,25,0,0,0,114,49,0,0,0,114, + 162,0,0,0,114,142,0,0,0,114,148,0,0,0,114,3, + 0,0,0,41,12,114,31,0,0,0,114,37,0,0,0,114, + 12,0,0,0,114,82,0,0,0,114,83,0,0,0,114,44, + 0,0,0,114,56,0,0,0,114,51,0,0,0,114,39,0, + 0,0,114,117,0,0,0,114,141,0,0,0,114,43,0,0, + 0,114,9,0,0,0,114,9,0,0,0,114,10,0,0,0, + 114,42,0,0,0,148,2,0,0,115,38,0,0,0,0,1, + 10,1,14,1,8,1,22,1,2,1,14,1,14,1,6,2, + 8,1,12,1,4,1,10,1,14,2,10,1,8,3,2,1, + 8,1,16,2,114,42,0,0,0,99,0,0,0,0,0,0, + 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,60, + 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, + 2,90,4,100,3,100,4,132,0,90,5,100,5,100,6,132, + 0,90,6,100,7,100,8,132,0,90,7,100,9,100,10,132, + 0,90,8,100,11,100,12,132,0,90,9,100,13,83,0,41, + 14,114,73,0,0,0,122,165,80,114,105,118,97,116,101,32, + 99,108,97,115,115,32,117,115,101,100,32,116,111,32,115,117, + 112,112,111,114,116,32,90,105,112,73,109,112,111,114,116,46, + 103,101,116,95,114,101,115,111,117,114,99,101,95,114,101,97, + 100,101,114,40,41,46,10,10,32,32,32,32,84,104,105,115, + 32,99,108,97,115,115,32,105,115,32,97,108,108,111,119,101, + 100,32,116,111,32,114,101,102,101,114,101,110,99,101,32,97, + 108,108,32,116,104,101,32,105,110,110,97,114,100,115,32,97, + 110,100,32,112,114,105,118,97,116,101,32,112,97,114,116,115, + 32,111,102,10,32,32,32,32,116,104,101,32,122,105,112,105, + 109,112,111,114,116,101,114,46,10,32,32,32,32,70,99,3, + 0,0,0,0,0,0,0,3,0,0,0,2,0,0,0,67, + 0,0,0,115,16,0,0,0,124,1,124,0,95,0,124,2, + 124,0,95,1,100,0,83,0,41,1,78,41,2,114,4,0, + 0,0,114,37,0,0,0,41,3,114,31,0,0,0,114,4, + 0,0,0,114,37,0,0,0,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,114,33,0,0,0,183,2,0,0, + 115,4,0,0,0,0,1,6,1,122,33,95,90,105,112,73, 109,112,111,114,116,82,101,115,111,117,114,99,101,82,101,97, - 100,101,114,46,114,101,115,111,117,114,99,101,95,112,97,116, - 104,99,2,0,0,0,0,0,0,0,4,0,0,0,8,0, - 0,0,67,0,0,0,115,72,0,0,0,124,0,106,0,160, - 1,100,1,100,2,161,2,125,2,124,2,155,0,100,2,124, - 1,155,0,157,3,125,3,122,16,124,0,106,2,160,3,124, - 3,161,1,1,0,87,0,110,22,4,0,116,4,107,10,114, - 66,1,0,1,0,1,0,89,0,100,3,83,0,88,0,100, - 4,83,0,41,5,78,114,77,0,0,0,114,100,0,0,0, - 70,84,41,5,114,36,0,0,0,114,17,0,0,0,114,4, - 0,0,0,114,51,0,0,0,114,20,0,0,0,41,4,114, - 30,0,0,0,114,53,0,0,0,114,161,0,0,0,114,11, + 100,101,114,46,95,95,105,110,105,116,95,95,99,2,0,0, + 0,0,0,0,0,5,0,0,0,8,0,0,0,67,0,0, + 0,115,92,0,0,0,124,0,106,0,160,1,100,1,100,2, + 161,2,125,2,124,2,155,0,100,2,124,1,155,0,157,3, + 125,3,100,3,100,4,108,2,109,3,125,4,1,0,122,18, + 124,4,124,0,106,4,160,5,124,3,161,1,131,1,87,0, + 83,0,4,0,116,6,107,10,114,86,1,0,1,0,1,0, + 116,7,124,3,131,1,130,1,89,0,110,2,88,0,100,0, + 83,0,41,5,78,114,78,0,0,0,114,100,0,0,0,114, + 0,0,0,0,41,1,218,7,66,121,116,101,115,73,79,41, + 8,114,37,0,0,0,114,18,0,0,0,90,2,105,111,114, + 163,0,0,0,114,4,0,0,0,114,52,0,0,0,114,21, + 0,0,0,218,17,70,105,108,101,78,111,116,70,111,117,110, + 100,69,114,114,111,114,41,5,114,31,0,0,0,218,8,114, + 101,115,111,117,114,99,101,218,16,102,117,108,108,110,97,109, + 101,95,97,115,95,112,97,116,104,114,12,0,0,0,114,163, 0,0,0,114,9,0,0,0,114,9,0,0,0,114,10,0, - 0,0,218,11,105,115,95,114,101,115,111,117,114,99,101,176, - 2,0,0,115,14,0,0,0,0,3,14,1,14,1,2,1, - 16,1,14,1,8,1,122,36,95,90,105,112,73,109,112,111, - 114,116,82,101,115,111,117,114,99,101,82,101,97,100,101,114, - 46,105,115,95,114,101,115,111,117,114,99,101,99,1,0,0, - 0,0,0,0,0,9,0,0,0,9,0,0,0,99,0,0, - 0,115,186,0,0,0,100,1,100,2,108,0,109,1,125,1, - 1,0,124,1,124,0,106,2,160,3,124,0,106,4,161,1, - 131,1,125,2,124,2,160,5,124,0,106,2,106,6,161,1, - 125,3,124,3,106,7,100,3,107,2,115,58,116,8,130,1, - 124,3,106,9,125,4,116,10,131,0,125,5,124,0,106,2, - 106,11,68,0,93,102,125,6,122,18,124,1,124,6,131,1, - 160,5,124,4,161,1,125,7,87,0,110,24,4,0,116,12, - 107,10,114,124,1,0,1,0,1,0,89,0,113,78,89,0, - 110,2,88,0,124,7,106,9,106,7,125,8,116,13,124,8, - 131,1,100,1,107,2,114,156,124,7,106,7,86,0,1,0, - 113,78,124,8,124,5,107,7,114,78,124,5,160,14,124,8, - 161,1,1,0,124,8,86,0,1,0,113,78,100,0,83,0, - 41,4,78,114,0,0,0,0,41,1,218,4,80,97,116,104, - 122,11,95,95,105,110,105,116,95,95,46,112,121,41,15,90, - 7,112,97,116,104,108,105,98,114,165,0,0,0,114,4,0, - 0,0,114,52,0,0,0,114,36,0,0,0,90,11,114,101, - 108,97,116,105,118,101,95,116,111,114,27,0,0,0,114,53, - 0,0,0,114,155,0,0,0,90,6,112,97,114,101,110,116, - 218,3,115,101,116,114,26,0,0,0,114,21,0,0,0,114, - 47,0,0,0,218,3,97,100,100,41,9,114,30,0,0,0, - 114,165,0,0,0,90,13,102,117,108,108,110,97,109,101,95, - 112,97,116,104,90,13,114,101,108,97,116,105,118,101,95,112, - 97,116,104,90,12,112,97,99,107,97,103,101,95,112,97,116, - 104,90,12,115,117,98,100,105,114,115,95,115,101,101,110,218, - 8,102,105,108,101,110,97,109,101,90,8,114,101,108,97,116, - 105,118,101,90,11,112,97,114,101,110,116,95,110,97,109,101, - 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,218, - 8,99,111,110,116,101,110,116,115,187,2,0,0,115,34,0, - 0,0,0,8,12,1,18,1,14,3,14,1,6,1,6,1, - 12,1,2,1,18,1,14,1,10,5,8,1,12,1,10,1, - 8,1,10,1,122,33,95,90,105,112,73,109,112,111,114,116, - 82,101,115,111,117,114,99,101,82,101,97,100,101,114,46,99, - 111,110,116,101,110,116,115,78,41,10,114,6,0,0,0,114, - 7,0,0,0,114,8,0,0,0,114,76,0,0,0,114,73, - 0,0,0,114,32,0,0,0,114,162,0,0,0,114,163,0, - 0,0,114,164,0,0,0,114,169,0,0,0,114,9,0,0, - 0,114,9,0,0,0,114,9,0,0,0,114,10,0,0,0, - 114,72,0,0,0,149,2,0,0,115,14,0,0,0,8,5, - 4,1,4,2,8,4,8,9,8,6,8,11,114,72,0,0, - 0,41,41,114,76,0,0,0,90,26,95,102,114,111,122,101, - 110,95,105,109,112,111,114,116,108,105,98,95,101,120,116,101, - 114,110,97,108,114,19,0,0,0,114,1,0,0,0,114,2, - 0,0,0,90,17,95,102,114,111,122,101,110,95,105,109,112, - 111,114,116,108,105,98,114,68,0,0,0,114,131,0,0,0, - 114,101,0,0,0,114,132,0,0,0,114,59,0,0,0,114, - 115,0,0,0,90,7,95,95,97,108,108,95,95,114,18,0, - 0,0,90,15,112,97,116,104,95,115,101,112,97,114,97,116, - 111,114,115,114,16,0,0,0,114,67,0,0,0,114,3,0, - 0,0,114,23,0,0,0,218,4,116,121,112,101,114,62,0, - 0,0,114,4,0,0,0,114,80,0,0,0,114,34,0,0, - 0,114,35,0,0,0,114,33,0,0,0,114,25,0,0,0, - 114,108,0,0,0,114,125,0,0,0,114,127,0,0,0,114, - 48,0,0,0,114,130,0,0,0,114,137,0,0,0,218,8, - 95,95,99,111,100,101,95,95,114,133,0,0,0,114,141,0, - 0,0,114,143,0,0,0,114,151,0,0,0,114,157,0,0, - 0,114,41,0,0,0,114,72,0,0,0,114,9,0,0,0, - 114,9,0,0,0,114,9,0,0,0,114,10,0,0,0,218, - 8,60,109,111,100,117,108,101,62,13,0,0,0,115,82,0, - 0,0,4,4,8,1,16,1,8,1,8,1,8,1,8,1, - 8,1,8,2,8,3,6,1,14,3,16,4,4,2,8,3, - 14,127,0,127,0,1,12,1,12,1,2,1,2,253,2,255, - 2,9,8,4,8,9,8,31,8,103,2,254,2,29,4,5, - 8,21,8,46,8,8,8,28,10,5,8,7,8,6,8,13, - 8,16,8,27, + 0,0,218,13,111,112,101,110,95,114,101,115,111,117,114,99, + 101,187,2,0,0,115,14,0,0,0,0,1,14,1,14,1, + 12,1,2,1,18,1,14,1,122,38,95,90,105,112,73,109, + 112,111,114,116,82,101,115,111,117,114,99,101,82,101,97,100, + 101,114,46,111,112,101,110,95,114,101,115,111,117,114,99,101, + 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, + 0,67,0,0,0,115,8,0,0,0,116,0,130,1,100,0, + 83,0,41,1,78,41,1,114,164,0,0,0,41,2,114,31, + 0,0,0,114,165,0,0,0,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,218,13,114,101,115,111,117,114,99, + 101,95,112,97,116,104,196,2,0,0,115,2,0,0,0,0, + 4,122,38,95,90,105,112,73,109,112,111,114,116,82,101,115, + 111,117,114,99,101,82,101,97,100,101,114,46,114,101,115,111, + 117,114,99,101,95,112,97,116,104,99,2,0,0,0,0,0, + 0,0,4,0,0,0,8,0,0,0,67,0,0,0,115,72, + 0,0,0,124,0,106,0,160,1,100,1,100,2,161,2,125, + 2,124,2,155,0,100,2,124,1,155,0,157,3,125,3,122, + 16,124,0,106,2,160,3,124,3,161,1,1,0,87,0,110, + 22,4,0,116,4,107,10,114,66,1,0,1,0,1,0,89, + 0,100,3,83,0,88,0,100,4,83,0,41,5,78,114,78, + 0,0,0,114,100,0,0,0,70,84,41,5,114,37,0,0, + 0,114,18,0,0,0,114,4,0,0,0,114,52,0,0,0, + 114,21,0,0,0,41,4,114,31,0,0,0,114,54,0,0, + 0,114,166,0,0,0,114,12,0,0,0,114,9,0,0,0, + 114,9,0,0,0,114,10,0,0,0,218,11,105,115,95,114, + 101,115,111,117,114,99,101,202,2,0,0,115,14,0,0,0, + 0,3,14,1,14,1,2,1,16,1,14,1,8,1,122,36, + 95,90,105,112,73,109,112,111,114,116,82,101,115,111,117,114, + 99,101,82,101,97,100,101,114,46,105,115,95,114,101,115,111, + 117,114,99,101,99,1,0,0,0,0,0,0,0,9,0,0, + 0,9,0,0,0,99,0,0,0,115,186,0,0,0,100,1, + 100,2,108,0,109,1,125,1,1,0,124,1,124,0,106,2, + 160,3,124,0,106,4,161,1,131,1,125,2,124,2,160,5, + 124,0,106,2,106,6,161,1,125,3,124,3,106,7,100,3, + 107,2,115,58,116,8,130,1,124,3,106,9,125,4,116,10, + 131,0,125,5,124,0,106,2,106,11,68,0,93,102,125,6, + 122,18,124,1,124,6,131,1,160,5,124,4,161,1,125,7, + 87,0,110,24,4,0,116,12,107,10,114,124,1,0,1,0, + 1,0,89,0,113,78,89,0,110,2,88,0,124,7,106,9, + 106,7,125,8,116,13,124,8,131,1,100,1,107,2,114,156, + 124,7,106,7,86,0,1,0,113,78,124,8,124,5,107,7, + 114,78,124,5,160,14,124,8,161,1,1,0,124,8,86,0, + 1,0,113,78,100,0,83,0,41,4,78,114,0,0,0,0, + 41,1,218,4,80,97,116,104,122,11,95,95,105,110,105,116, + 95,95,46,112,121,41,15,90,7,112,97,116,104,108,105,98, + 114,170,0,0,0,114,4,0,0,0,114,53,0,0,0,114, + 37,0,0,0,90,11,114,101,108,97,116,105,118,101,95,116, + 111,114,28,0,0,0,114,54,0,0,0,114,160,0,0,0, + 90,6,112,97,114,101,110,116,218,3,115,101,116,114,27,0, + 0,0,114,22,0,0,0,114,48,0,0,0,218,3,97,100, + 100,41,9,114,31,0,0,0,114,170,0,0,0,90,13,102, + 117,108,108,110,97,109,101,95,112,97,116,104,90,13,114,101, + 108,97,116,105,118,101,95,112,97,116,104,90,12,112,97,99, + 107,97,103,101,95,112,97,116,104,90,12,115,117,98,100,105, + 114,115,95,115,101,101,110,218,8,102,105,108,101,110,97,109, + 101,90,8,114,101,108,97,116,105,118,101,90,11,112,97,114, + 101,110,116,95,110,97,109,101,114,9,0,0,0,114,9,0, + 0,0,114,10,0,0,0,218,8,99,111,110,116,101,110,116, + 115,213,2,0,0,115,34,0,0,0,0,8,12,1,18,1, + 14,3,14,1,6,1,6,1,12,1,2,1,18,1,14,1, + 10,5,8,1,12,1,10,1,8,1,10,1,122,33,95,90, + 105,112,73,109,112,111,114,116,82,101,115,111,117,114,99,101, + 82,101,97,100,101,114,46,99,111,110,116,101,110,116,115,78, + 41,10,114,6,0,0,0,114,7,0,0,0,114,8,0,0, + 0,114,77,0,0,0,114,74,0,0,0,114,33,0,0,0, + 114,167,0,0,0,114,168,0,0,0,114,169,0,0,0,114, + 174,0,0,0,114,9,0,0,0,114,9,0,0,0,114,9, + 0,0,0,114,10,0,0,0,114,73,0,0,0,175,2,0, + 0,115,14,0,0,0,8,5,4,1,4,2,8,4,8,9, + 8,6,8,11,114,73,0,0,0,41,44,114,77,0,0,0, + 90,26,95,102,114,111,122,101,110,95,105,109,112,111,114,116, + 108,105,98,95,101,120,116,101,114,110,97,108,114,20,0,0, + 0,114,1,0,0,0,114,2,0,0,0,90,17,95,102,114, + 111,122,101,110,95,105,109,112,111,114,116,108,105,98,114,69, + 0,0,0,114,137,0,0,0,114,101,0,0,0,114,138,0, + 0,0,114,60,0,0,0,114,122,0,0,0,90,7,95,95, + 97,108,108,95,95,114,19,0,0,0,90,15,112,97,116,104, + 95,115,101,112,97,114,97,116,111,114,115,114,17,0,0,0, + 114,68,0,0,0,114,3,0,0,0,114,24,0,0,0,218, + 4,116,121,112,101,114,63,0,0,0,114,104,0,0,0,114, + 106,0,0,0,114,108,0,0,0,114,4,0,0,0,114,81, + 0,0,0,114,35,0,0,0,114,36,0,0,0,114,34,0, + 0,0,114,26,0,0,0,114,113,0,0,0,114,131,0,0, + 0,114,133,0,0,0,114,49,0,0,0,114,136,0,0,0, + 114,142,0,0,0,218,8,95,95,99,111,100,101,95,95,114, + 139,0,0,0,114,146,0,0,0,114,148,0,0,0,114,156, + 0,0,0,114,162,0,0,0,114,42,0,0,0,114,73,0, + 0,0,114,9,0,0,0,114,9,0,0,0,114,9,0,0, + 0,114,10,0,0,0,218,8,60,109,111,100,117,108,101,62, + 13,0,0,0,115,88,0,0,0,4,4,8,1,16,1,8, + 1,8,1,8,1,8,1,8,1,8,2,8,3,6,1,14, + 3,16,4,4,2,8,2,4,1,4,1,4,2,14,127,0, + 127,0,1,12,1,12,1,2,1,2,253,2,255,2,9,8, + 4,8,9,8,31,8,126,2,254,2,29,4,5,8,21,8, + 46,8,8,8,28,10,5,8,7,8,6,8,13,8,16,8, + 27, }; From webhook-mailer at python.org Tue Sep 25 17:44:57 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 25 Sep 2018 21:44:57 -0000 Subject: [Python-checkins] asyncio/docs: Replace Python 4.0 -> 3.10 (GH-9579) Message-ID: https://github.com/python/cpython/commit/fad6af2744c0b022568f7f4a8afc93fed056d4db commit: fad6af2744c0b022568f7f4a8afc93fed056d4db branch: master author: Yury Selivanov committer: GitHub date: 2018-09-25T17:44:52-04:00 summary: asyncio/docs: Replace Python 4.0 -> 3.10 (GH-9579) files: M Doc/library/asyncio-task.rst M Lib/asyncio/tasks.py diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index bb693d7a6475..1a504363946e 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -277,7 +277,7 @@ Sleeping when the coroutine completes. The *loop* argument is deprecated and scheduled for removal - in Python 4.0. + in Python 3.10. .. _asyncio_example_sleep: @@ -435,7 +435,7 @@ Timeouts If the wait is cancelled, the future *aw* is also cancelled. The *loop* argument is deprecated and scheduled for removal - in Python 4.0. + in Python 3.10. .. _asyncio_example_waitfor: @@ -487,7 +487,7 @@ Waiting Primitives done, pending = await asyncio.wait(aws) The *loop* argument is deprecated and scheduled for removal - in Python 4.0. + in Python 3.10. *timeout* (a float or int), if specified, can be used to control the maximum number of seconds to wait before returning. @@ -888,7 +888,7 @@ Generator-based Coroutines .. note:: Support for generator-based coroutines is **deprecated** and - is scheduled for removal in Python 4.0. + is scheduled for removal in Python 3.10. Generator-based coroutines predate async/await syntax. They are Python generators that use ``yield from`` expressions to await @@ -914,7 +914,7 @@ enforced. await old_style_coroutine() This decorator is **deprecated** and is scheduled for removal in - Python 4.0. + Python 3.10. This decorator should not be used for :keyword:`async def` coroutines. diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py index b52aad8c422d..743e82baff7a 100644 --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -384,8 +384,8 @@ def create_task(coro, *, name=None): if loop is None: loop = events.get_running_loop() else: - warnings.warn("The loop argument is deprecated and scheduled for" - "removal in Python 4.0.", + warnings.warn("The loop argument is deprecated and scheduled for " + "removal in Python 3.10.", DeprecationWarning, stacklevel=2) fs = {ensure_future(f, loop=loop) for f in set(fs)} @@ -414,8 +414,8 @@ def _release_waiter(waiter, *args): if loop is None: loop = events.get_running_loop() else: - warnings.warn("The loop argument is deprecated and scheduled for" - "removal in Python 4.0.", + warnings.warn("The loop argument is deprecated and scheduled for " + "removal in Python 3.10.", DeprecationWarning, stacklevel=2) if timeout is None: @@ -595,8 +595,8 @@ def __sleep0(): if loop is None: loop = events.get_running_loop() else: - warnings.warn("The loop argument is deprecated and scheduled for" - "removal in Python 4.0.", + warnings.warn("The loop argument is deprecated and scheduled for " + "removal in Python 3.10.", DeprecationWarning, stacklevel=2) future = loop.create_future() From webhook-mailer at python.org Tue Sep 25 18:00:19 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Tue, 25 Sep 2018 22:00:19 -0000 Subject: [Python-checkins] [3.7] asyncio/docs: Replace Python 4.0 -> 3.10 (GH-9580) Message-ID: https://github.com/python/cpython/commit/22a56958e72957e2ee6fb84ac5376c4ba642f4ce commit: 22a56958e72957e2ee6fb84ac5376c4ba642f4ce branch: 3.7 author: Yury Selivanov committer: GitHub date: 2018-09-25T18:00:15-04:00 summary: [3.7] asyncio/docs: Replace Python 4.0 -> 3.10 (GH-9580) files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 7fbfd426dba6..5cbdfeff32f5 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -271,7 +271,7 @@ Sleeping when the coroutine completes. The *loop* argument is deprecated and scheduled for removal - in Python 4.0. + in Python 3.10. .. _asyncio_example_sleep: @@ -429,7 +429,7 @@ Timeouts If the wait is cancelled, the future *aw* is also cancelled. The *loop* argument is deprecated and scheduled for removal - in Python 4.0. + in Python 3.10. .. _asyncio_example_waitfor: @@ -481,7 +481,7 @@ Waiting Primitives done, pending = await asyncio.wait(aws) The *loop* argument is deprecated and scheduled for removal - in Python 4.0. + in Python 3.10. *timeout* (a float or int), if specified, can be used to control the maximum number of seconds to wait before returning. @@ -857,7 +857,7 @@ Generator-based Coroutines .. note:: Support for generator-based coroutines is **deprecated** and - is scheduled for removal in Python 4.0. + is scheduled for removal in Python 3.10. Generator-based coroutines predate async/await syntax. They are Python generators that use ``yield from`` expressions to await @@ -883,7 +883,7 @@ enforced. await old_style_coroutine() This decorator is **deprecated** and is scheduled for removal in - Python 4.0. + Python 3.10. This decorator should not be used for :keyword:`async def` coroutines. From webhook-mailer at python.org Tue Sep 25 19:00:18 2018 From: webhook-mailer at python.org (Vinay Sajip) Date: Tue, 25 Sep 2018 23:00:18 -0000 Subject: [Python-checkins] bpo-34334: Don't log traceback twice in QueueHandler (GH-9537) Message-ID: https://github.com/python/cpython/commit/d345bb4d9b6e16c681cd8a4e1fff94ecd6b0bb09 commit: d345bb4d9b6e16c681cd8a4e1fff94ecd6b0bb09 branch: master author: Cheryl Sabella committer: Vinay Sajip date: 2018-09-26T00:00:08+01:00 summary: bpo-34334: Don't log traceback twice in QueueHandler (GH-9537) files: A Misc/NEWS.d/next/Library/2018-09-25-08-42-34.bpo-34334.rSPBW9.rst M Doc/library/logging.handlers.rst M Lib/logging/handlers.py M Lib/test/test_logging.py diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index bdf16a8177e9..dee9a84e3337 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -973,9 +973,9 @@ possible, while any potentially slow operations (such as sending an email via Prepares a record for queuing. The object returned by this method is enqueued. - The base implementation formats the record to merge the message - and arguments, and removes unpickleable items from the record - in-place. + The base implementation formats the record to merge the message, + arguments, and exception information, if present. It also + removes unpickleable items from the record in-place. You might want to override this method if you want to convert the record to a dict or JSON string, or send a modified copy diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py index 974c089d40ec..e213e438c31a 100644 --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -1374,13 +1374,14 @@ def prepare(self, record): # (if there's exception data), and also returns the formatted # message. We can then use this to replace the original # msg + args, as these might be unpickleable. We also zap the - # exc_info attribute, as it's no longer needed and, if not None, - # will typically not be pickleable. + # exc_info and exc_text attributes, as they are no longer + # needed and, if not None, will typically not be pickleable. msg = self.format(record) record.message = msg record.msg = msg record.args = None record.exc_info = None + record.exc_text = None return record def emit(self, record): diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index b9dad64a5949..d352e5fa3f3d 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -3345,6 +3345,21 @@ def test_queue_listener(self): self.assertFalse(handler.matches(levelno=logging.WARNING, message='4')) self.assertFalse(handler.matches(levelno=logging.ERROR, message='5')) self.assertTrue(handler.matches(levelno=logging.CRITICAL, message='6')) + handler.close() + + @unittest.skipUnless(hasattr(logging.handlers, 'QueueListener'), + 'logging.handlers.QueueListener required for this test') + def test_queue_listener_with_StreamHandler(self): + # Test that traceback only appends once (bpo-34334). + listener = logging.handlers.QueueListener(self.queue, self.root_hdlr) + listener.start() + try: + 1 / 0 + except ZeroDivisionError as e: + exc = e + self.que_logger.exception(self.next_message(), exc_info=exc) + listener.stop() + self.assertEqual(self.stream.getvalue().strip().count('Traceback'), 1) if hasattr(logging.handlers, 'QueueListener'): import multiprocessing diff --git a/Misc/NEWS.d/next/Library/2018-09-25-08-42-34.bpo-34334.rSPBW9.rst b/Misc/NEWS.d/next/Library/2018-09-25-08-42-34.bpo-34334.rSPBW9.rst new file mode 100644 index 000000000000..137a4f78f420 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-25-08-42-34.bpo-34334.rSPBW9.rst @@ -0,0 +1,2 @@ +In :class:`QueueHandler`, clear `exc_text` from :class:`LogRecord` to +prevent traceback from being written twice. From webhook-mailer at python.org Tue Sep 25 23:59:04 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 26 Sep 2018 03:59:04 -0000 Subject: [Python-checkins] bpo-34320: Fix dict(o) didn't copy order of dict subclass (GH-8624) Message-ID: https://github.com/python/cpython/commit/2aaf98c16ae3070378de523a173e29644037d8bd commit: 2aaf98c16ae3070378de523a173e29644037d8bd branch: master author: INADA Naoki committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-25T20:59:00-07:00 summary: bpo-34320: Fix dict(o) didn't copy order of dict subclass (GH-8624) When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)` should use it instead of dict ordering. https://bugs.python.org/issue34320 files: A Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst M Lib/test/test_builtin.py M Lib/test/test_call.py M Lib/test/test_dict.py M Objects/dictobject.c diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py index 46cf2d328362..dafcf004c095 100644 --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -1968,6 +1968,15 @@ class B: with self.assertRaises(TypeError): type('A', (B,), {'__slots__': '__weakref__'}) + def test_namespace_order(self): + # bpo-34320: namespace should preserve order + od = collections.OrderedDict([('a', 1), ('b', 2)]) + od.move_to_end('a') + expected = list(od.items()) + + C = type('C', (), od) + self.assertEqual(list(C.__dict__.items())[:2], [('b', 2), ('a', 1)]) + def load_tests(loader, tests, pattern): from doctest import DocTestSuite diff --git a/Lib/test/test_call.py b/Lib/test/test_call.py index ec9697a444f0..45b34e46a5ab 100644 --- a/Lib/test/test_call.py +++ b/Lib/test/test_call.py @@ -9,6 +9,23 @@ import collections import itertools + +class FunctionCalls(unittest.TestCase): + + def test_kwargs_order(self): + # bpo-34320: **kwargs should preserve order of passed OrderedDict + od = collections.OrderedDict([('a', 1), ('b', 2)]) + od.move_to_end('a') + expected = list(od.items()) + + def fn(**kw): + return kw + + res = fn(**od) + self.assertIsInstance(res, dict) + self.assertEqual(list(res.items()), expected) + + # The test cases here cover several paths through the function calling # code. They depend on the METH_XXX flag that is used to define a C # function, which can't be verified from Python. If the METH_XXX decl diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py index 38521bbf6630..90c0a3131a78 100644 --- a/Lib/test/test_dict.py +++ b/Lib/test/test_dict.py @@ -1222,6 +1222,36 @@ def iter_and_mutate(): self.assertRaises(RuntimeError, iter_and_mutate) + def test_dict_copy_order(self): + # bpo-34320 + od = collections.OrderedDict([('a', 1), ('b', 2)]) + od.move_to_end('a') + expected = list(od.items()) + + copy = dict(od) + self.assertEqual(list(copy.items()), expected) + + # dict subclass doesn't override __iter__ + class CustomDict(dict): + pass + + pairs = [('a', 1), ('b', 2), ('c', 3)] + + d = CustomDict(pairs) + self.assertEqual(pairs, list(dict(d).items())) + + class CustomReversedDict(dict): + def keys(self): + return reversed(list(dict.keys(self))) + + __iter__ = keys + + def items(self): + return reversed(dict.items(self)) + + d = CustomReversedDict(pairs) + self.assertEqual(pairs[::-1], list(dict(d).items())) + class CAPITest(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst b/Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst new file mode 100644 index 000000000000..ce5b3397e65f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst @@ -0,0 +1 @@ +Fix ``dict(od)`` didn't copy iteration order of OrderedDict. diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 413557d66741..fec69678c500 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -235,6 +235,8 @@ static Py_ssize_t lookdict_split(PyDictObject *mp, PyObject *key, static int dictresize(PyDictObject *mp, Py_ssize_t minused); +static PyObject* dict_iter(PyDictObject *dict); + /*Global counter used to set ma_version_tag field of dictionary. * It is incremented each time that a dictionary is created and each * time that a dictionary is modified. */ @@ -2379,7 +2381,7 @@ dict_merge(PyObject *a, PyObject *b, int override) return -1; } mp = (PyDictObject*)a; - if (PyDict_Check(b)) { + if (PyDict_Check(b) && (Py_TYPE(b)->tp_iter == (getiterfunc)dict_iter)) { other = (PyDictObject*)b; if (other == mp || other->ma_used == 0) /* a.update(a) or a.update({}); nothing to do */ From webhook-mailer at python.org Wed Sep 26 00:17:55 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 26 Sep 2018 04:17:55 -0000 Subject: [Python-checkins] bpo-34320: Fix dict(o) didn't copy order of dict subclass (GH-8624) Message-ID: https://github.com/python/cpython/commit/12e3e80241e91df79811f53ff372e28e9371bf9b commit: 12e3e80241e91df79811f53ff372e28e9371bf9b branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T21:17:52-07:00 summary: bpo-34320: Fix dict(o) didn't copy order of dict subclass (GH-8624) When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)` should use it instead of dict ordering. https://bugs.python.org/issue34320 (cherry picked from commit 2aaf98c16ae3070378de523a173e29644037d8bd) Co-authored-by: INADA Naoki files: A Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst M Lib/test/test_builtin.py M Lib/test/test_call.py M Lib/test/test_dict.py M Objects/dictobject.c diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py index 8f91bc9bf919..e7202cd70c9c 100644 --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -1961,6 +1961,15 @@ class B: with self.assertRaises(TypeError): type('A', (B,), {'__slots__': '__weakref__'}) + def test_namespace_order(self): + # bpo-34320: namespace should preserve order + od = collections.OrderedDict([('a', 1), ('b', 2)]) + od.move_to_end('a') + expected = list(od.items()) + + C = type('C', (), od) + self.assertEqual(list(C.__dict__.items())[:2], [('b', 2), ('a', 1)]) + def load_tests(loader, tests, pattern): from doctest import DocTestSuite diff --git a/Lib/test/test_call.py b/Lib/test/test_call.py index 3f9987c9e777..362c31c40c4a 100644 --- a/Lib/test/test_call.py +++ b/Lib/test/test_call.py @@ -9,6 +9,23 @@ import collections import itertools + +class FunctionCalls(unittest.TestCase): + + def test_kwargs_order(self): + # bpo-34320: **kwargs should preserve order of passed OrderedDict + od = collections.OrderedDict([('a', 1), ('b', 2)]) + od.move_to_end('a') + expected = list(od.items()) + + def fn(**kw): + return kw + + res = fn(**od) + self.assertIsInstance(res, dict) + self.assertEqual(list(res.items()), expected) + + # The test cases here cover several paths through the function calling # code. They depend on the METH_XXX flag that is used to define a C # function, which can't be verified from Python. If the METH_XXX decl diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py index 38521bbf6630..90c0a3131a78 100644 --- a/Lib/test/test_dict.py +++ b/Lib/test/test_dict.py @@ -1222,6 +1222,36 @@ def iter_and_mutate(): self.assertRaises(RuntimeError, iter_and_mutate) + def test_dict_copy_order(self): + # bpo-34320 + od = collections.OrderedDict([('a', 1), ('b', 2)]) + od.move_to_end('a') + expected = list(od.items()) + + copy = dict(od) + self.assertEqual(list(copy.items()), expected) + + # dict subclass doesn't override __iter__ + class CustomDict(dict): + pass + + pairs = [('a', 1), ('b', 2), ('c', 3)] + + d = CustomDict(pairs) + self.assertEqual(pairs, list(dict(d).items())) + + class CustomReversedDict(dict): + def keys(self): + return reversed(list(dict.keys(self))) + + __iter__ = keys + + def items(self): + return reversed(dict.items(self)) + + d = CustomReversedDict(pairs) + self.assertEqual(pairs[::-1], list(dict(d).items())) + class CAPITest(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst b/Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst new file mode 100644 index 000000000000..ce5b3397e65f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst @@ -0,0 +1 @@ +Fix ``dict(od)`` didn't copy iteration order of OrderedDict. diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 828eb99e2d03..eb635263b04c 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -235,6 +235,8 @@ static Py_ssize_t lookdict_split(PyDictObject *mp, PyObject *key, static int dictresize(PyDictObject *mp, Py_ssize_t minused); +static PyObject* dict_iter(PyDictObject *dict); + /*Global counter used to set ma_version_tag field of dictionary. * It is incremented each time that a dictionary is created and each * time that a dictionary is modified. */ @@ -2379,7 +2381,7 @@ dict_merge(PyObject *a, PyObject *b, int override) return -1; } mp = (PyDictObject*)a; - if (PyDict_Check(b)) { + if (PyDict_Check(b) && (Py_TYPE(b)->tp_iter == (getiterfunc)dict_iter)) { other = (PyDictObject*)b; if (other == mp || other->ma_used == 0) /* a.update(a) or a.update({}); nothing to do */ From webhook-mailer at python.org Wed Sep 26 01:19:11 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 26 Sep 2018 05:19:11 -0000 Subject: [Python-checkins] Fix pickletools doc for NEWFALSE. (GH-9432) Message-ID: https://github.com/python/cpython/commit/69d63bbbd6ea8fab7792a7e4de21a4662665cb5f commit: 69d63bbbd6ea8fab7792a7e4de21a4662665cb5f branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T22:19:08-07:00 summary: Fix pickletools doc for NEWFALSE. (GH-9432) Also make docs for NEWFALSE and NEWTRUE more consistent with docs for other opcodes. (cherry picked from commit 488cfb78c8b81075942b5e4cc9630e7a6dd9dc28) Co-authored-by: Krzysztof Wroblewski files: M Lib/pickletools.py diff --git a/Lib/pickletools.py b/Lib/pickletools.py index 8486cbf84368..ed8bee36e8c5 100644 --- a/Lib/pickletools.py +++ b/Lib/pickletools.py @@ -1325,9 +1325,7 @@ def __init__(self, name, code, arg, stack_before=[], stack_after=[pybool], proto=2, - doc="""True. - - Push True onto the stack."""), + doc="Push True onto the stack."), I(name='NEWFALSE', code='\x89', @@ -1335,9 +1333,7 @@ def __init__(self, name, code, arg, stack_before=[], stack_after=[pybool], proto=2, - doc="""True. - - Push False onto the stack."""), + doc="Push False onto the stack."), # Ways to spell Unicode strings. From webhook-mailer at python.org Wed Sep 26 02:38:40 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 26 Sep 2018 06:38:40 -0000 Subject: [Python-checkins] [3.6] bpo-34320: Fix dict(o) didn't copy order of dict subclass (GH-8624) (GH-9583) Message-ID: https://github.com/python/cpython/commit/d45a9613402b686f8afd3dd5b6acf8141f14d711 commit: d45a9613402b686f8afd3dd5b6acf8141f14d711 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-25T23:38:36-07:00 summary: [3.6] bpo-34320: Fix dict(o) didn't copy order of dict subclass (GH-8624) (GH-9583) When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)` should use it instead of dict ordering. https://bugs.python.org/issue34320 (cherry picked from commit 2aaf98c16ae3070378de523a173e29644037d8bd) Co-authored-by: INADA Naoki https://bugs.python.org/issue34320 files: A Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst M Lib/test/test_builtin.py M Lib/test/test_call.py M Lib/test/test_dict.py M Objects/dictobject.c diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py index e0dbe7849808..e885e6d4a49a 100644 --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -1841,6 +1841,15 @@ class B: with self.assertRaises(TypeError): type('A', (B,), {'__slots__': '__weakref__'}) + def test_namespace_order(self): + # bpo-34320: namespace should preserve order + od = collections.OrderedDict([('a', 1), ('b', 2)]) + od.move_to_end('a') + expected = list(od.items()) + + C = type('C', (), od) + self.assertEqual(list(C.__dict__.items())[:2], [('b', 2), ('a', 1)]) + def load_tests(loader, tests, pattern): from doctest import DocTestSuite diff --git a/Lib/test/test_call.py b/Lib/test/test_call.py index 2e8819be5dac..e71ede26466b 100644 --- a/Lib/test/test_call.py +++ b/Lib/test/test_call.py @@ -1,3 +1,4 @@ +import collections import datetime import unittest from test.support import cpython_only @@ -6,6 +7,23 @@ except ImportError: _testcapi = None + +class FunctionCalls(unittest.TestCase): + + def test_kwargs_order(self): + # bpo-34320: **kwargs should preserve order of passed OrderedDict + od = collections.OrderedDict([('a', 1), ('b', 2)]) + od.move_to_end('a') + expected = list(od.items()) + + def fn(**kw): + return kw + + res = fn(**od) + self.assertIsInstance(res, dict) + self.assertEqual(list(res.items()), expected) + + # The test cases here cover several paths through the function calling # code. They depend on the METH_XXX flag that is used to define a C # function, which can't be verified from Python. If the METH_XXX decl diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py index 639e05f74a8a..7e94ad279364 100644 --- a/Lib/test/test_dict.py +++ b/Lib/test/test_dict.py @@ -1176,6 +1176,37 @@ def iter_and_mutate(): self.assertRaises(RuntimeError, iter_and_mutate) + @support.cpython_only + def test_dict_copy_order(self): + # bpo-34320 + od = collections.OrderedDict([('a', 1), ('b', 2)]) + od.move_to_end('a') + expected = list(od.items()) + + copy = dict(od) + self.assertEqual(list(copy.items()), expected) + + # dict subclass doesn't override __iter__ + class CustomDict(dict): + pass + + pairs = [('a', 1), ('b', 2), ('c', 3)] + + d = CustomDict(pairs) + self.assertEqual(pairs, list(dict(d).items())) + + class CustomReversedDict(dict): + def keys(self): + return reversed(list(dict.keys(self))) + + __iter__ = keys + + def items(self): + return reversed(dict.items(self)) + + d = CustomReversedDict(pairs) + self.assertEqual(pairs[::-1], list(dict(d).items())) + class CAPITest(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst b/Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst new file mode 100644 index 000000000000..ce5b3397e65f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-08-02-22-34-59.bpo-34320.hNshAA.rst @@ -0,0 +1 @@ +Fix ``dict(od)`` didn't copy iteration order of OrderedDict. diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 0768d1154542..b55e49c4c9f7 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -237,6 +237,8 @@ static Py_ssize_t lookdict_split(PyDictObject *mp, PyObject *key, static int dictresize(PyDictObject *mp, Py_ssize_t minused); +static PyObject* dict_iter(PyDictObject *dict); + /*Global counter used to set ma_version_tag field of dictionary. * It is incremented each time that a dictionary is created and each * time that a dictionary is modified. */ @@ -2482,7 +2484,7 @@ dict_merge(PyObject *a, PyObject *b, int override) return -1; } mp = (PyDictObject*)a; - if (PyDict_Check(b)) { + if (PyDict_Check(b) && (Py_TYPE(b)->tp_iter == (getiterfunc)dict_iter)) { other = (PyDictObject*)b; if (other == mp || other->ma_used == 0) /* a.update(a) or a.update({}); nothing to do */ From solipsis at pitrou.net Wed Sep 26 05:15:01 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 26 Sep 2018 09:15:01 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=1 Message-ID: <20180926091501.1.4BCF17011D823FFE@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 3] memory blocks, sum=3 test_collections leaked [-7, 1, 0] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogUaHUXq', '--timeout', '7200'] From webhook-mailer at python.org Wed Sep 26 09:48:03 2018 From: webhook-mailer at python.org (Tal Einat) Date: Wed, 26 Sep 2018 13:48:03 -0000 Subject: [Python-checkins] bpo-31425: Expose AF_QIPCRTR in socket module (GH-3706) Message-ID: https://github.com/python/cpython/commit/bb8165172ac2ef8c7092e8e82928cc7f5f310ab3 commit: bb8165172ac2ef8c7092e8e82928cc7f5f310ab3 branch: master author: Bjorn Andersson committer: Tal Einat date: 2018-09-26T16:47:52+03:00 summary: bpo-31425: Expose AF_QIPCRTR in socket module (GH-3706) The AF_QIPCRTR address family was introduced in Linux v4.7. Co-authored-by: Bjorn Andersson files: A Misc/NEWS.d/next/Library/2017-10-24-10-18-35.bpo-31425.1lgw47.rst M Doc/library/socket.rst M Lib/test/test_socket.py M Modules/socketmodule.c M Modules/socketmodule.h M configure M configure.ac M pyconfig.h.in diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index 32e7c5e6c079..44370fef35c3 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -193,6 +193,13 @@ created. Socket addresses are represented as follows: - *addr* - Optional bytes-like object specifying the hardware physical address, whose interpretation depends on the device. +- :const:`AF_QIPCRTR` is a Linux-only socket based interface for communicating + with services running on co-processors in Qualcomm platforms. The address + family is represented as a ``(node, port)`` tuple where the *node* and *port* + are non-negative integers. + + .. versionadded:: 3.7 + If you use a hostname in the *host* portion of IPv4/v6 socket address, the program may show a nondeterministic behavior, as Python uses the first address returned from the DNS resolution. The socket address will be resolved @@ -481,6 +488,13 @@ Constants :const:`HCI_DATA_DIR` are not available for FreeBSD, NetBSD, or DragonFlyBSD. +.. data:: AF_QIPCRTR + + Constant for Qualcomm's IPC router protocol, used to communicate with + service providing remote processors. + + Availability: Linux >= 4.7. + Functions ^^^^^^^^^ diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index bd4fad1f6380..bbbf27b60978 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -94,6 +94,16 @@ def _have_socket_alg(): s.close() return True +def _have_socket_qipcrtr(): + """Check whether AF_QIPCRTR sockets are supported on this host.""" + try: + s = socket.socket(socket.AF_QIPCRTR, socket.SOCK_DGRAM, 0) + except (AttributeError, OSError): + return False + else: + s.close() + return True + def _have_socket_vsock(): """Check whether AF_VSOCK sockets are supported on this host.""" ret = get_cid() is not None @@ -113,6 +123,8 @@ def _is_fd_in_blocking_mode(sock): HAVE_SOCKET_ALG = _have_socket_alg() +HAVE_SOCKET_QIPCRTR = _have_socket_qipcrtr() + HAVE_SOCKET_VSOCK = _have_socket_vsock() # Size in bytes of the int type @@ -2054,6 +2066,34 @@ def _testSelect(self): self.data = b'select' self.cli.sendto(self.data, 0, (HOST, self.port)) + at unittest.skipUnless(HAVE_SOCKET_QIPCRTR, + 'QIPCRTR sockets required for this test.') +class BasicQIPCRTRTest(unittest.TestCase): + + def testCrucialConstants(self): + socket.AF_QIPCRTR + + def testCreateSocket(self): + with socket.socket(socket.AF_QIPCRTR, socket.SOCK_DGRAM) as s: + pass + + def testUnbound(self): + with socket.socket(socket.AF_QIPCRTR, socket.SOCK_DGRAM) as s: + self.assertEqual(s.getsockname()[1], 0) + + def testBindSock(self): + with socket.socket(socket.AF_QIPCRTR, socket.SOCK_DGRAM) as s: + support.bind_port(s, host=s.getsockname()[0]) + self.assertNotEqual(s.getsockname()[1], 0) + + def testInvalidBindSock(self): + with socket.socket(socket.AF_QIPCRTR, socket.SOCK_DGRAM) as s: + self.assertRaises(OSError, support.bind_port, s, host=-2) + + def testAutoBindSock(self): + with socket.socket(socket.AF_QIPCRTR, socket.SOCK_DGRAM) as s: + s.connect((123, 123)) + self.assertNotEqual(s.getsockname()[1], 0) @unittest.skipIf(fcntl is None, "need fcntl") @unittest.skipUnless(HAVE_SOCKET_VSOCK, @@ -5978,6 +6018,7 @@ def test_main(): tests.extend([BasicCANTest, CANTest]) tests.extend([BasicRDSTest, RDSTest]) tests.append(LinuxKernelCryptoAPI) + tests.append(BasicQIPCRTRTest) tests.extend([ BasicVSOCKTest, ThreadedVSOCKSocketStreamTest, diff --git a/Misc/NEWS.d/next/Library/2017-10-24-10-18-35.bpo-31425.1lgw47.rst b/Misc/NEWS.d/next/Library/2017-10-24-10-18-35.bpo-31425.1lgw47.rst new file mode 100644 index 000000000000..c5d646775b94 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2017-10-24-10-18-35.bpo-31425.1lgw47.rst @@ -0,0 +1,3 @@ +Add support for sockets of the AF_QIPCRTR address family, supported by the +Linux kernel. This is used to communicate with services, such as GPS or +radio, running on Qualcomm devices. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index adaefad556ca..9149641fce5a 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -7,8 +7,8 @@ This module provides an interface to Berkeley socket IPC. Limitations: - Only AF_INET, AF_INET6 and AF_UNIX address families are supported in a - portable manner, though AF_PACKET, AF_NETLINK and AF_TIPC are supported - under Linux. + portable manner, though AF_PACKET, AF_NETLINK, AF_QIPCRTR and AF_TIPC are + supported under Linux. - No read/write operations (use sendall/recv or makefile instead). - Additional restrictions apply on some non-Unix platforms (compensated for by socket.py). @@ -55,6 +55,8 @@ Module interface: the Ethernet protocol number to be received. For example: ("eth0",0x1234). Optional 3rd,4th,5th elements in the tuple specify packet-type and ha-type/addr. +- an AF_QIPCRTR socket address is a (node, port) tuple where the + node and port are non-negative integers. - an AF_TIPC socket address is expressed as (addr_type, v1, v2, v3 [, scope]); where addr_type can be one of: TIPC_ADDR_NAMESEQ, TIPC_ADDR_NAME, and TIPC_ADDR_ID; @@ -1293,6 +1295,14 @@ makesockaddr(SOCKET_T sockfd, struct sockaddr *addr, size_t addrlen, int proto) } #endif /* AF_NETLINK */ +#if defined(AF_QIPCRTR) + case AF_QIPCRTR: + { + struct sockaddr_qrtr *a = (struct sockaddr_qrtr *) addr; + return Py_BuildValue("II", a->sq_node, a->sq_port); + } +#endif /* AF_QIPCRTR */ + #if defined(AF_VSOCK) case AF_VSOCK: { @@ -1668,6 +1678,30 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, } #endif /* AF_NETLINK */ +#if defined(AF_QIPCRTR) + case AF_QIPCRTR: + { + struct sockaddr_qrtr* addr; + unsigned int node, port; + addr = (struct sockaddr_qrtr *)addr_ret; + if (!PyTuple_Check(args)) { + PyErr_Format( + PyExc_TypeError, + "getsockaddrarg: " + "AF_QIPCRTR address must be tuple, not %.500s", + Py_TYPE(args)->tp_name); + return 0; + } + if (!PyArg_ParseTuple(args, "II:getsockaddrarg", &node, &port)) + return 0; + addr->sq_family = AF_QIPCRTR; + addr->sq_node = node; + addr->sq_port = port; + *len_ret = sizeof(*addr); + return 1; + } +#endif /* AF_QIPCRTR */ + #if defined(AF_VSOCK) case AF_VSOCK: { @@ -2263,6 +2297,14 @@ getsockaddrlen(PySocketSockObject *s, socklen_t *len_ret) } #endif /* AF_NETLINK */ +#if defined(AF_QIPCRTR) + case AF_QIPCRTR: + { + *len_ret = sizeof (struct sockaddr_qrtr); + return 1; + } +#endif /* AF_QIPCRTR */ + #if defined(AF_VSOCK) case AF_VSOCK: { @@ -6983,6 +7025,11 @@ PyInit__socket(void) #endif #endif /* AF_NETLINK */ +#ifdef AF_QIPCRTR + /* Qualcomm IPCROUTER */ + PyModule_AddIntMacro(m, AF_QIPCRTR); +#endif + #ifdef AF_VSOCK PyModule_AddIntConstant(m, "AF_VSOCK", AF_VSOCK); PyModule_AddIntConstant(m, "SO_VM_SOCKETS_BUFFER_SIZE", 0); diff --git a/Modules/socketmodule.h b/Modules/socketmodule.h index 0b2edc158782..dff1f8f4e9ce 100644 --- a/Modules/socketmodule.h +++ b/Modules/socketmodule.h @@ -54,6 +54,15 @@ typedef int socklen_t; # undef AF_NETLINK #endif +#ifdef HAVE_LINUX_QRTR_H +# ifdef HAVE_ASM_TYPES_H +# include +# endif +# include +#else +# undef AF_QIPCRTR +#endif + #ifdef HAVE_BLUETOOTH_BLUETOOTH_H #include #include @@ -203,6 +212,9 @@ typedef union sock_addr { #ifdef HAVE_SOCKADDR_ALG struct sockaddr_alg alg; #endif +#ifdef AF_QIPCRTR + struct sockaddr_qrtr sq; +#endif #ifdef AF_VSOCK struct sockaddr_vm vm; #endif diff --git a/configure b/configure index 38546d6ca7b4..9d2c4e4ad7e6 100755 --- a/configure +++ b/configure @@ -8023,6 +8023,28 @@ fi done +# On Linux, qrtr.h requires asm/types.h +for ac_header in linux/qrtr.h +do : + ac_fn_c_check_header_compile "$LINENO" "linux/qrtr.h" "ac_cv_header_linux_qrtr_h" " +#ifdef HAVE_ASM_TYPES_H +#include +#endif +#ifdef HAVE_SYS_SOCKET_H +#include +#endif + +" +if test "x$ac_cv_header_linux_qrtr_h" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_LINUX_QRTR_H 1 +_ACEOF + +fi + +done + + for ac_header in linux/vm_sockets.h do : ac_fn_c_check_header_compile "$LINENO" "linux/vm_sockets.h" "ac_cv_header_linux_vm_sockets_h" " diff --git a/configure.ac b/configure.ac index 96331ec221be..2235a13d1160 100644 --- a/configure.ac +++ b/configure.ac @@ -2112,6 +2112,16 @@ AC_CHECK_HEADERS(linux/netlink.h,,,[ #endif ]) +# On Linux, qrtr.h requires asm/types.h +AC_CHECK_HEADERS(linux/qrtr.h,,,[ +#ifdef HAVE_ASM_TYPES_H +#include +#endif +#ifdef HAVE_SYS_SOCKET_H +#include +#endif +]) + AC_CHECK_HEADERS(linux/vm_sockets.h,,,[ #ifdef HAVE_SYS_SOCKET_H #include diff --git a/pyconfig.h.in b/pyconfig.h.in index 41e0479cad2e..254a2dcab4da 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -615,6 +615,9 @@ /* Define to 1 if you have the header file. */ #undef HAVE_LINUX_NETLINK_H +/* Define to 1 if you have the header file. */ +#undef HAVE_LINUX_QRTR_H + /* Define to 1 if you have the header file. */ #undef HAVE_LINUX_RANDOM_H From webhook-mailer at python.org Wed Sep 26 10:44:50 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 26 Sep 2018 14:44:50 -0000 Subject: [Python-checkins] Drop confusing commented out code in pystrtod.c (GH-6072) (GH-9587) Message-ID: https://github.com/python/cpython/commit/3707bcf02b7574448d7137935995146f6e8ea810 commit: 3707bcf02b7574448d7137935995146f6e8ea810 branch: 3.6 author: Victor Stinner committer: GitHub date: 2018-09-26T07:44:45-07:00 summary: Drop confusing commented out code in pystrtod.c (GH-6072) (GH-9587) Fix the following warning: Python/pystrtod.c: In function 'format_float_short': Python/pystrtod.c:1007:13: warning: 'strncpy' output truncated before terminating nul copying 3 bytes from a string of the same length [-Wstringop-truncation] strncpy(p, "ERR", 3); (cherry picked from commit 9fb84157595a385f15799e5d0729c1e1b0ba9d38) files: M Python/pystrtod.c diff --git a/Python/pystrtod.c b/Python/pystrtod.c index 58278c24be9c..1c635602340a 100644 --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -1061,8 +1061,6 @@ format_float_short(double d, char format_code, else { /* shouldn't get here: Gay's code should always return something starting with a digit, an 'I', or 'N' */ - strncpy(p, "ERR", 3); - /* p += 3; */ assert(0); } goto exit; From webhook-mailer at python.org Wed Sep 26 10:45:24 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 26 Sep 2018 14:45:24 -0000 Subject: [Python-checkins] Drop confusing commented out code in pystrtod.c (GH-6072) (GH-9586) Message-ID: https://github.com/python/cpython/commit/b54fc15e32a9faed9398ac8442728756cbbba89a commit: b54fc15e32a9faed9398ac8442728756cbbba89a branch: 3.7 author: Victor Stinner committer: GitHub date: 2018-09-26T07:45:19-07:00 summary: Drop confusing commented out code in pystrtod.c (GH-6072) (GH-9586) Fix the following warning: Python/pystrtod.c: In function 'format_float_short': Python/pystrtod.c:1007:13: warning: 'strncpy' output truncated before terminating nul copying 3 bytes from a string of the same length [-Wstringop-truncation] strncpy(p, "ERR", 3); (cherry picked from commit 9fb84157595a385f15799e5d0729c1e1b0ba9d38) files: M Python/pystrtod.c diff --git a/Python/pystrtod.c b/Python/pystrtod.c index 141a47a8999c..461e8dcb5e0c 100644 --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -1062,8 +1062,6 @@ format_float_short(double d, char format_code, else { /* shouldn't get here: Gay's code should always return something starting with a digit, an 'I', or 'N' */ - strncpy(p, "ERR", 3); - /* p += 3; */ Py_UNREACHABLE(); } goto exit; From webhook-mailer at python.org Wed Sep 26 11:13:33 2018 From: webhook-mailer at python.org (Ivan Levkivskyi) Date: Wed, 26 Sep 2018 15:13:33 -0000 Subject: [Python-checkins] Clarify that Type[SomeTypeVar] is legal (#9585) Message-ID: https://github.com/python/cpython/commit/130717fe58abb2ab9e7938207df0c130a2562747 commit: 130717fe58abb2ab9e7938207df0c130a2562747 branch: master author: Michael Lee committer: Ivan Levkivskyi date: 2018-09-26T16:13:28+01:00 summary: Clarify that Type[SomeTypeVar] is legal (#9585) Currently, the docs state that when doing `Type[X]`, X is only allowed to be a class, a union of classes, and Any. This pull request amends that sentence to clarify X may also be a typevar (or a union involving classes, Any, and TypeVars). files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 23a64156c592..268adc0c9de5 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -169,6 +169,8 @@ It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis for the list of arguments in the type hint: ``Callable[..., ReturnType]``. +.. _generics: + Generics -------- @@ -183,7 +185,7 @@ subscription to denote expected types for container elements. def notify_by_email(employees: Sequence[Employee], overrides: Mapping[str, str]) -> None: ... -Generics can be parametrized by using a new factory available in typing +Generics can be parameterized by using a new factory available in typing called :class:`TypeVar`. :: @@ -488,8 +490,9 @@ The module defines the following classes, functions and decorators: required to handle this particular case may change in future revisions of :pep:`484`. - The only legal parameters for :class:`Type` are classes, unions of classes, and - :data:`Any`. For example:: + The only legal parameters for :class:`Type` are classes, :data:`Any`, + :ref:`type variables `, and unions of any of these types. + For example:: def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... From webhook-mailer at python.org Wed Sep 26 11:18:30 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 26 Sep 2018 15:18:30 -0000 Subject: [Python-checkins] Drop confusing commented out code in pystrtod.c (GH-6072) (GH-9588) Message-ID: https://github.com/python/cpython/commit/911231e70843fb255fc395473e668361532e00a6 commit: 911231e70843fb255fc395473e668361532e00a6 branch: 2.7 author: Victor Stinner committer: GitHub date: 2018-09-26T08:18:23-07:00 summary: Drop confusing commented out code in pystrtod.c (GH-6072) (GH-9588) Fix the following warning: Python/pystrtod.c: In function 'format_float_short': Python/pystrtod.c:1007:13: warning: 'strncpy' output truncated before terminating nul copying 3 bytes from a string of the same length [-Wstringop-truncation] strncpy(p, "ERR", 3); (cherry picked from commit 9fb84157595a385f15799e5d0729c1e1b0ba9d38) files: M Python/pystrtod.c diff --git a/Python/pystrtod.c b/Python/pystrtod.c index ae6ab9c94fd7..29d79960ab92 100644 --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -1004,8 +1004,6 @@ format_float_short(double d, char format_code, else { /* shouldn't get here: Gay's code should always return something starting with a digit, an 'I', or 'N' */ - strncpy(p, "ERR", 3); - p += 3; assert(0); } goto exit; From webhook-mailer at python.org Wed Sep 26 11:21:18 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 26 Sep 2018 15:21:18 -0000 Subject: [Python-checkins] Clarify that Type[SomeTypeVar] is legal (GH-9585) Message-ID: https://github.com/python/cpython/commit/4ea64a25c207c18c3939d3899c8b7889970606c7 commit: 4ea64a25c207c18c3939d3899c8b7889970606c7 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-26T08:21:15-07:00 summary: Clarify that Type[SomeTypeVar] is legal (GH-9585) Currently, the docs state that when doing `Type[X]`, X is only allowed to be a class, a union of classes, and Any. This pull request amends that sentence to clarify X may also be a typevar (or a union involving classes, Any, and TypeVars). (cherry picked from commit 130717fe58abb2ab9e7938207df0c130a2562747) Co-authored-by: Michael Lee files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 23a64156c592..268adc0c9de5 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -169,6 +169,8 @@ It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis for the list of arguments in the type hint: ``Callable[..., ReturnType]``. +.. _generics: + Generics -------- @@ -183,7 +185,7 @@ subscription to denote expected types for container elements. def notify_by_email(employees: Sequence[Employee], overrides: Mapping[str, str]) -> None: ... -Generics can be parametrized by using a new factory available in typing +Generics can be parameterized by using a new factory available in typing called :class:`TypeVar`. :: @@ -488,8 +490,9 @@ The module defines the following classes, functions and decorators: required to handle this particular case may change in future revisions of :pep:`484`. - The only legal parameters for :class:`Type` are classes, unions of classes, and - :data:`Any`. For example:: + The only legal parameters for :class:`Type` are classes, :data:`Any`, + :ref:`type variables `, and unions of any of these types. + For example:: def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... From webhook-mailer at python.org Wed Sep 26 11:23:26 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 26 Sep 2018 15:23:26 -0000 Subject: [Python-checkins] Clarify that Type[SomeTypeVar] is legal (GH-9585) Message-ID: https://github.com/python/cpython/commit/dfa11135ce53967ac97550d0a9343d32c9958ac0 commit: dfa11135ce53967ac97550d0a9343d32c9958ac0 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-26T08:23:23-07:00 summary: Clarify that Type[SomeTypeVar] is legal (GH-9585) Currently, the docs state that when doing `Type[X]`, X is only allowed to be a class, a union of classes, and Any. This pull request amends that sentence to clarify X may also be a typevar (or a union involving classes, Any, and TypeVars). (cherry picked from commit 130717fe58abb2ab9e7938207df0c130a2562747) Co-authored-by: Michael Lee files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 91d8a5fcf788..e80cd3f2d75c 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -169,6 +169,8 @@ It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis for the list of arguments in the type hint: ``Callable[..., ReturnType]``. +.. _generics: + Generics -------- @@ -183,7 +185,7 @@ subscription to denote expected types for container elements. def notify_by_email(employees: Sequence[Employee], overrides: Mapping[str, str]) -> None: ... -Generics can be parametrized by using a new factory available in typing +Generics can be parameterized by using a new factory available in typing called :class:`TypeVar`. :: @@ -488,8 +490,9 @@ The module defines the following classes, functions and decorators: required to handle this particular case may change in future revisions of :pep:`484`. - The only legal parameters for :class:`Type` are classes, unions of classes, and - :data:`Any`. For example:: + The only legal parameters for :class:`Type` are classes, :data:`Any`, + :ref:`type variables `, and unions of any of these types. + For example:: def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... From webhook-mailer at python.org Wed Sep 26 12:09:42 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 26 Sep 2018 16:09:42 -0000 Subject: [Python-checkins] bpo-28655: Fix test_import.test_missing_source_legacy() (GH-9589) Message-ID: https://github.com/python/cpython/commit/43500a5907eb9ae2e470dcbffe73012cd456f5a1 commit: 43500a5907eb9ae2e470dcbffe73012cd456f5a1 branch: 3.6 author: Victor Stinner committer: GitHub date: 2018-09-26T09:09:32-07:00 summary: bpo-28655: Fix test_import.test_missing_source_legacy() (GH-9589) bpo-28655, bpo-33053: test_import.test_missing_source_legacy() now removes the .pyc file that it creates to avoid leaking a file. Fix extract from commit d5d9e02dd3c6df06a8dd9ce75ee9b52976420a8b. Co-Authored-By: Nick Coghlan files: M Lib/test/test_import/__init__.py diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py index b73a96f75700..6fb7cb0669cc 100644 --- a/Lib/test/test_import/__init__.py +++ b/Lib/test/test_import/__init__.py @@ -779,8 +779,11 @@ def test_missing_source_legacy(self): unload(TESTFN) importlib.invalidate_caches() m = __import__(TESTFN) - self.assertEqual(m.__file__, - os.path.join(os.curdir, os.path.relpath(pyc_file))) + try: + self.assertEqual(m.__file__, + os.path.join(os.curdir, os.path.relpath(pyc_file))) + finally: + os.remove(pyc_file) def test___cached__(self): # Modules now also have an __cached__ that points to the pyc file. From webhook-mailer at python.org Wed Sep 26 17:20:42 2018 From: webhook-mailer at python.org (Tal Einat) Date: Wed, 26 Sep 2018 21:20:42 -0000 Subject: [Python-checkins] bpo-31425: fix versionadded in docs and add attribution in NEWS (GH-9595) Message-ID: https://github.com/python/cpython/commit/f55c64c632af438d0daa043acdd95a5e74f31441 commit: f55c64c632af438d0daa043acdd95a5e74f31441 branch: master author: Tal Einat committer: GitHub date: 2018-09-27T00:20:38+03:00 summary: bpo-31425: fix versionadded in docs and add attribution in NEWS (GH-9595) files: M Doc/library/socket.rst M Misc/NEWS.d/next/Library/2017-10-24-10-18-35.bpo-31425.1lgw47.rst diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index 44370fef35c3..bfb315f1875a 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -198,7 +198,7 @@ created. Socket addresses are represented as follows: family is represented as a ``(node, port)`` tuple where the *node* and *port* are non-negative integers. - .. versionadded:: 3.7 + .. versionadded:: 3.8 If you use a hostname in the *host* portion of IPv4/v6 socket address, the program may show a nondeterministic behavior, as Python uses the first address diff --git a/Misc/NEWS.d/next/Library/2017-10-24-10-18-35.bpo-31425.1lgw47.rst b/Misc/NEWS.d/next/Library/2017-10-24-10-18-35.bpo-31425.1lgw47.rst index c5d646775b94..26294a9a4765 100644 --- a/Misc/NEWS.d/next/Library/2017-10-24-10-18-35.bpo-31425.1lgw47.rst +++ b/Misc/NEWS.d/next/Library/2017-10-24-10-18-35.bpo-31425.1lgw47.rst @@ -1,3 +1,3 @@ Add support for sockets of the AF_QIPCRTR address family, supported by the Linux kernel. This is used to communicate with services, such as GPS or -radio, running on Qualcomm devices. +radio, running on Qualcomm devices. Patch by Bjorn Andersson. \ No newline at end of file From webhook-mailer at python.org Wed Sep 26 20:00:26 2018 From: webhook-mailer at python.org (Ned Deily) Date: Thu, 27 Sep 2018 00:00:26 -0000 Subject: [Python-checkins] bpo-34370: Update Tk 8.6 used with macOS installers Message-ID: https://github.com/python/cpython/commit/adf493227f1efd5d6b34f46b854142bf3b5a411c commit: adf493227f1efd5d6b34f46b854142bf3b5a411c branch: 3.6 author: Ned Deily committer: Ned Deily date: 2018-09-26T01:13:54-04:00 summary: bpo-34370: Update Tk 8.6 used with macOS installers Have macOS 10.9+ installer builds for 3.7.1rc and 3.6.7rc use a development snapshot of Tk 8.6 (post-8.6.8) to mitigate certain scroller issues seen with IDLE and tkinter apps. files: A Misc/NEWS.d/next/macOS/2018-09-25-23-37-39.bpo-34370.FQhtAD.rst M Mac/BuildScript/build-installer.py diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index b97d55bb0306..2959bea48a18 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -239,9 +239,9 @@ def library_recipes(): }, ), dict( - name="Tk 8.6.8", - url="ftp://ftp.tcl.tk/pub/tcl//tcl8_6/tk8.6.8-src.tar.gz", - checksum='5e0faecba458ee1386078fb228d008ba', + name="Tk 8.6.8+", + url="http://core.tcl.tk/tk/tarball/16fdad9d/tk-16fdad9d.tar.gz", + checksum='b8e0df69021924e8392f03d506252bdb', patches=[ "tk868_on_10_8_10_9.patch", ], diff --git a/Misc/NEWS.d/next/macOS/2018-09-25-23-37-39.bpo-34370.FQhtAD.rst b/Misc/NEWS.d/next/macOS/2018-09-25-23-37-39.bpo-34370.FQhtAD.rst new file mode 100644 index 000000000000..b9b0c2c786f1 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2018-09-25-23-37-39.bpo-34370.FQhtAD.rst @@ -0,0 +1,3 @@ +Have macOS 10.9+ installer builds for 3.7.1rc and 3.6.7rc use a development +snapshot of Tk 8.6 (post-8.6.8) to mitigate certain scroller issues seen +with IDLE and tkinter apps. From webhook-mailer at python.org Wed Sep 26 20:09:02 2018 From: webhook-mailer at python.org (Ned Deily) Date: Thu, 27 Sep 2018 00:09:02 -0000 Subject: [Python-checkins] Post release bump Message-ID: https://github.com/python/cpython/commit/6475c053637f14a9951188c3b123c06659201aab commit: 6475c053637f14a9951188c3b123c06659201aab branch: 3.6 author: Ned Deily committer: Ned Deily date: 2018-09-26T20:08:16-04:00 summary: Post release bump files: M Include/patchlevel.h M README.rst diff --git a/Include/patchlevel.h b/Include/patchlevel.h index 45be992a21cd..e60fca5ac82e 100644 --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -23,7 +23,7 @@ #define PY_RELEASE_SERIAL 1 /* Version as a string */ -#define PY_VERSION "3.6.7rc1" +#define PY_VERSION "3.6.7rc1+" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/README.rst b/README.rst index 7917e9ee8859..94c1322cd560 100644 --- a/README.rst +++ b/README.rst @@ -1,5 +1,5 @@ -This is Python version 3.6.7 release candidate -============================================== +This is Python version 3.6.7rc1+ +================================ .. image:: https://travis-ci.org/python/cpython.svg?branch=3.6 :alt: CPython build status on Travis CI From webhook-mailer at python.org Wed Sep 26 20:12:57 2018 From: webhook-mailer at python.org (Ned Deily) Date: Thu, 27 Sep 2018 00:12:57 -0000 Subject: [Python-checkins] bpo-34370: Update Tk 8.6 used with macOS installers Message-ID: https://github.com/python/cpython/commit/d9cfe5ed2c2c61eeae915b76f5e10aadbbb28da6 commit: d9cfe5ed2c2c61eeae915b76f5e10aadbbb28da6 branch: 3.7 author: Ned Deily committer: Ned Deily date: 2018-09-26T01:06:13-04:00 summary: bpo-34370: Update Tk 8.6 used with macOS installers Have macOS 10.9+ installer builds for 3.7.1rc and 3.6.7rc use a development snapshot of Tk 8.6 (post-8.6.8) to mitigate certain scroller issues seen with IDLE and tkinter apps. files: A Misc/NEWS.d/next/macOS/2018-09-25-23-37-39.bpo-34370.FQhtAD.rst M Mac/BuildScript/build-installer.py diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index 15488780327e..7d3bed2f6457 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -242,29 +242,63 @@ def library_recipes(): "TCL_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tcl8.6'%(getVersion())), }, ), - dict( - name="Tk 8.6.8", - url="ftp://ftp.tcl.tk/pub/tcl//tcl8_6/tk8.6.8-src.tar.gz", - checksum='5e0faecba458ee1386078fb228d008ba', - patches=[ - "tk868_on_10_8_10_9.patch", - ], - buildDir="unix", - configure_pre=[ - '--enable-aqua', - '--enable-shared', - '--enable-threads', - '--libdir=/Library/Frameworks/Python.framework/Versions/%s/lib'%(getVersion(),), - ], - useLDFlags=False, - install='make TCL_LIBRARY=%(TCL_LIBRARY)s TK_LIBRARY=%(TK_LIBRARY)s && make install TCL_LIBRARY=%(TCL_LIBRARY)s TK_LIBRARY=%(TK_LIBRARY)s DESTDIR=%(DESTDIR)s'%{ - "DESTDIR": shellQuote(os.path.join(WORKDIR, 'libraries')), - "TCL_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tcl8.6'%(getVersion())), - "TK_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tk8.6'%(getVersion())), - }, - ), ]) + # temporary workaround in 3.7.1 for addressing bpo-34370: + # use development snapshot of Tk 8.6 branch (post 8.6.8) to pick up + # potential fixes for various scrolling problems seen with 8.6.8. + # However, the snapshot fails to build on 10.6. For the moment, + # continue to build the 3.7.x 10.6 variant with the standard + # 8.6.6 branch. + if getDeptargetTuple() < (10, 9): + result.extend([ + dict( + name="Tk 8.6.8", + url="ftp://ftp.tcl.tk/pub/tcl//tcl8_6/tk8.6.8-src.tar.gz", + checksum='5e0faecba458ee1386078fb228d008ba', + patches=[ + "tk868_on_10_8_10_9.patch", + ], + buildDir="unix", + configure_pre=[ + '--enable-aqua', + '--enable-shared', + '--enable-threads', + '--libdir=/Library/Frameworks/Python.framework/Versions/%s/lib'%(getVersion(),), + ], + useLDFlags=False, + install='make TCL_LIBRARY=%(TCL_LIBRARY)s TK_LIBRARY=%(TK_LIBRARY)s && make install TCL_LIBRARY=%(TCL_LIBRARY)s TK_LIBRARY=%(TK_LIBRARY)s DESTDIR=%(DESTDIR)s'%{ + "DESTDIR": shellQuote(os.path.join(WORKDIR, 'libraries')), + "TCL_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tcl8.6'%(getVersion())), + "TK_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tk8.6'%(getVersion())), + }, + ), + ]) + else: + result.extend([ + dict( + name="Tk 8.6.8+", + url="http://core.tcl.tk/tk/tarball/16fdad9d/tk-16fdad9d.tar.gz", + checksum='b8e0df69021924e8392f03d506252bdb', + patches=[ + "tk868_on_10_8_10_9.patch", + ], + buildDir="unix", + configure_pre=[ + '--enable-aqua', + '--enable-shared', + '--enable-threads', + '--libdir=/Library/Frameworks/Python.framework/Versions/%s/lib'%(getVersion(),), + ], + useLDFlags=False, + install='make TCL_LIBRARY=%(TCL_LIBRARY)s TK_LIBRARY=%(TK_LIBRARY)s && make install TCL_LIBRARY=%(TCL_LIBRARY)s TK_LIBRARY=%(TK_LIBRARY)s DESTDIR=%(DESTDIR)s'%{ + "DESTDIR": shellQuote(os.path.join(WORKDIR, 'libraries')), + "TCL_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tcl8.6'%(getVersion())), + "TK_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tk8.6'%(getVersion())), + }, + ), + ]) + if PYTHON_3: result.extend([ dict( diff --git a/Misc/NEWS.d/next/macOS/2018-09-25-23-37-39.bpo-34370.FQhtAD.rst b/Misc/NEWS.d/next/macOS/2018-09-25-23-37-39.bpo-34370.FQhtAD.rst new file mode 100644 index 000000000000..b9b0c2c786f1 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2018-09-25-23-37-39.bpo-34370.FQhtAD.rst @@ -0,0 +1,3 @@ +Have macOS 10.9+ installer builds for 3.7.1rc and 3.6.7rc use a development +snapshot of Tk 8.6 (post-8.6.8) to mitigate certain scroller issues seen +with IDLE and tkinter apps. From solipsis at pitrou.net Thu Sep 27 05:10:47 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 27 Sep 2018 09:10:47 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=4 Message-ID: <20180927091047.1.9D045882F3E01771@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_forkserver leaked [-1, -1, 2] memory blocks, sum=0 test_multiprocessing_spawn leaked [0, -2, 2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogQmiu6s', '--timeout', '7200'] From webhook-mailer at python.org Thu Sep 27 05:25:13 2018 From: webhook-mailer at python.org (Pablo Galindo) Date: Thu, 27 Sep 2018 09:25:13 -0000 Subject: [Python-checkins] Fix tests in test_socket to use correctly CMSG_LEN (GH-9594) Message-ID: https://github.com/python/cpython/commit/7291108d88ea31d205da4db19d202d6cbffc6d93 commit: 7291108d88ea31d205da4db19d202d6cbffc6d93 branch: master author: Pablo Galindo committer: GitHub date: 2018-09-27T10:25:03+01:00 summary: Fix tests in test_socket to use correctly CMSG_LEN (GH-9594) After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots regarding tests in test_socket that are using testFDPassSeparateMinSpace(), FreeBDS revision 337423 was pointed out to be the reason the test started to fail. A close examination of the manpage for cmsg_space(3) reveals that the number of file descriptors needs to be taken into account when using CMSG_LEN(). This commit fixes tests in test_socket to use correctly CMSG_LEN, taking into account the number of FDs. files: M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index bbbf27b60978..663a018dcfda 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -3223,10 +3223,11 @@ def _testFDPassSeparate(self): def testFDPassSeparateMinSpace(self): # Pass two FDs in two separate arrays, receiving them into the # minimum space for two arrays. - self.checkRecvmsgFDs(2, + num_fds = 2 + self.checkRecvmsgFDs(num_fds, self.doRecvmsg(self.serv_sock, len(MSG), socket.CMSG_SPACE(SIZEOF_INT) + - socket.CMSG_LEN(SIZEOF_INT)), + socket.CMSG_LEN(SIZEOF_INT * num_fds)), maxcmsgs=2, ignoreflags=socket.MSG_CTRUNC) @testFDPassSeparateMinSpace.client_skip From webhook-mailer at python.org Thu Sep 27 07:16:31 2018 From: webhook-mailer at python.org (Antoine Pitrou) Date: Thu, 27 Sep 2018 11:16:31 -0000 Subject: [Python-checkins] bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures (GH-9599) Message-ID: https://github.com/python/cpython/commit/a94ee12c26aa8dd7dce01373779df8055aff765b commit: a94ee12c26aa8dd7dce01373779df8055aff765b branch: master author: orlnub123 committer: Antoine Pitrou date: 2018-09-27T13:16:26+02:00 summary: bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures (GH-9599) Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted. This may not be sufficient on all systems. On POSIX for example, the actual waiting (e.g. in ``sem_timedwait``) is specified to rely on the CLOCK_REALTIME clock. files: A Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst M Lib/concurrent/futures/_base.py diff --git a/Lib/concurrent/futures/_base.py b/Lib/concurrent/futures/_base.py index d4416c62450e..8b9dc507138e 100644 --- a/Lib/concurrent/futures/_base.py +++ b/Lib/concurrent/futures/_base.py @@ -216,7 +216,7 @@ def as_completed(fs, timeout=None): before the given timeout. """ if timeout is not None: - end_time = timeout + time.time() + end_time = timeout + time.monotonic() fs = set(fs) total_futures = len(fs) @@ -235,7 +235,7 @@ def as_completed(fs, timeout=None): if timeout is None: wait_timeout = None else: - wait_timeout = end_time - time.time() + wait_timeout = end_time - time.monotonic() if wait_timeout < 0: raise TimeoutError( '%d (of %d) futures unfinished' % ( @@ -578,7 +578,7 @@ def map(self, fn, *iterables, timeout=None, chunksize=1): Exception: If fn(*args) raises for any values. """ if timeout is not None: - end_time = timeout + time.time() + end_time = timeout + time.monotonic() fs = [self.submit(fn, *args) for args in zip(*iterables)] @@ -593,7 +593,7 @@ def result_iterator(): if timeout is None: yield fs.pop().result() else: - yield fs.pop().result(end_time - time.time()) + yield fs.pop().result(end_time - time.monotonic()) finally: for future in fs: future.cancel() diff --git a/Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst b/Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst new file mode 100644 index 000000000000..6bbdea235f03 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst @@ -0,0 +1 @@ +Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted. From webhook-mailer at python.org Thu Sep 27 07:37:37 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 27 Sep 2018 11:37:37 -0000 Subject: [Python-checkins] bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures (GH-9599) Message-ID: https://github.com/python/cpython/commit/3a4aa6ac55e04c42757443d5b5854b6d893e0461 commit: 3a4aa6ac55e04c42757443d5b5854b6d893e0461 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-27T04:37:33-07:00 summary: bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures (GH-9599) Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted. This may not be sufficient on all systems. On POSIX for example, the actual waiting (e.g. in ``sem_timedwait``) is specified to rely on the CLOCK_REALTIME clock. (cherry picked from commit a94ee12c26aa8dd7dce01373779df8055aff765b) Co-authored-by: orlnub123 files: A Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst M Lib/concurrent/futures/_base.py diff --git a/Lib/concurrent/futures/_base.py b/Lib/concurrent/futures/_base.py index 6bace6c74641..4c140508c9ea 100644 --- a/Lib/concurrent/futures/_base.py +++ b/Lib/concurrent/futures/_base.py @@ -212,7 +212,7 @@ def as_completed(fs, timeout=None): before the given timeout. """ if timeout is not None: - end_time = timeout + time.time() + end_time = timeout + time.monotonic() fs = set(fs) total_futures = len(fs) @@ -231,7 +231,7 @@ def as_completed(fs, timeout=None): if timeout is None: wait_timeout = None else: - wait_timeout = end_time - time.time() + wait_timeout = end_time - time.monotonic() if wait_timeout < 0: raise TimeoutError( '%d (of %d) futures unfinished' % ( @@ -570,7 +570,7 @@ def map(self, fn, *iterables, timeout=None, chunksize=1): Exception: If fn(*args) raises for any values. """ if timeout is not None: - end_time = timeout + time.time() + end_time = timeout + time.monotonic() fs = [self.submit(fn, *args) for args in zip(*iterables)] @@ -585,7 +585,7 @@ def result_iterator(): if timeout is None: yield fs.pop().result() else: - yield fs.pop().result(end_time - time.time()) + yield fs.pop().result(end_time - time.monotonic()) finally: for future in fs: future.cancel() diff --git a/Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst b/Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst new file mode 100644 index 000000000000..6bbdea235f03 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst @@ -0,0 +1 @@ +Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted. From webhook-mailer at python.org Thu Sep 27 07:46:41 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 27 Sep 2018 11:46:41 -0000 Subject: [Python-checkins] bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures (GH-9599) Message-ID: https://github.com/python/cpython/commit/2b01121fd4200f1c27873422f7f72d02eec08630 commit: 2b01121fd4200f1c27873422f7f72d02eec08630 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-27T04:46:38-07:00 summary: bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures (GH-9599) Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted. This may not be sufficient on all systems. On POSIX for example, the actual waiting (e.g. in ``sem_timedwait``) is specified to rely on the CLOCK_REALTIME clock. (cherry picked from commit a94ee12c26aa8dd7dce01373779df8055aff765b) Co-authored-by: orlnub123 files: A Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst M Lib/concurrent/futures/_base.py diff --git a/Lib/concurrent/futures/_base.py b/Lib/concurrent/futures/_base.py index 4f22f7ee0e6d..61c81bb7fe68 100644 --- a/Lib/concurrent/futures/_base.py +++ b/Lib/concurrent/futures/_base.py @@ -212,7 +212,7 @@ def as_completed(fs, timeout=None): before the given timeout. """ if timeout is not None: - end_time = timeout + time.time() + end_time = timeout + time.monotonic() fs = set(fs) total_futures = len(fs) @@ -231,7 +231,7 @@ def as_completed(fs, timeout=None): if timeout is None: wait_timeout = None else: - wait_timeout = end_time - time.time() + wait_timeout = end_time - time.monotonic() if wait_timeout < 0: raise TimeoutError( '%d (of %d) futures unfinished' % ( @@ -570,7 +570,7 @@ def map(self, fn, *iterables, timeout=None, chunksize=1): Exception: If fn(*args) raises for any values. """ if timeout is not None: - end_time = timeout + time.time() + end_time = timeout + time.monotonic() fs = [self.submit(fn, *args) for args in zip(*iterables)] @@ -585,7 +585,7 @@ def result_iterator(): if timeout is None: yield fs.pop().result() else: - yield fs.pop().result(end_time - time.time()) + yield fs.pop().result(end_time - time.monotonic()) finally: for future in fs: future.cancel() diff --git a/Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst b/Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst new file mode 100644 index 000000000000..6bbdea235f03 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-27-09-45-00.bpo-34819.9ZaFyO.rst @@ -0,0 +1 @@ +Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted. From webhook-mailer at python.org Thu Sep 27 09:30:59 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 27 Sep 2018 13:30:59 -0000 Subject: [Python-checkins] Fix tests in test_socket to use correctly CMSG_LEN (GH-9594) Message-ID: https://github.com/python/cpython/commit/fe48b6df101aac10dc846fa6fd1a41f877e77025 commit: fe48b6df101aac10dc846fa6fd1a41f877e77025 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-27T06:30:55-07:00 summary: Fix tests in test_socket to use correctly CMSG_LEN (GH-9594) After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots regarding tests in test_socket that are using testFDPassSeparateMinSpace(), FreeBDS revision 337423 was pointed out to be the reason the test started to fail. A close examination of the manpage for cmsg_space(3) reveals that the number of file descriptors needs to be taken into account when using CMSG_LEN(). This commit fixes tests in test_socket to use correctly CMSG_LEN, taking into account the number of FDs. (cherry picked from commit 7291108d88ea31d205da4db19d202d6cbffc6d93) Co-authored-by: Pablo Galindo files: M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index f8559ac4eab9..fbbc9f9abfb0 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -2858,10 +2858,11 @@ def _testFDPassSeparate(self): def testFDPassSeparateMinSpace(self): # Pass two FDs in two separate arrays, receiving them into the # minimum space for two arrays. - self.checkRecvmsgFDs(2, + num_fds = 2 + self.checkRecvmsgFDs(num_fds, self.doRecvmsg(self.serv_sock, len(MSG), socket.CMSG_SPACE(SIZEOF_INT) + - socket.CMSG_LEN(SIZEOF_INT)), + socket.CMSG_LEN(SIZEOF_INT * num_fds)), maxcmsgs=2, ignoreflags=socket.MSG_CTRUNC) @testFDPassSeparateMinSpace.client_skip From webhook-mailer at python.org Thu Sep 27 09:30:59 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 27 Sep 2018 13:30:59 -0000 Subject: [Python-checkins] Fix tests in test_socket to use correctly CMSG_LEN (GH-9594) Message-ID: https://github.com/python/cpython/commit/addef07ca7d7b6971d59c062c3229e91a99e5f5e commit: addef07ca7d7b6971d59c062c3229e91a99e5f5e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-27T06:30:47-07:00 summary: Fix tests in test_socket to use correctly CMSG_LEN (GH-9594) After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots regarding tests in test_socket that are using testFDPassSeparateMinSpace(), FreeBDS revision 337423 was pointed out to be the reason the test started to fail. A close examination of the manpage for cmsg_space(3) reveals that the number of file descriptors needs to be taken into account when using CMSG_LEN(). This commit fixes tests in test_socket to use correctly CMSG_LEN, taking into account the number of FDs. (cherry picked from commit 7291108d88ea31d205da4db19d202d6cbffc6d93) Co-authored-by: Pablo Galindo files: M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index bd4fad1f6380..5f9891de4936 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -3183,10 +3183,11 @@ def _testFDPassSeparate(self): def testFDPassSeparateMinSpace(self): # Pass two FDs in two separate arrays, receiving them into the # minimum space for two arrays. - self.checkRecvmsgFDs(2, + num_fds = 2 + self.checkRecvmsgFDs(num_fds, self.doRecvmsg(self.serv_sock, len(MSG), socket.CMSG_SPACE(SIZEOF_INT) + - socket.CMSG_LEN(SIZEOF_INT)), + socket.CMSG_LEN(SIZEOF_INT * num_fds)), maxcmsgs=2, ignoreflags=socket.MSG_CTRUNC) @testFDPassSeparateMinSpace.client_skip From webhook-mailer at python.org Thu Sep 27 10:42:43 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Thu, 27 Sep 2018 14:42:43 -0000 Subject: [Python-checkins] bpo-32892: Use ast.Constant instead of specific constant AST types. (GH-9445) Message-ID: https://github.com/python/cpython/commit/3f22811fef73aec848d961593d95fa877f77ecbf commit: 3f22811fef73aec848d961593d95fa877f77ecbf branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-27T17:42:37+03:00 summary: bpo-32892: Use ast.Constant instead of specific constant AST types. (GH-9445) files: A Misc/NEWS.d/next/Library/2018-09-20-17-35-05.bpo-32892.TOUBdg.rst M Doc/library/ast.rst M Doc/whatsnew/3.8.rst M Include/Python-ast.h M Lib/ast.py M Lib/inspect.py M Lib/test/test_ast.py M Lib/test/test_fstring.py M Lib/test/test_future.py M Lib/test/test_pyclbr.py M Parser/Python.asdl M Parser/asdl_c.py M Python/Python-ast.c M Python/ast.c M Python/ast_opt.c M Python/ast_unparse.c M Python/compile.c M Python/symtable.c M Tools/clinic/clinic.py M Tools/parser/unparse.py diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 9ff422cdf503..97ce2f5281b5 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -78,8 +78,8 @@ Node classes node = ast.UnaryOp() node.op = ast.USub() - node.operand = ast.Num() - node.operand.n = 5 + node.operand = ast.Constant() + node.operand.value = 5 node.operand.lineno = 0 node.operand.col_offset = 0 node.lineno = 0 @@ -87,9 +87,16 @@ Node classes or the more compact :: - node = ast.UnaryOp(ast.USub(), ast.Num(5, lineno=0, col_offset=0), + node = ast.UnaryOp(ast.USub(), ast.Constant(5, lineno=0, col_offset=0), lineno=0, col_offset=0) +.. deprecated:: 3.8 + + Class :class:`ast.Constant` is now used for all constants. Old classes + :class:`ast.Num`, :class:`ast.Str`, :class:`ast.Bytes`, + :class:`ast.NameConstant` and :class:`ast.Ellipsis` are still available, + but they will be removed in future Python releases. + .. _abstract-grammar: @@ -239,7 +246,7 @@ and classes for traversing abstract syntax trees: def visit_Name(self, node): return copy_location(Subscript( value=Name(id='data', ctx=Load()), - slice=Index(value=Str(s=node.id)), + slice=Index(value=Constant(value=node.id)), ctx=node.ctx ), node) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 7fbe98105d12..a146249178f4 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -262,6 +262,11 @@ Deprecated (Contributed by Berker Peksag in :issue:`9372`.) +* :mod:`ast` classes ``Num``, ``Str``, ``Bytes``, ``NameConstant`` and + ``Ellipsis`` are considered deprecated and will be removed in future Python + versions. :class:`~ast.Constant` should be used instead. + (Contributed by Serhiy Storchaka in :issue:`32892`.) + Removed ======= diff --git a/Include/Python-ast.h b/Include/Python-ast.h index 8e0f750a8250..2913d1d2d0a9 100644 --- a/Include/Python-ast.h +++ b/Include/Python-ast.h @@ -208,11 +208,10 @@ enum _expr_kind {BoolOp_kind=1, BinOp_kind=2, UnaryOp_kind=3, Lambda_kind=4, IfExp_kind=5, Dict_kind=6, Set_kind=7, ListComp_kind=8, SetComp_kind=9, DictComp_kind=10, GeneratorExp_kind=11, Await_kind=12, Yield_kind=13, YieldFrom_kind=14, - Compare_kind=15, Call_kind=16, Num_kind=17, Str_kind=18, - FormattedValue_kind=19, JoinedStr_kind=20, Bytes_kind=21, - NameConstant_kind=22, Ellipsis_kind=23, Constant_kind=24, - Attribute_kind=25, Subscript_kind=26, Starred_kind=27, - Name_kind=28, List_kind=29, Tuple_kind=30}; + Compare_kind=15, Call_kind=16, FormattedValue_kind=17, + JoinedStr_kind=18, Constant_kind=19, Attribute_kind=20, + Subscript_kind=21, Starred_kind=22, Name_kind=23, + List_kind=24, Tuple_kind=25}; struct _expr { enum _expr_kind kind; union { @@ -297,14 +296,6 @@ struct _expr { asdl_seq *keywords; } Call; - struct { - object n; - } Num; - - struct { - string s; - } Str; - struct { expr_ty value; int conversion; @@ -315,14 +306,6 @@ struct _expr { asdl_seq *values; } JoinedStr; - struct { - bytes s; - } Bytes; - - struct { - singleton value; - } NameConstant; - struct { constant value; } Constant; @@ -566,23 +549,12 @@ expr_ty _Py_Compare(expr_ty left, asdl_int_seq * ops, asdl_seq * comparators, #define Call(a0, a1, a2, a3, a4, a5) _Py_Call(a0, a1, a2, a3, a4, a5) expr_ty _Py_Call(expr_ty func, asdl_seq * args, asdl_seq * keywords, int lineno, int col_offset, PyArena *arena); -#define Num(a0, a1, a2, a3) _Py_Num(a0, a1, a2, a3) -expr_ty _Py_Num(object n, int lineno, int col_offset, PyArena *arena); -#define Str(a0, a1, a2, a3) _Py_Str(a0, a1, a2, a3) -expr_ty _Py_Str(string s, int lineno, int col_offset, PyArena *arena); #define FormattedValue(a0, a1, a2, a3, a4, a5) _Py_FormattedValue(a0, a1, a2, a3, a4, a5) expr_ty _Py_FormattedValue(expr_ty value, int conversion, expr_ty format_spec, int lineno, int col_offset, PyArena *arena); #define JoinedStr(a0, a1, a2, a3) _Py_JoinedStr(a0, a1, a2, a3) expr_ty _Py_JoinedStr(asdl_seq * values, int lineno, int col_offset, PyArena *arena); -#define Bytes(a0, a1, a2, a3) _Py_Bytes(a0, a1, a2, a3) -expr_ty _Py_Bytes(bytes s, int lineno, int col_offset, PyArena *arena); -#define NameConstant(a0, a1, a2, a3) _Py_NameConstant(a0, a1, a2, a3) -expr_ty _Py_NameConstant(singleton value, int lineno, int col_offset, PyArena - *arena); -#define Ellipsis(a0, a1, a2) _Py_Ellipsis(a0, a1, a2) -expr_ty _Py_Ellipsis(int lineno, int col_offset, PyArena *arena); #define Constant(a0, a1, a2, a3) _Py_Constant(a0, a1, a2, a3) expr_ty _Py_Constant(constant value, int lineno, int col_offset, PyArena *arena); diff --git a/Lib/ast.py b/Lib/ast.py index bfe346bba8e3..de3df1473e5e 100644 --- a/Lib/ast.py +++ b/Lib/ast.py @@ -48,10 +48,8 @@ def literal_eval(node_or_string): node_or_string = node_or_string.body def _convert_num(node): if isinstance(node, Constant): - if isinstance(node.value, (int, float, complex)): + if type(node.value) in (int, float, complex): return node.value - elif isinstance(node, Num): - return node.n raise ValueError('malformed node or string: ' + repr(node)) def _convert_signed_num(node): if isinstance(node, UnaryOp) and isinstance(node.op, (UAdd, USub)): @@ -64,10 +62,6 @@ def _convert_signed_num(node): def _convert(node): if isinstance(node, Constant): return node.value - elif isinstance(node, (Str, Bytes)): - return node.s - elif isinstance(node, Num): - return node.n elif isinstance(node, Tuple): return tuple(map(_convert, node.elts)) elif isinstance(node, List): @@ -77,8 +71,6 @@ def _convert(node): elif isinstance(node, Dict): return dict(zip(map(_convert, node.keys), map(_convert, node.values))) - elif isinstance(node, NameConstant): - return node.value elif isinstance(node, BinOp) and isinstance(node.op, (Add, Sub)): left = _convert_signed_num(node.left) right = _convert_num(node.right) @@ -329,3 +321,66 @@ def generic_visit(self, node): else: setattr(node, field, new_node) return node + + +# The following code is for backward compatibility. +# It will be removed in future. + +def _getter(self): + return self.value + +def _setter(self, value): + self.value = value + +Constant.n = property(_getter, _setter) +Constant.s = property(_getter, _setter) + +class _ABC(type): + + def __instancecheck__(cls, inst): + if not isinstance(inst, Constant): + return False + if cls in _const_types: + try: + value = inst.value + except AttributeError: + return False + else: + return type(value) in _const_types[cls] + return type.__instancecheck__(cls, inst) + +def _new(cls, *args, **kwargs): + if cls in _const_types: + return Constant(*args, **kwargs) + return Constant.__new__(cls, *args, **kwargs) + +class Num(Constant, metaclass=_ABC): + _fields = ('n',) + __new__ = _new + +class Str(Constant, metaclass=_ABC): + _fields = ('s',) + __new__ = _new + +class Bytes(Constant, metaclass=_ABC): + _fields = ('s',) + __new__ = _new + +class NameConstant(Constant, metaclass=_ABC): + __new__ = _new + +class Ellipsis(Constant, metaclass=_ABC): + _fields = () + + def __new__(cls, *args, **kwargs): + if cls is Ellipsis: + return Constant(..., *args, **kwargs) + return Constant.__new__(cls, *args, **kwargs) + +_const_types = { + Num: (int, float, complex), + Str: (str,), + Bytes: (bytes,), + NameConstant: (type(None), bool), + Ellipsis: (type(...),), +} diff --git a/Lib/inspect.py b/Lib/inspect.py index e799a83a7404..5b7f526939b6 100644 --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -2005,14 +2005,8 @@ def wrap_value(s): except NameError: raise RuntimeError() - if isinstance(value, str): - return ast.Str(value) - if isinstance(value, (int, float)): - return ast.Num(value) - if isinstance(value, bytes): - return ast.Bytes(value) - if value in (True, False, None): - return ast.NameConstant(value) + if isinstance(value, (str, int, float, bytes, bool, type(None))): + return ast.Constant(value) raise RuntimeError() class RewriteSymbolics(ast.NodeTransformer): diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py index 72f8467847e8..10be02eee0c0 100644 --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -300,12 +300,16 @@ def test_field_attr_writable(self): def test_classattrs(self): x = ast.Num() - self.assertEqual(x._fields, ('n',)) + self.assertEqual(x._fields, ('value',)) + + with self.assertRaises(AttributeError): + x.value with self.assertRaises(AttributeError): x.n x = ast.Num(42) + self.assertEqual(x.value, 42) self.assertEqual(x.n, 42) with self.assertRaises(AttributeError): @@ -319,12 +323,102 @@ def test_classattrs(self): x = ast.Num(42, lineno=0) self.assertEqual(x.lineno, 0) - self.assertEqual(x._fields, ('n',)) + self.assertEqual(x._fields, ('value',)) + self.assertEqual(x.value, 42) self.assertEqual(x.n, 42) self.assertRaises(TypeError, ast.Num, 1, 2) self.assertRaises(TypeError, ast.Num, 1, 2, lineno=0) + self.assertEqual(ast.Num(42).n, 42) + self.assertEqual(ast.Num(4.25).n, 4.25) + self.assertEqual(ast.Num(4.25j).n, 4.25j) + self.assertEqual(ast.Str('42').s, '42') + self.assertEqual(ast.Bytes(b'42').s, b'42') + self.assertIs(ast.NameConstant(True).value, True) + self.assertIs(ast.NameConstant(False).value, False) + self.assertIs(ast.NameConstant(None).value, None) + + self.assertEqual(ast.Constant(42).value, 42) + self.assertEqual(ast.Constant(4.25).value, 4.25) + self.assertEqual(ast.Constant(4.25j).value, 4.25j) + self.assertEqual(ast.Constant('42').value, '42') + self.assertEqual(ast.Constant(b'42').value, b'42') + self.assertIs(ast.Constant(True).value, True) + self.assertIs(ast.Constant(False).value, False) + self.assertIs(ast.Constant(None).value, None) + self.assertIs(ast.Constant(...).value, ...) + + def test_realtype(self): + self.assertEqual(type(ast.Num(42)), ast.Constant) + self.assertEqual(type(ast.Num(4.25)), ast.Constant) + self.assertEqual(type(ast.Num(4.25j)), ast.Constant) + self.assertEqual(type(ast.Str('42')), ast.Constant) + self.assertEqual(type(ast.Bytes(b'42')), ast.Constant) + self.assertEqual(type(ast.NameConstant(True)), ast.Constant) + self.assertEqual(type(ast.NameConstant(False)), ast.Constant) + self.assertEqual(type(ast.NameConstant(None)), ast.Constant) + self.assertEqual(type(ast.Ellipsis()), ast.Constant) + + def test_isinstance(self): + self.assertTrue(isinstance(ast.Num(42), ast.Num)) + self.assertTrue(isinstance(ast.Num(4.2), ast.Num)) + self.assertTrue(isinstance(ast.Num(4.2j), ast.Num)) + self.assertTrue(isinstance(ast.Str('42'), ast.Str)) + self.assertTrue(isinstance(ast.Bytes(b'42'), ast.Bytes)) + self.assertTrue(isinstance(ast.NameConstant(True), ast.NameConstant)) + self.assertTrue(isinstance(ast.NameConstant(False), ast.NameConstant)) + self.assertTrue(isinstance(ast.NameConstant(None), ast.NameConstant)) + self.assertTrue(isinstance(ast.Ellipsis(), ast.Ellipsis)) + + self.assertTrue(isinstance(ast.Constant(42), ast.Num)) + self.assertTrue(isinstance(ast.Constant(4.2), ast.Num)) + self.assertTrue(isinstance(ast.Constant(4.2j), ast.Num)) + self.assertTrue(isinstance(ast.Constant('42'), ast.Str)) + self.assertTrue(isinstance(ast.Constant(b'42'), ast.Bytes)) + self.assertTrue(isinstance(ast.Constant(True), ast.NameConstant)) + self.assertTrue(isinstance(ast.Constant(False), ast.NameConstant)) + self.assertTrue(isinstance(ast.Constant(None), ast.NameConstant)) + self.assertTrue(isinstance(ast.Constant(...), ast.Ellipsis)) + + self.assertFalse(isinstance(ast.Str('42'), ast.Num)) + self.assertFalse(isinstance(ast.Num(42), ast.Str)) + self.assertFalse(isinstance(ast.Str('42'), ast.Bytes)) + self.assertFalse(isinstance(ast.Num(42), ast.NameConstant)) + self.assertFalse(isinstance(ast.Num(42), ast.Ellipsis)) + + self.assertFalse(isinstance(ast.Constant('42'), ast.Num)) + self.assertFalse(isinstance(ast.Constant(42), ast.Str)) + self.assertFalse(isinstance(ast.Constant('42'), ast.Bytes)) + self.assertFalse(isinstance(ast.Constant(42), ast.NameConstant)) + self.assertFalse(isinstance(ast.Constant(42), ast.Ellipsis)) + + self.assertFalse(isinstance(ast.Constant(), ast.Num)) + self.assertFalse(isinstance(ast.Constant(), ast.Str)) + self.assertFalse(isinstance(ast.Constant(), ast.Bytes)) + self.assertFalse(isinstance(ast.Constant(), ast.NameConstant)) + self.assertFalse(isinstance(ast.Constant(), ast.Ellipsis)) + + def test_subclasses(self): + class N(ast.Num): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.z = 'spam' + class N2(ast.Num): + pass + + n = N(42) + self.assertEqual(n.n, 42) + self.assertEqual(n.z, 'spam') + self.assertEqual(type(n), N) + self.assertTrue(isinstance(n, N)) + self.assertTrue(isinstance(n, ast.Num)) + self.assertFalse(isinstance(n, N2)) + self.assertFalse(isinstance(ast.Num(42), N)) + n = N(n=42) + self.assertEqual(n.n, 42) + self.assertEqual(type(n), N) + def test_module(self): body = [ast.Num(42)] x = ast.Module(body) @@ -446,17 +540,17 @@ def test_dump(self): node = ast.parse('spam(eggs, "and cheese")') self.assertEqual(ast.dump(node), "Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load()), " - "args=[Name(id='eggs', ctx=Load()), Str(s='and cheese')], " + "args=[Name(id='eggs', ctx=Load()), Constant(value='and cheese')], " "keywords=[]))])" ) self.assertEqual(ast.dump(node, annotate_fields=False), "Module([Expr(Call(Name('spam', Load()), [Name('eggs', Load()), " - "Str('and cheese')], []))])" + "Constant('and cheese')], []))])" ) self.assertEqual(ast.dump(node, include_attributes=True), "Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load(), " "lineno=1, col_offset=0), args=[Name(id='eggs', ctx=Load(), " - "lineno=1, col_offset=5), Str(s='and cheese', lineno=1, " + "lineno=1, col_offset=5), Constant(value='and cheese', lineno=1, " "col_offset=11)], keywords=[], " "lineno=1, col_offset=0), lineno=1, col_offset=0)])" ) @@ -465,8 +559,8 @@ def test_copy_location(self): src = ast.parse('1 + 1', mode='eval') src.body.right = ast.copy_location(ast.Num(2), src.body.right) self.assertEqual(ast.dump(src, include_attributes=True), - 'Expression(body=BinOp(left=Num(n=1, lineno=1, col_offset=0), ' - 'op=Add(), right=Num(n=2, lineno=1, col_offset=4), lineno=1, ' + 'Expression(body=BinOp(left=Constant(value=1, lineno=1, col_offset=0), ' + 'op=Add(), right=Constant(value=2, lineno=1, col_offset=4), lineno=1, ' 'col_offset=0))' ) @@ -477,11 +571,11 @@ def test_fix_missing_locations(self): self.assertEqual(src, ast.fix_missing_locations(src)) self.assertEqual(ast.dump(src, include_attributes=True), "Module(body=[Expr(value=Call(func=Name(id='write', ctx=Load(), " - "lineno=1, col_offset=0), args=[Str(s='spam', lineno=1, " + "lineno=1, col_offset=0), args=[Constant(value='spam', lineno=1, " "col_offset=6)], keywords=[], " "lineno=1, col_offset=0), lineno=1, col_offset=0), " "Expr(value=Call(func=Name(id='spam', ctx=Load(), lineno=1, " - "col_offset=0), args=[Str(s='eggs', lineno=1, col_offset=0)], " + "col_offset=0), args=[Constant(value='eggs', lineno=1, col_offset=0)], " "keywords=[], lineno=1, " "col_offset=0), lineno=1, col_offset=0)])" ) @@ -490,16 +584,16 @@ def test_increment_lineno(self): src = ast.parse('1 + 1', mode='eval') self.assertEqual(ast.increment_lineno(src, n=3), src) self.assertEqual(ast.dump(src, include_attributes=True), - 'Expression(body=BinOp(left=Num(n=1, lineno=4, col_offset=0), ' - 'op=Add(), right=Num(n=1, lineno=4, col_offset=4), lineno=4, ' + 'Expression(body=BinOp(left=Constant(value=1, lineno=4, col_offset=0), ' + 'op=Add(), right=Constant(value=1, lineno=4, col_offset=4), lineno=4, ' 'col_offset=0))' ) # issue10869: do not increment lineno of root twice src = ast.parse('1 + 1', mode='eval') self.assertEqual(ast.increment_lineno(src.body, n=3), src.body) self.assertEqual(ast.dump(src, include_attributes=True), - 'Expression(body=BinOp(left=Num(n=1, lineno=4, col_offset=0), ' - 'op=Add(), right=Num(n=1, lineno=4, col_offset=4), lineno=4, ' + 'Expression(body=BinOp(left=Constant(value=1, lineno=4, col_offset=0), ' + 'op=Add(), right=Constant(value=1, lineno=4, col_offset=4), lineno=4, ' 'col_offset=0))' ) @@ -514,10 +608,10 @@ def test_iter_child_nodes(self): self.assertEqual(len(list(ast.iter_child_nodes(node.body))), 4) iterator = ast.iter_child_nodes(node.body) self.assertEqual(next(iterator).id, 'spam') - self.assertEqual(next(iterator).n, 23) - self.assertEqual(next(iterator).n, 42) + self.assertEqual(next(iterator).value, 23) + self.assertEqual(next(iterator).value, 42) self.assertEqual(ast.dump(next(iterator)), - "keyword(arg='eggs', value=Str(s='leek'))" + "keyword(arg='eggs', value=Constant(value='leek'))" ) def test_get_docstring(self): @@ -627,9 +721,11 @@ class ASTValidatorTests(unittest.TestCase): def mod(self, mod, msg=None, mode="exec", *, exc=ValueError): mod.lineno = mod.col_offset = 0 ast.fix_missing_locations(mod) - with self.assertRaises(exc) as cm: + if msg is None: compile(mod, "", mode) - if msg is not None: + else: + with self.assertRaises(exc) as cm: + compile(mod, "", mode) self.assertIn(msg, str(cm.exception)) def expr(self, node, msg=None, *, exc=ValueError): @@ -927,9 +1023,9 @@ def test_compare(self): comp = ast.Compare(left, [ast.In()], [ast.Num(4), ast.Num(5)]) self.expr(comp, "different number of comparators and operands") comp = ast.Compare(ast.Num("blah"), [ast.In()], [left]) - self.expr(comp, "non-numeric", exc=TypeError) + self.expr(comp) comp = ast.Compare(left, [ast.In()], [ast.Num("blah")]) - self.expr(comp, "non-numeric", exc=TypeError) + self.expr(comp) def test_call(self): func = ast.Name("x", ast.Load()) @@ -950,8 +1046,10 @@ class subfloat(float): pass class subcomplex(complex): pass - for obj in "0", "hello", subint(), subfloat(), subcomplex(): - self.expr(ast.Num(obj), "non-numeric", exc=TypeError) + for obj in "0", "hello": + self.expr(ast.Num(obj)) + for obj in subint(), subfloat(), subcomplex(): + self.expr(ast.Num(obj), "invalid type", exc=TypeError) def test_attribute(self): attr = ast.Attribute(ast.Name("x", ast.Store()), "y", ast.Load()) @@ -993,7 +1091,7 @@ def test_tuple(self): self._sequence(ast.Tuple) def test_nameconstant(self): - self.expr(ast.NameConstant(4), "singleton must be True, False, or None") + self.expr(ast.NameConstant(4)) def test_stdlib_validates(self): stdlib = os.path.dirname(ast.__file__) @@ -1140,35 +1238,35 @@ def main(): #### EVERYTHING BELOW IS GENERATED ##### exec_results = [ -('Module', [('Expr', (1, 0), ('NameConstant', (1, 0), None))]), -('Module', [('Expr', (1, 0), ('Str', (1, 0), 'module docstring'))]), +('Module', [('Expr', (1, 0), ('Constant', (1, 0), None))]), +('Module', [('Expr', (1, 0), ('Constant', (1, 0), 'module docstring'))]), ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Pass', (1, 9))], [], None)]), -('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (1, 9), ('Str', (1, 9), 'function docstring'))], [], None)]), +('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (1, 9), ('Constant', (1, 9), 'function docstring'))], [], None)]), ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6), 'a', None)], None, [], [], None, []), [('Pass', (1, 10))], [], None)]), -('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6), 'a', None)], None, [], [], None, [('Num', (1, 8), 0)]), [('Pass', (1, 12))], [], None)]), +('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6), 'a', None)], None, [], [], None, [('Constant', (1, 8), 0)]), [('Pass', (1, 12))], [], None)]), ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], ('arg', (1, 7), 'args', None), [], [], None, []), [('Pass', (1, 14))], [], None)]), ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], ('arg', (1, 8), 'kwargs', None), []), [('Pass', (1, 17))], [], None)]), -('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6), 'a', None), ('arg', (1, 9), 'b', None), ('arg', (1, 14), 'c', None), ('arg', (1, 22), 'd', None), ('arg', (1, 28), 'e', None)], ('arg', (1, 35), 'args', None), [('arg', (1, 41), 'f', None)], [('Num', (1, 43), 42)], ('arg', (1, 49), 'kwargs', None), [('Num', (1, 11), 1), ('NameConstant', (1, 16), None), ('List', (1, 24), [], ('Load',)), ('Dict', (1, 30), [], [])]), [('Expr', (1, 58), ('Str', (1, 58), 'doc for f()'))], [], None)]), +('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6), 'a', None), ('arg', (1, 9), 'b', None), ('arg', (1, 14), 'c', None), ('arg', (1, 22), 'd', None), ('arg', (1, 28), 'e', None)], ('arg', (1, 35), 'args', None), [('arg', (1, 41), 'f', None)], [('Constant', (1, 43), 42)], ('arg', (1, 49), 'kwargs', None), [('Constant', (1, 11), 1), ('Constant', (1, 16), None), ('List', (1, 24), [], ('Load',)), ('Dict', (1, 30), [], [])]), [('Expr', (1, 58), ('Constant', (1, 58), 'doc for f()'))], [], None)]), ('Module', [('ClassDef', (1, 0), 'C', [], [], [('Pass', (1, 8))], [])]), -('Module', [('ClassDef', (1, 0), 'C', [], [], [('Expr', (1, 9), ('Str', (1, 9), 'docstring for class C'))], [])]), +('Module', [('ClassDef', (1, 0), 'C', [], [], [('Expr', (1, 9), ('Constant', (1, 9), 'docstring for class C'))], [])]), ('Module', [('ClassDef', (1, 0), 'C', [('Name', (1, 8), 'object', ('Load',))], [], [('Pass', (1, 17))], [])]), -('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Return', (1, 8), ('Num', (1, 15), 1))], [], None)]), +('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Return', (1, 8), ('Constant', (1, 15), 1))], [], None)]), ('Module', [('Delete', (1, 0), [('Name', (1, 4), 'v', ('Del',))])]), -('Module', [('Assign', (1, 0), [('Name', (1, 0), 'v', ('Store',))], ('Num', (1, 4), 1))]), -('Module', [('AugAssign', (1, 0), ('Name', (1, 0), 'v', ('Store',)), ('Add',), ('Num', (1, 5), 1))]), +('Module', [('Assign', (1, 0), [('Name', (1, 0), 'v', ('Store',))], ('Constant', (1, 4), 1))]), +('Module', [('AugAssign', (1, 0), ('Name', (1, 0), 'v', ('Store',)), ('Add',), ('Constant', (1, 5), 1))]), ('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Pass', (1, 11))], [])]), ('Module', [('While', (1, 0), ('Name', (1, 6), 'v', ('Load',)), [('Pass', (1, 8))], [])]), ('Module', [('If', (1, 0), ('Name', (1, 3), 'v', ('Load',)), [('Pass', (1, 5))], [])]), ('Module', [('With', (1, 0), [('withitem', ('Name', (1, 5), 'x', ('Load',)), ('Name', (1, 10), 'y', ('Store',)))], [('Pass', (1, 13))])]), ('Module', [('With', (1, 0), [('withitem', ('Name', (1, 5), 'x', ('Load',)), ('Name', (1, 10), 'y', ('Store',))), ('withitem', ('Name', (1, 13), 'z', ('Load',)), ('Name', (1, 18), 'q', ('Store',)))], [('Pass', (1, 21))])]), -('Module', [('Raise', (1, 0), ('Call', (1, 6), ('Name', (1, 6), 'Exception', ('Load',)), [('Str', (1, 16), 'string')], []), None)]), +('Module', [('Raise', (1, 0), ('Call', (1, 6), ('Name', (1, 6), 'Exception', ('Load',)), [('Constant', (1, 16), 'string')], []), None)]), ('Module', [('Try', (1, 0), [('Pass', (2, 2))], [('ExceptHandler', (3, 0), ('Name', (3, 7), 'Exception', ('Load',)), None, [('Pass', (4, 2))])], [], [])]), ('Module', [('Try', (1, 0), [('Pass', (2, 2))], [], [], [('Pass', (4, 2))])]), ('Module', [('Assert', (1, 0), ('Name', (1, 7), 'v', ('Load',)), None)]), ('Module', [('Import', (1, 0), [('alias', 'sys', None)])]), ('Module', [('ImportFrom', (1, 0), 'sys', [('alias', 'v', None)], 0)]), ('Module', [('Global', (1, 0), ['v'])]), -('Module', [('Expr', (1, 0), ('Num', (1, 0), 1))]), +('Module', [('Expr', (1, 0), ('Constant', (1, 0), 1))]), ('Module', [('Pass', (1, 0))]), ('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Break', (1, 11))], [])]), ('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Continue', (1, 11))], [])]), @@ -1181,40 +1279,40 @@ def main(): ('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [], 0)]))]), ('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))], 0)]))]), ('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [], 0)]))]), -('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Str', (2, 1), 'async function')), ('Expr', (3, 1), ('Await', (3, 1), ('Call', (3, 7), ('Name', (3, 7), 'something', ('Load',)), [], [])))], [], None)]), -('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 1), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), -('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 1), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), -('Module', [('Expr', (1, 0), ('Dict', (1, 0), [None, ('Num', (1, 10), 2)], [('Dict', (1, 3), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), -('Module', [('Expr', (1, 0), ('Set', (1, 0), [('Starred', (1, 1), ('Set', (1, 2), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Constant', (2, 1), 'async function')), ('Expr', (3, 1), ('Await', (3, 1), ('Call', (3, 7), ('Name', (3, 7), 'something', ('Load',)), [], [])))], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 1), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Constant', (2, 19), 1))], [('Expr', (3, 7), ('Constant', (3, 7), 2))])], [], None)]), +('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 1), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Constant', (2, 20), 1))])], [], None)]), +('Module', [('Expr', (1, 0), ('Dict', (1, 0), [None, ('Constant', (1, 10), 2)], [('Dict', (1, 3), [('Constant', (1, 4), 1)], [('Constant', (1, 6), 2)]), ('Constant', (1, 12), 3)]))]), +('Module', [('Expr', (1, 0), ('Set', (1, 0), [('Starred', (1, 1), ('Set', (1, 2), [('Constant', (1, 3), 1), ('Constant', (1, 6), 2)]), ('Load',)), ('Constant', (1, 10), 3)]))]), ('Module', [('AsyncFunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('ListComp', (2, 2), ('Name', (2, 2), 'i', ('Load',)), [('comprehension', ('Name', (2, 14), 'b', ('Store',)), ('Name', (2, 19), 'c', ('Load',)), [], 1)]))], [], None)]), ] single_results = [ -('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]), +('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Constant', (1, 0), 1), ('Add',), ('Constant', (1, 2), 2)))]), ] eval_results = [ -('Expression', ('NameConstant', (1, 0), None)), +('Expression', ('Constant', (1, 0), None)), ('Expression', ('BoolOp', (1, 0), ('And',), [('Name', (1, 0), 'a', ('Load',)), ('Name', (1, 6), 'b', ('Load',))])), ('Expression', ('BinOp', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Add',), ('Name', (1, 4), 'b', ('Load',)))), ('Expression', ('UnaryOp', (1, 0), ('Not',), ('Name', (1, 4), 'v', ('Load',)))), -('Expression', ('Lambda', (1, 0), ('arguments', [], None, [], [], None, []), ('NameConstant', (1, 7), None))), -('Expression', ('Dict', (1, 0), [('Num', (1, 2), 1)], [('Num', (1, 4), 2)])), +('Expression', ('Lambda', (1, 0), ('arguments', [], None, [], [], None, []), ('Constant', (1, 7), None))), +('Expression', ('Dict', (1, 0), [('Constant', (1, 2), 1)], [('Constant', (1, 4), 2)])), ('Expression', ('Dict', (1, 0), [], [])), -('Expression', ('Set', (1, 0), [('NameConstant', (1, 1), None)])), -('Expression', ('Dict', (1, 0), [('Num', (2, 6), 1)], [('Num', (4, 10), 2)])), +('Expression', ('Set', (1, 0), [('Constant', (1, 1), None)])), +('Expression', ('Dict', (1, 0), [('Constant', (2, 6), 1)], [('Constant', (4, 10), 2)])), ('Expression', ('ListComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))], 0)])), ('Expression', ('GeneratorExp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))], 0)])), -('Expression', ('Compare', (1, 0), ('Num', (1, 0), 1), [('Lt',), ('Lt',)], [('Num', (1, 4), 2), ('Num', (1, 8), 3)])), -('Expression', ('Call', (1, 0), ('Name', (1, 0), 'f', ('Load',)), [('Num', (1, 2), 1), ('Num', (1, 4), 2), ('Starred', (1, 10), ('Name', (1, 11), 'd', ('Load',)), ('Load',))], [('keyword', 'c', ('Num', (1, 8), 3)), ('keyword', None, ('Name', (1, 15), 'e', ('Load',)))])), -('Expression', ('Num', (1, 0), 10)), -('Expression', ('Str', (1, 0), 'string')), +('Expression', ('Compare', (1, 0), ('Constant', (1, 0), 1), [('Lt',), ('Lt',)], [('Constant', (1, 4), 2), ('Constant', (1, 8), 3)])), +('Expression', ('Call', (1, 0), ('Name', (1, 0), 'f', ('Load',)), [('Constant', (1, 2), 1), ('Constant', (1, 4), 2), ('Starred', (1, 10), ('Name', (1, 11), 'd', ('Load',)), ('Load',))], [('keyword', 'c', ('Constant', (1, 8), 3)), ('keyword', None, ('Name', (1, 15), 'e', ('Load',)))])), +('Expression', ('Constant', (1, 0), 10)), +('Expression', ('Constant', (1, 0), 'string')), ('Expression', ('Attribute', (1, 0), ('Name', (1, 0), 'a', ('Load',)), 'b', ('Load',))), ('Expression', ('Subscript', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Slice', ('Name', (1, 2), 'b', ('Load',)), ('Name', (1, 4), 'c', ('Load',)), None), ('Load',))), ('Expression', ('Name', (1, 0), 'v', ('Load',))), -('Expression', ('List', (1, 0), [('Num', (1, 1), 1), ('Num', (1, 3), 2), ('Num', (1, 5), 3)], ('Load',))), +('Expression', ('List', (1, 0), [('Constant', (1, 1), 1), ('Constant', (1, 3), 2), ('Constant', (1, 5), 3)], ('Load',))), ('Expression', ('List', (1, 0), [], ('Load',))), -('Expression', ('Tuple', (1, 0), [('Num', (1, 0), 1), ('Num', (1, 2), 2), ('Num', (1, 4), 3)], ('Load',))), -('Expression', ('Tuple', (1, 1), [('Num', (1, 1), 1), ('Num', (1, 3), 2), ('Num', (1, 5), 3)], ('Load',))), +('Expression', ('Tuple', (1, 0), [('Constant', (1, 0), 1), ('Constant', (1, 2), 2), ('Constant', (1, 4), 3)], ('Load',))), +('Expression', ('Tuple', (1, 1), [('Constant', (1, 1), 1), ('Constant', (1, 3), 2), ('Constant', (1, 5), 3)], ('Load',))), ('Expression', ('Tuple', (1, 0), [], ('Load',))), -('Expression', ('Call', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Name', (1, 0), 'a', ('Load',)), 'b', ('Load',)), 'c', ('Load',)), 'd', ('Load',)), [('Subscript', (1, 8), ('Attribute', (1, 8), ('Name', (1, 8), 'a', ('Load',)), 'b', ('Load',)), ('Slice', ('Num', (1, 12), 1), ('Num', (1, 14), 2), None), ('Load',))], [])), +('Expression', ('Call', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Name', (1, 0), 'a', ('Load',)), 'b', ('Load',)), 'c', ('Load',)), 'd', ('Load',)), [('Subscript', (1, 8), ('Attribute', (1, 8), ('Name', (1, 8), 'a', ('Load',)), 'b', ('Load',)), ('Slice', ('Constant', (1, 12), 1), ('Constant', (1, 14), 2), None), ('Load',))], [])), ] main() diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index 5e7efe25e39c..fec72e008e5b 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -116,9 +116,11 @@ def test_ast_line_numbers_multiple_formattedvalues(self): self.assertEqual(type(t.body[1]), ast.Expr) self.assertEqual(type(t.body[1].value), ast.JoinedStr) self.assertEqual(len(t.body[1].value.values), 4) - self.assertEqual(type(t.body[1].value.values[0]), ast.Str) + self.assertEqual(type(t.body[1].value.values[0]), ast.Constant) + self.assertEqual(type(t.body[1].value.values[0].value), str) self.assertEqual(type(t.body[1].value.values[1]), ast.FormattedValue) - self.assertEqual(type(t.body[1].value.values[2]), ast.Str) + self.assertEqual(type(t.body[1].value.values[2]), ast.Constant) + self.assertEqual(type(t.body[1].value.values[2].value), str) self.assertEqual(type(t.body[1].value.values[3]), ast.FormattedValue) self.assertEqual(t.body[1].lineno, 3) self.assertEqual(t.body[1].value.lineno, 3) @@ -183,9 +185,11 @@ def test_ast_line_numbers_nested(self): self.assertEqual(binop.right.col_offset, 7) # check the nested call location self.assertEqual(len(binop.right.values), 3) - self.assertEqual(type(binop.right.values[0]), ast.Str) + self.assertEqual(type(binop.right.values[0]), ast.Constant) + self.assertEqual(type(binop.right.values[0].value), str) self.assertEqual(type(binop.right.values[1]), ast.FormattedValue) - self.assertEqual(type(binop.right.values[2]), ast.Str) + self.assertEqual(type(binop.right.values[2]), ast.Constant) + self.assertEqual(type(binop.right.values[2].value), str) self.assertEqual(binop.right.values[0].lineno, 3) self.assertEqual(binop.right.values[1].lineno, 3) self.assertEqual(binop.right.values[2].lineno, 3) @@ -215,9 +219,11 @@ def test_ast_line_numbers_duplicate_expression(self): self.assertEqual(type(t.body[1].value), ast.JoinedStr) self.assertEqual(len(t.body[1].value.values), 5) self.assertEqual(type(t.body[1].value.values[0]), ast.FormattedValue) - self.assertEqual(type(t.body[1].value.values[1]), ast.Str) + self.assertEqual(type(t.body[1].value.values[1]), ast.Constant) + self.assertEqual(type(t.body[1].value.values[1].value), str) self.assertEqual(type(t.body[1].value.values[2]), ast.FormattedValue) - self.assertEqual(type(t.body[1].value.values[3]), ast.Str) + self.assertEqual(type(t.body[1].value.values[3]), ast.Constant) + self.assertEqual(type(t.body[1].value.values[3].value), str) self.assertEqual(type(t.body[1].value.values[4]), ast.FormattedValue) self.assertEqual(t.body[1].lineno, 3) self.assertEqual(t.body[1].value.lineno, 3) @@ -287,9 +293,11 @@ def test_ast_line_numbers_multiline_fstring(self): self.assertEqual(type(t.body[1]), ast.Expr) self.assertEqual(type(t.body[1].value), ast.JoinedStr) self.assertEqual(len(t.body[1].value.values), 3) - self.assertEqual(type(t.body[1].value.values[0]), ast.Str) + self.assertEqual(type(t.body[1].value.values[0]), ast.Constant) + self.assertEqual(type(t.body[1].value.values[0].value), str) self.assertEqual(type(t.body[1].value.values[1]), ast.FormattedValue) - self.assertEqual(type(t.body[1].value.values[2]), ast.Str) + self.assertEqual(type(t.body[1].value.values[2]), ast.Constant) + self.assertEqual(type(t.body[1].value.values[2].value), str) # NOTE: the following invalid behavior is described in bpo-16806. # - line number should be the *first* line (3), not the *last* (8) # - column offset should not be -1 diff --git a/Lib/test/test_future.py b/Lib/test/test_future.py index 660701b4b3d8..904e8a9d5dfc 100644 --- a/Lib/test/test_future.py +++ b/Lib/test/test_future.py @@ -230,7 +230,7 @@ def test_annotations(self): eq("lukasz.langa.pl") eq("call.me(maybe)") eq("1 .real") - eq("1.0 .real") + eq("1.0.real") eq("....__class__") eq("list[str]") eq("dict[str, int]") diff --git a/Lib/test/test_pyclbr.py b/Lib/test/test_pyclbr.py index eaab591f74ef..9e970d9df041 100644 --- a/Lib/test/test_pyclbr.py +++ b/Lib/test/test_pyclbr.py @@ -144,7 +144,8 @@ def defined_in(item, module): def test_easy(self): self.checkModule('pyclbr') - self.checkModule('ast') + # XXX: Metaclasses are not supported + # self.checkModule('ast') self.checkModule('doctest', ignore=("TestResults", "_SpoofOut", "DocTestCase", '_DocTestSuite')) self.checkModule('difflib', ignore=("Match",)) diff --git a/Misc/NEWS.d/next/Library/2018-09-20-17-35-05.bpo-32892.TOUBdg.rst b/Misc/NEWS.d/next/Library/2018-09-20-17-35-05.bpo-32892.TOUBdg.rst new file mode 100644 index 000000000000..9be4bf89008d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-20-17-35-05.bpo-32892.TOUBdg.rst @@ -0,0 +1,4 @@ +The parser now represents all constants as :class:`ast.Constant` instead of +using specific constant AST types (``Num``, ``Str``, ``Bytes``, +``NameConstant`` and ``Ellipsis``). These classes are considered deprecated +and will be removed in future Python versions. diff --git a/Parser/Python.asdl b/Parser/Python.asdl index f470ad13b655..eee982be1c95 100644 --- a/Parser/Python.asdl +++ b/Parser/Python.asdl @@ -1,8 +1,5 @@ --- ASDL's 7 builtin types are: --- identifier, int, string, bytes, object, singleton, constant --- --- singleton: None, True or False --- constant can be None, whereas None means "no value" for object. +-- ASDL's 5 builtin types are: +-- identifier, int, string, object, constant module Python { @@ -75,13 +72,8 @@ module Python -- x < 4 < 3 and (x < 4) < 3 | Compare(expr left, cmpop* ops, expr* comparators) | Call(expr func, expr* args, keyword* keywords) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? | FormattedValue(expr value, int? conversion, expr? format_spec) | JoinedStr(expr* values) - | Bytes(bytes s) - | NameConstant(singleton value) - | Ellipsis | Constant(constant value) -- the following expression can appear in assignment context diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py index 44e3d40c6155..4c280a96c30e 100644 --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -855,17 +855,6 @@ def visitModule(self, mod): /* Conversion Python -> AST */ -static int obj2ast_singleton(PyObject *obj, PyObject** out, PyArena* arena) -{ - if (obj != Py_None && obj != Py_True && obj != Py_False) { - PyErr_SetString(PyExc_ValueError, - "AST singleton must be True, False, or None"); - return 1; - } - *out = obj; - return 0; -} - static int obj2ast_object(PyObject* obj, PyObject** out, PyArena* arena) { if (obj == Py_None) @@ -883,13 +872,11 @@ def visitModule(self, mod): static int obj2ast_constant(PyObject* obj, PyObject** out, PyArena* arena) { - if (obj) { - if (PyArena_AddPyObject(arena, obj) < 0) { - *out = NULL; - return -1; - } - Py_INCREF(obj); + if (PyArena_AddPyObject(arena, obj) < 0) { + *out = NULL; + return -1; } + Py_INCREF(obj); *out = obj; return 0; } @@ -903,24 +890,6 @@ def visitModule(self, mod): return obj2ast_object(obj, out, arena); } -static int obj2ast_string(PyObject* obj, PyObject** out, PyArena* arena) -{ - if (!PyUnicode_CheckExact(obj) && !PyBytes_CheckExact(obj)) { - PyErr_SetString(PyExc_TypeError, "AST string must be of type str"); - return 1; - } - return obj2ast_object(obj, out, arena); -} - -static int obj2ast_bytes(PyObject* obj, PyObject** out, PyArena* arena) -{ - if (!PyBytes_CheckExact(obj)) { - PyErr_SetString(PyExc_TypeError, "AST bytes must be of type bytes"); - return 1; - } - return obj2ast_object(obj, out, arena); -} - static int obj2ast_int(PyObject* obj, int* out, PyArena* arena) { int i; diff --git a/Python/Python-ast.c b/Python/Python-ast.c index 6a2f28e0e712..bbe8e69fdd50 100644 --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -284,16 +284,6 @@ static char *Call_fields[]={ "args", "keywords", }; -static PyTypeObject *Num_type; -_Py_IDENTIFIER(n); -static char *Num_fields[]={ - "n", -}; -static PyTypeObject *Str_type; -_Py_IDENTIFIER(s); -static char *Str_fields[]={ - "s", -}; static PyTypeObject *FormattedValue_type; _Py_IDENTIFIER(conversion); _Py_IDENTIFIER(format_spec); @@ -306,15 +296,6 @@ static PyTypeObject *JoinedStr_type; static char *JoinedStr_fields[]={ "values", }; -static PyTypeObject *Bytes_type; -static char *Bytes_fields[]={ - "s", -}; -static PyTypeObject *NameConstant_type; -static char *NameConstant_fields[]={ - "value", -}; -static PyTypeObject *Ellipsis_type; static PyTypeObject *Constant_type; static char *Constant_fields[]={ "value", @@ -736,17 +717,6 @@ static PyObject* ast2obj_int(long b) /* Conversion Python -> AST */ -static int obj2ast_singleton(PyObject *obj, PyObject** out, PyArena* arena) -{ - if (obj != Py_None && obj != Py_True && obj != Py_False) { - PyErr_SetString(PyExc_ValueError, - "AST singleton must be True, False, or None"); - return 1; - } - *out = obj; - return 0; -} - static int obj2ast_object(PyObject* obj, PyObject** out, PyArena* arena) { if (obj == Py_None) @@ -764,13 +734,11 @@ static int obj2ast_object(PyObject* obj, PyObject** out, PyArena* arena) static int obj2ast_constant(PyObject* obj, PyObject** out, PyArena* arena) { - if (obj) { - if (PyArena_AddPyObject(arena, obj) < 0) { - *out = NULL; - return -1; - } - Py_INCREF(obj); + if (PyArena_AddPyObject(arena, obj) < 0) { + *out = NULL; + return -1; } + Py_INCREF(obj); *out = obj; return 0; } @@ -784,24 +752,6 @@ static int obj2ast_identifier(PyObject* obj, PyObject** out, PyArena* arena) return obj2ast_object(obj, out, arena); } -static int obj2ast_string(PyObject* obj, PyObject** out, PyArena* arena) -{ - if (!PyUnicode_CheckExact(obj) && !PyBytes_CheckExact(obj)) { - PyErr_SetString(PyExc_TypeError, "AST string must be of type str"); - return 1; - } - return obj2ast_object(obj, out, arena); -} - -static int obj2ast_bytes(PyObject* obj, PyObject** out, PyArena* arena) -{ - if (!PyBytes_CheckExact(obj)) { - PyErr_SetString(PyExc_TypeError, "AST bytes must be of type bytes"); - return 1; - } - return obj2ast_object(obj, out, arena); -} - static int obj2ast_int(PyObject* obj, int* out, PyArena* arena) { int i; @@ -943,22 +893,11 @@ static int init_types(void) if (!Compare_type) return 0; Call_type = make_type("Call", expr_type, Call_fields, 3); if (!Call_type) return 0; - Num_type = make_type("Num", expr_type, Num_fields, 1); - if (!Num_type) return 0; - Str_type = make_type("Str", expr_type, Str_fields, 1); - if (!Str_type) return 0; FormattedValue_type = make_type("FormattedValue", expr_type, FormattedValue_fields, 3); if (!FormattedValue_type) return 0; JoinedStr_type = make_type("JoinedStr", expr_type, JoinedStr_fields, 1); if (!JoinedStr_type) return 0; - Bytes_type = make_type("Bytes", expr_type, Bytes_fields, 1); - if (!Bytes_type) return 0; - NameConstant_type = make_type("NameConstant", expr_type, - NameConstant_fields, 1); - if (!NameConstant_type) return 0; - Ellipsis_type = make_type("Ellipsis", expr_type, NULL, 0); - if (!Ellipsis_type) return 0; Constant_type = make_type("Constant", expr_type, Constant_fields, 1); if (!Constant_type) return 0; Attribute_type = make_type("Attribute", expr_type, Attribute_fields, 3); @@ -2089,44 +2028,6 @@ Call(expr_ty func, asdl_seq * args, asdl_seq * keywords, int lineno, int return p; } -expr_ty -Num(object n, int lineno, int col_offset, PyArena *arena) -{ - expr_ty p; - if (!n) { - PyErr_SetString(PyExc_ValueError, - "field n is required for Num"); - return NULL; - } - p = (expr_ty)PyArena_Malloc(arena, sizeof(*p)); - if (!p) - return NULL; - p->kind = Num_kind; - p->v.Num.n = n; - p->lineno = lineno; - p->col_offset = col_offset; - return p; -} - -expr_ty -Str(string s, int lineno, int col_offset, PyArena *arena) -{ - expr_ty p; - if (!s) { - PyErr_SetString(PyExc_ValueError, - "field s is required for Str"); - return NULL; - } - p = (expr_ty)PyArena_Malloc(arena, sizeof(*p)); - if (!p) - return NULL; - p->kind = Str_kind; - p->v.Str.s = s; - p->lineno = lineno; - p->col_offset = col_offset; - return p; -} - expr_ty FormattedValue(expr_ty value, int conversion, expr_ty format_spec, int lineno, int col_offset, PyArena *arena) @@ -2163,57 +2064,6 @@ JoinedStr(asdl_seq * values, int lineno, int col_offset, PyArena *arena) return p; } -expr_ty -Bytes(bytes s, int lineno, int col_offset, PyArena *arena) -{ - expr_ty p; - if (!s) { - PyErr_SetString(PyExc_ValueError, - "field s is required for Bytes"); - return NULL; - } - p = (expr_ty)PyArena_Malloc(arena, sizeof(*p)); - if (!p) - return NULL; - p->kind = Bytes_kind; - p->v.Bytes.s = s; - p->lineno = lineno; - p->col_offset = col_offset; - return p; -} - -expr_ty -NameConstant(singleton value, int lineno, int col_offset, PyArena *arena) -{ - expr_ty p; - if (!value) { - PyErr_SetString(PyExc_ValueError, - "field value is required for NameConstant"); - return NULL; - } - p = (expr_ty)PyArena_Malloc(arena, sizeof(*p)); - if (!p) - return NULL; - p->kind = NameConstant_kind; - p->v.NameConstant.value = value; - p->lineno = lineno; - p->col_offset = col_offset; - return p; -} - -expr_ty -Ellipsis(int lineno, int col_offset, PyArena *arena) -{ - expr_ty p; - p = (expr_ty)PyArena_Malloc(arena, sizeof(*p)); - if (!p) - return NULL; - p->kind = Ellipsis_kind; - p->lineno = lineno; - p->col_offset = col_offset; - return p; -} - expr_ty Constant(constant value, int lineno, int col_offset, PyArena *arena) { @@ -3289,24 +3139,6 @@ ast2obj_expr(void* _o) goto failed; Py_DECREF(value); break; - case Num_kind: - result = PyType_GenericNew(Num_type, NULL, NULL); - if (!result) goto failed; - value = ast2obj_object(o->v.Num.n); - if (!value) goto failed; - if (_PyObject_SetAttrId(result, &PyId_n, value) == -1) - goto failed; - Py_DECREF(value); - break; - case Str_kind: - result = PyType_GenericNew(Str_type, NULL, NULL); - if (!result) goto failed; - value = ast2obj_string(o->v.Str.s); - if (!value) goto failed; - if (_PyObject_SetAttrId(result, &PyId_s, value) == -1) - goto failed; - Py_DECREF(value); - break; case FormattedValue_kind: result = PyType_GenericNew(FormattedValue_type, NULL, NULL); if (!result) goto failed; @@ -3335,28 +3167,6 @@ ast2obj_expr(void* _o) goto failed; Py_DECREF(value); break; - case Bytes_kind: - result = PyType_GenericNew(Bytes_type, NULL, NULL); - if (!result) goto failed; - value = ast2obj_bytes(o->v.Bytes.s); - if (!value) goto failed; - if (_PyObject_SetAttrId(result, &PyId_s, value) == -1) - goto failed; - Py_DECREF(value); - break; - case NameConstant_kind: - result = PyType_GenericNew(NameConstant_type, NULL, NULL); - if (!result) goto failed; - value = ast2obj_singleton(o->v.NameConstant.value); - if (!value) goto failed; - if (_PyObject_SetAttrId(result, &PyId_value, value) == -1) - goto failed; - Py_DECREF(value); - break; - case Ellipsis_kind: - result = PyType_GenericNew(Ellipsis_type, NULL, NULL); - if (!result) goto failed; - break; case Constant_kind: result = PyType_GenericNew(Constant_type, NULL, NULL); if (!result) goto failed; @@ -6606,54 +6416,6 @@ obj2ast_expr(PyObject* obj, expr_ty* out, PyArena* arena) if (*out == NULL) goto failed; return 0; } - isinstance = PyObject_IsInstance(obj, (PyObject*)Num_type); - if (isinstance == -1) { - return 1; - } - if (isinstance) { - object n; - - if (_PyObject_LookupAttrId(obj, &PyId_n, &tmp) < 0) { - return 1; - } - if (tmp == NULL) { - PyErr_SetString(PyExc_TypeError, "required field \"n\" missing from Num"); - return 1; - } - else { - int res; - res = obj2ast_object(tmp, &n, arena); - if (res != 0) goto failed; - Py_CLEAR(tmp); - } - *out = Num(n, lineno, col_offset, arena); - if (*out == NULL) goto failed; - return 0; - } - isinstance = PyObject_IsInstance(obj, (PyObject*)Str_type); - if (isinstance == -1) { - return 1; - } - if (isinstance) { - string s; - - if (_PyObject_LookupAttrId(obj, &PyId_s, &tmp) < 0) { - return 1; - } - if (tmp == NULL) { - PyErr_SetString(PyExc_TypeError, "required field \"s\" missing from Str"); - return 1; - } - else { - int res; - res = obj2ast_string(tmp, &s, arena); - if (res != 0) goto failed; - Py_CLEAR(tmp); - } - *out = Str(s, lineno, col_offset, arena); - if (*out == NULL) goto failed; - return 0; - } isinstance = PyObject_IsInstance(obj, (PyObject*)FormattedValue_type); if (isinstance == -1) { return 1; @@ -6748,64 +6510,6 @@ obj2ast_expr(PyObject* obj, expr_ty* out, PyArena* arena) if (*out == NULL) goto failed; return 0; } - isinstance = PyObject_IsInstance(obj, (PyObject*)Bytes_type); - if (isinstance == -1) { - return 1; - } - if (isinstance) { - bytes s; - - if (_PyObject_LookupAttrId(obj, &PyId_s, &tmp) < 0) { - return 1; - } - if (tmp == NULL) { - PyErr_SetString(PyExc_TypeError, "required field \"s\" missing from Bytes"); - return 1; - } - else { - int res; - res = obj2ast_bytes(tmp, &s, arena); - if (res != 0) goto failed; - Py_CLEAR(tmp); - } - *out = Bytes(s, lineno, col_offset, arena); - if (*out == NULL) goto failed; - return 0; - } - isinstance = PyObject_IsInstance(obj, (PyObject*)NameConstant_type); - if (isinstance == -1) { - return 1; - } - if (isinstance) { - singleton value; - - if (_PyObject_LookupAttrId(obj, &PyId_value, &tmp) < 0) { - return 1; - } - if (tmp == NULL) { - PyErr_SetString(PyExc_TypeError, "required field \"value\" missing from NameConstant"); - return 1; - } - else { - int res; - res = obj2ast_singleton(tmp, &value, arena); - if (res != 0) goto failed; - Py_CLEAR(tmp); - } - *out = NameConstant(value, lineno, col_offset, arena); - if (*out == NULL) goto failed; - return 0; - } - isinstance = PyObject_IsInstance(obj, (PyObject*)Ellipsis_type); - if (isinstance == -1) { - return 1; - } - if (isinstance) { - - *out = Ellipsis(lineno, col_offset, arena); - if (*out == NULL) goto failed; - return 0; - } isinstance = PyObject_IsInstance(obj, (PyObject*)Constant_type); if (isinstance == -1) { return 1; @@ -8244,18 +7948,10 @@ PyInit__ast(void) if (PyDict_SetItemString(d, "Compare", (PyObject*)Compare_type) < 0) return NULL; if (PyDict_SetItemString(d, "Call", (PyObject*)Call_type) < 0) return NULL; - if (PyDict_SetItemString(d, "Num", (PyObject*)Num_type) < 0) return NULL; - if (PyDict_SetItemString(d, "Str", (PyObject*)Str_type) < 0) return NULL; if (PyDict_SetItemString(d, "FormattedValue", (PyObject*)FormattedValue_type) < 0) return NULL; if (PyDict_SetItemString(d, "JoinedStr", (PyObject*)JoinedStr_type) < 0) return NULL; - if (PyDict_SetItemString(d, "Bytes", (PyObject*)Bytes_type) < 0) return - NULL; - if (PyDict_SetItemString(d, "NameConstant", (PyObject*)NameConstant_type) < - 0) return NULL; - if (PyDict_SetItemString(d, "Ellipsis", (PyObject*)Ellipsis_type) < 0) - return NULL; if (PyDict_SetItemString(d, "Constant", (PyObject*)Constant_type) < 0) return NULL; if (PyDict_SetItemString(d, "Attribute", (PyObject*)Attribute_type) < 0) diff --git a/Python/ast.c b/Python/ast.c index b2fcb219752d..587f838f8d8e 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -295,23 +295,6 @@ validate_expr(expr_ty exp, expr_context_ty ctx) return 0; } return 1; - case Num_kind: { - PyObject *n = exp->v.Num.n; - if (!PyLong_CheckExact(n) && !PyFloat_CheckExact(n) && - !PyComplex_CheckExact(n)) { - PyErr_SetString(PyExc_TypeError, "non-numeric type in Num"); - return 0; - } - return 1; - } - case Str_kind: { - PyObject *s = exp->v.Str.s; - if (!PyUnicode_CheckExact(s)) { - PyErr_SetString(PyExc_TypeError, "non-string type in Str"); - return 0; - } - return 1; - } case JoinedStr_kind: return validate_exprs(exp->v.JoinedStr.values, Load, 0); case FormattedValue_kind: @@ -320,14 +303,6 @@ validate_expr(expr_ty exp, expr_context_ty ctx) if (exp->v.FormattedValue.format_spec) return validate_expr(exp->v.FormattedValue.format_spec, Load); return 1; - case Bytes_kind: { - PyObject *b = exp->v.Bytes.s; - if (!PyBytes_CheckExact(b)) { - PyErr_SetString(PyExc_TypeError, "non-bytes type in Bytes"); - return 0; - } - return 1; - } case Attribute_kind: return validate_expr(exp->v.Attribute.value, Load); case Subscript_kind: @@ -339,10 +314,8 @@ validate_expr(expr_ty exp, expr_context_ty ctx) return validate_exprs(exp->v.List.elts, ctx, 0); case Tuple_kind: return validate_exprs(exp->v.Tuple.elts, ctx, 0); - /* These last cases don't have any checking. */ + /* This last case doesn't have any checking. */ case Name_kind: - case NameConstant_kind: - case Ellipsis_kind: return 1; default: PyErr_SetString(PyExc_SystemError, "unexpected expression"); @@ -1040,19 +1013,23 @@ set_context(struct compiling *c, expr_ty e, expr_context_ty ctx, const node *n) break; case Dict_kind: case Set_kind: - case Num_kind: - case Str_kind: - case Bytes_kind: case JoinedStr_kind: case FormattedValue_kind: expr_name = "literal"; break; - case NameConstant_kind: - expr_name = "keyword"; - break; - case Ellipsis_kind: - expr_name = "Ellipsis"; + case Constant_kind: { + PyObject *value = e->v.Constant.value; + if (value == Py_None || value == Py_False || value == Py_True) { + expr_name = "keyword"; + } + else if (value == Py_Ellipsis) { + expr_name = "Ellipsis"; + } + else { + expr_name = "literal"; + } break; + } case Compare_kind: expr_name = "comparison"; break; @@ -2091,11 +2068,11 @@ ast_for_atom(struct compiling *c, const node *n) size_t len = strlen(s); if (len >= 4 && len <= 5) { if (!strcmp(s, "None")) - return NameConstant(Py_None, LINENO(n), n->n_col_offset, c->c_arena); + return Constant(Py_None, LINENO(n), n->n_col_offset, c->c_arena); if (!strcmp(s, "True")) - return NameConstant(Py_True, LINENO(n), n->n_col_offset, c->c_arena); + return Constant(Py_True, LINENO(n), n->n_col_offset, c->c_arena); if (!strcmp(s, "False")) - return NameConstant(Py_False, LINENO(n), n->n_col_offset, c->c_arena); + return Constant(Py_False, LINENO(n), n->n_col_offset, c->c_arena); } name = new_identifier(s, c); if (!name) @@ -2144,10 +2121,10 @@ ast_for_atom(struct compiling *c, const node *n) Py_DECREF(pynum); return NULL; } - return Num(pynum, LINENO(n), n->n_col_offset, c->c_arena); + return Constant(pynum, LINENO(n), n->n_col_offset, c->c_arena); } case ELLIPSIS: /* Ellipsis */ - return Ellipsis(LINENO(n), n->n_col_offset, c->c_arena); + return Constant(Py_Ellipsis, LINENO(n), n->n_col_offset, c->c_arena); case LPAR: /* some parenthesized expressions */ ch = CHILD(n, 1); @@ -4751,7 +4728,7 @@ typedef struct { expr_ty's, and then after that start dynamically allocating, doubling the number allocated each time. Note that the f-string f'{0}a{1}' contains 3 expr_ty's: 2 FormattedValue's, and one - Str for the literal 'a'. So you add expr_ty's about twice as + Constant for the literal 'a'. So you add expr_ty's about twice as fast as you add exressions in an f-string. */ Py_ssize_t allocated; /* Number we've allocated. */ @@ -4903,7 +4880,7 @@ FstringParser_Dealloc(FstringParser *state) ExprList_Dealloc(&state->expr_list); } -/* Make a Str node, but decref the PyUnicode object being added. */ +/* Make a Constant node, but decref the PyUnicode object being added. */ static expr_ty make_str_node_and_del(PyObject **str, struct compiling *c, const node* n) { @@ -4914,7 +4891,7 @@ make_str_node_and_del(PyObject **str, struct compiling *c, const node* n) Py_DECREF(s); return NULL; } - return Str(s, LINENO(n), n->n_col_offset, c->c_arena); + return Constant(s, LINENO(n), n->n_col_offset, c->c_arena); } /* Add a non-f-string (that is, a regular literal string). str is @@ -5002,11 +4979,11 @@ FstringParser_ConcatFstring(FstringParser *state, const char **str, break; /* We know we have an expression. Convert any existing string - to a Str node. */ + to a Constant node. */ if (!state->last_str) { /* Do nothing. No previous literal. */ } else { - /* Convert the existing last_str literal to a Str node. */ + /* Convert the existing last_str literal to a Constant node. */ expr_ty str = make_str_node_and_del(&state->last_str, c, n); if (!str || ExprList_Append(&state->expr_list, str) < 0) return -1; @@ -5033,7 +5010,7 @@ FstringParser_ConcatFstring(FstringParser *state, const char **str, } /* Convert the partial state reflected in last_str and expr_list to an - expr_ty. The expr_ty can be a Str, or a JoinedStr. */ + expr_ty. The expr_ty can be a Constant, or a JoinedStr. */ static expr_ty FstringParser_Finish(FstringParser *state, struct compiling *c, const node *n) @@ -5055,7 +5032,7 @@ FstringParser_Finish(FstringParser *state, struct compiling *c, return make_str_node_and_del(&state->last_str, c, n); } - /* Create a Str node out of last_str, if needed. It will be the + /* Create a Constant node out of last_str, if needed. It will be the last node in our expression list. */ if (state->last_str) { expr_ty str = make_str_node_and_del(&state->last_str, c, n); @@ -5206,9 +5183,9 @@ parsestr(struct compiling *c, const node *n, int *bytesmode, int *rawmode, /* Accepts a STRING+ atom, and produces an expr_ty node. Run through each STRING atom, and process it as needed. For bytes, just - concatenate them together, and the result will be a Bytes node. For + concatenate them together, and the result will be a Constant node. For normal strings and f-strings, concatenate them together. The result - will be a Str node if there were no f-strings; a FormattedValue + will be a Constant node if there were no f-strings; a FormattedValue node if there's just an f-string (with no leading or trailing literals), or a JoinedStr node if there are multiple f-strings or any literals involved. */ @@ -5279,7 +5256,7 @@ parsestrplus(struct compiling *c, const node *n) /* Just return the bytes object and we're done. */ if (PyArena_AddPyObject(c->c_arena, bytes_str) < 0) goto error; - return Bytes(bytes_str, LINENO(n), n->n_col_offset, c->c_arena); + return Constant(bytes_str, LINENO(n), n->n_col_offset, c->c_arena); } /* We're not a bytes string, bytes_str should never have been set. */ @@ -5304,9 +5281,6 @@ _PyAST_GetDocString(asdl_seq *body) return NULL; } expr_ty e = st->v.Expr.value; - if (e->kind == Str_kind) { - return e->v.Str.s; - } if (e->kind == Constant_kind && PyUnicode_CheckExact(e->v.Constant.value)) { return e->v.Constant.value; } diff --git a/Python/ast_opt.c b/Python/ast_opt.c index 5e57638e3d01..1f9cb773ea7b 100644 --- a/Python/ast_opt.c +++ b/Python/ast_opt.c @@ -5,47 +5,6 @@ #include "ast.h" -/* TODO: is_const and get_const_value are copied from Python/compile.c. - It should be deduped in the future. Maybe, we can include this file - from compile.c? -*/ -static int -is_const(expr_ty e) -{ - switch (e->kind) { - case Constant_kind: - case Num_kind: - case Str_kind: - case Bytes_kind: - case Ellipsis_kind: - case NameConstant_kind: - return 1; - default: - return 0; - } -} - -static PyObject * -get_const_value(expr_ty e) -{ - switch (e->kind) { - case Constant_kind: - return e->v.Constant.value; - case Num_kind: - return e->v.Num.n; - case Str_kind: - return e->v.Str.s; - case Bytes_kind: - return e->v.Bytes.s; - case Ellipsis_kind: - return Py_Ellipsis; - case NameConstant_kind: - return e->v.NameConstant.value; - default: - Py_UNREACHABLE(); - } -} - static int make_const(expr_ty node, PyObject *val, PyArena *arena) { @@ -81,7 +40,7 @@ fold_unaryop(expr_ty node, PyArena *arena, int optimize) { expr_ty arg = node->v.UnaryOp.operand; - if (!is_const(arg)) { + if (arg->kind != Constant_kind) { /* Fold not into comparison */ if (node->v.UnaryOp.op == Not && arg->kind == Compare_kind && asdl_seq_LEN(arg->v.Compare.ops) == 1) { @@ -123,7 +82,7 @@ fold_unaryop(expr_ty node, PyArena *arena, int optimize) [UAdd] = PyNumber_Positive, [USub] = PyNumber_Negative, }; - PyObject *newval = ops[node->v.UnaryOp.op](get_const_value(arg)); + PyObject *newval = ops[node->v.UnaryOp.op](arg->v.Constant.value); return make_const(node, newval, arena); } @@ -259,12 +218,12 @@ fold_binop(expr_ty node, PyArena *arena, int optimize) expr_ty lhs, rhs; lhs = node->v.BinOp.left; rhs = node->v.BinOp.right; - if (!is_const(lhs) || !is_const(rhs)) { + if (lhs->kind != Constant_kind || rhs->kind != Constant_kind) { return 1; } - PyObject *lv = get_const_value(lhs); - PyObject *rv = get_const_value(rhs); + PyObject *lv = lhs->v.Constant.value; + PyObject *rv = rhs->v.Constant.value; PyObject *newval; switch (node->v.BinOp.op) { @@ -316,7 +275,7 @@ make_const_tuple(asdl_seq *elts) { for (int i = 0; i < asdl_seq_LEN(elts); i++) { expr_ty e = (expr_ty)asdl_seq_GET(elts, i); - if (!is_const(e)) { + if (e->kind != Constant_kind) { return NULL; } } @@ -328,7 +287,7 @@ make_const_tuple(asdl_seq *elts) for (int i = 0; i < asdl_seq_LEN(elts); i++) { expr_ty e = (expr_ty)asdl_seq_GET(elts, i); - PyObject *v = get_const_value(e); + PyObject *v = e->v.Constant.value; Py_INCREF(v); PyTuple_SET_ITEM(newval, i, v); } @@ -357,16 +316,16 @@ fold_subscr(expr_ty node, PyArena *arena, int optimize) arg = node->v.Subscript.value; slice = node->v.Subscript.slice; if (node->v.Subscript.ctx != Load || - !is_const(arg) || + arg->kind != Constant_kind || /* TODO: handle other types of slices */ slice->kind != Index_kind || - !is_const(slice->v.Index.value)) + slice->v.Index.value->kind != Constant_kind) { return 1; } idx = slice->v.Index.value; - newval = PyObject_GetItem(get_const_value(arg), get_const_value(idx)); + newval = PyObject_GetItem(arg->v.Constant.value, idx->v.Constant.value); return make_const(node, newval, arena); } diff --git a/Python/ast_unparse.c b/Python/ast_unparse.c index 725ce31fe3c0..56ea3c424c5b 100644 --- a/Python/ast_unparse.c +++ b/Python/ast_unparse.c @@ -567,8 +567,6 @@ append_fstring_element(_PyUnicodeWriter *writer, expr_ty e, bool is_format_spec) switch (e->kind) { case Constant_kind: return append_fstring_unicode(writer, e->v.Constant.value); - case Str_kind: - return append_fstring_unicode(writer, e->v.Str.s); case JoinedStr_kind: return append_joinedstr(writer, e, is_format_spec); case FormattedValue_kind: @@ -690,13 +688,12 @@ static int append_ast_attribute(_PyUnicodeWriter *writer, expr_ty e) { const char *period; - APPEND_EXPR(e->v.Attribute.value, PR_ATOM); + expr_ty v = e->v.Attribute.value; + APPEND_EXPR(v, PR_ATOM); /* Special case: integers require a space for attribute access to be - unambiguous. Floats and complex numbers don't but work with it, too. */ - if (e->v.Attribute.value->kind == Num_kind || - e->v.Attribute.value->kind == Constant_kind) - { + unambiguous. */ + if (v->kind == Constant_kind && PyLong_CheckExact(v->v.Constant.value)) { period = " ."; } else { @@ -841,21 +838,14 @@ append_ast_expr(_PyUnicodeWriter *writer, expr_ty e, int level) case Call_kind: return append_ast_call(writer, e); case Constant_kind: + if (e->v.Constant.value == Py_Ellipsis) { + APPEND_STR_FINISH("..."); + } return append_repr(writer, e->v.Constant.value); - case Num_kind: - return append_repr(writer, e->v.Num.n); - case Str_kind: - return append_repr(writer, e->v.Str.s); case JoinedStr_kind: return append_joinedstr(writer, e, false); case FormattedValue_kind: return append_formattedvalue(writer, e, false); - case Bytes_kind: - return append_repr(writer, e->v.Bytes.s); - case Ellipsis_kind: - APPEND_STR_FINISH("..."); - case NameConstant_kind: - return append_repr(writer, e->v.NameConstant.value); /* The following exprs can be assignment targets. */ case Attribute_kind: return append_ast_attribute(writer, e); diff --git a/Python/compile.c b/Python/compile.c index 3a45804580ed..096b762f36bb 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -1398,43 +1398,6 @@ compiler_addop_j(struct compiler *c, int opcode, basicblock *b, int absolute) } \ } -static int -is_const(expr_ty e) -{ - switch (e->kind) { - case Constant_kind: - case Num_kind: - case Str_kind: - case Bytes_kind: - case Ellipsis_kind: - case NameConstant_kind: - return 1; - default: - return 0; - } -} - -static PyObject * -get_const_value(expr_ty e) -{ - switch (e->kind) { - case Constant_kind: - return e->v.Constant.value; - case Num_kind: - return e->v.Num.n; - case Str_kind: - return e->v.Str.s; - case Bytes_kind: - return e->v.Bytes.s; - case Ellipsis_kind: - return Py_Ellipsis; - case NameConstant_kind: - return e->v.NameConstant.value; - default: - Py_UNREACHABLE(); - } -} - /* Search if variable annotations are present statically in a block. */ static int @@ -2568,7 +2531,7 @@ static int compiler_return(struct compiler *c, stmt_ty s) { int preserve_tos = ((s->v.Return.value != NULL) && - !is_const(s->v.Return.value)); + (s->v.Return.value->kind != Constant_kind)); if (c->u->u_ste->ste_type != FunctionBlock) return compiler_error(c, "'return' outside function"); if (s->v.Return.value != NULL && @@ -3054,7 +3017,7 @@ compiler_visit_stmt_expr(struct compiler *c, expr_ty value) return 1; } - if (is_const(value)) { + if (value->kind == Constant_kind) { /* ignore constant statement */ return 1; } @@ -3502,7 +3465,7 @@ are_all_items_const(asdl_seq *seq, Py_ssize_t begin, Py_ssize_t end) Py_ssize_t i; for (i = begin; i < end; i++) { expr_ty key = (expr_ty)asdl_seq_GET(seq, i); - if (key == NULL || !is_const(key)) + if (key == NULL || key->kind != Constant_kind) return 0; } return 1; @@ -3522,7 +3485,7 @@ compiler_subdict(struct compiler *c, expr_ty e, Py_ssize_t begin, Py_ssize_t end return 0; } for (i = begin; i < end; i++) { - key = get_const_value((expr_ty)asdl_seq_GET(e->v.Dict.keys, i)); + key = ((expr_ty)asdl_seq_GET(e->v.Dict.keys, i))->v.Constant.value; Py_INCREF(key); PyTuple_SET_ITEM(keys, i - begin, key); } @@ -4244,8 +4207,8 @@ compiler_visit_keyword(struct compiler *c, keyword_ty k) static int expr_constant(expr_ty e) { - if (is_const(e)) { - return PyObject_IsTrue(get_const_value(e)); + if (e->kind == Constant_kind) { + return PyObject_IsTrue(e->v.Constant.value); } return -1; } @@ -4505,25 +4468,10 @@ compiler_visit_expr1(struct compiler *c, expr_ty e) case Constant_kind: ADDOP_LOAD_CONST(c, e->v.Constant.value); break; - case Num_kind: - ADDOP_LOAD_CONST(c, e->v.Num.n); - break; - case Str_kind: - ADDOP_LOAD_CONST(c, e->v.Str.s); - break; case JoinedStr_kind: return compiler_joined_str(c, e); case FormattedValue_kind: return compiler_formatted_value(c, e); - case Bytes_kind: - ADDOP_LOAD_CONST(c, e->v.Bytes.s); - break; - case Ellipsis_kind: - ADDOP_LOAD_CONST(c, Py_Ellipsis); - break; - case NameConstant_kind: - ADDOP_LOAD_CONST(c, e->v.NameConstant.value); - break; /* The following exprs can be assignment targets. */ case Attribute_kind: if (e->v.Attribute.ctx != AugStore) diff --git a/Python/symtable.c b/Python/symtable.c index e7216147a8a4..16b706b36338 100644 --- a/Python/symtable.c +++ b/Python/symtable.c @@ -1461,11 +1461,6 @@ symtable_visit_expr(struct symtable *st, expr_ty e) VISIT_SEQ(st, expr, e->v.JoinedStr.values); break; case Constant_kind: - case Num_kind: - case Str_kind: - case Bytes_kind: - case Ellipsis_kind: - case NameConstant_kind: /* Nothing to do here. */ break; /* The following exprs can be assignment targets. */ diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index a6a43d1361d0..ca8096f43a6f 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -3840,9 +3840,6 @@ def bad_node(self, node): # "starred": "a = [1, 2, 3]; *a" visit_Starred = bad_node - # allow ellipsis, for now - # visit_Ellipsis = bad_node - blacklist = DetectBadNodes() blacklist.visit(module) bad = blacklist.bad @@ -3868,10 +3865,15 @@ def bad_node(self, node): py_default = 'None' c_default = "NULL" elif (isinstance(expr, ast.BinOp) or - (isinstance(expr, ast.UnaryOp) and not isinstance(expr.operand, ast.Num))): + (isinstance(expr, ast.UnaryOp) and + not (isinstance(expr.operand, ast.Num) or + (hasattr(ast, 'Constant') and + isinstance(expr.operand, ast.Constant) and + type(expr.operand.value) in (int, float, complex))) + )): c_default = kwargs.get("c_default") if not (isinstance(c_default, str) and c_default): - fail("When you specify an expression (" + repr(default) + ") as your default value,\nyou MUST specify a valid c_default.") + fail("When you specify an expression (" + repr(default) + ") as your default value,\nyou MUST specify a valid c_default." + ast.dump(expr)) py_default = default value = unknown elif isinstance(expr, ast.Attribute): @@ -3946,6 +3948,11 @@ def bad_node(self, node): self.function.parameters[parameter_name] = p def parse_converter(self, annotation): + if (hasattr(ast, 'Constant') and + isinstance(annotation, ast.Constant) and + type(annotation.value) is str): + return annotation.value, True, {} + if isinstance(annotation, ast.Str): return annotation.s, True, {} diff --git a/Tools/parser/unparse.py b/Tools/parser/unparse.py index 7e1cc4ea5db9..82c3c7768072 100644 --- a/Tools/parser/unparse.py +++ b/Tools/parser/unparse.py @@ -329,12 +329,6 @@ def _AsyncWith(self, t): self.leave() # expr - def _Bytes(self, t): - self.write(repr(t.s)) - - def _Str(self, tree): - self.write(repr(tree.s)) - def _JoinedStr(self, t): self.write("f") string = io.StringIO() @@ -352,10 +346,6 @@ def _fstring_JoinedStr(self, t, write): meth = getattr(self, "_fstring_" + type(value).__name__) meth(value, write) - def _fstring_Str(self, t, write): - value = t.s.replace("{", "{{").replace("}", "}}") - write(value) - def _fstring_Constant(self, t, write): assert isinstance(t.value, str) value = t.value.replace("{", "{{").replace("}", "}}") @@ -384,6 +374,7 @@ def _Name(self, t): def _write_constant(self, value): if isinstance(value, (float, complex)): + # Substitute overflowing decimal literal for AST infinities. self.write(repr(value).replace("inf", INFSTR)) else: self.write(repr(value)) @@ -398,16 +389,11 @@ def _Constant(self, t): else: interleave(lambda: self.write(", "), self._write_constant, value) self.write(")") + elif value is ...: + self.write("...") else: self._write_constant(t.value) - def _NameConstant(self, t): - self.write(repr(t.value)) - - def _Num(self, t): - # Substitute overflowing decimal literal for AST infinities. - self.write(repr(t.n).replace("inf", INFSTR)) - def _List(self, t): self.write("[") interleave(lambda: self.write(", "), self.dispatch, t.elts) @@ -539,8 +525,7 @@ def _Attribute(self,t): # Special case: 3.__abs__() is a syntax error, so if t.value # is an integer literal then we need to either parenthesize # it or add an extra space to get 3 .__abs__(). - if ((isinstance(t.value, ast.Num) and isinstance(t.value.n, int)) - or (isinstance(t.value, ast.Constant) and isinstance(t.value.value, int))): + if isinstance(t.value, ast.Constant) and isinstance(t.value.value, int): self.write(" ") self.write(".") self.write(t.attr) From webhook-mailer at python.org Thu Sep 27 11:25:53 2018 From: webhook-mailer at python.org (Ned Deily) Date: Thu, 27 Sep 2018 15:25:53 -0000 Subject: [Python-checkins] Fix markup for xml.sax entry in 3.7.1 notes (GH-9602) Message-ID: https://github.com/python/cpython/commit/273fc220b25933e443c82af6888eb1871d032fb8 commit: 273fc220b25933e443c82af6888eb1871d032fb8 branch: 3.7 author: Andr?s Delfino committer: Ned Deily date: 2018-09-27T11:25:47-04:00 summary: Fix markup for xml.sax entry in 3.7.1 notes (GH-9602) files: M Doc/whatsnew/3.7.rst diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 534b9a07a081..87a3ddbf1ecf 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2521,5 +2521,5 @@ In 3.7.1 the C API for Context Variables :ref:`was updated ` to use :c:type:`PyObject` pointers. See also :issue:`34762`. -:mod:`xml.dom.minidom` and mod:`xml.sax` modules no longer process +:mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process external entities by default. See also :issue:`17239`. From webhook-mailer at python.org Thu Sep 27 14:55:58 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Thu, 27 Sep 2018 18:55:58 -0000 Subject: [Python-checkins] bpo-34762: Update PyContext* to PyObject* in asyncio and decimal (GH-9609) Message-ID: https://github.com/python/cpython/commit/994269ccee5574f03cda6b018399347fc52bf330 commit: 994269ccee5574f03cda6b018399347fc52bf330 branch: master author: Yury Selivanov committer: GitHub date: 2018-09-27T14:55:55-04:00 summary: bpo-34762: Update PyContext* to PyObject* in asyncio and decimal (GH-9609) This fixes various compiler warnings. files: M Modules/_asynciomodule.c M Modules/_decimal/_decimal.c diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index fc91ebd854d0..6bf0fd6b0c94 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -61,7 +61,7 @@ typedef enum { PyObject_HEAD \ PyObject *prefix##_loop; \ PyObject *prefix##_callback0; \ - PyContext *prefix##_context0; \ + PyObject *prefix##_context0; \ PyObject *prefix##_callbacks; \ PyObject *prefix##_exception; \ PyObject *prefix##_result; \ @@ -81,7 +81,7 @@ typedef struct { PyObject *task_fut_waiter; PyObject *task_coro; PyObject *task_name; - PyContext *task_context; + PyObject *task_context; int task_must_cancel; int task_log_destroy_pending; } TaskObj; @@ -340,7 +340,7 @@ get_event_loop(void) static int -call_soon(PyObject *loop, PyObject *func, PyObject *arg, PyContext *ctx) +call_soon(PyObject *loop, PyObject *func, PyObject *arg, PyObject *ctx) { PyObject *handle; PyObject *stack[3]; @@ -451,7 +451,7 @@ future_schedule_callbacks(FutureObj *fut) PyObject *cb = PyTuple_GET_ITEM(cb_tup, 0); PyObject *ctx = PyTuple_GET_ITEM(cb_tup, 1); - if (call_soon(fut->fut_loop, cb, (PyObject *)fut, (PyContext *)ctx)) { + if (call_soon(fut->fut_loop, cb, (PyObject *)fut, ctx)) { /* If an error occurs in pure-Python implementation, all callbacks are cleared. */ Py_CLEAR(fut->fut_callbacks); @@ -619,7 +619,7 @@ future_get_result(FutureObj *fut, PyObject **result) } static PyObject * -future_add_done_callback(FutureObj *fut, PyObject *arg, PyContext *ctx) +future_add_done_callback(FutureObj *fut, PyObject *arg, PyObject *ctx) { if (!future_is_alive(fut)) { PyErr_SetString(PyExc_RuntimeError, "uninitialized Future object"); @@ -906,16 +906,15 @@ _asyncio_Future_add_done_callback_impl(FutureObj *self, PyObject *fn, /*[clinic end generated code: output=7ce635bbc9554c1e input=15ab0693a96e9533]*/ { if (context == NULL) { - context = (PyObject *)PyContext_CopyCurrent(); + context = PyContext_CopyCurrent(); if (context == NULL) { return NULL; } - PyObject *res = future_add_done_callback( - self, fn, (PyContext *)context); + PyObject *res = future_add_done_callback(self, fn, context); Py_DECREF(context); return res; } - return future_add_done_callback(self, fn, (PyContext *)context); + return future_add_done_callback(self, fn, context); } /*[clinic input] diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 5bce780cb7fb..1e58d3d5b779 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -122,7 +122,7 @@ incr_false(void) } -static PyContextVar *current_context_var; +static PyObject *current_context_var; /* Template for creating new thread contexts, calling Context() without * arguments and initializing the module_context on first access. */ @@ -1500,7 +1500,7 @@ init_current_context(void) } CTX(tl_context)->status = 0; - PyContextToken *tok = PyContextVar_Set(current_context_var, tl_context); + PyObject *tok = PyContextVar_Set(current_context_var, tl_context); if (tok == NULL) { Py_DECREF(tl_context); return NULL; @@ -1561,7 +1561,7 @@ PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) Py_INCREF(v); } - PyContextToken *tok = PyContextVar_Set(current_context_var, v); + PyObject *tok = PyContextVar_Set(current_context_var, v); Py_DECREF(v); if (tok == NULL) { return NULL; From webhook-mailer at python.org Thu Sep 27 15:33:29 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Thu, 27 Sep 2018 19:33:29 -0000 Subject: [Python-checkins] [3.7] bpo-34762: Update PyContext* refs to PyObject* in asyncio and decimal (GH-9610) Message-ID: https://github.com/python/cpython/commit/24cb7de15d3a5979425b281ab4f600f7c2b401f2 commit: 24cb7de15d3a5979425b281ab4f600f7c2b401f2 branch: 3.7 author: Yury Selivanov committer: GitHub date: 2018-09-27T15:33:23-04:00 summary: [3.7] bpo-34762: Update PyContext* refs to PyObject* in asyncio and decimal (GH-9610) files: M Modules/_asynciomodule.c M Modules/_decimal/_decimal.c diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index ebda10404eb5..0c161fb2bc7d 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -59,7 +59,7 @@ typedef enum { PyObject_HEAD \ PyObject *prefix##_loop; \ PyObject *prefix##_callback0; \ - PyContext *prefix##_context0; \ + PyObject *prefix##_context0; \ PyObject *prefix##_callbacks; \ PyObject *prefix##_exception; \ PyObject *prefix##_result; \ @@ -78,7 +78,7 @@ typedef struct { FutureObj_HEAD(task) PyObject *task_fut_waiter; PyObject *task_coro; - PyContext *task_context; + PyObject *task_context; int task_must_cancel; int task_log_destroy_pending; } TaskObj; @@ -337,7 +337,7 @@ get_event_loop(void) static int -call_soon(PyObject *loop, PyObject *func, PyObject *arg, PyContext *ctx) +call_soon(PyObject *loop, PyObject *func, PyObject *arg, PyObject *ctx) { PyObject *handle; PyObject *stack[3]; @@ -448,7 +448,7 @@ future_schedule_callbacks(FutureObj *fut) PyObject *cb = PyTuple_GET_ITEM(cb_tup, 0); PyObject *ctx = PyTuple_GET_ITEM(cb_tup, 1); - if (call_soon(fut->fut_loop, cb, (PyObject *)fut, (PyContext *)ctx)) { + if (call_soon(fut->fut_loop, cb, (PyObject *)fut, ctx)) { /* If an error occurs in pure-Python implementation, all callbacks are cleared. */ Py_CLEAR(fut->fut_callbacks); @@ -616,7 +616,7 @@ future_get_result(FutureObj *fut, PyObject **result) } static PyObject * -future_add_done_callback(FutureObj *fut, PyObject *arg, PyContext *ctx) +future_add_done_callback(FutureObj *fut, PyObject *arg, PyObject *ctx) { if (!future_is_alive(fut)) { PyErr_SetString(PyExc_RuntimeError, "uninitialized Future object"); @@ -903,16 +903,15 @@ _asyncio_Future_add_done_callback_impl(FutureObj *self, PyObject *fn, /*[clinic end generated code: output=7ce635bbc9554c1e input=15ab0693a96e9533]*/ { if (context == NULL) { - context = (PyObject *)PyContext_CopyCurrent(); + context = PyContext_CopyCurrent(); if (context == NULL) { return NULL; } - PyObject *res = future_add_done_callback( - self, fn, (PyContext *)context); + PyObject *res = future_add_done_callback(self, fn, context); Py_DECREF(context); return res; } - return future_add_done_callback(self, fn, (PyContext *)context); + return future_add_done_callback(self, fn, context); } /*[clinic input] diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 5bce780cb7fb..1e58d3d5b779 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -122,7 +122,7 @@ incr_false(void) } -static PyContextVar *current_context_var; +static PyObject *current_context_var; /* Template for creating new thread contexts, calling Context() without * arguments and initializing the module_context on first access. */ @@ -1500,7 +1500,7 @@ init_current_context(void) } CTX(tl_context)->status = 0; - PyContextToken *tok = PyContextVar_Set(current_context_var, tl_context); + PyObject *tok = PyContextVar_Set(current_context_var, tl_context); if (tok == NULL) { Py_DECREF(tl_context); return NULL; @@ -1561,7 +1561,7 @@ PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) Py_INCREF(v); } - PyContextToken *tok = PyContextVar_Set(current_context_var, v); + PyObject *tok = PyContextVar_Set(current_context_var, v); Py_DECREF(v); if (tok == NULL) { return NULL; From webhook-mailer at python.org Thu Sep 27 15:48:33 2018 From: webhook-mailer at python.org (Yury Selivanov) Date: Thu, 27 Sep 2018 19:48:33 -0000 Subject: [Python-checkins] bpo-34802: Fix asyncio.iscoroutine() docs (GH-9611) Message-ID: https://github.com/python/cpython/commit/59ee5b12938efbf534f2a19300a847bf6b23a77d commit: 59ee5b12938efbf534f2a19300a847bf6b23a77d branch: master author: Yury Selivanov committer: GitHub date: 2018-09-27T15:48:30-04:00 summary: bpo-34802: Fix asyncio.iscoroutine() docs (GH-9611) files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 1a504363946e..d7102b807b60 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -924,8 +924,7 @@ enforced. Return ``True`` if *obj* is a :ref:`coroutine object `. This method is different from :func:`inspect.iscoroutine` because - it returns ``True`` for generator-based coroutines decorated with - :func:`@coroutine `. + it returns ``True`` for generator-based coroutines. .. function:: iscoroutinefunction(func) From webhook-mailer at python.org Thu Sep 27 15:53:22 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 27 Sep 2018 19:53:22 -0000 Subject: [Python-checkins] bpo-34802: Fix asyncio.iscoroutine() docs (GH-9611) Message-ID: https://github.com/python/cpython/commit/85ccedc5b57ddda198e7176ba787e3896435c504 commit: 85ccedc5b57ddda198e7176ba787e3896435c504 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-27T12:53:18-07:00 summary: bpo-34802: Fix asyncio.iscoroutine() docs (GH-9611) (cherry picked from commit 59ee5b12938efbf534f2a19300a847bf6b23a77d) Co-authored-by: Yury Selivanov files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 5cbdfeff32f5..198bd7d07a01 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -893,8 +893,7 @@ enforced. Return ``True`` if *obj* is a :ref:`coroutine object `. This method is different from :func:`inspect.iscoroutine` because - it returns ``True`` for generator-based coroutines decorated with - :func:`@coroutine `. + it returns ``True`` for generator-based coroutines. .. function:: iscoroutinefunction(func) From webhook-mailer at python.org Thu Sep 27 15:54:38 2018 From: webhook-mailer at python.org (Berker Peksag) Date: Thu, 27 Sep 2018 19:54:38 -0000 Subject: [Python-checkins] bpo-34248: Add filename to error raised in {gnu, ndbm}.open() (GH-8590) Message-ID: https://github.com/python/cpython/commit/9df346bf98069a87de14a3c2f69009d800994c63 commit: 9df346bf98069a87de14a3c2f69009d800994c63 branch: master author: Zsolt Cserna committer: Berker Peksag date: 2018-09-27T22:54:34+03:00 summary: bpo-34248: Add filename to error raised in {gnu,ndbm}.open() (GH-8590) Report the filename to the exception when raising {gdbm,dbm.ndbm}.error in dbm.gnu.open() and dbm.ndbm.open() functions, so it gets printed when the exception is raised, and can also be obtained by the filename attribute of the exception object. files: A Misc/NEWS.d/next/Library/2018-07-31-23-00-09.bpo-34248.5U6wwc.rst M Lib/test/test_dbm_gnu.py M Lib/test/test_dbm_ndbm.py M Modules/_dbmmodule.c M Modules/_gdbmmodule.c diff --git a/Lib/test/test_dbm_gnu.py b/Lib/test/test_dbm_gnu.py index c96eff5a319e..16b7fe6fac17 100644 --- a/Lib/test/test_dbm_gnu.py +++ b/Lib/test/test_dbm_gnu.py @@ -144,6 +144,13 @@ def test_nonascii_filename(self): self.assertTrue(b'key' in db) self.assertEqual(db[b'key'], b'value') + def test_nonexisting_file(self): + nonexisting_file = 'nonexisting-file' + with self.assertRaises(gdbm.error) as cm: + gdbm.open(nonexisting_file) + self.assertIn(nonexisting_file, str(cm.exception)) + self.assertEqual(cm.exception.filename, nonexisting_file) + if __name__ == '__main__': unittest.main() diff --git a/Lib/test/test_dbm_ndbm.py b/Lib/test/test_dbm_ndbm.py index 49b88f5cccef..bd411da652e6 100644 --- a/Lib/test/test_dbm_ndbm.py +++ b/Lib/test/test_dbm_ndbm.py @@ -105,6 +105,12 @@ def test_nonascii_filename(self): self.assertTrue(b'key' in db) self.assertEqual(db[b'key'], b'value') + def test_nonexisting_file(self): + nonexisting_file = 'nonexisting-file' + with self.assertRaises(dbm.ndbm.error) as cm: + dbm.ndbm.open(nonexisting_file) + self.assertIn(nonexisting_file, str(cm.exception)) + self.assertEqual(cm.exception.filename, nonexisting_file) if __name__ == '__main__': diff --git a/Misc/NEWS.d/next/Library/2018-07-31-23-00-09.bpo-34248.5U6wwc.rst b/Misc/NEWS.d/next/Library/2018-07-31-23-00-09.bpo-34248.5U6wwc.rst new file mode 100644 index 000000000000..55215f60550b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-31-23-00-09.bpo-34248.5U6wwc.rst @@ -0,0 +1,3 @@ +Report filename in the exception raised when the database file cannot be opened +by :func:`dbm.gnu.open` and :func:`dbm.ndbm.open` due to OS-related error. +Patch by Zsolt Cserna. diff --git a/Modules/_dbmmodule.c b/Modules/_dbmmodule.c index 65761d83d380..081184a0b322 100644 --- a/Modules/_dbmmodule.c +++ b/Modules/_dbmmodule.c @@ -62,7 +62,7 @@ newdbmobject(const char *file, int flags, int mode) dp->di_size = -1; /* See issue #19296 */ if ( (dp->di_dbm = dbm_open((char *)file, flags, mode)) == 0 ) { - PyErr_SetFromErrno(DbmError); + PyErr_SetFromErrnoWithFilename(DbmError, file); Py_DECREF(dp); return NULL; } diff --git a/Modules/_gdbmmodule.c b/Modules/_gdbmmodule.c index 10560040e446..ceb744b99ba8 100644 --- a/Modules/_gdbmmodule.c +++ b/Modules/_gdbmmodule.c @@ -75,7 +75,7 @@ newdbmobject(const char *file, int flags, int mode) errno = 0; if ((dp->di_dbm = gdbm_open((char *)file, 0, flags, mode, NULL)) == 0) { if (errno != 0) - PyErr_SetFromErrno(DbmError); + PyErr_SetFromErrnoWithFilename(DbmError, file); else PyErr_SetString(DbmError, gdbm_strerror(gdbm_errno)); Py_DECREF(dp); From webhook-mailer at python.org Fri Sep 28 01:57:33 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 28 Sep 2018 05:57:33 -0000 Subject: [Python-checkins] bpo-34736: improve error message for invalid length b64decode inputs (GH-9563) Message-ID: https://github.com/python/cpython/commit/1fba2ffc37da52c08db51fe4360459990b0311c9 commit: 1fba2ffc37da52c08db51fe4360459990b0311c9 branch: master author: Tal Einat committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-27T22:57:22-07:00 summary: bpo-34736: improve error message for invalid length b64decode inputs (GH-9563) Improvements: 1. Include the number of valid data characters in the error message. 2. Mention "number of data characters" rather than "length". https://bugs.python.org/issue34736 files: M Lib/test/test_binascii.py M Modules/binascii.c diff --git a/Lib/test/test_binascii.py b/Lib/test/test_binascii.py index c5fcc1ac1647..572e50c3e25e 100644 --- a/Lib/test/test_binascii.py +++ b/Lib/test/test_binascii.py @@ -3,6 +3,7 @@ import unittest import binascii import array +import re # Note: "*_hex" functions are aliases for "(un)hexlify" b2a_functions = ['b2a_base64', 'b2a_hex', 'b2a_hqx', 'b2a_qp', 'b2a_uu', @@ -127,7 +128,10 @@ def assertIncorrectPadding(data): # Test base64 with invalid number of valid characters (1 mod 4) def assertInvalidLength(data): - with self.assertRaisesRegex(binascii.Error, r'(?i)invalid.+length'): + n_data_chars = len(re.sub(br'[^A-Za-z0-9/+]', br'', data)) + expected_errmsg_re = \ + r'(?i)Invalid.+number of data characters.+' + str(n_data_chars) + with self.assertRaisesRegex(binascii.Error, expected_errmsg_re): binascii.a2b_base64(self.type2test(data)) assertInvalidLength(b'a') diff --git a/Modules/binascii.c b/Modules/binascii.c index 5667018d6e1b..19589f906f72 100644 --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -438,6 +438,7 @@ binascii_a2b_base64_impl(PyObject *module, Py_buffer *data) { const unsigned char *ascii_data; unsigned char *bin_data; + unsigned char *bin_data_start; int leftbits = 0; unsigned char this_ch; unsigned int leftchar = 0; @@ -461,6 +462,7 @@ binascii_a2b_base64_impl(PyObject *module, Py_buffer *data) bin_data = _PyBytesWriter_Alloc(&writer, bin_len); if (bin_data == NULL) return NULL; + bin_data_start = bin_data; for( ; ascii_len > 0; ascii_len--, ascii_data++) { this_ch = *ascii_data; @@ -516,9 +518,11 @@ binascii_a2b_base64_impl(PyObject *module, Py_buffer *data) ** This is an invalid length, as there is no possible input that ** could encoded into such a base64 string. */ - PyErr_SetString(Error, - "Invalid base64-encoded string: " - "length cannot be 1 more than a multiple of 4"); + PyErr_Format(Error, + "Invalid base64-encoded string: " + "number of data characters (%d) cannot be 1 more " + "than a multiple of 4", + (bin_data - bin_data_start) / 3 * 4 + 1); } else { PyErr_SetString(Error, "Incorrect padding"); } From webhook-mailer at python.org Fri Sep 28 02:12:57 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 28 Sep 2018 06:12:57 -0000 Subject: [Python-checkins] bpo-34736: improve error message for invalid length b64decode inputs (GH-9563) Message-ID: https://github.com/python/cpython/commit/7e35081bc828291da5793db49ab45dee4fda5043 commit: 7e35081bc828291da5793db49ab45dee4fda5043 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-27T23:12:54-07:00 summary: bpo-34736: improve error message for invalid length b64decode inputs (GH-9563) Improvements: 1. Include the number of valid data characters in the error message. 2. Mention "number of data characters" rather than "length". https://bugs.python.org/issue34736 (cherry picked from commit 1fba2ffc37da52c08db51fe4360459990b0311c9) Co-authored-by: Tal Einat files: M Lib/test/test_binascii.py M Modules/binascii.c diff --git a/Lib/test/test_binascii.py b/Lib/test/test_binascii.py index 7418a9ce9848..8e37d55a64f9 100644 --- a/Lib/test/test_binascii.py +++ b/Lib/test/test_binascii.py @@ -3,6 +3,7 @@ import unittest import binascii import array +import re # Note: "*_hex" functions are aliases for "(un)hexlify" b2a_functions = ['b2a_base64', 'b2a_hex', 'b2a_hqx', 'b2a_qp', 'b2a_uu', @@ -127,7 +128,10 @@ def assertIncorrectPadding(data): # Test base64 with invalid number of valid characters (1 mod 4) def assertInvalidLength(data): - with self.assertRaisesRegex(binascii.Error, r'(?i)invalid.+length'): + n_data_chars = len(re.sub(br'[^A-Za-z0-9/+]', br'', data)) + expected_errmsg_re = \ + r'(?i)Invalid.+number of data characters.+' + str(n_data_chars) + with self.assertRaisesRegex(binascii.Error, expected_errmsg_re): binascii.a2b_base64(self.type2test(data)) assertInvalidLength(b'a') diff --git a/Modules/binascii.c b/Modules/binascii.c index 2df80affb695..d0d3f7d34b8f 100644 --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -438,6 +438,7 @@ binascii_a2b_base64_impl(PyObject *module, Py_buffer *data) { const unsigned char *ascii_data; unsigned char *bin_data; + unsigned char *bin_data_start; int leftbits = 0; unsigned char this_ch; unsigned int leftchar = 0; @@ -461,6 +462,7 @@ binascii_a2b_base64_impl(PyObject *module, Py_buffer *data) bin_data = _PyBytesWriter_Alloc(&writer, bin_len); if (bin_data == NULL) return NULL; + bin_data_start = bin_data; for( ; ascii_len > 0; ascii_len--, ascii_data++) { this_ch = *ascii_data; @@ -516,9 +518,11 @@ binascii_a2b_base64_impl(PyObject *module, Py_buffer *data) ** This is an invalid length, as there is no possible input that ** could encoded into such a base64 string. */ - PyErr_SetString(Error, - "Invalid base64-encoded string: " - "length cannot be 1 more than a multiple of 4"); + PyErr_Format(Error, + "Invalid base64-encoded string: " + "number of data characters (%d) cannot be 1 more " + "than a multiple of 4", + (bin_data - bin_data_start) / 3 * 4 + 1); } else { PyErr_SetString(Error, "Incorrect padding"); } From solipsis at pitrou.net Fri Sep 28 05:07:50 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 28 Sep 2018 09:07:50 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=-2 Message-ID: <20180928090750.1.762E25A5BD53A326@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_collections leaked [0, -7, 1] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogKJ5bcf', '--timeout', '7200'] From webhook-mailer at python.org Fri Sep 28 05:51:09 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 28 Sep 2018 09:51:09 -0000 Subject: [Python-checkins] bpo-34521: Fix tests in test_multiprocessing_spawn to use correctly CMSG_SPACE (GH-9613) Message-ID: https://github.com/python/cpython/commit/077061a7b24917aaf31057885c69919c5a553c88 commit: 077061a7b24917aaf31057885c69919c5a553c88 branch: master author: Pablo Galindo committer: Victor Stinner date: 2018-09-28T02:51:05-07:00 summary: bpo-34521: Fix tests in test_multiprocessing_spawn to use correctly CMSG_SPACE (GH-9613) After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots regarding tests in test_multiprocessing_spawn and after examining similar failures in test_socket, some errors in the calculation of ancillary data buffers were found in multiprocessing.reduction. CMSG_LEN() can often be used as the buffer size for recvmsg() to receive a single item of ancillary data, but RFC 3542 requires portable applications to use CMSG_SPACE() and thus include space for padding, even when the item will be the last in the buffer. The failures we experience are due to the usage of CMSG_LEN() instead of CMSG_SPACE(). files: M Lib/multiprocessing/reduction.py diff --git a/Lib/multiprocessing/reduction.py b/Lib/multiprocessing/reduction.py index 473fd59df61b..5593f0682f7f 100644 --- a/Lib/multiprocessing/reduction.py +++ b/Lib/multiprocessing/reduction.py @@ -154,7 +154,7 @@ def recvfds(sock, size): '''Receive an array of fds over an AF_UNIX socket.''' a = array.array('i') bytes_size = a.itemsize * size - msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_LEN(bytes_size)) + msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_SPACE(bytes_size)) if not msg and not ancdata: raise EOFError try: From webhook-mailer at python.org Fri Sep 28 07:31:56 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 28 Sep 2018 11:31:56 -0000 Subject: [Python-checkins] bpo-34521: Fix tests in test_multiprocessing_spawn to use correctly CMSG_SPACE (GH-9613) Message-ID: https://github.com/python/cpython/commit/5d33ee1595767d68800fda13bc68b7b01dba5117 commit: 5d33ee1595767d68800fda13bc68b7b01dba5117 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-28T04:31:51-07:00 summary: bpo-34521: Fix tests in test_multiprocessing_spawn to use correctly CMSG_SPACE (GH-9613) After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots regarding tests in test_multiprocessing_spawn and after examining similar failures in test_socket, some errors in the calculation of ancillary data buffers were found in multiprocessing.reduction. CMSG_LEN() can often be used as the buffer size for recvmsg() to receive a single item of ancillary data, but RFC 3542 requires portable applications to use CMSG_SPACE() and thus include space for padding, even when the item will be the last in the buffer. The failures we experience are due to the usage of CMSG_LEN() instead of CMSG_SPACE(). (cherry picked from commit 077061a7b24917aaf31057885c69919c5a553c88) Co-authored-by: Pablo Galindo files: M Lib/multiprocessing/reduction.py diff --git a/Lib/multiprocessing/reduction.py b/Lib/multiprocessing/reduction.py index c043c9a0dc16..f1b9d3ac99ce 100644 --- a/Lib/multiprocessing/reduction.py +++ b/Lib/multiprocessing/reduction.py @@ -150,7 +150,7 @@ def recvfds(sock, size): '''Receive an array of fds over an AF_UNIX socket.''' a = array.array('i') bytes_size = a.itemsize * size - msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_LEN(bytes_size)) + msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_SPACE(bytes_size)) if not msg and not ancdata: raise EOFError try: From webhook-mailer at python.org Fri Sep 28 09:08:00 2018 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 28 Sep 2018 13:08:00 -0000 Subject: [Python-checkins] bpo-34521: Fix tests in test_multiprocessing_spawn to use correctly CMSG_SPACE (GH-9613) (GH-9619) Message-ID: https://github.com/python/cpython/commit/007fda436f707ac95c2fa8f8886efd9e09d5b630 commit: 007fda436f707ac95c2fa8f8886efd9e09d5b630 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Victor Stinner date: 2018-09-28T06:07:56-07:00 summary: bpo-34521: Fix tests in test_multiprocessing_spawn to use correctly CMSG_SPACE (GH-9613) (GH-9619) After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots regarding tests in test_multiprocessing_spawn and after examining similar failures in test_socket, some errors in the calculation of ancillary data buffers were found in multiprocessing.reduction. CMSG_LEN() can often be used as the buffer size for recvmsg() to receive a single item of ancillary data, but RFC 3542 requires portable applications to use CMSG_SPACE() and thus include space for padding, even when the item will be the last in the buffer. The failures we experience are due to the usage of CMSG_LEN() instead of CMSG_SPACE(). (cherry picked from commit 077061a7b24917aaf31057885c69919c5a553c88) Co-authored-by: Pablo Galindo files: M Lib/multiprocessing/reduction.py diff --git a/Lib/multiprocessing/reduction.py b/Lib/multiprocessing/reduction.py index deca19ccad7b..0f5432593a58 100644 --- a/Lib/multiprocessing/reduction.py +++ b/Lib/multiprocessing/reduction.py @@ -150,7 +150,7 @@ def recvfds(sock, size): '''Receive an array of fds over an AF_UNIX socket.''' a = array.array('i') bytes_size = a.itemsize * size - msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_LEN(bytes_size)) + msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_SPACE(bytes_size)) if not msg and not ancdata: raise EOFError try: From webhook-mailer at python.org Fri Sep 28 11:40:13 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 28 Sep 2018 15:40:13 -0000 Subject: [Python-checkins] bpo-34687: Update asyncio doc for ProactorEventLoop (GH-9623) Message-ID: https://github.com/python/cpython/commit/37aae9dcf18753b8ffda99d1a5758a90af852464 commit: 37aae9dcf18753b8ffda99d1a5758a90af852464 branch: master author: Victor Stinner committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-28T08:40:08-07:00 summary: bpo-34687: Update asyncio doc for ProactorEventLoop (GH-9623) Since ProactorEventLoop is now the default in 3.8, remove examples using it explicitly on Windows. https://bugs.python.org/issue34687 files: M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-protocol.rst M Doc/library/asyncio-subprocess.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 50d6ea4b1389..76c8ce9943ef 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1432,15 +1432,6 @@ on all platforms. Availability: Windows. - An example how to use :class:`ProactorEventLoop` on Windows:: - - import asyncio - import sys - - if sys.platform == 'win32': - loop = asyncio.ProactorEventLoop() - asyncio.set_event_loop(loop) - .. seealso:: `MSDN documentation on I/O Completion Ports diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst index bdfdcf7ddb6e..14ec31a6d5d2 100644 --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -1030,10 +1030,6 @@ The subprocess is created by th :meth:`loop.subprocess_exec` method:: data = bytes(protocol.output) return data.decode('ascii').rstrip() - if sys.platform == "win32": - asyncio.set_event_loop_policy( - asyncio.WindowsProactorEventLoopPolicy()) - date = asyncio.run(get_date()) print(f"Current date: {date}") diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 0bcf66175ce3..a2cf51734848 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -344,10 +344,6 @@ function:: await proc.wait() return line - if sys.platform == "win32": - asyncio.set_event_loop_policy( - asyncio.WindowsProactorEventLoopPolicy()) - date = asyncio.run(get_date()) print(f"Current date: {date}") From webhook-mailer at python.org Fri Sep 28 15:39:51 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Fri, 28 Sep 2018 19:39:51 -0000 Subject: [Python-checkins] bpo-34797: Convert heapq to the argument clinic (GH-9560) Message-ID: https://github.com/python/cpython/commit/e2f48bf0e851dc79d888c70fd24c631a8a9e1030 commit: e2f48bf0e851dc79d888c70fd24c631a8a9e1030 branch: master author: Pablo Galindo committer: Raymond Hettinger date: 2018-09-28T12:39:43-07:00 summary: bpo-34797: Convert heapq to the argument clinic (GH-9560) files: A Modules/clinic/_heapqmodule.c.h M Modules/_heapqmodule.c diff --git a/Modules/_heapqmodule.c b/Modules/_heapqmodule.c index b499e1f668aa..a84cade3aaa1 100644 --- a/Modules/_heapqmodule.c +++ b/Modules/_heapqmodule.c @@ -8,6 +8,13 @@ annotated by Fran?ois Pinard, and converted to C by Raymond Hettinger. #include "Python.h" +#include "clinic/_heapqmodule.c.h" + +/*[clinic input] +module _heapq +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=d7cca0a2e4c0ceb3]*/ + static int siftdown(PyListObject *heap, Py_ssize_t startpos, Py_ssize_t pos) { @@ -96,14 +103,20 @@ siftup(PyListObject *heap, Py_ssize_t pos) return siftdown(heap, startpos, pos); } -static PyObject * -heappush(PyObject *self, PyObject *args) -{ - PyObject *heap, *item; +/*[clinic input] +_heapq.heappush - if (!PyArg_UnpackTuple(args, "heappush", 2, 2, &heap, &item)) - return NULL; + heap: object + item: object + / +Push item onto heap, maintaining the heap invariant. +[clinic start generated code]*/ + +static PyObject * +_heapq_heappush_impl(PyObject *module, PyObject *heap, PyObject *item) +/*[clinic end generated code: output=912c094f47663935 input=7913545cb5118842]*/ +{ if (!PyList_Check(heap)) { PyErr_SetString(PyExc_TypeError, "heap argument must be a list"); return NULL; @@ -117,9 +130,6 @@ heappush(PyObject *self, PyObject *args) Py_RETURN_NONE; } -PyDoc_STRVAR(heappush_doc, -"heappush(heap, item) -> None. Push item onto heap, maintaining the heap invariant."); - static PyObject * heappop_internal(PyObject *heap, int siftup_func(PyListObject *, Py_ssize_t)) { @@ -157,22 +167,26 @@ heappop_internal(PyObject *heap, int siftup_func(PyListObject *, Py_ssize_t)) return returnitem; } +/*[clinic input] +_heapq.heappop + + heap: object + / + +Pop the smallest item off the heap, maintaining the heap invariant. +[clinic start generated code]*/ + static PyObject * -heappop(PyObject *self, PyObject *heap) +_heapq_heappop(PyObject *module, PyObject *heap) +/*[clinic end generated code: output=e1bbbc9866bce179 input=9bd36317b806033d]*/ { return heappop_internal(heap, siftup); } -PyDoc_STRVAR(heappop_doc, -"Pop the smallest item off the heap, maintaining the heap invariant."); - static PyObject * -heapreplace_internal(PyObject *args, int siftup_func(PyListObject *, Py_ssize_t)) +heapreplace_internal(PyObject *heap, PyObject *item, int siftup_func(PyListObject *, Py_ssize_t)) { - PyObject *heap, *item, *returnitem; - - if (!PyArg_UnpackTuple(args, "heapreplace", 2, 2, &heap, &item)) - return NULL; + PyObject *returnitem; if (!PyList_Check(heap)) { PyErr_SetString(PyExc_TypeError, "heap argument must be a list"); @@ -194,31 +208,52 @@ heapreplace_internal(PyObject *args, int siftup_func(PyListObject *, Py_ssize_t) return returnitem; } + +/*[clinic input] +_heapq.heapreplace + + heap: object + item: object + / + +Pop and return the current smallest value, and add the new item. + +This is more efficient than heappop() followed by heappush(), and can be +more appropriate when using a fixed-size heap. Note that the value +returned may be larger than item! That constrains reasonable uses of +this routine unless written as part of a conditional replacement: + + if item > heap[0]: + item = heapreplace(heap, item) +[clinic start generated code]*/ + static PyObject * -heapreplace(PyObject *self, PyObject *args) +_heapq_heapreplace_impl(PyObject *module, PyObject *heap, PyObject *item) +/*[clinic end generated code: output=82ea55be8fbe24b4 input=e57ae8f4ecfc88e3]*/ { - return heapreplace_internal(args, siftup); + return heapreplace_internal(heap, item, siftup); } -PyDoc_STRVAR(heapreplace_doc, -"heapreplace(heap, item) -> value. Pop and return the current smallest value, and add the new item.\n\ -\n\ -This is more efficient than heappop() followed by heappush(), and can be\n\ -more appropriate when using a fixed-size heap. Note that the value\n\ -returned may be larger than item! That constrains reasonable uses of\n\ -this routine unless written as part of a conditional replacement:\n\n\ - if item > heap[0]:\n\ - item = heapreplace(heap, item)\n"); +/*[clinic input] +_heapq.heappushpop + + heap: object + item: object + / + +Push item on the heap, then pop and return the smallest item from the heap. + +The combined action runs more efficiently than heappush() followed by +a separate call to heappop(). +[clinic start generated code]*/ static PyObject * -heappushpop(PyObject *self, PyObject *args) +_heapq_heappushpop_impl(PyObject *module, PyObject *heap, PyObject *item) +/*[clinic end generated code: output=67231dc98ed5774f input=eb48c90ba77b2214]*/ { - PyObject *heap, *item, *returnitem; + PyObject *returnitem; int cmp; - if (!PyArg_UnpackTuple(args, "heappushpop", 2, 2, &heap, &item)) - return NULL; - if (!PyList_Check(heap)) { PyErr_SetString(PyExc_TypeError, "heap argument must be a list"); return NULL; @@ -252,11 +287,6 @@ heappushpop(PyObject *self, PyObject *args) return returnitem; } -PyDoc_STRVAR(heappushpop_doc, -"heappushpop(heap, item) -> value. Push item on the heap, then pop and return the smallest item\n\ -from the heap. The combined action runs more efficiently than\n\ -heappush() followed by a separate call to heappop()."); - static Py_ssize_t keep_top_bit(Py_ssize_t n) { @@ -353,15 +383,22 @@ heapify_internal(PyObject *heap, int siftup_func(PyListObject *, Py_ssize_t)) Py_RETURN_NONE; } +/*[clinic input] +_heapq.heapify + + heap: object + / + +Transform list into a heap, in-place, in O(len(heap)) time. +[clinic start generated code]*/ + static PyObject * -heapify(PyObject *self, PyObject *heap) +_heapq_heapify(PyObject *module, PyObject *heap) +/*[clinic end generated code: output=11483f23627c4616 input=872c87504b8de970]*/ { return heapify_internal(heap, siftup); } -PyDoc_STRVAR(heapify_doc, -"Transform list into a heap, in-place, in O(len(heap)) time."); - static int siftdown_max(PyListObject *heap, Py_ssize_t startpos, Py_ssize_t pos) { @@ -450,48 +487,68 @@ siftup_max(PyListObject *heap, Py_ssize_t pos) return siftdown_max(heap, startpos, pos); } + +/*[clinic input] +_heapq._heappop_max + + heap: object + / + +Maxheap variant of heappop. +[clinic start generated code]*/ + static PyObject * -heappop_max(PyObject *self, PyObject *heap) +_heapq__heappop_max(PyObject *module, PyObject *heap) +/*[clinic end generated code: output=acd30acf6384b13c input=62ede3ba9117f541]*/ { return heappop_internal(heap, siftup_max); } -PyDoc_STRVAR(heappop_max_doc, "Maxheap variant of heappop."); +/*[clinic input] +_heapq._heapreplace_max + + heap: object + item: object + / + +Maxheap variant of heapreplace. +[clinic start generated code]*/ static PyObject * -heapreplace_max(PyObject *self, PyObject *args) +_heapq__heapreplace_max_impl(PyObject *module, PyObject *heap, + PyObject *item) +/*[clinic end generated code: output=8ad7545e4a5e8adb input=6d8f25131e0f0e5f]*/ { - return heapreplace_internal(args, siftup_max); + return heapreplace_internal(heap, item, siftup_max); } -PyDoc_STRVAR(heapreplace_max_doc, "Maxheap variant of heapreplace"); +/*[clinic input] +_heapq._heapify_max + + heap: object + / + +Maxheap variant of heapify. +[clinic start generated code]*/ static PyObject * -heapify_max(PyObject *self, PyObject *heap) +_heapq__heapify_max(PyObject *module, PyObject *heap) +/*[clinic end generated code: output=1c6bb6b60d6a2133 input=cdfcc6835b14110d]*/ { return heapify_internal(heap, siftup_max); } -PyDoc_STRVAR(heapify_max_doc, "Maxheap variant of heapify."); static PyMethodDef heapq_methods[] = { - {"heappush", (PyCFunction)heappush, - METH_VARARGS, heappush_doc}, - {"heappushpop", (PyCFunction)heappushpop, - METH_VARARGS, heappushpop_doc}, - {"heappop", (PyCFunction)heappop, - METH_O, heappop_doc}, - {"heapreplace", (PyCFunction)heapreplace, - METH_VARARGS, heapreplace_doc}, - {"heapify", (PyCFunction)heapify, - METH_O, heapify_doc}, - {"_heappop_max", (PyCFunction)heappop_max, - METH_O, heappop_max_doc}, - {"_heapreplace_max",(PyCFunction)heapreplace_max, - METH_VARARGS, heapreplace_max_doc}, - {"_heapify_max", (PyCFunction)heapify_max, - METH_O, heapify_max_doc}, - {NULL, NULL} /* sentinel */ + _HEAPQ_HEAPPUSH_METHODDEF + _HEAPQ_HEAPPUSHPOP_METHODDEF + _HEAPQ_HEAPPOP_METHODDEF + _HEAPQ_HEAPREPLACE_METHODDEF + _HEAPQ_HEAPIFY_METHODDEF + _HEAPQ__HEAPPOP_MAX_METHODDEF + _HEAPQ__HEAPIFY_MAX_METHODDEF + _HEAPQ__HEAPREPLACE_MAX_METHODDEF + {NULL, NULL} /* sentinel */ }; PyDoc_STRVAR(module_doc, diff --git a/Modules/clinic/_heapqmodule.c.h b/Modules/clinic/_heapqmodule.c.h new file mode 100644 index 000000000000..5e346df241f0 --- /dev/null +++ b/Modules/clinic/_heapqmodule.c.h @@ -0,0 +1,172 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +PyDoc_STRVAR(_heapq_heappush__doc__, +"heappush($module, heap, item, /)\n" +"--\n" +"\n" +"Push item onto heap, maintaining the heap invariant."); + +#define _HEAPQ_HEAPPUSH_METHODDEF \ + {"heappush", (PyCFunction)_heapq_heappush, METH_FASTCALL, _heapq_heappush__doc__}, + +static PyObject * +_heapq_heappush_impl(PyObject *module, PyObject *heap, PyObject *item); + +static PyObject * +_heapq_heappush(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *heap; + PyObject *item; + + if (!_PyArg_UnpackStack(args, nargs, "heappush", + 2, 2, + &heap, &item)) { + goto exit; + } + return_value = _heapq_heappush_impl(module, heap, item); + +exit: + return return_value; +} + +PyDoc_STRVAR(_heapq_heappop__doc__, +"heappop($module, heap, /)\n" +"--\n" +"\n" +"Pop the smallest item off the heap, maintaining the heap invariant."); + +#define _HEAPQ_HEAPPOP_METHODDEF \ + {"heappop", (PyCFunction)_heapq_heappop, METH_O, _heapq_heappop__doc__}, + +PyDoc_STRVAR(_heapq_heapreplace__doc__, +"heapreplace($module, heap, item, /)\n" +"--\n" +"\n" +"Pop and return the current smallest value, and add the new item.\n" +"\n" +"This is more efficient than heappop() followed by heappush(), and can be\n" +"more appropriate when using a fixed-size heap. Note that the value\n" +"returned may be larger than item! That constrains reasonable uses of\n" +"this routine unless written as part of a conditional replacement:\n" +"\n" +" if item > heap[0]:\n" +" item = heapreplace(heap, item)"); + +#define _HEAPQ_HEAPREPLACE_METHODDEF \ + {"heapreplace", (PyCFunction)_heapq_heapreplace, METH_FASTCALL, _heapq_heapreplace__doc__}, + +static PyObject * +_heapq_heapreplace_impl(PyObject *module, PyObject *heap, PyObject *item); + +static PyObject * +_heapq_heapreplace(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *heap; + PyObject *item; + + if (!_PyArg_UnpackStack(args, nargs, "heapreplace", + 2, 2, + &heap, &item)) { + goto exit; + } + return_value = _heapq_heapreplace_impl(module, heap, item); + +exit: + return return_value; +} + +PyDoc_STRVAR(_heapq_heappushpop__doc__, +"heappushpop($module, heap, item, /)\n" +"--\n" +"\n" +"Push item on the heap, then pop and return the smallest item from the heap.\n" +"\n" +"The combined action runs more efficiently than heappush() followed by\n" +"a separate call to heappop()."); + +#define _HEAPQ_HEAPPUSHPOP_METHODDEF \ + {"heappushpop", (PyCFunction)_heapq_heappushpop, METH_FASTCALL, _heapq_heappushpop__doc__}, + +static PyObject * +_heapq_heappushpop_impl(PyObject *module, PyObject *heap, PyObject *item); + +static PyObject * +_heapq_heappushpop(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *heap; + PyObject *item; + + if (!_PyArg_UnpackStack(args, nargs, "heappushpop", + 2, 2, + &heap, &item)) { + goto exit; + } + return_value = _heapq_heappushpop_impl(module, heap, item); + +exit: + return return_value; +} + +PyDoc_STRVAR(_heapq_heapify__doc__, +"heapify($module, heap, /)\n" +"--\n" +"\n" +"Transform list into a heap, in-place, in O(len(heap)) time."); + +#define _HEAPQ_HEAPIFY_METHODDEF \ + {"heapify", (PyCFunction)_heapq_heapify, METH_O, _heapq_heapify__doc__}, + +PyDoc_STRVAR(_heapq__heappop_max__doc__, +"_heappop_max($module, heap, /)\n" +"--\n" +"\n" +"Maxheap variant of heappop."); + +#define _HEAPQ__HEAPPOP_MAX_METHODDEF \ + {"_heappop_max", (PyCFunction)_heapq__heappop_max, METH_O, _heapq__heappop_max__doc__}, + +PyDoc_STRVAR(_heapq__heapreplace_max__doc__, +"_heapreplace_max($module, heap, item, /)\n" +"--\n" +"\n" +"Maxheap variant of heapreplace."); + +#define _HEAPQ__HEAPREPLACE_MAX_METHODDEF \ + {"_heapreplace_max", (PyCFunction)_heapq__heapreplace_max, METH_FASTCALL, _heapq__heapreplace_max__doc__}, + +static PyObject * +_heapq__heapreplace_max_impl(PyObject *module, PyObject *heap, + PyObject *item); + +static PyObject * +_heapq__heapreplace_max(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *heap; + PyObject *item; + + if (!_PyArg_UnpackStack(args, nargs, "_heapreplace_max", + 2, 2, + &heap, &item)) { + goto exit; + } + return_value = _heapq__heapreplace_max_impl(module, heap, item); + +exit: + return return_value; +} + +PyDoc_STRVAR(_heapq__heapify_max__doc__, +"_heapify_max($module, heap, /)\n" +"--\n" +"\n" +"Maxheap variant of heapify."); + +#define _HEAPQ__HEAPIFY_MAX_METHODDEF \ + {"_heapify_max", (PyCFunction)_heapq__heapify_max, METH_O, _heapq__heapify_max__doc__}, +/*[clinic end generated code: output=0bb0dd0df473ab14 input=a9049054013a1b77]*/ From webhook-mailer at python.org Sat Sep 29 01:48:50 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sat, 29 Sep 2018 05:48:50 -0000 Subject: [Python-checkins] Fix incorrect error handling in _pickle.Unpickler.__init__() (GH-9630) Message-ID: https://github.com/python/cpython/commit/4b430e5f6954ef4b248e95bfb4087635dcdefc6d commit: 4b430e5f6954ef4b248e95bfb4087635dcdefc6d branch: master author: Zackery Spytz committer: Serhiy Storchaka date: 2018-09-29T08:48:46+03:00 summary: Fix incorrect error handling in _pickle.Unpickler.__init__() (GH-9630) _pickle.Unpickler.__init__() should return -1 if Pdata_New() fails, not 1. files: M Modules/_pickle.c diff --git a/Modules/_pickle.c b/Modules/_pickle.c index 3588e33f0971..0e3bd225dbd8 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -6792,7 +6792,7 @@ _pickle_Unpickler___init___impl(UnpicklerObject *self, PyObject *file, self->stack = (Pdata *)Pdata_New(); if (self->stack == NULL) - return 1; + return -1; self->memo_size = 32; self->memo = _Unpickler_NewMemo(self->memo_size); From webhook-mailer at python.org Sat Sep 29 02:01:51 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 29 Sep 2018 06:01:51 -0000 Subject: [Python-checkins] Fix incorrect error handling in _pickle.Unpickler.__init__() (GH-9630) Message-ID: https://github.com/python/cpython/commit/758ad54dac6360657173d1c56f60510e5cdd08b6 commit: 758ad54dac6360657173d1c56f60510e5cdd08b6 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-28T23:01:48-07:00 summary: Fix incorrect error handling in _pickle.Unpickler.__init__() (GH-9630) _pickle.Unpickler.__init__() should return -1 if Pdata_New() fails, not 1. (cherry picked from commit 4b430e5f6954ef4b248e95bfb4087635dcdefc6d) Co-authored-by: Zackery Spytz files: M Modules/_pickle.c diff --git a/Modules/_pickle.c b/Modules/_pickle.c index 34ce38c5d3b2..ba8962d686de 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -6804,7 +6804,7 @@ _pickle_Unpickler___init___impl(UnpicklerObject *self, PyObject *file, self->stack = (Pdata *)Pdata_New(); if (self->stack == NULL) - return 1; + return -1; self->memo_size = 32; self->memo = _Unpickler_NewMemo(self->memo_size); From webhook-mailer at python.org Sat Sep 29 02:12:38 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 29 Sep 2018 06:12:38 -0000 Subject: [Python-checkins] Fix incorrect error handling in _pickle.Unpickler.__init__() (GH-9630) Message-ID: https://github.com/python/cpython/commit/207bb5fda52c352d20be1ca4e7fc85cb513d7cf4 commit: 207bb5fda52c352d20be1ca4e7fc85cb513d7cf4 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-28T23:12:35-07:00 summary: Fix incorrect error handling in _pickle.Unpickler.__init__() (GH-9630) _pickle.Unpickler.__init__() should return -1 if Pdata_New() fails, not 1. (cherry picked from commit 4b430e5f6954ef4b248e95bfb4087635dcdefc6d) Co-authored-by: Zackery Spytz files: M Modules/_pickle.c diff --git a/Modules/_pickle.c b/Modules/_pickle.c index 93bc1c6fee7f..0a7e20e10592 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -6713,7 +6713,7 @@ _pickle_Unpickler___init___impl(UnpicklerObject *self, PyObject *file, self->stack = (Pdata *)Pdata_New(); if (self->stack == NULL) - return 1; + return -1; self->memo_size = 32; self->memo = _Unpickler_NewMemo(self->memo_size); From solipsis at pitrou.net Sat Sep 29 05:08:38 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 29 Sep 2018 09:08:38 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=2 Message-ID: <20180929090838.1.F488D7B4315A2FA1@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_asyncio leaked [3, 0, 0] memory blocks, sum=3 test_collections leaked [-7, 1, 0] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_fork leaked [2, 0, -1] memory blocks, sum=1 test_multiprocessing_forkserver leaked [-2, 2, 0] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogs7i4k5', '--timeout', '7200'] From webhook-mailer at python.org Sat Sep 29 06:50:42 2018 From: webhook-mailer at python.org (Eric V. Smith) Date: Sat, 29 Sep 2018 10:50:42 -0000 Subject: [Python-checkins] Fix astuple in dataclasses documentation (GH-9631) Message-ID: https://github.com/python/cpython/commit/508d8205121868ef24cca99f1f952558b1ff2f2e commit: 508d8205121868ef24cca99f1f952558b1ff2f2e branch: master author: ??? (Bang Seongbeom) committer: Eric V. Smith date: 2018-09-29T06:50:31-04:00 summary: Fix astuple in dataclasses documentation (GH-9631) files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index acfd13a712b9..072a500d9f87 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -308,7 +308,7 @@ Module-level decorators, classes, and functions Raises :exc:`TypeError` if ``instance`` is not a dataclass instance. -.. function:: astuple(*, tuple_factory=tuple) +.. function:: astuple(instance, *, tuple_factory=tuple) Converts the dataclass ``instance`` to a tuple (by using the factory function ``tuple_factory``). Each dataclass is converted From webhook-mailer at python.org Sat Sep 29 06:55:17 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 29 Sep 2018 10:55:17 -0000 Subject: [Python-checkins] Fix astuple in dataclasses documentation (GH-9631) Message-ID: https://github.com/python/cpython/commit/4c1b2ad44f67ed19b7acf4fc09510ec3f5d081bc commit: 4c1b2ad44f67ed19b7acf4fc09510ec3f5d081bc branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-29T03:55:14-07:00 summary: Fix astuple in dataclasses documentation (GH-9631) (cherry picked from commit 508d8205121868ef24cca99f1f952558b1ff2f2e) Co-authored-by: ??? (Bang Seongbeom) files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 2efa2c51a459..fe0feeda8b87 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -309,7 +309,7 @@ Module-level decorators, classes, and functions Raises :exc:`TypeError` if ``instance`` is not a dataclass instance. -.. function:: astuple(*, tuple_factory=tuple) +.. function:: astuple(instance, *, tuple_factory=tuple) Converts the dataclass ``instance`` to a tuple (by using the factory function ``tuple_factory``). Each dataclass is converted From webhook-mailer at python.org Sat Sep 29 12:07:19 2018 From: webhook-mailer at python.org (Brian Curtin) Date: Sat, 29 Sep 2018 16:07:19 -0000 Subject: [Python-checkins] bpo-31370: Remove references to threadless builds (#8805) Message-ID: https://github.com/python/cpython/commit/eef059657d6b10babdb4831e1148d60cc644ee9a commit: eef059657d6b10babdb4831e1148d60cc644ee9a branch: master author: Zackery Spytz committer: Brian Curtin date: 2018-09-29T10:07:11-06:00 summary: bpo-31370: Remove references to threadless builds (#8805) Support for threadless builds was removed in a6a4dc81. files: M Doc/c-api/init.rst M Doc/library/queue.rst M Lib/test/_test_multiprocessing.py M Python/ceval.c diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 694b4669eea8..e51a2555f6ad 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -842,18 +842,18 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyEval_SaveThread() - Release the global interpreter lock (if it has been created and thread - support is enabled) and reset the thread state to *NULL*, returning the - previous thread state (which is not *NULL*). If the lock has been created, - the current thread must have acquired it. + Release the global interpreter lock (if it has been created) and reset the + thread state to *NULL*, returning the previous thread state (which is not + *NULL*). If the lock has been created, the current thread must have + acquired it. .. c:function:: void PyEval_RestoreThread(PyThreadState *tstate) - Acquire the global interpreter lock (if it has been created and thread - support is enabled) and set the thread state to *tstate*, which must not be - *NULL*. If the lock has been created, the current thread must not have - acquired it, otherwise deadlock ensues. + Acquire the global interpreter lock (if it has been created) and set the + thread state to *tstate*, which must not be *NULL*. If the lock has been + created, the current thread must not have acquired it, otherwise deadlock + ensues. .. c:function:: PyThreadState* PyThreadState_Get() diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 1520faa9b83f..6106d0cd381f 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -11,9 +11,7 @@ The :mod:`queue` module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads. The :class:`Queue` class in this -module implements all the required locking semantics. It depends on the -availability of thread support in Python; see the :mod:`threading` -module. +module implements all the required locking semantics. The module implements three types of queue, which differ only in the order in which the entries are retrieved. In a :abbr:`FIFO (first-in, first-out)` diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index bb9eb240a8a2..d728091bab1b 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -30,9 +30,6 @@ _multiprocessing = test.support.import_module('_multiprocessing') # Skip tests if sem_open implementation is broken. test.support.import_module('multiprocessing.synchronize') -# import threading after _multiprocessing to raise a more relevant error -# message: "No module named _multiprocessing". _multiprocessing is not compiled -# without thread support. import threading import multiprocessing.connection diff --git a/Python/ceval.c b/Python/ceval.c index d0f9915b4f0a..1a8c9e236493 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -237,7 +237,7 @@ PyEval_ReInitThreads(void) } /* This function is used to signal that async exceptions are waiting to be - raised, therefore it is also useful in non-threaded builds. */ + raised. */ void _PyEval_SignalAsyncExc(void) @@ -245,10 +245,6 @@ _PyEval_SignalAsyncExc(void) SIGNAL_ASYNC_EXC(); } -/* Functions save_thread and restore_thread are always defined so - dynamically loaded modules needn't be compiled separately for use - with and without threads: */ - PyThreadState * PyEval_SaveThread(void) { From webhook-mailer at python.org Sat Sep 29 12:15:59 2018 From: webhook-mailer at python.org (Brian Curtin) Date: Sat, 29 Sep 2018 16:15:59 -0000 Subject: [Python-checkins] bpo-27351: Fix ConfigParser.read() documentation and docstring (GH-8123) Message-ID: https://github.com/python/cpython/commit/e45473e3ca31e5b78dc85cab575f5bb60d5b7f8f commit: e45473e3ca31e5b78dc85cab575f5bb60d5b7f8f branch: master author: Zackery Spytz committer: Brian Curtin date: 2018-09-29T10:15:55-06:00 summary: bpo-27351: Fix ConfigParser.read() documentation and docstring (GH-8123) Switch "list" with "iterable" to match with the implementation. files: M Doc/library/configparser.rst M Lib/configparser.py diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index 1e5f25f36919..0ae466e7d4f8 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -963,16 +963,17 @@ ConfigParser Objects .. method:: read(filenames, encoding=None) - Attempt to read and parse a list of filenames, returning a list of + Attempt to read and parse an iterable of filenames, returning a list of filenames which were successfully parsed. If *filenames* is a string, a :class:`bytes` object or a :term:`path-like object`, it is treated as a single filename. If a file named in *filenames* cannot be opened, that - file will be ignored. This is designed so that you can specify a list of - potential configuration file locations (for example, the current - directory, the user's home directory, and some system-wide directory), - and all existing configuration files in the list will be read. + file will be ignored. This is designed so that you can specify an + iterable of potential configuration file locations (for example, the + current directory, the user's home directory, and some system-wide + directory), and all existing configuration files in the iterable will be + read. If none of the named files exist, the :class:`ConfigParser` instance will contain an empty dataset. An application which requires diff --git a/Lib/configparser.py b/Lib/configparser.py index 4a16101c7a3a..79a991084b8f 100644 --- a/Lib/configparser.py +++ b/Lib/configparser.py @@ -80,7 +80,7 @@ Return list of configuration options for the named section. read(filenames, encoding=None) - Read and parse the list of named configuration files, given by + Read and parse the iterable of named configuration files, given by name. A single filename is also allowed. Non-existing files are ignored. Return list of successfully read files. @@ -677,13 +677,13 @@ def options(self, section): return list(opts.keys()) def read(self, filenames, encoding=None): - """Read and parse a filename or a list of filenames. + """Read and parse a filename or an iterable of filenames. Files that cannot be opened are silently ignored; this is - designed so that you can specify a list of potential + designed so that you can specify an iterable of potential configuration file locations (e.g. current directory, user's home directory, systemwide directory), and all existing - configuration files in the list will be read. A single + configuration files in the iterable will be read. A single filename may also be given. Return list of successfully read files. From webhook-mailer at python.org Sat Sep 29 12:33:08 2018 From: webhook-mailer at python.org (Brian Curtin) Date: Sat, 29 Sep 2018 16:33:08 -0000 Subject: [Python-checkins] bpo-27351: Fix ConfigParser.read() documentation and docstring (GH-8123) Message-ID: https://github.com/python/cpython/commit/b0b8f9bd4e6f78ac7383b4e56cfb6cbacc77da89 commit: b0b8f9bd4e6f78ac7383b4e56cfb6cbacc77da89 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Brian Curtin date: 2018-09-29T10:33:05-06:00 summary: bpo-27351: Fix ConfigParser.read() documentation and docstring (GH-8123) Switch "list" with "iterable" to match with the implementation. (cherry picked from commit e45473e3ca31e5b78dc85cab575f5bb60d5b7f8f) Co-authored-by: Zackery Spytz files: M Doc/library/configparser.rst M Lib/configparser.py diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index 4e55623f5360..0bd03b6259ee 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -995,16 +995,17 @@ ConfigParser Objects .. method:: read(filenames, encoding=None) - Attempt to read and parse a list of filenames, returning a list of + Attempt to read and parse an iterable of filenames, returning a list of filenames which were successfully parsed. If *filenames* is a string, a :class:`bytes` object or a :term:`path-like object`, it is treated as a single filename. If a file named in *filenames* cannot be opened, that - file will be ignored. This is designed so that you can specify a list of - potential configuration file locations (for example, the current - directory, the user's home directory, and some system-wide directory), - and all existing configuration files in the list will be read. + file will be ignored. This is designed so that you can specify an + iterable of potential configuration file locations (for example, the + current directory, the user's home directory, and some system-wide + directory), and all existing configuration files in the iterable will be + read. If none of the named files exist, the :class:`ConfigParser` instance will contain an empty dataset. An application which requires diff --git a/Lib/configparser.py b/Lib/configparser.py index 445fa99ccb4b..4caf3faadc75 100644 --- a/Lib/configparser.py +++ b/Lib/configparser.py @@ -80,7 +80,7 @@ Return list of configuration options for the named section. read(filenames, encoding=None) - Read and parse the list of named configuration files, given by + Read and parse the iterable of named configuration files, given by name. A single filename is also allowed. Non-existing files are ignored. Return list of successfully read files. @@ -676,13 +676,13 @@ def options(self, section): return list(opts.keys()) def read(self, filenames, encoding=None): - """Read and parse a filename or a list of filenames. + """Read and parse a filename or an iterable of filenames. Files that cannot be opened are silently ignored; this is - designed so that you can specify a list of potential + designed so that you can specify an iterable of potential configuration file locations (e.g. current directory, user's home directory, systemwide directory), and all existing - configuration files in the list will be read. A single + configuration files in the iterable will be read. A single filename may also be given. Return list of successfully read files. From webhook-mailer at python.org Sat Sep 29 12:39:42 2018 From: webhook-mailer at python.org (Brian Curtin) Date: Sat, 29 Sep 2018 16:39:42 -0000 Subject: [Python-checkins] bpo-27351: Fix ConfigParser.read() documentation and docstring (GH-8123) Message-ID: https://github.com/python/cpython/commit/3cd5e8e83c9785d9f505138903c7a50dc964101e commit: 3cd5e8e83c9785d9f505138903c7a50dc964101e branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Brian Curtin date: 2018-09-29T10:39:39-06:00 summary: bpo-27351: Fix ConfigParser.read() documentation and docstring (GH-8123) Switch "list" with "iterable" to match with the implementation. (cherry picked from commit e45473e3ca31e5b78dc85cab575f5bb60d5b7f8f) Co-authored-by: Zackery Spytz files: M Doc/library/configparser.rst M Lib/configparser.py diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index ba107274f7be..4abc6aa75606 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -982,15 +982,16 @@ ConfigParser Objects .. method:: read(filenames, encoding=None) - Attempt to read and parse a list of filenames, returning a list of + Attempt to read and parse an iterable of filenames, returning a list of filenames which were successfully parsed. If *filenames* is a string or :term:`path-like object`, it is treated as a single filename. If a file named in *filenames* cannot be opened, that - file will be ignored. This is designed so that you can specify a list of - potential configuration file locations (for example, the current - directory, the user's home directory, and some system-wide directory), - and all existing configuration files in the list will be read. + file will be ignored. This is designed so that you can specify an + iterable of potential configuration file locations (for example, the + current directory, the user's home directory, and some system-wide + directory), and all existing configuration files in the iterable will be + read. If none of the named files exist, the :class:`ConfigParser` instance will contain an empty dataset. An application which requires diff --git a/Lib/configparser.py b/Lib/configparser.py index 230ab2b017ea..0e529e9693ab 100644 --- a/Lib/configparser.py +++ b/Lib/configparser.py @@ -80,7 +80,7 @@ Return list of configuration options for the named section. read(filenames, encoding=None) - Read and parse the list of named configuration files, given by + Read and parse the iterable of named configuration files, given by name. A single filename is also allowed. Non-existing files are ignored. Return list of successfully read files. @@ -677,13 +677,13 @@ def options(self, section): return list(opts.keys()) def read(self, filenames, encoding=None): - """Read and parse a filename or a list of filenames. + """Read and parse a filename or an iterable of filenames. Files that cannot be opened are silently ignored; this is - designed so that you can specify a list of potential + designed so that you can specify an iterable of potential configuration file locations (e.g. current directory, user's home directory, systemwide directory), and all existing - configuration files in the list will be read. A single + configuration files in the iterable will be read. A single filename may also be given. Return list of successfully read files. From webhook-mailer at python.org Sat Sep 29 12:57:38 2018 From: webhook-mailer at python.org (Brian Curtin) Date: Sat, 29 Sep 2018 16:57:38 -0000 Subject: [Python-checkins] [2.7] bpo-13407: Mention that bz2/tarfile doesn't support multi-stream bzip2 files (GH-8428) Message-ID: https://github.com/python/cpython/commit/8d3b0f49021e6cd25030a1eb979218cfceb44061 commit: 8d3b0f49021e6cd25030a1eb979218cfceb44061 branch: 2.7 author: Andr?s Delfino committer: Brian Curtin date: 2018-09-29T10:57:35-06:00 summary: [2.7] bpo-13407: Mention that bz2/tarfile doesn't support multi-stream bzip2 files (GH-8428) * bpo-13407: Mention that tarfile doesn't support multistream bzip2 files * Add mention to bz2 module also files: A Misc/NEWS.d/next/Documentation/2018-09-29-13-40-06.bpo-13407.BRI-_D.rst M Doc/library/bz2.rst M Doc/library/tarfile.rst diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst index e764f3939544..478a8423f3ff 100644 --- a/Doc/library/bz2.rst +++ b/Doc/library/bz2.rst @@ -35,6 +35,10 @@ Here is a summary of the features offered by the bz2 module: * Thread safety uses individual locking mechanism. +.. note:: + Handling of multi-stream bzip2 files is not supported. Modules such as + `bz2file `_ let you overcome this. + (De)compression of files ------------------------ diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst index c819bf500a19..5789e5efd5a9 100644 --- a/Doc/library/tarfile.rst +++ b/Doc/library/tarfile.rst @@ -37,6 +37,10 @@ Some facts and figures: character devices and block devices and is able to acquire and restore file information like timestamp, access permissions and owner. +.. note:: + Handling of multi-stream bzip2 files is not supported. Modules such as + `bz2file `_ let you overcome this. + .. function:: open(name=None, mode='r', fileobj=None, bufsize=10240, \*\*kwargs) diff --git a/Misc/NEWS.d/next/Documentation/2018-09-29-13-40-06.bpo-13407.BRI-_D.rst b/Misc/NEWS.d/next/Documentation/2018-09-29-13-40-06.bpo-13407.BRI-_D.rst new file mode 100644 index 000000000000..e1766932c86c --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-09-29-13-40-06.bpo-13407.BRI-_D.rst @@ -0,0 +1,2 @@ +Add a note to :mod:`bz2` and :mod:`tarfile` stating that handling of +multi-stream bzip2 files is not supported. From webhook-mailer at python.org Sat Sep 29 17:30:42 2018 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sat, 29 Sep 2018 21:30:42 -0000 Subject: [Python-checkins] Speed-up math.dist() by 30% (GH-9628) Message-ID: https://github.com/python/cpython/commit/df8101517aa1c917fdf8aeb466e480c26d4e878c commit: df8101517aa1c917fdf8aeb466e480c26d4e878c branch: master author: Raymond Hettinger committer: GitHub date: 2018-09-29T14:30:38-07:00 summary: Speed-up math.dist() by 30% (GH-9628) files: M Modules/clinic/mathmodule.c.h M Modules/mathmodule.c diff --git a/Modules/clinic/mathmodule.c.h b/Modules/clinic/mathmodule.c.h index c4d278634101..b40a227dbea5 100644 --- a/Modules/clinic/mathmodule.c.h +++ b/Modules/clinic/mathmodule.c.h @@ -294,8 +294,9 @@ math_dist(PyObject *module, PyObject *const *args, Py_ssize_t nargs) PyObject *p; PyObject *q; - if (!_PyArg_ParseStack(args, nargs, "O!O!:dist", - &PyTuple_Type, &p, &PyTuple_Type, &q)) { + if (!_PyArg_UnpackStack(args, nargs, "dist", + 2, 2, + &p, &q)) { goto exit; } return_value = math_dist_impl(module, p, q); @@ -522,4 +523,4 @@ math_isclose(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject exit: return return_value; } -/*[clinic end generated code: output=d936137c1189b89b input=a9049054013a1b77]*/ +/*[clinic end generated code: output=239c51a5acefbafb input=a9049054013a1b77]*/ diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index e872e473f5fe..e956314e27fc 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -2101,8 +2101,8 @@ vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) /*[clinic input] math.dist - p: object(subclass_of='&PyTuple_Type') - q: object(subclass_of='&PyTuple_Type') + p: object + q: object / Return the Euclidean distance between two points p and q. @@ -2116,7 +2116,7 @@ Roughly equivalent to: static PyObject * math_dist_impl(PyObject *module, PyObject *p, PyObject *q) -/*[clinic end generated code: output=56bd9538d06bbcfe input=937122eaa5f19272]*/ +/*[clinic end generated code: output=56bd9538d06bbcfe input=8c83c07c7a524664]*/ { PyObject *item; double max = 0.0; @@ -2126,6 +2126,11 @@ math_dist_impl(PyObject *module, PyObject *p, PyObject *q) double diffs_on_stack[NUM_STACK_ELEMS]; double *diffs = diffs_on_stack; + if (!PyTuple_Check(p) || !PyTuple_Check(q)) { + PyErr_SetString(PyExc_TypeError, "dist argument must be a tuple"); + return NULL; + } + m = PyTuple_GET_SIZE(p); n = PyTuple_GET_SIZE(q); if (m != n) { From webhook-mailer at python.org Sun Sep 30 01:28:43 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 30 Sep 2018 05:28:43 -0000 Subject: [Python-checkins] bpo-34849: Don't log wating for selector.select in asyncio loop iteration (GH-9641) Message-ID: https://github.com/python/cpython/commit/d5bd036138881bb90a803397d992870a46fbdc2d commit: d5bd036138881bb90a803397d992870a46fbdc2d branch: master author: Andrew Svetlov committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2018-09-29T22:28:40-07:00 summary: bpo-34849: Don't log wating for selector.select in asyncio loop iteration (GH-9641) The waiting is pretty normal for any asyncio program, logging its time just adds a noise to logs without any useful information provided. https://bugs.python.org/issue34849 files: A Misc/NEWS.d/next/Library/2018-09-30-08-08-14.bpo-34849.NXK9Ff.rst M Lib/asyncio/base_events.py M Lib/test/test_asyncio/test_base_events.py diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index 492e377d09e0..780a06192dcf 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -1719,28 +1719,7 @@ def _run_once(self): when = self._scheduled[0]._when timeout = min(max(0, when - self.time()), MAXIMUM_SELECT_TIMEOUT) - if self._debug and timeout != 0: - t0 = self.time() - event_list = self._selector.select(timeout) - dt = self.time() - t0 - if dt >= 1.0: - level = logging.INFO - else: - level = logging.DEBUG - nevent = len(event_list) - if timeout is None: - logger.log(level, 'poll took %.3f ms: %s events', - dt * 1e3, nevent) - elif nevent: - logger.log(level, - 'poll %.3f ms took %.3f ms: %s events', - timeout * 1e3, dt * 1e3, nevent) - elif dt >= 1.0: - logger.log(level, - 'poll %.3f ms took %.3f ms: timeout', - timeout * 1e3, dt * 1e3) - else: - event_list = self._selector.select(timeout) + event_list = self._selector.select(timeout) self._process_events(event_list) # Handle 'later' callbacks that are ready. diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py index 95f4f6b82d53..42244699372a 100644 --- a/Lib/test/test_asyncio/test_base_events.py +++ b/Lib/test/test_asyncio/test_base_events.py @@ -371,31 +371,6 @@ def test_set_debug(self): self.loop.set_debug(False) self.assertFalse(self.loop.get_debug()) - @mock.patch('asyncio.base_events.logger') - def test__run_once_logging(self, m_logger): - def slow_select(timeout): - # Sleep a bit longer than a second to avoid timer resolution - # issues. - time.sleep(1.1) - return [] - - # logging needs debug flag - self.loop.set_debug(True) - - # Log to INFO level if timeout > 1.0 sec. - self.loop._selector.select = slow_select - self.loop._process_events = mock.Mock() - self.loop._run_once() - self.assertEqual(logging.INFO, m_logger.log.call_args[0][0]) - - def fast_select(timeout): - time.sleep(0.001) - return [] - - self.loop._selector.select = fast_select - self.loop._run_once() - self.assertEqual(logging.DEBUG, m_logger.log.call_args[0][0]) - def test__run_once_schedule_handle(self): handle = None processed = False diff --git a/Misc/NEWS.d/next/Library/2018-09-30-08-08-14.bpo-34849.NXK9Ff.rst b/Misc/NEWS.d/next/Library/2018-09-30-08-08-14.bpo-34849.NXK9Ff.rst new file mode 100644 index 000000000000..6f5321ce4cf7 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-30-08-08-14.bpo-34849.NXK9Ff.rst @@ -0,0 +1,3 @@ +Don't log wating for ``selector.select`` in asyncio loop iteration. The +waiting is pretty normal for any asyncio program, logging its time just adds +a noise to logs without any useful information provided. From solipsis at pitrou.net Sun Sep 30 05:09:16 2018 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 30 Sep 2018 09:09:16 +0000 Subject: [Python-checkins] Daily reference leaks (4243df51fe43): sum=8 Message-ID: <20180930090916.1.579376A4336157DC@psf.io> results for 4243df51fe43 on branch "default" -------------------------------------------- test_asyncio leaked [0, 3, 0] memory blocks, sum=3 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_forkserver leaked [-1, 0, 2] memory blocks, sum=1 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogDk_bm5', '--timeout', '7200'] From webhook-mailer at python.org Sun Sep 30 14:07:10 2018 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sun, 30 Sep 2018 18:07:10 -0000 Subject: [Python-checkins] bpo-34854: Fix compiling string annotations containing lambdas. (GH-9645) Message-ID: https://github.com/python/cpython/commit/2a2940e5c3e6d92f4fac5e9d361a1e224bb2f12e commit: 2a2940e5c3e6d92f4fac5e9d361a1e224bb2f12e branch: master author: Serhiy Storchaka committer: GitHub date: 2018-09-30T21:07:05+03:00 summary: bpo-34854: Fix compiling string annotations containing lambdas. (GH-9645) * Compiling a string annotation containing a lambda with keyword-only argument without default value caused a crash. * Remove the final "*" (it is incorrect syntax) in the representation of lambda without *args and keyword-only arguments when compile from AST. * Improve the representation of lambda without arguments. files: A Misc/NEWS.d/next/Core and Builtins/2018-09-30-19-27-13.bpo-34854.6TKTcB.rst M Lib/test/test_future.py M Python/ast_unparse.c diff --git a/Lib/test/test_future.py b/Lib/test/test_future.py index 904e8a9d5dfc..4f2f9d272e7f 100644 --- a/Lib/test/test_future.py +++ b/Lib/test/test_future.py @@ -178,11 +178,12 @@ def test_annotations(self): eq('-1') eq('~int and not v1 ^ 123 + v2 | True') eq('a + (not b)') + eq('lambda: None') eq('lambda arg: None') eq('lambda a=True: a') eq('lambda a, b, c=True: a') eq("lambda a, b, c=True, *, d=1 << v2, e='str': a") - eq("lambda a, b, c=True, *vararg, d=v1 << 2, e='str', **kwargs: a + b") + eq("lambda a, b, c=True, *vararg, d, e='str', **kwargs: a + b") eq('lambda x: lambda y: x + y') eq('1 if True else 2') eq('str or None if int or True else str or bytes or None') diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-30-19-27-13.bpo-34854.6TKTcB.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-30-19-27-13.bpo-34854.6TKTcB.rst new file mode 100644 index 000000000000..4e04e1f157cd --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-30-19-27-13.bpo-34854.6TKTcB.rst @@ -0,0 +1,2 @@ +Fixed a crash in compiling string annotations containing a lambda with a +keyword-only argument that doesn't have a default value. diff --git a/Python/ast_unparse.c b/Python/ast_unparse.c index 56ea3c424c5b..916ad5f97f0c 100644 --- a/Python/ast_unparse.c +++ b/Python/ast_unparse.c @@ -212,7 +212,7 @@ append_ast_args(_PyUnicodeWriter *writer, arguments_ty args) } /* vararg, or bare '*' if no varargs but keyword-only arguments present */ - if (args->vararg || args->kwonlyargs) { + if (args->vararg || asdl_seq_LEN(args->kwonlyargs)) { APPEND_STR_IF_NOT_FIRST(", "); APPEND_STR("*"); if (args->vararg) { @@ -229,8 +229,11 @@ append_ast_args(_PyUnicodeWriter *writer, arguments_ty args) di = i - arg_count + default_count; if (di >= 0) { - APPEND_STR("="); - APPEND_EXPR((expr_ty)asdl_seq_GET(args->kw_defaults, di), PR_TEST); + expr_ty default_ = (expr_ty)asdl_seq_GET(args->kw_defaults, di); + if (default_) { + APPEND_STR("="); + APPEND_EXPR(default_, PR_TEST); + } } } @@ -248,7 +251,7 @@ static int append_ast_lambda(_PyUnicodeWriter *writer, expr_ty e, int level) { APPEND_STR_IF(level > PR_TEST, "("); - APPEND_STR("lambda "); + APPEND_STR(asdl_seq_LEN(e->v.Lambda.args->args) ? "lambda " : "lambda"); APPEND(args, e->v.Lambda.args); APPEND_STR(": "); APPEND_EXPR(e->v.Lambda.body, PR_TEST); From webhook-mailer at python.org Sun Sep 30 14:19:20 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 30 Sep 2018 18:19:20 -0000 Subject: [Python-checkins] bpo-34854: Fix compiling string annotations containing lambdas. (GH-9645) Message-ID: https://github.com/python/cpython/commit/0f161b307969f86b4f8f31baf38f53f5462a9874 commit: 0f161b307969f86b4f8f31baf38f53f5462a9874 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-30T11:19:15-07:00 summary: bpo-34854: Fix compiling string annotations containing lambdas. (GH-9645) * Compiling a string annotation containing a lambda with keyword-only argument without default value caused a crash. * Remove the final "*" (it is incorrect syntax) in the representation of lambda without *args and keyword-only arguments when compile from AST. * Improve the representation of lambda without arguments. (cherry picked from commit 2a2940e5c3e6d92f4fac5e9d361a1e224bb2f12e) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Core and Builtins/2018-09-30-19-27-13.bpo-34854.6TKTcB.rst M Lib/test/test_future.py M Python/ast_unparse.c diff --git a/Lib/test/test_future.py b/Lib/test/test_future.py index 61cd63479d85..70da0cf57040 100644 --- a/Lib/test/test_future.py +++ b/Lib/test/test_future.py @@ -178,11 +178,12 @@ def test_annotations(self): eq('-1') eq('~int and not v1 ^ 123 + v2 | True') eq('a + (not b)') + eq('lambda: None') eq('lambda arg: None') eq('lambda a=True: a') eq('lambda a, b, c=True: a') eq("lambda a, b, c=True, *, d=1 << v2, e='str': a") - eq("lambda a, b, c=True, *vararg, d=v1 << 2, e='str', **kwargs: a + b") + eq("lambda a, b, c=True, *vararg, d, e='str', **kwargs: a + b") eq('lambda x: lambda y: x + y') eq('1 if True else 2') eq('str or None if int or True else str or bytes or None') diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-30-19-27-13.bpo-34854.6TKTcB.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-30-19-27-13.bpo-34854.6TKTcB.rst new file mode 100644 index 000000000000..4e04e1f157cd --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-30-19-27-13.bpo-34854.6TKTcB.rst @@ -0,0 +1,2 @@ +Fixed a crash in compiling string annotations containing a lambda with a +keyword-only argument that doesn't have a default value. diff --git a/Python/ast_unparse.c b/Python/ast_unparse.c index 725ce31fe3c0..43453f567b05 100644 --- a/Python/ast_unparse.c +++ b/Python/ast_unparse.c @@ -212,7 +212,7 @@ append_ast_args(_PyUnicodeWriter *writer, arguments_ty args) } /* vararg, or bare '*' if no varargs but keyword-only arguments present */ - if (args->vararg || args->kwonlyargs) { + if (args->vararg || asdl_seq_LEN(args->kwonlyargs)) { APPEND_STR_IF_NOT_FIRST(", "); APPEND_STR("*"); if (args->vararg) { @@ -229,8 +229,11 @@ append_ast_args(_PyUnicodeWriter *writer, arguments_ty args) di = i - arg_count + default_count; if (di >= 0) { - APPEND_STR("="); - APPEND_EXPR((expr_ty)asdl_seq_GET(args->kw_defaults, di), PR_TEST); + expr_ty default_ = (expr_ty)asdl_seq_GET(args->kw_defaults, di); + if (default_) { + APPEND_STR("="); + APPEND_EXPR(default_, PR_TEST); + } } } @@ -248,7 +251,7 @@ static int append_ast_lambda(_PyUnicodeWriter *writer, expr_ty e, int level) { APPEND_STR_IF(level > PR_TEST, "("); - APPEND_STR("lambda "); + APPEND_STR(asdl_seq_LEN(e->v.Lambda.args->args) ? "lambda " : "lambda"); APPEND(args, e->v.Lambda.args); APPEND_STR(": "); APPEND_EXPR(e->v.Lambda.body, PR_TEST); From webhook-mailer at python.org Sun Sep 30 17:17:21 2018 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Sun, 30 Sep 2018 21:17:21 -0000 Subject: [Python-checkins] bpo-34850: Replace is with == in idlelib.iomenu (GH-9649) Message-ID: https://github.com/python/cpython/commit/5fa247d60d4f3f2b8c8ae8cb57363aca234344c2 commit: 5fa247d60d4f3f2b8c8ae8cb57363aca234344c2 branch: master author: Terry Jan Reedy committer: GitHub date: 2018-09-30T17:17:17-04:00 summary: bpo-34850: Replace is with == in idlelib.iomenu (GH-9649) Patch by Serhiy Storchaka (in PR #9642). files: M Lib/idlelib/iomenu.py diff --git a/Lib/idlelib/iomenu.py b/Lib/idlelib/iomenu.py index 63d107dd9003..fcd8dcc1393d 100644 --- a/Lib/idlelib/iomenu.py +++ b/Lib/idlelib/iomenu.py @@ -40,7 +40,7 @@ # resulting codeset may be unknown to Python. We ignore all # these problems, falling back to ASCII locale_encoding = locale.nl_langinfo(locale.CODESET) - if locale_encoding is None or locale_encoding is '': + if locale_encoding is None or locale_encoding == '': # situation occurs on Mac OS X locale_encoding = 'ascii' codecs.lookup(locale_encoding) @@ -50,7 +50,7 @@ # bugs that can cause ValueError. try: locale_encoding = locale.getdefaultlocale()[1] - if locale_encoding is None or locale_encoding is '': + if locale_encoding is None or locale_encoding == '': # situation occurs on Mac OS X locale_encoding = 'ascii' codecs.lookup(locale_encoding) From webhook-mailer at python.org Sun Sep 30 17:35:57 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 30 Sep 2018 21:35:57 -0000 Subject: [Python-checkins] bpo-34850: Replace is with == in idlelib.iomenu (GH-9649) Message-ID: https://github.com/python/cpython/commit/214c0b3d153c4bad14086888b9de0826a7abc083 commit: 214c0b3d153c4bad14086888b9de0826a7abc083 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-30T14:35:53-07:00 summary: bpo-34850: Replace is with == in idlelib.iomenu (GH-9649) Patch by Serhiy Storchaka (in PR GH-9642). (cherry picked from commit 5fa247d60d4f3f2b8c8ae8cb57363aca234344c2) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/iomenu.py diff --git a/Lib/idlelib/iomenu.py b/Lib/idlelib/iomenu.py index 63d107dd9003..fcd8dcc1393d 100644 --- a/Lib/idlelib/iomenu.py +++ b/Lib/idlelib/iomenu.py @@ -40,7 +40,7 @@ # resulting codeset may be unknown to Python. We ignore all # these problems, falling back to ASCII locale_encoding = locale.nl_langinfo(locale.CODESET) - if locale_encoding is None or locale_encoding is '': + if locale_encoding is None or locale_encoding == '': # situation occurs on Mac OS X locale_encoding = 'ascii' codecs.lookup(locale_encoding) @@ -50,7 +50,7 @@ # bugs that can cause ValueError. try: locale_encoding = locale.getdefaultlocale()[1] - if locale_encoding is None or locale_encoding is '': + if locale_encoding is None or locale_encoding == '': # situation occurs on Mac OS X locale_encoding = 'ascii' codecs.lookup(locale_encoding) From webhook-mailer at python.org Sun Sep 30 17:40:59 2018 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 30 Sep 2018 21:40:59 -0000 Subject: [Python-checkins] bpo-34850: Replace is with == in idlelib.iomenu (GH-9649) Message-ID: https://github.com/python/cpython/commit/cb0bec37dd8279555bc01fa03a259eaf7dbb6d5d commit: cb0bec37dd8279555bc01fa03a259eaf7dbb6d5d branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2018-09-30T14:40:56-07:00 summary: bpo-34850: Replace is with == in idlelib.iomenu (GH-9649) Patch by Serhiy Storchaka (in PR GH-9642). (cherry picked from commit 5fa247d60d4f3f2b8c8ae8cb57363aca234344c2) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/iomenu.py diff --git a/Lib/idlelib/iomenu.py b/Lib/idlelib/iomenu.py index 63d107dd9003..fcd8dcc1393d 100644 --- a/Lib/idlelib/iomenu.py +++ b/Lib/idlelib/iomenu.py @@ -40,7 +40,7 @@ # resulting codeset may be unknown to Python. We ignore all # these problems, falling back to ASCII locale_encoding = locale.nl_langinfo(locale.CODESET) - if locale_encoding is None or locale_encoding is '': + if locale_encoding is None or locale_encoding == '': # situation occurs on Mac OS X locale_encoding = 'ascii' codecs.lookup(locale_encoding) @@ -50,7 +50,7 @@ # bugs that can cause ValueError. try: locale_encoding = locale.getdefaultlocale()[1] - if locale_encoding is None or locale_encoding is '': + if locale_encoding is None or locale_encoding == '': # situation occurs on Mac OS X locale_encoding = 'ascii' codecs.lookup(locale_encoding)